Worth testing surfaces with a surface plate, or other flat plate (gauge plate) and Engineers' blue. It may sound "old fashioned", but it was how we checked for flatness or high spots in the 1960s workshop. - I guess today you have a laser thingummy or something.... but if old methods work, why not use them?
Have you tried a steel rule across the surface? (Use a light source and look for light bleed beneath). Or roll an old car piston gudgeon pin (or other ground steel parallel pin), with marking blue on it, across the surface? - 2 directions, perpendicular....
Sounds to me as if the milling was not as flat as you need?
K2