Depending on your feeds/speeds and DOC/WOC, I think you're getting some vibration. I suspect the head is bouncing up and down, or the column is flexing - the marks that appear as a radial pattern in the upper part of the second photo look like the cutter is doing a 'bite-skip-bite-skip' routine. The fly cutter may simply be too much of a shock load for your mill frame.
The other thing is you didn't mention - or I missed it - what you're using for lubricant or coolant. The second photo appears to have some galling/smearing that could be an indicator of chips sticking to the cutter or the cutter getting hot.
About the only thing I can suggest - short of seriously reinforcing the column with triangulated braces - is to take another light cut after tightening everything up. Set the cutter to cut about 0.005-0.010 deep and tighten the Z-gibs until it won't move. Tighten the Y-gibs same way. Take a cut in X using plenty of lube (WD40 or kerosene if you're not running flood) and see if it's any better.
Do you have a big bullnose endmill? say, about 1/2"? If that surfaces better than the fly cutter (ignore pattern - just the surface finish/scratches) then I think I'm right about the frame simply being too floppy. What RPM and IPM for those cuts?