Is that "grainy" as in little bits being torn out of the surface? Get a magnifier, and have a hard look at the cutter tip to see if there is build-up. Carbide tools, with the trick chip-breakers, are built to serve in hard industrial usage. There is actually a (tiny) radius at the cutting edge, making it strong. It needs some force to push into the metal. The finish can be great if the cut is deep enough to let the tool work as designed. It's hot, and hard, and sometimes blue chips.
Although a very sharp edge on carbide is somewhat fragile, Stefan Gotteswinter chooses to "modify" some of his carbide tools by sharpening much of the top radius away, so he can take fine finishing cuts. He is careful to have the back edge radius retained. One has to take care to keep the finish cut gentle enough so as not to damage the now sharper carbide edge. Search out Stefan's video on what he does to carbide tools, and the one about his homebrew slow-speed carbide grinder.