It’s an interesting question. I guess I could try a shim behind the washer to see, or I could dial the business end in on the 4-jaw and see how the face around the threaded hole runs.…I am curious if you think the problem lies in the mounting hole or if you feel the perimeter of the saw itself is not concentric.
I surface ground my washer, but could make one with a slight taper or maybe I could grind the back of the hole saw to true it. I’ll report back if I look into it further.