The threads of the holder should have nothing to do with the trueness of the ID of the collet. It's the runout of the ID of the collet that one is concerned with. (We are assuming the collet chuck is true, and the collet is correctly made.) Is the collet clean? If there is swarf (little chips or metal slivers) in the collet, it could affect runout. You could clean it out with some thin cardboard, like from a cracker box, or a sheet of paper. If you have compressed air, you could blow it out (being careful not to get junk in your eyes).
What does the bottom of your indicator plunger look like? Is it smooth, or does it have some faceting? My old Federal C21 had some slight faceting due to wear and being over 60 years old, sometimes it would oddly indicate. New indicator tips are inexpensive to source, or you could make your own. The thread should be 4-48. When you measured the runout was there any oil on the workpiece? Some people say not to use oil, others do. I find a very thin coat of light machine oil reduces tip stiction.
However, can we revisit what you are trying to accomplish again? If I recall correctly it has to do with valves. If you are facing the part of the valves that face the engine, not the ground edges that seal to the head, then you shouldn't care if there is 0.002 runout or 0.004. It won't matter. If you are trying to correct the ground part of the valve, then I wouldn't use the lathe at all. That one typically "grinds to fit" the head using valve grinding compound. What is the "end task" that you are trying to do?