I have been slow to get back on this problem, but here's where I am:
1) Todd at DRO has been as helpful as could be, but has not seen the problem I describe. He has sent a replacement scale to try...have not done that yet pending some new insight that I need.
2) I have gone over about 4" of X-axis travel measuring against the lathe dial indication, the DRO reading, and a 0.0001" dial indicator, resetting it at the end of its travel, about 0.050" as I recall. Might be 0.01".
3) I find that things look normal most of the time, and over long distances the DRO reading correlates with actual travel very well. However over certain even short distances, as short as 0.030", there can be an error of about 0.003". I tried very hard to capture a record of this, logging all the data. Do you know how many 0.0001's there are in 4 inches?
I got to where I had narrowed down the error, thought I had it all captured, and decided to blow any chips off the scale. Huh, problem changed, would not repeat. Without redoing the entire experiment, the particular error I was after went away, but the issue still exists in places, random places.
4) I am thinking that this is not a linear or non-linear compensation problem, as the DRO reads correctly over long distances, and I also don't think this is a scale resolution issue, though it's clear that a one micron scale would be superior.
5) I don't think there's any basic issue with the D80 DRO. I like it, it's easy to use (except compensation) and easy to read.
6) So I am suspicious about the ability of the magnetic scales to resist small ferrous chip contamination, mainly based on the change when I blew off the scale.
I am attaching two pictures, in hopes that someone here will suggest something great:
7) The first picture is the scale and reader installed on the tailstock of the carriage, which is attractive with these smaller scales as you don't lose much tailstock quill travel. I installed the scales "vertically", I would call it. You can see that it somewhat exposes the scale to chip collection, I suppose.
8) The next picture is of the scale with a magnetic "developer" held against it. This just shows the large scale magnetic patterns. There are two tracks, one at short spacing and one at much larger distance between transitions or zones. The reader has a green light that shows red when it's over the larger distance zone, by the way.
I know this is hard to read, and I apologize. It is simply reflective of how confused I actually am. I hope that someone here might really truly understand and be able to explain what is actually recorded on the scale, how the reader interprets that, and if it is possible that ferrous material could give a transient error. The error always seems to show a smaller than actual distance, by the way.
If contamination by magnetic material is the issue, then I could see some possible solutions:
1) Relocate the scale elsewhere.
2) Cover the scale better.
3) Give up and go to glass.
The difficulty with just relocation is that it seems like eventually there will be some collection of particles, so maybe not really a solution.
Anyway, I solicit your thoughts and opinions. Thanks.