I made a steady rest with ball bearing fingers some time back. I've noticed that swarf can get between the bearings and work, which can result in some irregularities in the turned piece because the swarf moves the work (and steady rest) around a bit. It also can push the fingers away from the work because the peak forces involved can be rather high.
To circumvent that I have made some ad-hoc shields out of cardboard or plastic lids. Slipped over the work so they're between the steady rest and cutter. Not perfect but better than without it. I also tried using my shop vac to suck the swarf away before it gets in there but it's a pain to set up -- and noisy, too.
The other style of finger wipes off the swarf, but of course it rubs more. Neither approach has zero impact on the work.