First off, Assuming your spindle runs true. If not or you don't know, start from there.
Since you mentioned tapping on the chuck gives you no improvement, assuming you are loosening the chuck from the adapter slightly when doing so then retighten, I'm assuming your chuck mates the the adaptor's register pretty well. If that's the case as you know it won't do anything.
Now you can make your chuck an "tap-tru" but it's not recommeneded & I don't recommend it. I've done it to the stock 3-jaw that came with my lathe. But I only keep it around for dirty work like heaving sand, yada yada. I skimmed the register on the adaptor so it was slightly smaller than the register on the chuck. So now it's a loose fit. Now I can tap the chuck to run true. But if you do any heavy cutting or bump it pretty hard, it may run out again. That is why it's not recommened. For light use it will be fine.
Check the runout of your chuck using a a guage pin, shank of an endmill, or even drill rod. I use linear shafting. Remove the piece, chuck it back up & indicate again. Does it repeat your measurement fairly well or is it all over the place? This will tell you if you can just regrind the jaws or if your chuck is very worn.
If it repeats fairly well, what you can do in the mean time (should you decide to try & fix the issue), you might get away with indexing the part before removing it. Say you're working on a piece & you need to remove it for some reason. Mark a line on the part as well as on the chuck. I just use one of the jaws for indexing. When you put it back in the chuck, align the marks & then indicate. You can turn the part CW or CCW to get closer to where you were if off. But again, this only works if your scroll is not worn out.