This is probably the weakest part in Bridgeport Mill heads. According to H&W Machine, this little part is almost always broke. BP thinks you are going to use a small machine screw to extract the part (which you have done). The problem is how to get the remaining broken part out of the fairly deep hole it is in. I had the same problem a while back. I tried piano wire, tweezers, tapping on the head with a soft hammer, etc. Finally got mine out by magnetizing a piece of drill rod that would fit loosely into the bore. It still took lots of wiggling of the drill rod and light tapping on the head to get the broken part to find its way out. A little WD-40 might help if there is petrified oil or grease holding back the broken part.
When you get it out, you can turn a new part yourself. Otherwise H&W Machine (and others) carry replacement parts.
Terry S.