According to the various MSD sheets, toxicity of carbon fiber, either inhaled or orally ingested, is not known. Carbon by itself is rather inert. Inhaling large quantities over an extended period of time, as when mining coal, will cause black lung disease. As far as oral ingestion, carbon in the form of activated charcoal is uses extensively for removal of impurities from drinking water. I know that I have ingested my share in eating grilled brats, hamburgers, and steaks. I have also inhaled quite a bit of carbon in the form of soot from diesel engines and from smoldering campfire.
I would be more concerned with the epoxy binders use in the carbon fiber material.