I worked for six years as an analytical chemist for a battery manufacturer and dealt with many chemicals that could be dangerous if not handled properly. One task that fell on me was disposing of 10 lbs. of sodium cyanide that the machine ship no longer wanted. PPE in the '70's consisted of safety glasses and a lab coat. I did some research and determined that cyanide could be completely neutralized with common bleach, converting it into NaCl, CO2, N2, and H2O. This was done in a fume hood , slowly adding the bleach to prevent excessive foaming from the gas generation over a period of several days.
The point of this story is proper handling of hazardous materials is the key. This includes respect for the material, knowledge of its properties, and its impact on the environment, on living things, and immediate and long term consequences of mishandling. Much of the fear of various substances is propagated by lawyers, looking to become overnight multimillionaires through class action lawsuits. This not the best source for understanding the impact of a material. MSDS sheets are a good guide. They not only detail safe exposure limits but also proper containment practices. When dealing with any hazardous material, it is also a good idea to understand the best way to deal with an accident resulting in inhalation, ingestion, or contact with skin or eyes. For new products that information is often on the label. For older materials, one should understand the proper mitigation before using.
Over the past sixty years, I have dealt with many substances that are considered extremely hazardous and I would be lying if I said there hadn't been any accidents. The only time that I had a visit to the ER was when I accidentally got some hydrofluoric acid on my fingers. Aside from being a strong acid, it has the ability to dissolve bone. The acid had gotten in around my fingernails and despite the immediate flushing with water, continued to burn. The trip to the ER was a precaution, the best action being to pack the affected area with a baking soda slurry. (If we had latex or nitrile gloves back then, it would have prevented tha trip.) I continually handle materials which are potentially hazardous without fear but with knowledge and respect.