It isn't the most toxic substance. There are plenty of toxic substances, and they occasionally poison people. Plutonium is one of the easiest to keep under control. Plutonium symbolizes nuclear war. Purely symbolic arguments have no force.
Nuclear reactors are likely to have accidents with severe consequences for humanity.
Chernobyl was the worst. There are now many hundreds of reactor years of experience, and Chernobyl was the only accident that injured the public. That's a good record for a source of energy. Radiation from operating nuclear reactors, uranium mining and other nuclear activities harm humanity. There is now enough experience so that the harm is extremely small compared to other human activities and especially compared to other sources of energy.