There is no policy including a ban on nuclear energy that can prevent a determined country from launching a nuclear war. So far deterrents have sufficed, and a total ban would give an enormous advantage to the first country to violate it. Plutonium is the most toxic substance in existence. It isn't the most toxic substance. There are plenty of toxic substances, and they occasionally poison people.
Plutonium is one of the easiest to keep under control.
Plutonium symbolizes nuclear war. Purely symbolic arguments have no force. Nuclear reactors are likely to have accidents with severe consequences for humanity. Chernobyl was the worst. There are now many hundreds of reactor years of experience, and Chernobyl was the only accident that injured the public.