Some people have allergies or reactions to certain plants. Figuring out which plants people react to and why is no easy task. The fact that allergies change over a lifetime contributes to the challenge. Everyone has skin reactions to some plants, such as poison ivy. Other plants cause sickness or death when ingested in part or whole.
Some gardeners try to avoid all potentially poisonous plants.
I say, impossible. While certainly I won't grow some plants until my children are grown, they should neither be afraid or fooled. I teach them which plants are safe to eat and touch and to not test the rest. They have a strong understanding and respect for the uses of plants. Check your plant encyclopedia for plants with poisonous parts.