Given your concerns as to your son's safety when he is with his father I suggest that you consult with a family law attorney to give you some more guidance as to possible edits and modifications to the parenting plan in place assuming the father is willing to sign a change to the document.