I am so sorry for your situation.  Having someone declared an unfit parent can be a difficult - but not impossible - task.  The burden of proof is obviously on you.  Generally speaking courts do not like to take sides in these matters but the focus is always on what is in the best interests of the children.  If the drugs do not show up in testing then you are going to have to prove your case by other means: actions, in placing the children in a dangerous situation; in showing abuse, etc.  You may have to call the police when he gets violent again.  Apply for a restraining order.  I think that it would be best for you to consult with an attorney in your area on the matter.  This way you know what you are up against.  Good luck to you.