Protect the kids at all costs. The behaviour you describe is abusive. Child support is not optional. Unless he is competing for the preparation H man of the year award, he would not even suggest messing with that. It doesn't matter what happened between you and your ex, a man takes care of his kids.
Nothing gets in the way of that.
A father's house is always open to his children. As long as he is competent, and his children or even his grandchildren are incapable of sustaining themselves, he is compelled to support them. I can completely relate. The only difference being i left him for no one and he found another mate right away. I'm relieved to be away from him because i think that is where we were headed.