So long as you have not given up your children for adoption you have the opportunity to regain physical and legal custody of them. In order to do so I suggest that you consult with a family law attorney to see about getting a stipulation from the father for your cusotdy and if he refuses, the filing of a petition with the court seeking such.