Please, please, please go and speak with an attorney about establishing your rights now in a written agreement.  Modification of the agreement can be had later on at any time should you both agree to it.  Should he not then you will have to take him to court to do so.  The court's determine these issues "in the best interest of the children."  You seem to have the same feeling about the situation now but you may wish to establish at least one or two nights with you now as well.  Good luck.