Assuming that there is trustworthy adult at your home at the time of the scheduled drop off I see no reason why the "ex" is refusing to do such other than to be "difficult".Unless there is a court order in place stating that you have to be physically present at the time of the drop off a trustworthy adult can be present to take charge of the minor children until you get home.To enforce the drop off you should consult with a family law attorney to write the "ex" and if he refuses to abide, then a petition with the court to enforce matters seems to be the result.