This is what Joseph supposed that Mary had done. There is NO Biblical allowance for two Christians who enter into a covenant marriage before God and law to ever be divorced. This simply means you may allow them to leave if they have made up their mind. But, you are called to peace. Every effort must be made to keep the marriage together.
Marriage is an unbreakable lifetime contract to the believer.
The unbeliever may not have the same understanding or conviction, so may decide to leave despite all the believers pleadings and peacemaking. In this case, a believer is not under bondage and may be remarried, but only in the Lord. We know as Christians we are commanded to marry only in the Lord. You see, they were obviously married while they were both unsaved because their children were unclean. But, now they are holy because one of them has since become saved.