Adultery is voluntary sexual conduct between a married person and someone else who is not his or her spouse. hat you describe would not be adultery, but would provide support for the claim that there are "irreconcilable differences"--a severe, permanent breakdown in the marital relationship--which also forms grounds for divorce.