When you have an affair with someone, you become invested in them either physically, mentally, or emotionally. You build a soul tie with them that may mimic a marriage bond and can be hard to break.