If you love someone and trust them, and then find out that they have been sexually unfaithful to you, it is emotionally devastating to say the least. God wants to protect people from this - therefore he forbids adultery. To allow it would make every marriage insecure Women in general pay the greater price in a relationship. Women are usually emotionally more sensitive than men and often have a greater need for love and security. To the extent that adultery and divorce are accepted in society - to that extent women will be exploited and robbed of the security and true love they need.
Men also suffer when marriages break down.
If faithfulness in marriage was not important, then no marriage would be safe. There are always going to be times when the thought comes that things could be better with someone else. If there was no safeguard against this, families would break down, and so would the moral fabric of society. Perhaps you yourself have already been a victim of some of this. It hurts the children Children love to see that their parents love each other.