I make no judgment on which is the most harmful. Both backgrounds are very tragic. This couple has small children. Children know when their parents are living separate lives and no longer sleep in the same bedroom. As children grow older, they understand the significance.
Children also sense when one partner is unfaithful to the other.
They feel incredible guilt if parents are staying together only for their sakes. Parents may deceive other people, but not their own children. Persevere with your love. A bad marriage need not remain bad forever. Generations of unhappiness hound the human family.