When the traveler returns, they find out that their special lover is no longer that, in fact they are someone elses lover now. The traveler feels extremely hurt, because they were faithful the entire time they traveled. But, the traveler was the first to leave and didn't invite their companion to join them. This had harmed the companion first so much that they couldn't deal. The one who did not travel was acting ethically by living their own life and finding a new lover even though the traveler was harmed.
People have to live for themselves much of the time even if others are harmed by their actions.