In the first incarnation, which appears to be a caveman, a man's family is taken from him by raiders due to his cowardice and hesitation. Before his wife is taken away, she says, "Don't lose the children!"
The next incarnation is in Ancient Rome in which he, Hector, is a slave to a "foolish master" who loses his fortune and is compelled to kill himself by his creditors and orders Hector to join him. Hector longs to be free to find the children and wife he had before he became a slave, but he has fallen in love with another slave and forgets his waiting family.
Third incarnation: He is a Scottish crusader on his way home to his children. The master from his life in Rome as a slave is now a crusader trying to decide whether to become a priest. They travel together until Hector finds his soul mate from the life in Rome. She is a widow and wishes Hector to join her family, but his duties to the children in Scotland pull at him.
Fourth incarnation: Hector is finally forced to confront his capacity for cowardly indecision. He is a Portuguese man in The Renaissance shipwrecked on the coast of Africa. He is the master in this life, his wife from the first incarnation shipwrecked with him as his spurned lover, and the raider who spirited her away is her steadfast friend.
Fifth incarnation: He is a modern man in New York, paying the consequences of cowardly indecision and gaining the strength to address the children he lost lifetimes ago. He is joined in this life by his master/slave/friend/soul mate, and former wife Janet and her husband/raider from lifetimes past. They support him but are people who are trying to find their own way, just as in the past lives.

Question: Why does the main character not want to join his soulmate and her family?
Answer: his children
Alternative answer: he feels duty-bound to his children in Scotland