The fear is that children will grow up in an environment confused about who their parents are and seeing their parents having sex with strangers, and become some kind of warped individual. First of all, in most groups children always know who their biological or adoptive parents are. This is important for various reasons, including biologically in terms of health issues, legally in terms of who has responsibility for the child's care and support, and emotionally since children usually prefer their parents to other less involved caretakers. It is the parent's responsibility to provide an environment that nurtures their children and provides the required consistency and safety for them. Other adults involved can be important role models for the children and if they are a consistent loving presence in the child's life, serve to expand the child's education about life and love and the variety of ways people can be.
Children benefit from seeing loving and responsible relationships as models.
It's important for them to see that how people who care about each other express their affection and deal with problems. Having more adults around is not a problem unless they are poor models for the children. Whether one or two or more, it is unhealthy adults who cause problems in children's lives. In other parts of the world, people still live in tribes and large extended families. A lot of sex is not a bad thing if that is an authentic expression of your relationship with another.