Also, some men and boys will not go out with their chests uncovered. The more Orthodox a movement is, the greater the concern about modesty in swimming attire. Hence, it is difficult to give a broad rule. Similarly, men should wear boxer-style trunks. Lastly, remember that it is not a good idea in general to start dressing very young children in ways that mimic sexually suggestive adult dress.
Many people do so for other reasons as well, such as to protect one's skin from the sun.
There is less of a concern around the immediate family.