Generally speaking, men don’t need to wear anything under their board shorts as men’s board shorts typically have some type of liner for additional support. If your board shorts don’t have a liner, you can also choose to wear swim briefs.