In general, sure. But sometimes no. Public baths are separated into women and men’s sections and you’re kinda expected to go in the nude. In the private baths, you can go in however you’d like. (If you’re having a first date there, I recommend a swimsuit - might be speaking from personal experience.)