Ask for a guided tour during those hours to make sure the gym isn't too crowded. Ask about the classes offered and ask to sit in on ones that you might want to join. Check out the equipment to see if it's in good shape and easy to use. Ask about fees for such extras as a masseuse, personal trainers and child care. Atmosphere also will vary.
Some women prefer The Lady's Spa because it's for women only.
Others join a gym to socialize. Last but not least - Make sure the gym is squeaky clean.