Swimming in a pool without chlorine is definitely not a good idea. Chlorine is a disinfectant that kills harmful microorganisms and bacteria in the water, making it safe for swimming.
Without it, you're essentially swimming in a petri dish of germs and bacteria. So, it's best to stick with a properly chlorinated pool.