Yes, you can swim without one especially if you're bald or have short hair. However, most pools require caps and without a cap your hair will get wet and chlorine can damage it over time. Your head may also feel cooler without a cap.