Even though this sign was there, plenty of swimmers and surfers were in the water on the far right