No, wearing spectacles or contact lenses during water sports are very risky. While the spectacle can break or get lost while you are enjoying your ride, the soft contact lenses are more likely to remain in your eyes. However, they are porous and can absorb chemicals and bacteria, increasing the risk of eye irritation and infection.
Hence, it is advisable to wear swim goggles if you are retaining your contact lenses during the water sports.