Contacts are generally safe to wear*, but you will want to close your eyes if you anticipate a splash coming, as there is a chance of them getting flushed out with your eyes opens.
As for glasses, if they can pass The Shake Test, you are welcome to wear them on the water*. To perform The Shake Test, put your glasses on, look straight down at the ground, and shake your head vigorously. If your glasses stay securely on, they’ve passed, and are probably safe to wear on the water. If they do not pass, it is not recommended to wear them on the water.
To minimize the chances of your glasses being donated to the river, make sure that you look straight ahead going through rapids. Looking straight ahead means that any splashes will be more likely to push your glasses further onto your face than washing them off. If you turn your head, there is a greater chance that the splash will get under the lenses and wash the glasses off your face.
*Bring personal items at your own risk