Spending the time to remove your contacts when you're fatigued might feel like a nuisance. However, sleeping with them limits oxygen supply to the eye and exposes you to infection. And so it is best advised to remove your contact lenses before you fall asleep.