Stress can increase cortisol levels in the body. High cortisol levels have been frequently associated with a decrease in sexual desire. Also, stress can alter the mood, which is also important in maintaining sexual function. Chronic stress can cause adrenal fatigue, which can lead to a decrease in relevant sex hormone production.