A man’s sex drive may depend on his level of testosterone. Also, many medications can affect sex, drive such as antidepressants, antihypertensives, and sedatives. Chronic opioid use can lead to a lower testosterone and a decrease in sex drive.