Most men experience ED from time to time, affecting about one-quarter of men under 40, with this rate climbing as you get older. In some cases, the urge to have sex, called sexual desire or libido, may decline along with the ability to achieve an erection.