The sense of ineffectiveness can simply be overwhelming. After a loss, a man can try to comfort his partner. Nothing seems to quite make up for the loss of a potential child. The overwhelming sense of loss of the unknown can eat away at your psyche, as easily as termites eat through wood, leaving behind a fragile shell, with all of your future plans taken away, leaving nothing but a vacuous void. The loss of what might have been can be as great as any pain you have ever felt.
Sex can seem rather unappetizing, too.
After all, that is what started the whole fiasco in the first place. Your partner may seem just as pretty and beautiful, but she might not seem as tantalizingly sexy right now. Maybe your partner thinks you are a big loser for not being able to reproduce properly. Heck, everybody else is having babies and you can't. Now all you might have is a sense of loss that only time can heal.