If your lack of sex life has become a constant source of conflict or contempt, or if your partner doesn't want to discuss the issue or make any changes, it's time to consider ending the relationship. While sex isn't the most crucial thing in a long-term relationship, it is an important way to feel connected and loved.