If you had an affair and you can't get over it, it sounds like there is some need to break it off. If you are married, then I would definitely want to put some energy into saving the relationship. Maybe you should talk to a priest, a friend or a therapist, and then maybe bring your partner in to the therapist to see if you can improve your marriage. If you want to end the relationship and forget about the guy, then maybe you need to have a ritual burial or put the affair away. It can be great without sex.
Sex is a very personal priority.
The sexual appetites of the individuals need to fit together. It has to be mutual and satisfying. If someone wants a sexual relationship and the other person doesn't, then that can cause a problem. He thinks that he's not satisfying me. Guys can be very sensitive about whether or not you are responding enough.