It is very difficult for me to speak for all men and what they might remember. I will say that if it turns out to be a memorable relationship, then they will likely remember the first time you had sex. But if it was a go-nowhere situation, then they will probably not remember it. Women have much more symbolic meaning attached to the first time. They are surrendering and giving up something very important.
Guys are pleased and happy about it, but it is just a sexual event.
If you are looking for the perfect woman, you will be looking for a long time. Our needs and desires change through time. Stop thinking about perfection and start thinking about what you want in a partner and in a relationship. There will be frustrations and arguments, but that is all part of humanity. It sounds like you want to become lovers again in a marriage that has become routine.