Actually, your point doesn't follow.

Who knows more about self-gift than a celibate?

And by the same argument, and perhaps to a far greater degree, why should the Church listen to a man or a woman who has had multiple partners before or even after marriage?

Not exactly good people to listen to with respect to intimate and whole and exclusive self-gift.

The difference between a celibate and a person living married life _properly_ is exactly N = 1.  

A person who couldn't hold out for marriage...or a person who has more than one partner has no credibility.

I suspect if you could find a representative sample...the people who were virgins before they were married and remained monogomous agree far more with Church teaching than not.