Since this is supposedly an adult discussion we need to take into account two factors involved--sexual arousal involves two parts of the brain--the brain stem-vagal nervous system over which we have no direct voluntary control--and the neo-cortex, our higher brain that is involved in our ability to love, our social behavior and our rational decisions.  Men are more quickly aroused and also, as a group, give each other facile permission to act on that arousal--and this has been true for thousands of years.   Men don't get pregnant, carry a baby for nine months and form a bond to their babies even before birth.  And this is especially true if a man isn't in a loving, committed relationship.  
The question is--why haven't our religious/moral leaders--bishops in this case--never held laymen accountable?  Maybe sex isn't society's business, but children without fathers and a healthy support system are society's business because we all suffer the consequences.