> We know that there are sex specific physical characteristics that differ between men and women. Nobody is arguing that's not true. Extending these to mental differences however is a stretch and not easily observed.

Yes, it is easily observed and has been extensively studied. Wikipedia has several articles about "Sex differences in ..." on different topics. For example, see "Sex differences in intelligence" [1] or "Sex differences in human psychology". Lots of studies are linked. One that I recall is "Sex differences in human neonatal social perception" [2]:

> Sexual dimorphism in sociability has been documented in humans. [...] 102 human neonates, who by definition have not yet been influenced by social and cultural factors, were tested to see if there was a difference in looking time at a face (social object) and a mobile (physical-mechanical object). Results showed that the male infants showed a stronger interest in the physical-mechanical mobile while the female infants showed a stronger interest in the face. The results of this research clearly demonstrate that sex differences are in part biological in origin.

Another example is "Sex differences in parking are affected by biological and social factors" [3], which found that men are better at parking cars:

> The stereotype of women’s limited parking
skills is deeply anchored in modern culture. Although laboratory tests prove men’s average superiority in visuospatial tasks and parking requires complex, spatial skills, underlying mechanisms remain unexplored. Here, we investigated performance of beginners (nine women, eight men) and more experienced drivers (21 women, 27 men) at different parking manoeuvres. Furthermore, subjects conducted the mental rotation test and self-assessed their parking skills. We show that men park more accurately and especially faster than women.

Another known result is that male IQ is higher on average has higher variance than female IQ. The intelligence study "Sex differences on the progressive matrices: a meta-analysis" [4] concluded:

> Results showed that there is no difference among
children aged 6–14 years, but that males obtain higher [mean IQ] from the age of 15 through to old age. Among adults, the male advantage is 0.33d equivalent to 5 IQ points. These results disconfirm the frequent assertion than there are no sex differences on the progressive matrices and support a developmental theory that a male advantage appears from the age of 15 years.

"Brother–sister differences in the g factor in intelligence" [5] claims that male IQ is not higher on average, but has "substantially greater variance". If male IQ had greater variance, this means we would expect that there are more men than women with very high IQs (as well as more men with very low IQs). The section about current research in [1] has more.

These are just a small sample of peer-reviewed research articles examining mental differences between men and women. There may not be consensus on all topics, but there is on some, such as the higher variance of male IQs, and it's being actively studied.

[1] https://en.wikipedia.org/wiki/Sex_differences_in_intelligenc...

[2] http://www.sciencedirect.com/science/article/pii/S0163638300...

[3] http://www.ruhr-uni-bochum.de/biopsy/papers/Wolf_Ocklenburg2...

[4] http://www.uam.es/personal_pdi/psicologia/pei/download/Lynn2...

[5] http://www.sciencedirect.com/science/article/pii/S0160289606...