You also need to consider that most shoes will stretch at least half a size. And, unlined shoes will stretch more than a lined shoe. Some of the newer models are being made with synthetic leather, which stretches very little. There are some women's shoes on the market, not a lot but some. Since women's feet are much different than men's, this is a good place to start.
Women typically have narrower heels, high arches and smaller feet.
This is not to say that you can't find a man's shoe that fits right. They're not made for walking around in them. So to get the most out of them, only wear them while climbing. They'll last longer. You can also have your climbing shoes resoled when they start to wear out.