I don't spend too much time on my physical appearance, but I'm a model and happen to be tall and slender, with fine bone structure. However, I often encounter women who act catty with me upon meeting me. I think it's a matter of jealousy and insecurity. Most women size other women up in terms of physical appearance because that's the way most men size women up. Everybody wants to be attractive.
Most women see men on many different levels - physical, emotional, mental.
I guess if you are going to size women up that way, so are we. That's like Donald Trump saying he doesn't worry about money much. I spent 20 years yo-yo dieting to try and achieve what society says is attractive, but I couldn't do it. I can understand the jealousy women feel toward attractive women. I feel it myself.