For many years I was the one with no money. Recently, at a rather advanced age (late forties) I hit it big with one of my startups. I'm now wealthy. So I can give you a perspective from both sides.

First, when I was in my 20s, in the first dotcom bubble, many of my friends became ridiculously wealthy. I probably should have felt envy, but at that time I was very self absorbed and focused on my own projects. Also, I was very naive and thought there was not much utility to money beyond 40K to 50K a year. At the time I was making 100K a year and felt good enough about it.

However, as the years progressed and I entered my thirties my youthful dreams of having a successful software business were crushed. I began to see how money liberated my friends from the kind of drudgery I was condemned too. This was brought particularly into focus on one instance, when a friend of mine who was very successful was invited to a tradeshow where I happened to be exhibiting on behalf of an employer. My friend came to visit me at my humble little booth, and he was followed by a literal entourage of hanger-ons. After we had a chat he asked me to join him for an after party and told someone in his crowd to put me on a list so I could attend. Unfortunately, after the exhibit floor closed I was tied up doing manual labor ... packing demo hardware, hauling things to storage, etc. I was not able to get to the party until around midnight. My friend was totally oblivious to my peon duties and just assumed I was at some other party. I never told him the truth ... that I was just packing boxes ... not at another party. I had a few drinks and appetizers with him, but was so exhausted I fell asleep while sitting on a sofa near the bar.

The next day, to add insult to injury, I was waiting for a shuttle to the airport and my friend had a black car. He took me with him and dropped me off at the gate. I did not ask him about how he was getting home ... but I think he basically had some kind of speedy pass to bypass checkin.

These kinds of things happened from time to time in San Francisco ... but it never really bothered me to the point of depression. I just felt a bit frustrated that my journey was taking so much longer. I had 7 failed startups before I hit it big.

Now let me tell you a few advantages of hitting it big later:
1) I did not pick up any gold-digging women in my youth, which caused complications for several of my friends. When you get rich, you attract many women who are far more attractive than you could if you are not rich. Your brain gets hooked on that level of attractiveness and you have a sort of hedonic adaptation I think. Many of my friends ended up with women that are essentially cost centers. They don't provide much value add beyond companionship. I was lucky enough to meet a woman when I was broke. She basically invested in me, and we got married before I hit it big. She makes a large salary on her own and does the lion's share of domestic work too. She's also attractive. I met her because I was trying to make a living. She's not the kind of woman you meet on the party circuit.
2) Similarly with friends. All my friends are people I've known for a while. Not people that need things from me. I have friends that have hanger ons that are not good for them mentally as they validate everything they do without critical thinking. It can be dangerous.
3) My tastes and expenses are very low as a result of decades of cheap living. Although I'm probably about the same level of material wealth as my rich friends now, I think I have more optionality as I can withstand those 1 in 100 geopolitical changes and still have enough to provide for my modest lifestyle till death. I have some friends that are one tech/crypto/real estate bubble pop away from not being able to maintain their jetset lifestyle. There's more misery downgrading your lifestyle then joy from upgrading it. Its Hedonic adaptation in action.
4) As a result of good fortune I realized the real value of money is just the optionality and freedom from stress. Big houses and nice cars and stuff give diminishing returns. The irony is, everything I have now, I had before. And to be honest, my money sometimes adds huge stress to my life. I'm always wondering how to preserve it. I'm worried my cash positions are eroding due to inflation. I'm worried my stocks will crash due to fed action. I'm worried my crypto will go down. I'm worried I'm not invested in commodities and oil. I'm worried I didn't buy farm land ... what if everything collapses and I have to grow food for my kids. Should I buy some property outside the USA in case there's a collapse here. All these things were not even options before so I didn't worry. Now I worry all the time.

Edit: Your friends should have the sense not to talk about money around you when you don't have much. I actually make this error myself and need to do better. 90% of the time money and belongings are a useless topic.