There's a few layers to this.

If you want higher than market (average) returns, there are known factors that you invest in that have higher expected (but not guaranteed) returns:

* https://en.wikipedia.org/wiki/Fama–French_three-factor_model

Buffett's historical success has been quantified in light of this:

* https://www.aqr.com/Insights/Research/Journal-Article/Buffet...

* https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3197185

Also worth noting that Benjamin Graham, in his very last published interview (Financial Analysts Journal, 1976), stated:

> In general, no. I am no longer an advocate of elaborate techniques of security analysis in order to find superior value opportunities. This was a rewarding activity, say, 40 years ago, when our textbook "Graham and Dodd" was first published; but the situation has changed a great deal since then. In the old days any well-trained security analyst could do a good professional job of selecting undervalued issues through detailed studies; but in the light of the enormous amount of research now being carried on, I doubt whether in most cases such extensive efforts will generate sufficiently superior selections to justify their cost. To that very limited extent I'm on the side of the "efficient market" school of thought now generally accepted by the professors.

* https://valuehunter.files.wordpress.com/2009/05/conversation...

Now there are lots of folks that don't bother listening to his advice, and take an active role in the market. But empirical studies have shown that, generally speaking, the more active a person is the worse their results tend to be. We've know this for at least fifty years:

* https://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street

And even the professional folks who spend their working life generally get it wrong:

* https://www.ifa.com/articles/despite_brief_reprieve_2018_spi...

Over the course of 20+ years of investing, you're more likely (>85%) to make more money by simply buying passive funds that follow major indexes (S&P 500, Russell 3000, MSCI EAFE), why bother with the extra risk? Is the extra potential (not guaranteed) return really needed to meet your financial goals? If you can (e.g.) retire comfortably getting 'only' 6% returns, why bother chasing (say) 8-9%? Having more money isn't necessarily a bad thing, but what are you trading-off chasing for it?

The odds are not on your side if you think that you're one of the folks that can beat the market, especially over a time period of 20-30 saving for retirement (and then 20-30 years in retirement). The extra returns are generally not 'free': you're taking extra risk for them.

Of course one often hears about the 'superinvestors' that made it, but what about the (probably many more) people that did not:

* https://xkcd.com/1827/

As for luck versus skill, this has been studied:

> When we turn to individual funds, the challenge is to distinguish skill from luck. With 3,156 funds in our full ($5 million AUM) sample, some do extraor- dinarily well and some do extraordinarily poorly just by chance. To distinguish between luck and skill, we compare the distribution of t(α) estimates from ac- tual fund returns with the distribution from bootstrap simulations in which all funds have zero true α. The tests on net returns say that few funds have enough skill to cover costs. The distribution of three-factor t(α) estimates from net fund returns is almost always to the left of the zero α distribution. The extreme right tail of the three-factor t(α) estimates for net fund returns, however, is roughly in line with the simulated distribution. This suggests that some managers do have sufficient skill to cover costs. But the estimate of net return three-factor true α is about zero even for the portfolio of funds in the top percentiles of historical three-factor t(α) estimates, and the estimate of four-factor true α is negative. Moreover, the estimate of true α for funds in the top percentiles is no better than the estimated α (also near zero) for large, efficiently managed passive funds.

* https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1356021

* https://mba.tuck.dartmouth.edu/bespeneckbo/default/AFA611-Ec...

Basically: the larger the portfolio, the more skill you need to beat the market (especially on a risk-adjusted basis) over time. Get 'too big' and you tend to run out of runway on your skill (at which point just go passive). This cross-over point is different for everyone, but you won't necessarily know where your's is ahead of time, and so initial (skilled) success may end unexpectedly and suddenly.