An astute observation. In an ideal "web" (as imagined in the 1990's), every business might have a website, there would be news sources[1], and many nerds would have websites, but beyond that, to the non-nerd, after a while, it's not very interesting. Despite what people once might have thought in the early-mid 90's, every living person is not going to create their own website. (Or even a blog, as people thought in the 2000's). The web is finite and that is bad news for search engines.[2] UGC and "social media" have been a way for certain companies to mask this truth.

1. As I remember it, news was one of the early internet centralisation points. As dial-up telephone charges were expensive, we patiently waited for someone at a large university to download the news and forward it. I am not a great source of internet history, others will may correct me here, but one of the largest operations like this, downloading Usenet news and making it available, ended up becoming what some called the first "ISP". That was UUNet. The takeway from this footnote is that "news" showed to be an early centralisation point, high traffic. Everyone wants "the news".

2. The trend today with Google and Bing, and those who use their feeds, is to limit the number of unique search results any user can retrieve. Around 250-300 max but with many searches one is lucky to get 50-100. The search engines are trying to market themselves as a way to "get answers" instead of a way to discover what websites exist on the web today. We all know what this looks like on Google and Bing. The companies place their own "web properties" in the results, i.e., many of the "results" are links to the companies own servers, and they scrape other websites to provide "instant" answers. The user never leaves the search results page, never even visits another website. DDG, following the lead of Google, calls this "instant answers" and "zero-click info". This statement from DDG sums up the present day popular search engines:

"When people search, we believe they're really looking for answers, as opposed to just links."

source: https://help.duckduckgo.com/duckduckgo-help-pages/results/so...

(Personally I do want "just links". I have written scripts to get them.)

It is up to the reader to decide whether this is intentional or not, but either way, unlike in the 1990's and early 2000's, search engines are limiting how much of the web users can actually "see" at one time. Regardless of intent, that is the effect. If, hypothetically, the web was not growing very much, no one could detect that using a search engine. The web today is portrayed by search engines as some sort of oracle that can provide answers. For easy questions, sure. For more difficult ones, we can fabricate answers but that does not mean they are good ones. Than add in "AI" hype. What happens when people lose all critical thinking ability. They just accept the oracle's answer as "good enough". We can already see this happenig with young people. You can end up with a Wizard of Oz scenario, but no one ever discovers the tiny man behind the curtain. The truth is that the web is still a motley collection of websites, along with some very large "walled gardens" of UGC that draw the lion's share of daily traffic.