I think part of the reason is that the basic skillset needed to produce any value has been increasing over the last 10-20 years. For those who master this skillset, the result is increased productivity (often by a multiple), but for others it provides a barrier to entry.

A few examples:
- 2000 sysadmin requirements: Good work ethic, basic IT knowledge, join some MS sponsored training programme.
- 2020 "devops" requirements: Medium+ developer skills, broad skills within linux, networking and cloud technologies (multiple vendors), expert at ansible, terraform, k8s, bash, AD integration and often other technologies. Also, the same work ethic and ability to be systematic that was required 20 years ago.

- 2000 data analyst: MSc form some STEM field, basic knowledge of SAS or SPSS, some presentation skills.
- 2020 data scientist: Still needing the stats knowledge, but now also requiring a much broader IT skillset. Ideally a fully Python developer skillset, including the ability to collect and manipulate data, train a model, present output (graphs etc), and deliver the model as a microservice directly into a k8s production environment.

- 2000 app developer: Competency in at least 1 programming language (perl, java, php, VB(.Net not available yet)). Bonus point ("full stack") for being able to do all the html/css as well as database integration.
- 2020 developer: Same as in 2020, but now do the same in a containerized environment in ways that should be inherently scalable (even when not strictly needed), and fully master a dev-test-prod deployment process on top of k8s or similar, often in an environment where it is not practically possible to run all services on the local machine.

10 years ago, there were still a lot of jobs requiring only the year 2000 skillset, but now there are not many such jobs any more. And a lot of the applicants are either stuck in the legacy skillset or are fresh out of school and have not had time to learn the essential skills.

Those who have mastered the modern stack can sometimes be 5x more productive than similar roles in 2020, those who have not may be useless, or even produce negative value.

I think part of the problem, is that there are way too many organizations out there that want to be "like google", and use a highly scalable, highly advanced tech stack, even if they are only computing similar data volumes that they did in 2020. Most don't realize that only the top 1-5% (my estimate) of tech people have the talent to properly leverage the modern stacks, while probably more than 50% could be productive in a simpler setup.

A simpler setup also has the advantage of allowing people of average+ talent to spend more time focusing on business requirements rather than struggling with the tech stack itself.

As was mentioned in this thread:
https://news.ycombinator.com/item?id=29188355

, this is not the first time we've been in this situation. The industry seems to swing back-and-forth betweeen complex code-heavy development and "no-code"/"visual" approaches. Over the last 5 years, the pendulum has been on the code side, maybe a bit too far, so 5 years from now, I would not be surprised if the pendulum swings too far in the other direction again.