I think the situation can be described pretty accurately like this:

1. A developers job is more complex than most, and takes a very long time to learn, because you have to become proficient in so many aspects of the job. e.g.

- technical stack which is huge (languages, protocols, how hardware works, networking etc),

- plus technical tooling (ides, source control, platforms such as clouds, programming frameworks and so on and on)

- programming models (desktop ui, web ui, state management, apis consumption, api creation, security models, and so on)

- teamwork (project management systems, soft skills, team dynamics, professional writing and so on)

- user level software (operating systems, office software, communication software, anything users use that you need to interact with)

- understanding your industry (how all the above changes and evolves over time)

- one or more business domains - to know what software to write and the nuances of how to create it to be truly useful

- all the other areas, such as high performance, or high quality, or design for testability, or how systems fit into companies management frameworks, or designing a system for manageability over the span of it's lifetime, or for easy extensibility and so on and so on

2. Skill levels

- This is a hard ask, many people don't have the kind of mind to ever do this well, and will be mediocre forever

- Learning time, it tends to take 5-10 years for talented people to become proficient in ALL the above areas they need to combine to truly be a journeyman developer

- True mastery requires a commitment to learn to a deep level on so many area, and 10-20 years is probably realistic for smart people with the right mindset, to truly get to a level of masters of so many related disciplines

3. Skills triangle

- The industry has been growing so fast. There are ten or a hundred times more recently started developers than there are developers with decades of experience because as an industry software development is growing by orders of magnitude as every company becomes an IT company where in the past it was rare.

- People often move up and out. Of the rare developers with the capacity to become real masters, who then have the 10-20 years of experience to have time to learn it all deeply - many of them have left the field by that time. This is for various reasons such as: having made their fortune and moving to investing, getting sick of computing or the industry and wanting a change, moving across to management, not updating their skills to keep up with technology etc

- Quick napkin guesses to put metrics to these
-- 1/10 developers with more than 10 years serious development experience (90% less than 10 years, because of fast growing industry)
-- 1/3 developers with capacity to really master it
-- 1/3 of those remaining in development instead of moving up and out

= on these completely made up but semi realistically guessed figures, 1/10 * 1/3 * 1/3 calculates out to having 90 non master developers for every master developer in the industry

So a true master developer is very, very, very rare.

~

As well as being very rare, they are other factors which mean their experience and guidance is not always utilised:

- management that often suffers from lack of experience technically

- peverse commercial incentives as one of the other comments here mentioned

- fashion storms of technology that occur due to the hype of a significant proportion of the hundred times as many less experienced developers filling the blogsphere and management mindshare with hype regarding the latest thing they've re-discovered that is already known if they cared to try to learn more deeply instead of try to 'thought lead'

- ageism and overemphasis on the 'new shiny' vs tried and true techniques

Last points might seem a bit bitter but please stay on topic, they are mentioned because they are significant, real reasons why there is so much crap software in the world!