I agree that it comes down to both poor management and poorly performing/trained engineers, but I've also seen a TON of dysfunction created by engineers that are perceived as competent. I'm not sure whether you might have actually been referring to that, but I'll continue assuming I'm referring to something slightly different.

At nearly every company I've worked for, there's usually an "inner-circle" of highly regarded staff engineers whom are considered to be always correct by both management while the rest of engineering are treated by both groups as peons. On first glance, this might seem like a normal dynamic that occurs in any hierarchy based on competency and experience. Indeed, some people will be given more say than others because reasons. That's okay. What's not okay is when the gap between the inner-circle and the peons is so hollowed out that there's virtually no power structure to keep the inner-circle in check. I've seen this first-hand and second-hand when a company chooses its inner-circle based on a combination of nepotism, whomever has been around the longest, and whether an engineer has some kind of name for themselves outside of the company. It's not that these factors can't or shouldn't be a part of the equation, but they lead to rampancy when competency isn't thoroughly checked. Just because a person is vouched for doesn't mean they should be handed the "key to the city", and neither does whether the person stuck around and managed to survive every layoff, or whether they have a well-known blog or are considered a guru in some web framework. By that "key", I mean the metaphorical one they are given that allows them to dictate every level of engineering process, which sometimes goes as far as forcing every engineer to use the same IDE. Maybe they can have that key once they've proven themselves at the present company, but not merely because someone has an existing opinion about them. When a company gives their inner-circle way too much power and makes the rest of their engineers peons, this is where developer experience can decline and the amount of wasted resources increases.

Most of the problems I've seen that make work a nightmare for engineers isn't even caused by management. Management notoriously asks unrealistic things of engineers, and this is a pain to deal with, but I've hardly ever seen this impossible to work with; it's the nature of the beast for engineers, more or less. The breakdown in engineer performance, from what I've seen, comes from the amount of bullshit that inner-circles add to the engineering process in order to solve problems. What the "staff" engineers whom are part of these inner-circles often fail to consider is that every link you add to the chain adds a new opportunity for the chain to break. This is engineering 101, but too many engineers with godlike clout either fail to consider it or pretend that it's not a problem. Maybe it's cognitive dissonance, or maybe it's ego. The rationalization is usually the idea that all of these checks and "standards" will protect the codebase in some way.

Whatever the case, what we get are these humongous toolchains just to build a fucking JavaScript application that runs in a browser. What we get are CI tasks that randomly break because someone believed that a bajillion tests relying on third party services would reduce the amount of bugs. What we get are tests that run incredibly slow and are unpredictable. What we get is outdated documentation because someone in the inner-circle thought it was a great idea to host the documentation as a full website, and of course only they have the knowledge and ability to deploy that thing. What we get are engineers frequently making "mistakes" and holding up code review because there's constantly new "best practices" being silently handed down by inner-circle engineers who want to write code and can't be bothered communicating effectively. What we get are so many lint rules, checks, and pre-commit hooks frequently added and modified that the engineers outside the inner-circle are afforded no creativity and must spend extra time fixing errors that aren't even real errors with the code, but are errors based on someone's opinion.

Ultimately, more time is spent fixing self-inflicted issues and far less time delivering value to customers. And software isn't even better today. I've spotted more glaring problems with both websites and native apps in 2023 than any other year prior. The quality of our output is a joke, but the world has accepted that all software is broken.

The best thing tech companies can do is to break up their inner-circles of genius engineers, separate the wheat from the chaff, hire staff engineers who are averse to adding complexity, and give power back to the rest of their engineers. Engineering shouldn't be a combination of bureaucracy and implementing demands in the form of code; engineering should be about working on real problems. As an example, frequently baffling your engineers with uninformative errors caused by a custom in-house syntax transpiler that only one or two people seem to understand is not solving a real problem. That's called bullshit.