Here are some reasons for it. FWIW I explored a hypothetical alternative non-web architecture that could avoid some of these issues here [1].

1. Encapsulated GUI components are a non-negotiable requirement for most projects, even apparently "static" stuff like technical manuals (e.g. search widgets), but there is/was no widely accepted standard for server side UI components. The closest was things like JSP taglets, but that is now considered legacy technology, gone in favor of React SSR. Why? Well, because components are most useful when you can instantiate and mutate them, which means they're most useful on the client, but server side tag libraries lost all the componentization when crossing the wire leaving you with a "tag soup".

2. HTML and CSS, being as they are committee driven and implemented multiple times in security sensitive C++, evolve very slowly. In practice people's ideas about how to present content, even so-called "static" content, change faster than the standards can keep up, so people fall back to JS to cross the gap. But then you get the impedance mismatch that comes from mixing several different programming technologies together (HTML, JS/TS, CSS, C++), and the DX goes to hell.

3. Why is it a "mess", well that's mostly an unarticulated social preference. Developers seem to prefer open source bazaars on the frontend, even if it means a horrific DX in which solutions have to be stitched together out of lots of tiny half-abandoned libraries. This is probably a legacy of the churn of the 2000-2010 era in which many large, well thought out proprietary frontend app frameworks ended up being abandoned by their owners or experiencing severe strategic product management errors. Delphi, VB6, Flash, Silverlight, .NET WinForms, .NET WPF, Java Swing, JavaFX, GTK, (to some extent) Cocoa and Qt and so on ... these all provided much more seamless and coherent platforms for writing apps but ended up leaving stranded user bases behind after the backing companies didn't execute properly on sandboxing/security/deployment/cross platform support.

Or to put it crudely, a part of the reason people target the web is the lack of product managers involved in defining the platform. Chrome has them in theory but in practice a lot of stuff they add to the platform is deliberately unambitious incrementalism, and the web's enormous base of valuable but unmaintained content means they aren't able to break backwards compatibility on the core tech despite having huge budgets. The downside is that anything not provided by the base platform experiences the opposite effect where the gaps get filled by enthusiastic volunteers who don't plan together or even stick around very long.

There's nothing fundamental about this choice, as the wholesale adoption of iOS and cloud tech shows. Devs will buy into fully proprietary platforms in a heartbeat if it's convenient to do so, but this is partly because those vendors have proven to be quite good about backwards compatibility and incremental development: there has never been an "AWS 2" effort and Cocoa's evolution has been quite smooth from the NeXTStep days.

One way to escape the complexity of front-end development is to write for non-HTML platforms instead. Historically this was quite painful because most work gets done on desktops, but whilst desktop development could be quite pleasant desktop distribution was extremely painful. I've spent a couple of years working on that problem and it's now way easier than it once was (check out [2]) so deployment complexity for devs is increasingly no longer a concern. If you want to write in Jetpack Compose or JavaFX or Flutter, or indeed Electron, then you can do that and the whole deploy/update story has got nice and easy. The big gap that remains is certificate cost, but we're looking at fixing that by signing for you if your app fits inside a sandbox. We're scouting around to understand demand at the moment.

[1] https://docs.google.com/document/d/1oDBw4fWyRNug3_f5mXWdlgDI...

[2] https://conveyor.hydraulic.dev/