This immediately makes me think about a path not (widely) taken in software: the path of higher level languages and execution environments with strong first class support for composability, easy message passing, reflection, automatic memory management, etc., across the entire system and even across networks.

Examples include Smalltalk, LISP machines, the JVM, and the CLR. There are probably others. Edit: is Erlang one of these? I seem to recall it having some of this.

If you've never operated in one of these environments, it will be hard to imagine what they bring to the table. Here's an incomplete list:

- All errors cause traceable exceptions and everything can be debugged.

- Statistics and diagnostics are a built-in feature of the runtime environment. Everything either always is or can easily be instrumented and profiled. The developer usually has to do nothing to allow this. Extremely detailed maps of memory and CPU usage can be produced for anything that is running.

- Software can be hot upgraded at the object/module level in running applications and services. If you design things to take advantage of this, full restarts can be reserved only for very major upgrades of the runtime or hardware.

- Everything is memory safe. Buffer overflows and similar "segmentation fault" errors are almost impossible.

- Composability -- applications and modules incorporating one another and building on one another -- is reasonably easy and commonplace.

- Modularity and code reuse is way easier.

- Inter-process communication (IPC) and the network equivalent (RPC) are easy, sometimes automatic and trivial.

- Function definitions and calling conventions are rich, typed, and support runtime reflection.

- Software can easily and efficiently pass complex data structures across module and even program boundaries with little to no API glue code.

- Module/object and program state can be serialized, moved, stored, deserialized, restored, etc., even across machines... or in some cases across architectures!

I'm sure there are more highlights. Those are the ones I can recall now.

It's not that these systems are totally dead. The JVM and the CLR are used heavily in business. But they are absolutely not trendy and have fallen out of favor among most "hackers" and new startups/projects in favor of the modern Unix way.

The modern Unix way, also known as "cloud native," might be described as: balls of random shit bundled up as containers or VM images communicating over bespoke APIs using inefficient text-based protocols over HTTP. There are of course many variations of this, but this is the basic theme.

I suppose this paradigm gives you some of what I listed above, but it does so in a very bespoke, inefficient, and labor intensive way. You end up having to create and maintain complicated API glue layers, implement connectors between applications that speak endless bespoke protocols, and of course every application runs in a completely different runtime (or no runtime at all) and can in no way be investigated using any set of common debugging or diagnostic tools.

This is definitely a case of "worse is better." Having used environments like the CLR and JVM and played with the likes of Smalltalk, it seems obvious to me that higher level environments save massive amounts of programmer labor and pain, but we don't like them. Why?

I find it hard to answer this question, but like many cases of "worse is better" it seems like a revealed preference. The better thing just never "takes."

Edit: I've got some speculations. It could be some combination of:

- Worse isn't better, but worse is free and unencumbered. Many of the better environments cost money or come with some other license or limitations.

- It could be that programmers are "macho" and don't like the fact that these environments are easier. Real Men(tm) code in C with no bounds checking. I sometimes see this attitude on display with regard to Rust. "We don't need safety!"

- The clunkiness and inefficiency of "weakly coupled wads of shit" encourages profligate waste of cloud resources and the use of managed services to avoid having to manually babysit this stuff, which is very profitable for vendors and cloud providers.

- By making it easier to build and sustain complex systems, these environments tend to foster runaway complexity growth. Look at abominations like enterprise Java for examples. The clunkiness and fragility of "worse is better" limits complexity, which turns out to be a good thing.

- The examples I listed were all too closely bound to one language or paradigm. Maybe something like WASM will finally make this mainstream?