> Even "schema-less" data storage solutions like MongoDB are then also built on similar lies, because while you can throw arbitrary data at them to store it, querying the data later will still depend on some assumptions about the data structure.

They are. I have been complaining loudly about this for years :)

> I'm inclined to agree, and yet, RESTful APIs are still very popular and seem to have much success (though some push for more "immutable" APIs that are versioned, which is nice), as do "schema-less" data stores in certain domains. Edit: you can also say that about everything from CLI tools or even dynamic programming languages as a whole. I think there's more to it.

The part that I think you're missing is the difference between structure and schema. It's entirely valid to leave the schema implicit in many circumstances (but not all - generally not in a database, for example), but you still need the general structure of how data is represented.

When you deserialize something from JSON, CBOR, whatever, you will get back a bunch of structured data. You may not know what it means semantically, but it's clear what the correct (memory) representation is for each bit of data, and that's exactly the most fragile part of dealing with data, which is now solved for you.

You do not get the same benefit when passing around strings in ad-hoc formats that need custom parsers; it is very easy to mess up that fragile part of data handling, which is why parsers are so infamously difficult to write, whereas working with parsed data structures is generally considered much simpler.

Likewise, from a usability perspective, it's fairly trivial to let the user pass eg. a dotpath to some tool to select some nested data from its input; but letting the user pass an entire parsing specification as an argument is not viable, and that's why most tools just don't allow that, and instead come with one or more built-in formats. If those supported formats don't match between two tools you're using, well, sucks to be you. They just won't interoperate now.

Essentially, "standardized structure but flexible schema" is the 'happy compromise' where the most fragile part is taken care of for you, but the least predictable and most variable part is still entirely customizable. You can see this work successfully in eg. nushell or Powershell, or even to a more limited degree in tools like `jq`.

> I suspect that this is where opinions might differ. Half-assing things and counting on vaguely compatible (at a given point in time) implementations has been the approach on which many pieces of software are built to solve numerous problems. On one hand, that is ignorant, but on the other hand - how often is the output from "ls -la" going to change its format, really? The majority of CLI tool writers seem to understand how brittle and bad everything is and thus treat their data formats carefully, adding a whole bunch of different flags as needed, or allowing for custom formats, like: "lsblk --output UUID,LABEL,SIZE"

Speaking as someone who is currently working on a project that needs to parse a lot of CLI tools: the formats still change all the time, often in subtle ways that a human wouldn't notice but a parser would, and there's effectively zero consistency between tools, with a ton of edgecases, some of them with security impact (eg. item separators). It's an absolute nightmare to work with, and interop is just bad.

> For many out there, that's good enough.

I think there's a strong selection bias here. It's not without reason that so many people have an aversion to terminals today - they just aren't very good. And it's not just their terminal-ness either, because many people who cannot deal with a standard (eg. bash) shell will happily use terminal-like input systems in specialized software.

It's certainly true that most people who use terminals on a daily basis, consider this paradigm good enough. But that's probably because those who don't, just stop using terminals. We should be striving to make tech better and more accessible, not just "good enough for a bunch of Linux nerds", and certainly not upholding "good enough" paradigms as some sort of virtue of computing, which is what happens when people say "text is the universal interface".

Of course changing this is a long process, and there are very real practical barriers to adoption of other interop models. But that's not a reason to downgrade the 'ideal' to fit the current reality, only a reason to acknowledge that we're just not there yet.