While I don't disagree with your sentiment I have a few comments:

"Why are we still dealing with over half a century of cruft?". IMHO that's because we have software running that's over half a century, with organizations depending on it, and so it needs to continue running. This means we need to continue providing that software with the environment it expects. For over a decade, a significant part of my recurring revenue came from helping companies make sure their ancient software would continue to run on newer machines (and they only got newer machines because it was no longer possible to keep the older hardware running). And this was on PC hardware, from my short experience in finance, I reckon there's a bunch of 390 software still running emulated on modern z/OS machines.

"tooling shouldn't need to parse strings to do something useful" The thing is tooling either needs to process some binary format, which is more efficient but also more obscure, or to parse strings. All the tooling that deals with json, yaml, etc is still parsing strings, and I prefer that to a binary format in most cases. I know a defined format like yaml is simpler to parse than free-form text, but it maintains some of the same constraints (notably the need to escape things).

Your comment about escape sequences to print color is very relevant. I feel two concurrent yet opposite feelings here: I get all warm inside from nostalgia since I spent what probably was an unhealthy amount of time learning about ansi sequences back in the day of BBSs, and I also despise that completely and would love to be able to have color on a textual computer interface without the need for that. I believe that's possible, but most likely, because the effort to achieve that is significantly bigger than the effort to keep hacking what we have, is why we don't have a new textual computer interface.
Challenges I can imagine: while I despise dealing with escape sequences (because I invariable got them wrong the first time, always), it's either something like that, or a side channel to convey formatting information. Or we move to html altogether. I guess this is the approach of text-intensive applications based on web frameworks (like I suppose most modern editors). There's the option emacs takes, which is to just have the text be plain text (whatever that means, really, we like to think it's a simple format because we can cat it and wysiwyg, but for the computer, it's all a bunch of bytes anyway) and use modes to alter how the editor shows you that text. As an emacs user I'm biased to think that's a better approach than html or a side channel, but I don't see that becoming the norm in the short term.

"and junior developers shouldn't need to waste hours scrolling an obtuse man page until they resort to a half-assed SO response with broken parameters to extract a tar archive" Now that is one hill I'm willing to die on: if someone can't figure out how to extract a tar archive just from looking at the synopsis on the man page and scrolling through the options, unless the archive was maliciously named to hide the fact that it's gzipped or something, I think that person would be a pre-junior developer and still have a way to go to become junior. Note I'm not saying all developers must use the command line: if you prefer to open archives with a GUI tool that's fine, but if for whatever reason you need to open it from the command line and you can't figure that out from the man page, I think it says more about them than about tar. Perhaps it's just a poor choice of example, as I agree there are very poor man pages out there, but for the basic use cases, I think tar is as understandable as it can get.

"There is more to shells and text interfaces than working within constraints set 50 years ago." Completely agree, which is why I often use the emacs shell, which is not a POSIX shell, and you know what? I don't care. I go to zsh when I need that, but often times, the emacs shell gets me what I need with less friction. And I'm sure the same would be possible with other shells. I wish we had more new shells that attempt to keep what seems useful about 'the old shells' but discards all that's not strictly needed and adds new, useful features.