I feel like a lot of these problems have been dealt with already in the lightweight markup ecosystem, and the other problems are solved in the VSC extensions.

While the "Big XML Specs" (aka DITA, S1000D, DocBook) allow you to re-use content, you can get the same functionality from Asciidoc or ReStructuredText. You can ALSO get the same functionality from Markdown, but it requires extensions, and the problem with THAT is that the functionality will change from instance to instance. Adoc and RST have transclusion and conditionals in core. I think Asciidoc is the right way to go here, but my reasoning might not be applicable for someone outside of the hard industries (manufacturing, aerospace, defense).

But just because you have transclusion and conditionals, that doesn't mean you're ready to start re-using content. Content re-use, the concept often called Component Content Systems, introduces a level of complexity that's almost always underestimated, and it's inherently dependent on information architecture in a way that a unified or natural document isn't.

Information architecture is going to be driven by product architecture, and if the product architecture is missing or wrong, then you probably shouldn't try to architect the documentation. Re-use only works when the product makes sense.

Here's an example.

Let's say my org makes planes. We make a product called SuperPlane. It has three variants, A, B, and C. Each of them is composed of modules: Wing, Fuel, Cockpit, Propulsion, Empennage. The product, in this case, is architected. SuperPlane modules share enough commonality so that the documentation for each module can be shared across all variants. So I have a "book" for A, for B, for C, and a single document module for Wing. Inside of the document module there are blocks of conditional content, for each of the variants. So when I run a book for A, it pulls in the Wing section, the it's customized for A as the conditionals are processed. Same for B and C: multiple deliverables, re-using the same modules.

My information architecture - filenaming, chunking, conditions - rides from the product architecture. The product architecture is solid, and so am I.

Now let's say our Maximum Leader has decided to acquire a car company, and has decided to make the flying car the D variant of SuperPlane. It's got an integrated wing and fuel module, so now, we lose content history with our A-B-C Wing and Fuel modules, in spite of the fact it's supposed to be equivalent. Its cockpit module shares no interfaces or commonality with the other cockpuit modules - so the module for Cockpit has a gigantic conditional section that largely parallels the others, in the same module, which makes assessing change impact really difficult. With the introduction of Variant D Superplane, the Superplane product is no longer architected, and our information architecture got blown up as well.

This is the core reason why component content systems (CCSs) are so hard. The end product depends on lots of stuff outside of the tech writers' wheelhouse, and unlike a document - which you can look at any old time you like - when the information architecture is busted in a CCS you might not even be able to get your content out of it. CCSs replace the natural language of document structure with the constructed language of information architecture, which is a bigger leap than I think many people realize; it fundamentally changes what your documents actually are. The original concept of CCSs did not come from a linguistic or information background, they were largely psychology academics and/or dealing with an EXTREMELY limited corpus, and you can see the trace of this in the awesome pattern of failure for most CCS systems to the present day. This is markup-agnostic; it doesn't matter if you're doing it in S1000D, DITA, DocBook, ReStructuredText, or Asciidoc.

So how do you check product architecture?

That is complicated, and it's going to depend on your industry. In a codebase, there are a LOT of tools for assessing variance, far too detailed a topic to get into here. In hardware and manufacturing, this is the sandbox of config management, but something I've found useful is warehousing all the product data I can get my hands on, then assessing an equivalence factor for any arbitrary grouping of final assemblies. When the equivalence factor is outside of a given range, I know that I might have problems if I treat those assemblies as "variants" of some fictional "product". Once it ventures outside of an acceptable equivalence, I will often assess that "variant" as its own pseudo-product for the purpose of information architecture. This saves so, so, so very many headaches - and if they ever do decide to rebaseline everything, we can make a task to reassess and, if commonality is now found, pull stuff together (within reason, some regulatory problems with wiping out procedures willy nilly).