Here's the insidious part of technical debt. There's two types:

Known and unknown.

If you are not knowingly accumulating technical debt, then 90% of the time you are accumulating it unknowingly. This is the most insidious form of debt.

I have searched my entire career in software engineering for the solution to unknown technical debt. The problem isn't just solving it, it's also clearly defining what is happening to your software over the years even when you spend the utmost care in not allowing technical debt to accumulate.

I think I sort of nailed down what's going on in a fuzzy way. I also sort of have a fuzzy solution to the problem. Allow me to elucidate:

Eventually most projects will hit a point where you realize that your initial design was poor. You are given a new feature and your old design is simply not flexible enough to accommodate the change. In order to put the feature in, you either do a massive rewrite or you hack it in.

In this way the technical debt that was accumulated unknowingly may force the programmer to begin accumulating debt knowingly. In fact they often forget about the unknown technical debt. They think that the only proper solution is to contemplate between the cost of a hacky shortcut or a massive rewrite without thinking about what caused the need for the massive change in design. What exactly is the thing that is causing the "need" for the massive rewrite?

What's happening is, whenever you design something you are predicting the future. You are assuming your design can handle certain current requirements and future ones. However, the future is actually unknown, there is very little chance you can ever predict the future in an accurate way. Thus by your lack of ability to predict the future, your design will usually in the majority of cases fail to predict the future and this problem will always occur.

That is all that is happening with unknown technical debt.

The best solution (not a full solution) to this is to design all your projects with the maximum flexibility possible. That means every corner of your code that can be factored into multiple modules or be easily factorable in the future. By coding this way, you maximize the reusability and reconfigurability of your code giving the best possible insurance for an unknown future.

If you heard of ravioli code, this is it. It has bad connotations but ravioli code is my best known solution to unknown technical debt. Most people feel ravioli code is hard to read or maintain... In a way it is, but the key here is deal with the program in layers of abstraction. Your complicated ravioli code of 100 primitives at the lowest layer must be able to "compose" into higher level layers where you only have to deal with a manageable 20 primitives.

So the key isn't just ravioli code, but ravioli code with highly compose-able modules you can use to simplify your code into layers. Think of it like a slider between flexibility and simplicity. The higher the flexibility (more primitives) the more complex your program is; the lower the flexibility (less primitives) the more simple your program is... A good design is one where at any moment in the projects life time you can move from layer to layer and readjust the flexibility and simplicity of the program at will.

Unknown to most people there is a programming style that does the above almost mechanically:

Typed Functional programming.

You may have done some FP and seen it get messy (especially with JS), but to really see why this is the solution you should take a look at Typed FP using the point free style. The point free style will help you understand the true nature of a compose-able primitive.