I don't want to impose what it should be, as I'm asking myself this question every day.

But I can list a few thoughts. The actual memory model is (seemingly) a flat address space, in which processes allocate segments, and everything else arises from there. There's some remapping behind the scenes, some segments are shared, some not, but this is beside the point.

And a pattern I see often is arena allocation, where you allocate a large segment and then "virtually" allocate smaller segments inside, down to individual scalar variables. This can happen several times with a lot of memory, but tends to be shallow.

For this and many other reasons, I see mutable memory is best structured as a tree, a shallow tree, almost relational, but still permitting nesting, where every item has one owner, one caller, and anything else is routed through that tree. Of course once this semantic model is established, you can reduce and optimize many calls into more direct calls. But you get clear intercept points when you need to decorate, block, remap and so on behaviors in the system. No "action from distance" surprises.

Everything should be pass by copy. For larger structures, we need copy-on-write optimization. Which means underneath the mutable tree there is a Directed Acyclic Graph of immutable segments supporting copy-on-write.

This is also not new, it's how forking works on Unix for example. Every modern OS utilizes copy-on-write on the file system, in memory, it's also utilized, if we think about it, in network caching systems. But to the process it looks like normal flat address space, where everyone owns their own copy of the content.

For collecting unused segments in COW you'd naively need refcounting. There are other approaches that borrow from GC algorithms, in a much simpler and more performant way, because, well, there are no cycles in a DAG. The best part is that once you decide to use copy semantics, you have a plethora of algorithms at your disposal to try and switch between adaptively (including... just copying the thing if it's small enough, which turns out is the fastest approach), while maintaining the same exact semantics for the programmer who doesn't have to care how it works under the hood.

Another benefit of copy semantics is that things are copied between processes, or especially between machines in a network, and now your language locally has the same semantics as when it talks to the network, which removes barriers and unifies interfaces. "Write once, run at any scale" if you will.

Anyway, this is not comprehensive. And just my opinion. But the problem with graphs is that it is the data structure with least discipline, and most complications. And in most cases those complications do not pay off. Not only they require complex GC, but they result in heavily entangled code, tons of shared mutable state, that gave the whole OOP space a bad name (that it doesn't deserve). This was never a problem with objects, because objects were never supposed to share mutable state. It's a problem of graphs and handles, a system that pretends everyone "has" a mutable piece of state... which it doesn't have, as ownership must be exclusive to be reliable.

Discipline of dependency management (packages etc.) often removes cycles and results in a tree or a DAG. This is not coincidental. And there's no reason we can't replicate this at the lower levels to individual objects.

Of course, we still need graphs. But not everywhere, and not all the time.