Rust &TheMachine
By: Prolific K
Tags: Rust

Good Stack-Keeping

Ownership is the abstraction that involves several behaviors such as managing memory and using pointers. When the compiler enforces ownership, it also ensures that we never create invalid pointers or leak memory. The memory management part is actually quite simple and rarely results in compiler fights. If there were no pointers, the drop mechanics of RAII would be completely trivial.

However, we sometimes want to see the same memory in multiple locations to write an elegant algorithm or we want to avoid copying large values into every function call. We need pointers. Pointer invalidation occurs when things are moved or deallocated with outstanding pointers. Excluding moves, everything else about valid ownership rules boils down to preserving pops-before relationships in stack memory.

First, we’re going to look at how ownership and references work for stack values. Second, by reasoning that every heap segment has an owner on the stack, we will map all lifetime & ownership into the stack. This neat mapping then makes it clear that every timing to drop memory is decidable up front and that at no time will we hold invalid references.

Starting With Correct Pops-Before Relationships

Note: Micro-arch application convention (including x86) grows stacks down in memory addresses, opposite of conceptual CS stacks. This article uses “upstream” and “downstream” semantics, where calling a function is going “downstream” and returning is “upstream”, back where you came from.

References naturally travel downstream and refer to upstream. &T always points through upstream stack memory to a T and always pops before the T it refers to.

In the trivial (easy) case, &T lives downstream and points directly to an upstream T

lives downstream and points directly to an upstream Ownership composition in structs preserves the trivial case. While &T can refer to a field T on a struct, and while we can move &T into a different struct later, whatever owns &T will be downstream of whatever owns the T it refers to

can refer to a field on a struct, and while we can move into a different struct later, whatever owns will be downstream of whatever owns the it refers to Consequently, T always gets popped after all outstanding &T ’s

always gets popped after all outstanding ’s It can be difficult to convince the compiler that the composed case doesn’t break the law, but the trivial case will always be pretty easy to pull off

Preserving Pops-Before Relationships

The compiler will not allow program execution to result in references upstream of what they refer to. This could be done by attempting to return a borrow to scope data, trying to move a reference into a struct that will outlive what the reference refers to, or attempting to move a value with an outstanding reference.

All function references and owned values can be passed in as arguments or returned accordingly:

&T ’s are passed into functions, downstream, referring to upstream caller data

’s are passed into functions, downstream, referring to upstream caller data T ’s can be passed downstream, into functions. They will either will be returned in the products or dropped when the stack is popped

’s can be passed downstream, into functions. They will either will be returned in the products or dropped when the stack is popped T ’s created in the current function can be returned upstream to the caller (left on the now taller stack) or dropped

’s created in the current function can be returned upstream to the caller (left on the now taller stack) or dropped &T ’s referring to data of the current scope cannot be returned back upstream to the caller. The data they would refer to has to be moved to be returned. The data therefore must be either dropped or moved and the &T can’t remain valid either way

’s referring to data of the current scope cannot be returned back upstream to the caller. The data they would refer to has to be moved to be returned. The data therefore must be either dropped or moved and the can’t remain valid either way &T ’s passed into the function refer to upstream data and can be returned upstream

Because these relationships are preserved through function calls, they become transitive all the way through the call stack. In general, all of your references point upstream and are sent or carried downstream. You can also remember this as just two rules for function calls:

References and owned values in

Owned values and references to still-alive upstream data out

There you have it. We have defined the correct order for all stack memory to pop and have a set of rules that say we will not ever break that situation. Now let’s look at heap.

Boxes etc are Owned Heap

Box<T> is an owned pointer to a segment of heap with some value inside. Box<T> is a combination of a Box on the stack and a T on the heap. It’s an actual real-life heap segment pointer that you own. You can pass it, move it, mutate what it points to, return it. When it dies, the associated heap and whatever it owns are deallocated. Box references, ( &Box ) can be passed downstream, but like any other reference, cannot be returned upstream of the Box they refer to.

You put that T on the heap when you call Box::new(myT) . It was on the stack before you made a box. It is on the heap after you box it up. Arc::new(myT) is another container type that owns heap data, placing data on the heap by moving it during construction. Vec::with_capacity(usize) is a growable segment of heap where you put your owned data onto the heap a few at a time, using methods like push . You own the Vec . The Vec owns its heap.

Stack Owns Heap

Stack Owning Heap Dies → Heap Dies

The Box holding the heap will live until the thing (or last copy of the thing) keeping it alive dies. For the most part, your heap memory values are just the appendices of stack values and reasoning about stack life has already been shown to be very easy.

Even if you do something like Box<…Box<Box<T>>…> and move pointers themselves over to the heap, it will ultimately be kept alive by the outer-most Box , which is still on the stack, still in a function scope that will pop the box, and all the boxes it holds, when it returns.

Even if you start tossing references up onto the heap by constructing Box::new(&myT) , you will not be able to do this with data from the current scope and still return the box. Owning a heap location that stores a reference to current scope data doesn’t create a way to return &T ’s to current scope data back upstream.

The completeness of this abstraction lets us get away with almost never manually allocating or explicitly freeing. You can allocate via std::mem, but it’s not the usual way.

Reference Counted Lifetimes

Rc and Arc (Atomic reference counter) deserve special focus in the context of heap and stack. We don’t know how long these values will live, so they are reference counted and live on the heap. Arc is counted atomically and can be shared across threads. The Rc guts live on the heap. The Rc you pass and return on the stack is just a handle. This handle can be cloned numerous times, and only the last drop will trigger drop of the inner owned data. They do not imply any shared mutability, but you can put mutability inside them.

Borrow Lifetime = Immobile Lifetime

You cannot move a value that has an outstanding borrow. A borrowing reference to a moved value would be a dangling pointer under the hood. Instead, either copy / clone the value or clone & pass a reference counter, an Rc or Arc of the value. Another option is to move the owned value upstream so that the value you are trying to move becomes just another reference.

Ownership Composition Leads to Lifetime Annotations

If you create something, which has lifetime, but it owns a reference to something else that can have a different lifetime, you will begin needing annotations. This is composition of ownership. You must both maintain pops-before relationships and convince the compiler that you have done so.

The most obvious example is if you build a Struct that contains a reference to borrowed data. Immediately the compiler starts asking for lifetimes. If you have two references, the compiler wants to know if you intend both of those borrows to expect and track the same lifetime for both or a different lifetime for each. Annotations just help it decide this accounting.