> A stack of transformers isn’t performing a sequence of fundamental transformations as much as it as performing a sequence of additions, each layer adding new paint to a shared canvas.

I don't know if "additions" is a good mental model.

If you have layers 1..N that you're training via backprop, then layer N has no reason to "push" some of its computation "back" to layer N-1 if that computation could be done fully independently using the information already available at layer N-1. Instead, you'd just get a wider parameter space + post-pruning embedding vector at layer N, to do more parallel work and produce more parallel outputs at layer N.

The only reason you end up with more than a single hidden layer doing anything other than pure passthrough, is that a given layer K is constrained in the operations it can perform. If a layer K requires inputs that are more than just a single linear AddMM+softmax of the input layer away, then layer K can't do those operations "on its own", and needs some other layer to supply a pre-transformed input for layer K to do "the rest" of the work on. In practice, layer K thus acts as a loss function to train layer K-1 to compute those nonlinear inputs that layer K needs; and so on, pushing back "dependent responsibilities" for computing the outputs, all the way back to the input layer.

(You might have the intuition that a layer might just "run out of parameter space" and so need to "slide" some of the computation backward in time to a previous layer — but no, there'd be no reason to do this, as the abstract NN passthrough nodes required to propagate that already-complete computation from a previous layer, take up just as much space in a Transformer's Q/K embedding vectors as abstract NN nodes that are actually doing nontrivial computation do.)

So fundamentally, each layer is doing something to the previous layer that's not just "addition" (an operation which is one AddMM+softmax away.)

...but that being said, it can be something conceptually equally-trivial to addition. For example, in an image-generation model, "alpha-blending by weighted averaging in an oppositional-color-space projection" is conceptually trivial, but isn't a simple AddMM+softmax, and so requires another NN layer each time any object must be composited on top of any other existing alpha-blended object.

---

The interesting intuition that this "dependency graph" mental model of a Transformer gives you is that, despite the analogy used in the paper:

> The canvas (input) is passed along a series of painters. Some painters specialize in birds, while others are better at painting wheels. Each painter receives the canvas from the painter below her, then she decides whether to add a few strokes to the painting or just pass it along to the painter above her

...it's actually still possible for several layers to all know how to paint a bird and a wheel and other things too.

The constraint is that those things all need to be able to be done in parallel and independently to each-other, for them to be "scheduled" together as parallel operations within a single layer. A given layer can't do two things at the same time — or at least can't finish two things at the same time — if those two things are interdependent in a way that requires a nontrivial (not just AddMM+softmax) amount of math to merge.

Whereas, if any object depends on another object, then the work for one or the other object has to be "pushed backward in time" at training time to some previous layer, so that the outputs of that operation can be used as inputs. (Thus "painting" in a very literal sense — paint has to already be on the canvas by the time you want to blend on top of it!)

When that "pushing computation backward in time" happens often during training, in an biased way (i.e. with a causal correlation in the computational-dependency order that the occlusion/lighting/reflection/etc effect requires the parts of the scene be composed in), then due to the "scheduling constraint", some particular layers might end up trained more often to do a particular thing; and so end up better at doing that thing; and so end up being "the place" where that thing happens.

But just the same, if the "pushing computation backward in time" happens often during training in an unbiased way — and/or if you paint birds-on-birds-on-birds in your image, such that there's no way to get one layer to be "the" bird expert; then many layers will end up being trained to do the task, slowly, as at one time or another the "responsibility" for learning that sub-task falls one or more different layers for each training example.