this quadratic tension mechanism because if you you know double the 512 you need you know four times the amount of compute and memory so how does this paper go about reducing that quadratic dependency the goal right here is of course to get this to some o of n right because then as we double the input length we simply need to double the compute requirements and that would be fantastic and that's what this paper does and it does so without you know sacrificing the properties of the transformer so here's the architecture that big bird proposes