are basically this thing right here and this thing right here because those weren't intended and are at the edges of the sequence so these were the the tricks of big bird to quickly uh summarize uh big bird is basically taking a transformer saying well why do we need all of this attention all of this full attention maybe we only need some of that and can already do a big job a good job especially you know considering the attention mechanism goes over multiple layers so we don't need a routing from each token to each token