attention is that probably neighbor information is very important and that has been shown empirically and then the global attention the reasoning behind this is that some of the tokens that are fixed by the developers are so important that it it's very beneficial that each other node is connected to them and that they are connected to each other node the result of that is the big bird attention mechanism which is basically long former which already had these two plus the random attention this achieves a linear