big bird right the big big big bird but big bird so big bird explicitly used random connections in the attention mechanism and their argument was well if we use different random connections in each layer then information can travel pretty fast through the network so what's the problem with these neighborhoods right here what's the problem with neighborhood attention like this the problem is that you break the long range dependencies so let's see what happens if information needs to go from this pixel or to this pixel or this node to this