or whatever your nlp tasks because you know more attention better and i also am completely willing to believe that you know using the full attention matrix while it is of course more accurate it won't hurt too much to leave some of that attention away because essentially all the path lengths are just becoming too or even with the random attention are really short or logarithmic to route information from a node to some other node so the loss that you incur is kind of in a logarithmic scale in terms of performance