long former paper so i've made a video or base i think even two videos on the long former which used exactly the window attention in combination with the global attention and uh if you want to know more about that go watch these videos but the new thing in big bird right here is this re edition of the random attention again the the window here in is is has exactly the same properties as the random attention so you have instead of a fully connected graph you have a sparsely connected graph now if you have random attention the