hope this was clear that this combination of random attention window attention and global attention is what is called big bird okay they have some engineering tricks that go along with this but in concept you can imagine big bird being long former plus these random attention right here and you know as an engineer as an nlp engineer that makes kind of total sense i you know i totally believe that a the introduction the addition of these random attention of these random attention patterns can absolutely help your classification