attending to 512 tokens of course the advantage here is that they now have 4 009 and 96 sequence length so they have the freedom to not attend to as many tokens as they have in the input length but you know to put it in perspective this here uses more memory and more compute on it on its face than bert because bert attends to as many tokens but has a smaller input sequence and you know i i there's sort of a thing where in order to make these sparse attention things work you have to go pretty pretty you know high in the number of things you