is it turns out that it you're still well you know the original bert attended to 512 tokens and then you look at the window and the window was still 512 tokens it's just that the global attention was even more so ultimately they ended up using more memory than the original bird and here if i look at the parameters of their um thing and they have multiple experiments right here and i believe this is the the base version so this is the base version they also have this large version but here this is the 12 layer um version and you can see they