your sequence length you'll get some n squared amount of computation and memory requirements for this so this is a problem and usually this means that you know this sequence length in bert this is limited to something like 512 tokens which is okay for some applications but if you want to summarize you know entire articles entire books even or do question answering with lots of context it's not really enough so people have been thinking about how to scale this input how to scale this and of course the main culprit is