deep learning models for nlp unfortunately one of their core limitations is the quadratic dependency mainly in terms of memory on the sequence length due to their full attention mechanism so really briefly the full attention mechanism and i've done you know numerous videos about attention mechanism bert attention is all you need and so on so if you want a detailed explanation of what that is just go look up the corresponding videos but briefly what you'll have in nlp is a set of tokens a sequence of tokens as an input and you