about transformers before and the attention is all you need video so what a transformer does is it uses attention and if for those who forgot what attention is if you have like a token sequence ABCDE then a classic model to use that would be an LST M so the other stem would go here it would like have a vector representation a hidden state and then it would take this a it would take this hidden state and compute a new hidden state and then it will go on and take be and incorporate this into the hidden state the hidden state kind of always