now these position encodings they are different for the queries keys and values but um ultimately we don't it doesn't make too much of a difference so here is a contrast between what a traditional attention layer would do and what they would do so a traditional attention layer gets the input x and transforms it by means of these linear transformations right here into the queries these are the queries let's call them q into the keys and into the values okay then it does a matrix multiplication with the keys and the queries and puts