query key multiplication right here but we also multiply the query by these uh positional embeddings for q we also multiply the keys by the positional embeddings for k and all of this together so this is a big plus right here all of this together is routed through the softmax okay and now the diagram is a little bit complicated uh now you can see the softmax aggregates information from here and from this learn position embeddings i would rather have they would just use it like they did in the formula uh do v plus r and