the output locations you want to know how should i aggregate information from that lower layer and you do this by this thing here this thing here is this dynamically constructed attention matrix using also the softmax okay so how should you aggregate information this comes from this query at the output position and the keys at the input position and now you add to that this method this thing right here which is again an inner product between the carrier query and the positional encodings okay so the positional encodings are going to