and then the relevant score is just the dot product of those two right and to visualize that you first you combine the key the query and the key values and that gives you the relevant scores you you use the softmax normalize them and then you do a weighted average of the values the third version of each token to get your output now to explain this in a little bit more detail I'm going to go through it in pseudocode so this looks like Python it wouldn't actually run but I think it's close enough to help people understand what's going on