importantly when they produce the keys and values the H is they'll look at here to update the G you're only allowed to look at H is before you in the permutation order but to update the H you're allowed to look at everything before including the position you're currently after so that's kind of the that's a it's an engineering solution to the problem introduced by their augmentation I think it's a pretty pretty neat solution pretty cool so the rest of the paper here is incorporating ideas from transformer Excel so transformer Excel is one of