um we can augment this by so-called positional embeddings uh a positional embedding you might know from the sequence transformers so if i have a sequence my cat is tall i don't even know what that means for a cat but okay what in a positional encoding so if you use a transformer and you transform this as we said into a sequence of equal length and then transform is basically information routing the transformer simply sees the lower layer sequence as a set not as a sequence it has no notion of what's neighboring to