usually one of the downsides of the Transformers is done via a quadratic attention mechanism so if I just list one feature channel go over here if I just list one feature Channel right here this is height times width of the image right this is this is the entire image unrolled in one vector height times width and here I unroll it again height times width then this this matrix that I can build right here which is called the attention matrix right here it will tell me which parts of the sequence attends to which other parts okay so if you have