attention mechanism um but then in a lot of experiments they don't actually have a random attention mechanism i believe they have to do that because that's kind of what makes them different from the long former in principle but still um yeah so the last novelty let's say is an engineering novelty in that they now always consider not single for example they don't consider single random attention they always consider these in blocks and that's because our current hardware is really bad at sparse stuff really bad at