transformers are quickly coming for your favorite models yesterday they replaced lstms in nlp they used to be good at nlp but blah we now have transformers think again today we're going to see that maybe in the near future transformers will replace convolutions in image processing so this paper is a step in toward towards this direction you just wonder what is it going to be tomorrow maybe linear regression is going to be replaced just by giant transformers trained on 5 000 tpus uh who knows we'll see in any case we're looking at