Before revision: Transformers are feedforward networksthat can process input tokens in parallel.
After revision: Transformers have been successfully applied to sequential, auto-regressive tasks despite being feedforward networks. Unlike recurrent neural networks, Transformers use attention to capture temporal relations while processing input tokens in parallel.