Before revision:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including MT (EnDe, EnFr), language modeling(LM1B), abstractive summarization (CNN/Dailymail), dialogue generation (PersonaChat) and Multi-task language understanding (GLUE, SuperGLUE ).
After revision:  Our experimental results show thatSynthesizer is competitive against vanilla Transformer models across a range of tasks, including machine translation, language modeling, text generation and GLUE/SuperGLUE benchmarks. When composed with dot product attention, we find that Synthesizers consistently outperform Transformers. Moreover, we conduct additional comparisons of Synthesizers against Dynamic Convolutions, showing that simple Random Synthesizer is not only 60