Before revision:  On standard language modeling benchmarks such as enwik8 and Wiki-103  datasets, our model obtains better perplexity and bits-per-character (bpc) while using 2.5x-10x less training time and cost compared to top-performing Transformer models.
After revision:  On standard language modeling benchmarks such as enwik8 and Wiki-103 and Billion Word datasets, our model obtains better perplexity and bits-per-character (bpc) while using 2.5x-10x less training time and cost compared to top-performing Transformer models.