Before revision:  For BERT, our model achieves parity with Transformers on pretraining perplexity and is better on some downstream  tasks.
After revision:  For BERT, our model achieves parity with Transformers on pretraining perplexity and is better on some downstream NLP tasks.