Before revision:  ERNIE-DOC achieves SOTA language modeling result of  16.8 ppl on WikiText-103 and outperforms competitive pretraining models  on most language understanding tasks  such as text classification , question answeringby a large margin .
After revision:  ERNIE-DOC achieves SOTA language modeling result of 16.8 ppl on WikiText-103 . Moreover, it outperformed competitive pretraining models on most language understanding tasks such as text classification , question answeringby a large margin .