Before revision:  Various experiments  on both English and Chinese document-level tasks are conducted. ERNIE-DOC achieves SOTA language modeling result of  16.8 ppl on WikiText-103 and outperforms competitive pretraining models  on most language understanding tasks  such as text classification , question answeringby a large margin .
After revision:  Various experiments on both English and Chinese document-level tasks . ERNIE-Doc improved the state-of-the-art language modeling result of 16.8 ppl on WikiText-103 and outperforms competitive pretraining models on most language understanding tasks such as text classification , question answeringby a large margin .