Before revision:  In this paper, we propose ERNIE-DOC , a document-level language pretraining model based on Recurrence Transformers.
After revision:  In this paper, we propose ERNIE-Doc , a document-level language pretraining model based on Recurrence Transformers.