Before revision:   We show that these two techniques significantly improve the efficiency of model pre-training and the performance of both natural languageunderstand (NLU) and natural langauge generation (NLG)  tasks.
After revision:  We show that these two techniques significantly improve the efficiency of model pre-training and the performance of both natural languageunderstand (NLU) and natural langauge generation (NLG) downstream tasks.