Before revision:   We show that these two techniques significantly improve the efficiency of model pre-training and the performance of both natural languageunderstand (NLU) and natural langauge generation (NLG)  tasks.
After revision:  In addition, a new virtual adversarial training method is used for fine-tuning to improve models' generalization. We show that these two techniques significantly improve the efficiency of model pre-training and the performance of both natural languageunderstand (NLU) and natural langauge generation (NLG) tasks.