Before revision: For natural language processing ' text-to-text' tasks, the prevailing approaches heavily rely on pretraining large self-supervised models on massive external datasources, which incurs exceptional pretraining data requirements and a diminished ability to pretrain over small datasets.
After revision: For natural language processing ` text-to-text' tasks, the prevailing approaches heavily rely on pretraining large self-supervised models on massive external datasources, which incurs exceptional pretraining data requirements and a diminished ability to pretrain over small datasets.