Before revision: For natural language processing (NLP) taskssuch as sentiment or topic classification, currently prevailing approaches heavily rely on pretraining large self-supervised models on massive external data resources. However, this methodology is being critiqued for: exceptional compute and pretraining data requirements ;
After revision: For natural language processing (NLP) taskssuch as sentiment or topic classification, currently prevailing approaches heavily rely on pretraining large self-supervised models on massive external data sources, which incurs exceptional pretraining data requirements ;