Before revision: Since the introduction of deep pretrained language models , most task-specific NLP models were reduced to simple lightweight layers.
After revision: The introduction of pretrained language models , most task-specific NLP models were reduced to simple lightweight layers.