Commonsense Knowledge Transfer for Pre-trained Language Models (2023)

Abstract: Despite serving as the foundation models for a wide range of NLP benchmarks, pre-trained language models have shown limited capabilities of acquiring implicit commonsense knowledge from self-supervision alone, compared to learning linguistic and factual knowledge that appear more explicitly in the surface patterns in text. In this work, we introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model. It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model and then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction, which align human language with the underlying commonsense knowledge. Empirical results show that our approach consistently improves the model’s performance on downstream tasks that require commonsense reasoning. Moreover, we find that the improvement is more significant in the few-shot setting. This suggests that our approach helps language models better transfer to downstream tasks without extensive supervision by injecting commonsense knowledge into their parameters.

## introduction recent advances in pre-trained language models have transformed the landscape of natural language processing. self-supervised pre-training objectives including masked language modeling @xcite and masked span infilling @xcite enable pre-trained models to acquire linguistic @xcite factual knowledge @xcite by modeling the distribution of naturally occurring texts. however, most of these objectives are limited to exploiting the surface form of human language, and the lack of grounded supervision calls into question how well these representations can ever capture meaning @xcite , not to mention the underlying commonsense knowledge which is often reasoned implicitly and does not appear in the surface form of human language @xcite @xcite . on the other hand, commonsense reasoning is important for building generalizable models because it enables the model to reason about a great number of events, causes, and effects, while observing only a small fraction of them. the ineffectiveness of self-supervised language model pre-training on acquiring commonsense knowledge makes them require a relatively large number of labeled examples to succeed in a downstream task and prune to overfit task-specific correlations @xcite . therefore, equipping pre-trained language models with commonsense reasoning ability has at-tracted much attention. to this end, two distinct lines of research focus on improving commonsense reasoning ability of pre-trained language models. the first one focuses on incorporating external commonsense knowledge graph for commonsense reasoning @xcite @xcite while the other attempts to inject commonsense knowledge into the parameters of pretrained models @xcite @xcite . in this work we focus on the second type of method because it alleviates the need for external knowledge bases for training and inference on downstream tasks, thus simpler, more efficient, and not limited by the coverage issue of external knowledge bases. prior work injects commonsense knowledge into pre-trained models either on symbolic commonsense knowledge graphs with manually defined rules @xcite or masked language modeling @xcite or on general text corpus with concept-centric self-supervised objectives @xcite . the former method is limited by the coverage of knowledge graphs and human-written rules. it also fails to make use of large-scale diverse natural text corpus. therefore, the training is limited to short and synthetic commonsense tuples, which affects its generalization ability on diverse downstream tasks. the latter method, however, only captures surface-level order relations between concepts and fails to learn commonsense relations between concepts such as cause, effect, intent, requirement, etc., which are crucial for commonsense reasoning but often implicitly reasoned, thus do not appear in the surface form of natural language. in this work, we propose commonsense knowledge transfer, an alternative framework to refine a general purpose pre-trained model's commonsense reasoning ability. in contrast to previous work, it aims to transfer the commonsense knowledge stored in a neural commonsense knowledge model (e.g., comet @xcite ) to a general purpose pre-trained model on large scale general text corpus. in this way, our approach combines the best of both worlds from prior art: the dense and informative commonsense knowledge from commonsense knowledge graphs and the accessibility of large-scale diverse general corpus. commonsense knowledge transfer is conceptually related to knowledge distillation (kd) @xcite since they both aim to trans-fer knowledge from a knowledge-rich model to another model that lacks it. however, different from conventional kd, in commonsense knowledge transfer, the source model (i.e., neural commonsense model) and the target model (i.e., pretrained model) are heterogeneous. moreover, instead of simply mimicking the teacher model, commonsense knowledge transfer requires the target model to learn specialized knowledge from the source model while retaining its own capability. this poses unique challenges since the knowledge transfer can not be accomplished by simply matching the logits or feature distribution between the student and the teacher. to this end, we propose to first extract commonsense knowledge in textual form from the source model and then exploit the extracted knowledge to form self-supervised training data for the target model. as illustrated in figure 1 , commonsense knowledge transfer first exploits general texts to form queries for retrieving commonsense knowledge from the neural commonsense knowledge model. then it refines a pretrained model with two self-supervised objectives that align the surface form of human language with its underlying commonsense inference: commonsense text infilling and commonsense relation prediction. the former objective concatenates natural text with its commonsense inference to form an input example, masks certain spans in it, and trains the model to reconstruct the original input. the latter method instead trains the model to distinguish valid commonsense inference from carefully constructed spurious commonsense inference given the original text and commonsense relation. refining a pre-trained model by multi-tasking on both generation (former) and understanding (latter) tasks enables the model to better adapt to different kinds of downstream tasks. we refine t5 @xcite with commonsense knowledge transfer and fine-tune the resulting model downstream tasks requiring commonsense reasoning ability in both the fully supervised setting and few-shot settings where only a percentage of labeled examples are available. experimental results show substantial improvements in downstream tasks requiring commonsense reasoning, especially in the few-shot setting, demonstrating the effectiveness of our approach. ## methodology our proposed commonsense knowledge transfer framework consists of a neural commonsense knowledge model (e.g., comet) and a pre-trained model (e.g., t5). the goal of commonsense knowledge transfer is to transfer the commonsense knowledge from the neural commonsense knowledge model (i.e., source model) to the pre-trained model (i.e., target model) so that it can generalize better to downstream tasks requiring commonsense reasoning ability. compared to conventional knowledge transfer methods such as knowledge distillation, commonsense knowledge transfer faces a unique challenge: the source model and the target model are heterogeneous because they are trained on different data with different objectives. as such, we can not simply feed a batch of data to both of the models and train the target model to match the source model's logits or feature distribution. to alleviate this problem, we propose a two-stage knowledge transfer scheme as illustrated in figure 1 . to be specific, we first use natural texts to form queries for retrieving commonsense knowledge (in text form) from the neural commonsense knowledge model. we then construct training data with two novel commonsense-related self-supervised objectives based on the retrieved commonsense knowledge and the corresponding natural text. finally, we train the target model on the constructed training data to inject commonsense knowledge retrieved from the source model. we describe our method to extract commonsense knowledge from a neural commonsense knowledge model and the proposed commonsense-related self-supervised objectives in detail in this section. ## conclusion we introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained model. our method extracts commonsense knowledge from the source model to construct self-supervised training data for the target model. empirical results show that our approach consistently outperforms previous methods for improving the commonsense reasoning ability of pre-trained models that exploit either symbolic knowledge graphs or texts alone.