Analyzing Commonsense Emergence in Few-shot Knowledge Models

tuned on knowledge graph ( KG ) tuples — showed that considerable amounts of commonsense knowledge can be encoded in the parameters of large language models [ Bosselut et al. , 2019 ] . However , as parallel studies show that LMs are poor hypothesizers of declarative commonsense relationships [ Petroni et al. , 2019 ] on their own , it remains unclear whether this knowledge is learned during pretraining or from fine-tuning on KG examples . To investigate this question , we train commonsense knowledge models in few-shot settings to study the emergence of their commonsense representation abilities . Our results show that commonsense knowledge models can rapidly adapt from limited examples , indicating that KG fine-tuning serves to learn an interface to encoded knowledge learned during pretraining . Importantly , our analysis of absolute , angular , and distributional parameter changes during few-shot fine-tuning provides novel insights into how this interface is learned . 1 . Introduction . Driven by advances in pretrained language models [ Liu et al. , 2019 , Raffel et al. , 2019 ] , recent NLP systems have demonstrated considerable improvement [ Clark et al. , 2020 , Lourie et al. , 2021 ] on benchmarks that test the commonsense representation and reasoning abilities of NLP systems [ Talmor et al. , 2019 , Sap et al. , 2019b , Bhagavatula et al. , 2020 , Sakaguchi et al. , 2020 ] . Despite this success , it remains unclear how much commonsense knowledge is actually directly encoded by these models . When prompted out-of-the-box to complete commonsense declarative relationships ( as depicted in Figure 1 ) , they exhibit limited ability to map their language modeling abilities to this task [ Petroni et al. , 2019 , Zhou et al. , 2020 ] . However , the zero-shot expression of declarative commonsense knowledge is but one use of language that these models are pretrained to manifest . Infilling evaluations that entangle this ability with general language expression may be narrow tests of true commonsense representation ability . Indeed , commonsense knowledge models [ Bosselut et al. , 2019 ] — which finetune pretrained LMs on examples from commonsense knowledge graphs [ Speer et al. , 2017 , Sap et al. , 2019a , Jiang et al. , 2021 ] — learn to express declarative commonsense relationships much more effectively . This learning procedure allows these models to recalibrate to the actual task of hypothesizing declarative knowledge . Simultaneously , these systems are evaluated in challenging train/test settings ( where head entities tested on can not be seen during training ) , indicating they may learn to transfer implicit representations of declarative knowledge that are learned during pretraining [ Hwang et al. , 2021 ] . However , current commonsense knowledge models are trained on a large number of examples from knowledge graphs . Consequently , this comprehensive finetuning obfuscates whether these systems exhibit knowledge that was encoded during language pretraining [ Geva et al. , 2020 , Dai et al. , 2021 ] , or whether pretraining merely provides a favorable initialization to learn to generalize from examples in the KG . In this work , we explore this question by training commonsense knowledge models in a few-shot setting . We vary the training budgets of KG examples that they receive , and evaluate their capacity to hypothesize commonsense knowledge after training on each of these budgets . Using this framework , we discover whether the finetuning process causes the model to learn new knowledge from the KG , or whether the model learns an interface to existing knowledge it already encodes . Our results demonstrate that few-shot commonsense knowledge models trained on knowledge graphs in few-shots — up to 10,000× fewer tuples than the full KG — exhibit strong performance relative to zero-shot LMs , and approach the performance of fully-trained knowledge models . Our analysis supports the hypothesis that few-shot finetuning allows the model to adapt knowledge it already encodes implicitly . We find that few-shot training mainly changes parameters in the attention heads of the decoder transformer blocks . The large feed-forward networks that serve as an implicit memory storage for information learned during pretraining [ Geva et al. , 2020 ] are minimally updated . Furthermore , we observe that larger knowledge models exhibit most of their parameter change across a more concentrated set of parameters , implying they learn knowledge during pretraining in a less entangled manner due to their capacity . Finally , we find that using natural language prompts to represent relation inputs accelerates commonsense emergence . Our code , few-shot training sets , and models are at https : //github.com/allenai/few-shot-comet/ . 2 . Background . In this work , we investigate few-shot performance and parameter change of commonsense knowledge models to shed light on the dynamics of commonsense representation learning . Below , we describe background concepts to contextualize this analysis . Commonsense Knowledge Graphs are structured , relational representations of commonsense knowledge . In our study , we use Atomic2020 [ Hwang et al. , 2021 ] ,1 a commonsense KG with 1.33M inferential knowledge tuples about entities and events . It represents a large-scale commonsense repository of textual descriptions that encode social and physical aspects of common human experiences . Across its 1.33M tuples , it captures information about 23 relationship types . Example head entities and relations are shown in Table 1 . Atomic2020 is split into training , development , and test subsets such that no head entities in one set appear in any other . This property allows models trained on this resource to be evaluated on their capacity to generalize commonsense relationships to new entities , events , and situations . Commonsense Knowledge Models represent facts by learning to encode a commonsense KG [ Bosselut et al. , 2019 ] . They are pre-trained as language models and fine-tuned on knowledge graph tuples to learn to hypothesize knowledge relationships through language generation . After training on a large collection of tuples from a KG , they learn the structure and relationships encoded by that KG . Furthermore , because they are seeded with pretrained language models , they learn to generalize the relationships to other entities about which the LM implicitly encodes knowledge [ Petroni et al. , 2019 , Roberts et al. , 2020 ] . Consequently , they can be used to produce precise knowledge on-demand for any entity that can be expressed through language . 3 . Experimental Setup . In this section , we outline our experimental setting for training few-shot knowledge models . Input Each tuple in the Atomic2020 knowledge graph is composed of a { head h , relation r , tail t } triplet . The head and tail entities in the triplet are natural language words or phrases . 1 . Downloadable at https : //allenai.org/data/atomic-2020 Commonsense knowledge models are trained by providing the tokens of h and r as inputs to the model and learning to generate the tokens of t. In this work , rather than initializing r with a new token and random embedding [ Bosselut et al. , 2019 ] , we automatically format the input tuples into natural language prompts to represent the relation [ Feldman et al. , 2019 , Jiang et al. , 2020 ] . Table 2 shows examples of such prompts . Training We source training tuples from the Atomic2020 training set [ Hwang et al. , 2021 ] . When constructing few-shot training sets , we set a target number n of examples to randomly sample from the knowledge graph for each relation ( i.e. , n = 3 =⇒ 3 examples × 23 relations = 69 total training examples ) . Once a training set of examples is produced , the model is trained on this subset to minimize the negative log-likelihood of the tokens of the tail entity for each tuple . We use the AdaFactor optimizer [ Shazeer and Stern , 2018 ] with a constant learning rate of 0.001 , a mini-batch size of 4 , and train the model for 3 epochs . Unless stated otherwise ( §5.3 ) , we use T5-11B [ Raffel et al. , 2019 ] as a seed language model for all experiments . To pick hyperparameters during fine-tuning , we run 5 different sets of hyperparameters and use a validation set of equivalent size to the training set ( n examples / relation ) to evaluate performance [ Perez et al. , 2021 ] . Evaluation We evaluate the knowledge hypothesized by the trained few-shot models using human and automatic evaluations . For the human evaluation ( Accept % in Table 3 ) , we use the procedure described in [ Hwang et al. , 2021 ] . We ask annotators to label the plausibility generated tuples using a 4-point Likert scale : { always/often true ( +2 ) , sometimes/likely true ( +1 ) , false/untrue ( −1 ) , nonsensical ( −2 ) } . We collect 3 annotations per relation , convert each annotation to an acceptability label ( i.e. , { +1 , +2 } → 3 , { −1 , −2 } → 7 ) and use the majority label as the acceptability judgment . Evaluation agreement is measured using Fleiss ’ s κ = 0.49 for the acceptability judgment . We also evaluate hypothesized knowledge tuples using automatic metrics : BLEU-1 [ Papineni et al. , 2002 ] , METEOR [ Banerjee and Lavie , 2005 ] , ROUGE-L [ Lin , 2004 ] , and CIDEr [ Vedantam et al. , 2015 ] . For few-shot experiments , we report average performance across 5 runs with different training sets . 4 . Do few-shot knowledge models learn ? . We evaluate the general few-shot commonsense interface learning capability of large-scale language models . We train COMeT models as described in §3 and set the number of training examples per relation to n = 3 . We report several zero-shot and few-shot augmentation baselines ( e.g. , GPT- { 2 XL , 3 } ) , which prepend n training examples ( for the same relation ) to the start of the input sequence as a prompt , and condition the generation on these additional examples . Few-shot augmentation baselines are not fine-tuned , but receive examples using the same prompt formats as the finetuning baselines ( Tables 2 , 8 ) . Similar to fine-tuning , we report average performance across 5 different example sets used for prompting . As T5 models are not pretrained using a causal language modeling objective , reporting zero-shot evaluation performance for a multi-word generation task would be an unfair comparison , so we only report limited numbers in Table 6 of Appendix A . We also report the results of several fully supervised COMeT knowledge models . Findings We find that both the few-shot augmentation and few-shot learning settings are able to produce high quality commonsense knowledge tuples , indicating that large LMs are also efficient few-shot learners ( in addition to showcasing impressive few-shot prompting abilities ) . Using only n = 3 tuples per relation , both GPT-3 and COMeT ( T5 ) produce highquality tuples that are accepted as plausible by human evaluators more than 70 % of the time — 75.7 % for COMeT ( T5 ) . While this performance falls short of the fully supervised knowledge model performance — 86.4 % — it demonstrates that the commonsense representation abilities of language models extends much further than their zero-shot prompting abilities . Furthermore , these abilities can be measured by finetuning on limited examples , supporting the hypothesis that early-stage finetuning serves to learn an interface to knowledge learned during pretraining . In the following section , we support this hypothesis by investigating how parameters of large language models change during finetuning ( across different training budgets n §5.2 , different model size §5.3 , and different prompting strategies §5.4 ) .

Summary: This paper analyzes the emergence of commonsense knowledge in few-shot knowledge models. In particular, it attempts to answer whether and when is this knowledge learned: during pretraining or from fine-tuning on KG examples. To investigate this, various commonsense knowledge models are trained in few-shot settings. The results and analysis in the paper shows that commonsense knowledge models can adapt from limited examples. Moreover, similar to a recent line of work, this work confirms that KG fine-tuning learns an interface to the commonsense knowledge learned during pretraining. Finally, the paper also presents an analysis of absolute, angular, and distributional parameter changes during few-shot fine-tuning. However, the insights here are less convincing to me.