Analyzing Commonsense Emergence in Few-shot Knowledge Models

tuned on knowledge graph ( KG ) tuples — showed that considerable amounts of commonsense knowledge can be encoded in the parameters of large language models [ Bosselut et al. , 2019 ] . However , as parallel studies show that LMs are poor hypothesizers of declarative commonsense relationships [ Petroni et al. , 2019 ] on their own , it remains unclear whether this knowledge is learned during pretraining or from fine-tuning on KG examples . To investigate this question , we train commonsense knowledge models in few-shot settings to study the emergence of their commonsense representation abilities . Our results show that commonsense knowledge models can rapidly adapt from limited examples , indicating that KG fine-tuning serves to learn an interface to encoded knowledge learned during pretraining . Importantly , our analysis of absolute , angular , and distributional parameter changes during few-shot fine-tuning provides novel insights into how this interface is learned . 1 . Introduction . Driven by advances in pretrained language models [ Liu et al. , 2019 , Raffel et al. , 2019 ] , recent NLP systems have demonstrated considerable improvement [ Clark et al. , 2020 , Lourie et al. , 2021 ] on benchmarks that test the commonsense representation and reasoning abilities of NLP systems [ Talmor et al. , 2019 , Sap et al. , 2019b , Bhagavatula et al. , 2020 , Sakaguchi et al. , 2020 ] . Despite this success , it remains unclear how much commonsense knowledge is actually directly encoded by these models . When prompted out-of-the-box to complete commonsense declarative relationships ( as depicted in Figure 1 ) , they exhibit limited ability to map their language modeling abilities to this task [ Petroni et al. , 2019 , Zhou et al. , 2020 ] . However , the zero-shot expression of declarative commonsense knowledge is but one use of language that these models are pretrained to manifest . Infilling evaluations that entangle this ability with general language expression may be narrow tests of true commonsense representation ability . Indeed , commonsense knowledge models [ Bosselut et al. , 2019 ] — which finetune pretrained LMs on examples from commonsense knowledge graphs [ Speer et al. , 2017 , Sap et al. , 2019a , Jiang et al. , 2021 ] — learn to express declarative commonsense relationships much more effectively . This learning procedure allows these models to recalibrate to the actual task of hypothesizing declarative knowledge . Simultaneously , these systems are evaluated in challenging train/test settings ( where head entities tested on can not be seen during training ) , indicating they may learn to transfer implicit representations of declarative knowledge that are learned during pretraining [ Hwang et al. , 2021 ] . However , current commonsense knowledge models are trained on a large number of examples from knowledge graphs . Consequently , this comprehensive finetuning obfuscates whether these systems exhibit knowledge that was encoded during language pretraining [ Geva et al. , 2020 , Dai et al. , 2021 ] , or whether pretraining merely provides a favorable initialization to learn to generalize from examples in the KG . In this work , we explore this question by training commonsense knowledge models in a few-shot setting . We vary the training budgets of KG examples that they receive , and evaluate their capacity to hypothesize commonsense knowledge after training on each of these budgets . Using this framework , we discover whether the finetuning process causes the model to learn new knowledge from the KG , or whether the model learns an interface to existing knowledge it already encodes . Our results demonstrate that few-shot commonsense knowledge models trained on knowledge graphs in few-shots — up to 10,000× fewer tuples than the full KG — exhibit strong performance relative to zero-shot LMs , and approach the performance of fully-trained knowledge models . Our analysis supports the hypothesis that few-shot finetuning allows the model to adapt knowledge it already encodes implicitly . We find that few-shot training mainly changes parameters in the attention heads of the decoder transformer blocks . The large feed-forward networks that serve as an implicit memory storage for information learned during pretraining [ Geva et al. , 2020 ] are minimally updated . Furthermore , we observe that larger knowledge models exhibit most of their parameter change across a more concentrated set of parameters , implying they learn knowledge during pretraining in a less entangled manner due to their capacity . Finally , we find that using natural language prompts to represent relation inputs accelerates commonsense emergence . Our code , few-shot training sets , and models are at https : //github.com/allenai/few-shot-comet/ . 2 . Background . In this work , we investigate few-shot performance and parameter change of commonsense knowledge models to shed light on the dynamics of commonsense representation learning . Below , we describe background concepts to contextualize this analysis . Commonsense Knowledge Graphs are structured , relational representations of commonsense knowledge . In our study , we use Atomic2020 [ Hwang et al. , 2021 ] ,1 a commonsense KG with 1.33M inferential knowledge tuples about entities and events . It represents a large-scale commonsense repository of textual descriptions that encode social and physical aspects of common human experiences . Across its 1.33M tuples , it captures information about 23 relationship types . Example head entities and relations are shown in Table 1 . Atomic2020 is split into training , development , and test subsets such that no head entities in one set appear in any other . This property allows models trained on this resource to be evaluated on their capacity to generalize commonsense relationships to new entities , events , and situations . Commonsense Knowledge Models represent facts by learning to encode a commonsense KG [ Bosselut et al. , 2019 ] . They are pre-trained as language models and fine-tuned on knowledge graph tuples to learn to hypothesize knowledge relationships through language generation . After training on a large collection of tuples from a KG , they learn the structure and relationships encoded by that KG . Furthermore , because they are seeded with pretrained language models , they learn to generalize the relationships to other entities about which the LM implicitly encodes knowledge [ Petroni et al. , 2019 , Roberts et al. , 2020 ] . Consequently , they can be used to produce precise knowledge on-demand for any entity that can be expressed through language . 3 . Experimental Setup . In this section , we outline our experimental setting for training few-shot knowledge models . Input Each tuple in the Atomic2020 knowledge graph is composed of a { head h , relation r , tail t } triplet . The head and tail entities in the triplet are natural language words or phrases . 1 . Downloadable at https : //allenai.org/data/atomic-2020 Commonsense knowledge models are trained by providing the tokens of h and r as inputs to the model and learning to generate the tokens of t. In this work , rather than initializing r with a new token and random embedding [ Bosselut et al. , 2019 ] , we automatically format the input tuples into natural language prompts to represent the relation [ Feldman et al. , 2019 , Jiang et al. , 2020 ] . Table 2 shows examples of such prompts . Training We source training tuples from the Atomic2020 training set [ Hwang et al. , 2021 ] . When constructing few-shot training sets , we set a target number n of examples to randomly sample from the knowledge graph for each relation ( i.e. , n = 3 =⇒ 3 examples × 23 relations = 69 total training examples ) . Once a training set of examples is produced , the model is trained on this subset to minimize the negative log-likelihood of the tokens of the tail entity for each tuple . We use the AdaFactor optimizer [ Shazeer and Stern , 2018 ] with a constant learning rate of 0.001 , a mini-batch size of 4 , and train the model for 3 epochs . Unless stated otherwise ( §5.3 ) , we use T5-11B [ Raffel et al. , 2019 ] as a seed language model for all experiments . To pick hyperparameters during fine-tuning , we run 5 different sets of hyperparameters and use a validation set of equivalent size to the training set ( n examples / relation ) to evaluate performance [ Perez et al. , 2021 ] . Evaluation We evaluate the knowledge hypothesized by the trained few-shot models using human and automatic evaluations . For the human evaluation ( Accept % in Table 3 ) , we use the procedure described in [ Hwang et al. , 2021 ] . We ask annotators to label the plausibility generated tuples using a 4-point Likert scale : { always/often true ( +2 ) , sometimes/likely true ( +1 ) , false/untrue ( −1 ) , nonsensical ( −2 ) } . We collect 3 annotations per relation , convert each annotation to an acceptability label ( i.e. , { +1 , +2 } → 3 , { −1 , −2 } → 7 ) and use the majority label as the acceptability judgment . Evaluation agreement is measured using Fleiss ’ s κ = 0.49 for the acceptability judgment . We also evaluate hypothesized knowledge tuples using automatic metrics : BLEU-1 [ Papineni et al. , 2002 ] , METEOR [ Banerjee and Lavie , 2005 ] , ROUGE-L [ Lin , 2004 ] , and CIDEr [ Vedantam et al. , 2015 ] . For few-shot experiments , we report average performance across 5 runs with different training sets . 4 . Do few-shot knowledge models learn ? . We evaluate the general few-shot commonsense interface learning capability of large-scale language models . We train COMeT models as described in §3 and set the number of training examples per relation to n = 3 . We report several zero-shot and few-shot augmentation baselines ( e.g. , GPT- { 2 XL , 3 } ) , which prepend n training examples ( for the same relation ) to the start of the input sequence as a prompt , and condition the generation on these additional examples . Few-shot augmentation baselines are not fine-tuned , but receive examples using the same prompt formats as the finetuning baselines ( Tables 2 , 8 ) . Similar to fine-tuning , we report average performance across 5 different example sets used for prompting . As T5 models are not pretrained using a causal language modeling objective , reporting zero-shot evaluation performance for a multi-word generation task would be an unfair comparison , so we only report limited numbers in Table 6 of Appendix A . We also report the results of several fully supervised COMeT knowledge models . Findings We find that both the few-shot augmentation and few-shot learning settings are able to produce high quality commonsense knowledge tuples , indicating that large LMs are also efficient few-shot learners ( in addition to showcasing impressive few-shot prompting abilities ) . Using only n = 3 tuples per relation , both GPT-3 and COMeT ( T5 ) produce highquality tuples that are accepted as plausible by human evaluators more than 70 % of the time — 75.7 % for COMeT ( T5 ) . While this performance falls short of the fully supervised knowledge model performance — 86.4 % — it demonstrates that the commonsense representation abilities of language models extends much further than their zero-shot prompting abilities . Furthermore , these abilities can be measured by finetuning on limited examples , supporting the hypothesis that early-stage finetuning serves to learn an interface to knowledge learned during pretraining . In the following section , we support this hypothesis by investigating how parameters of large language models change during finetuning ( across different training budgets n §5.2 , different model size §5.3 , and different prompting strategies §5.4 ) .

Summary: This paper analyzes fine-tuning of pre-trained generative language models on a commonsense knowledge task, investigating how much the strong performance of the models is due to pre-training or to task-specific learning (via fine-tuning or prompting/in-context learning). The task is to predict natural text for the tail of (head, relation, tail) relationships from the Atomic_20^20 knowledge graph (KG), e.g. ("PersonX goes to the mall", xIntent) -> "to buy clothes". The paper shows that a T5-based model obtains a substantial fraction of the performance of fine-tuning on the full Atomic dataset, using few-shot learning with only a small number of examples (3) per relation. Few-shot learning also obtains a substantial improvement over zero-shot performance for this model. The paper also presents several measures of parameter differences between pre-trained and fine-tuned models and performs an analysis with them, showing that more parameters are updated in the models' decoders than in their encoders. Taken together with past work, the evaluation results and analysis support the paper's claim that most of the knowledge useful for the KG task was already present in the pre-trained models.