Analyzing Commonsense Emergence in Few-shot Knowledge Models
Anonymous authors
Abstract
Recently, commonsense knowledge models — pretrained language models (LM) ﬁnetuned on knowledge graph (KG) tuples — showed that considerable amounts of commonsense knowledge can be encoded in the parameters of large language models [Bosselut et al., 2019].
However, as parallel studies show that LMs are poor hypothesizers of declarative commonsense relationships [Petroni et al., 2019] on their own, it remains unclear whether this knowledge is learned during pretraining or from ﬁne-tuning on KG examples.
To investigate this question, we train commonsense knowledge models in few-shot settings to study the emergence of their commonsense representation abilities.
Our results show that commonsense knowledge models can rapidly adapt from limited examples, indicating that
KG ﬁne-tuning serves to learn an interface to encoded knowledge learned during pretraining.
Importantly, our analysis of absolute, angular, and distributional parameter changes during few-shot ﬁne-tuning provides novel insights into how this interface is learned.
1. Introduction
Driven by advances in pretrained language models [Liu et al., 2019, Raﬀel et al., 2019], recent
NLP systems have demonstrated considerable improvement [Clark et al., 2020, Lourie et al., 2021] on benchmarks that test the commonsense representation and reasoning abilities of
NLP systems [Talmor et al., 2019, Sap et al., 2019b, Bhagavatula et al., 2020, Sakaguchi et al., 2020].
Despite this success, it remains unclear how much commonsense knowledge is actually directly encoded by these models.
When prompted out-of-the-box to complete commonsense declarative relationships (as depicted in Figure 1), they exhibit limited ability to map their language modeling abilities to this task
[Petroni et al., 2019, Zhou et al., 2020].
However, the zero-shot expression of declarative commonsense knowledge is but one use of language that these models are pretrained to manifest.
Inﬁlling evaluations that entangle this ability with general language expression may be narrow tests of true commonsense representation ability.
Indeed, commonsense knowledge models [Bosselut et al., 2019] — which ﬁnetune pretrained LMs on examples from commonsense knowledge graphs [Speer et al., 2017, Sap et al., 2019a, Jiang et al., 2021] — learn to express declarative commonsense relationships much more eﬀectively.
This learning procedure allows these models to recalibrate to the actual task of hypothesizing declarative knowledge.
Simultaneously, these systems are evaluated in challenging train/test settings (where head entities tested on cannot be seen during training), indicating they may learn to transfer implicit representations of declarative knowledge that are learned during pretraining [Hwang et al., 2021].
However, current commonsense knowledge models are trained on a large number of examples from knowledge graphs.
Consequently, this comprehensive ﬁnetuning obfuscates whether these systems exhibit knowledge that was encoded during language pretraining [Geva et al., 2020, Dai et al., 2021], or whether pretraining merely provides a favorable initialization to learn to generalize from examples in the KG.
In this work, we explore this question by training commonsense knowledge models in a few-shot setting.
We vary the training budgets of KG examples that they receive, and evaluate their capacity to hypothesize commonsense knowledge after training on each of these budgets.
Using this framework, we discover whether the ﬁnetuning process causes the model to learn new knowledge from the KG, or whether the model learns an interface to existing knowledge it already encodes.
Our results demonstrate that few-shot commonsense knowledge models trained on knowledge graphs in few-shots — up to 10,000 × fewer tuples than the full KG — exhibit strong performance relative to zero-shot LMs, and approach the performance of fully-trained knowledge models.
Our analysis supports the hypothesis that few-shot ﬁnetuning allows the model to adapt knowledge it already encodes implicitly.
We ﬁnd that few-shot training mainly changes parameters in the attention heads of the decoder transformer blocks.
The large feed-forward networks that serve as an implicit memory storage for information learned during pretraining [Geva et al., 2020] are minimally updated.
Furthermore, we observe that larger knowledge models exhibit most of their parameter change across a more concentrated set of parameters, implying they learn knowledge during pretraining in a less entangled manner due to their capacity.
Finally, we ﬁnd that using natural language prompts to
[Table]
COMeT (T5) produces diverse and novel tail hypotheses despite learning from few examples and is able to learn the task interface more easily than GPT-3, which tends to copy the input or generate unnecessary full sentences.
Assessments ( (cid:51) , (cid:55) ) are from human evaluators.
represent relation inputs accelerates commonsense emergence.
We will release our code, few-shot training sets
[Table]
and models for further study of this phenomenon
[Table]
2. Background
In this work, we investigate few-shot performance and parameter change of commonsense knowledge models to shed light on the dynamics of commonsense representation learning.
Below, we describe background concepts to contextualize this analysis.
Commonsense Knowledge Graphs are structured, relational representations of commonsense knowledge.
In our study, we use Atomic 2020 [Hwang et al., 2021], 1 a commonsense KG with 1.33M inferential knowledge tuples about entities and events.
It represents a large-scale commonsense repository of textual descriptions that encode social and physical aspects of common human experiences.
Across its 1.33M tuples, it captures information aboutrelationship types.
Example head entities and relations are shown in Table 1.
Atomic 2020 is split into training, development, and test subsets such that no head entities in one set appear in any other.
This property allows models trained on this resource to be evaluated on their capacity to generalize commonsense relationships to new entities, events, and situations.
Commonsense Knowledge Models represent facts by learning to encode a commonsense
KG [Bosselut et al., 2019].
They are pre-trained as language models and ﬁne-tuned on knowledge graph tuples to learn to hypothesize knowledge relationships through language generation.
After training on a large collection of tuples from a KG, they learn the structure and relationships encoded by that KG.
Furthermore, because they are seeded with pretrained language models, they learn to generalize the relationships to other entities about which the LM implicitly encodes knowledge [Petroni et al., 2019, Roberts et al., 2020].
Consequently, they can be used to produce precise knowledge on-demand for any entity that can be expressed through language. Relation r Prompt [Table]
seen
[Table]
isAfter Something [Table] happens
[Table]
As a result
[Table]
others
[Table]
Prompts signiﬁcantly speed up transfer in few-shot learning settings.
[Table]
In this section, we outline our experimental setting for training few-shot knowledge models.
Input
Each tuple in the Atomic 2020 knowledge graph is composed of a {head h , relation r , tail t } triplet.
The head and tail entities in the triplet are natural language words or phrases.
Commonsense knowledge models are trained by providing the tokens of h and r as inputs to the model and learning to generate the tokens of t .
In this work, rather than initializing r with a new token and random embedding [Bosselut et al., 2019], we automatically format the input tuples into natural language prompts to represent the relation [Feldman et al., 2019, Jiang et al., 2020].
Table 2 shows examples of such prompts.
Training
We source training tuples from the Atomic 2020 training set [Hwang et al., 2021].
When constructing few-shot training sets, we set a target number n of examples to randomly sample from the knowledge graph for each relation ( i.e. , n = 3 = ⇒ 3 examples ×relations = 69 total training examples).
Once a training set of examples is produced, the model is trained on this subset to minimize the negative log-likelihood of the tokens of the tail entity for each tuple.
We use the AdaFactor optimizer [Shazeer and Stern, 2018] with a constant learning rate of 0.001, a mini-batch size of 4, and train the model for 3 epochs.
Unless stated otherwise (§5.3), we use T5-11B
[Raﬀel et al., 2019] as a seed language model for all experiments.
To pick hyperparameters during ﬁne-tuning, we run 5 diﬀerent sets of hyperparameters and use a validation set of equivalent size to the training set ( n examples / relation) to evaluate performance [Perez et al., 2021].
Evaluation
We evaluate the knowledge hypothesized by the trained few-shot models using human and automatic evaluations.
For the human evaluation (Accept % in Table 3), we use the procedure described in [Hwang et al., 2021].
We ask annotators to label the plausibility generated tuples using a 4-point Likert scale: { always/often true (+2) , sometimes/likely true (+1) , false/untrue ( − 1) , nonsensical ( − 2) }.
We collect 3 annotations per relation, convert each annotation to an acceptability label ( i.e. , { +1 , +2 } → (cid:51) , {− 1 , − 2 } → (cid:55) ) and use the majority label as the acceptability judgment.
Evaluation agreement is measured using Fleiss’s κ = 0 .
49 for the acceptability judgment.
We also evaluate hypothesized knowledge tuples using automatic metrics: BLEU-1 [Papineni et al., 2002], METEOR [Banerjee and Lavie, [Table]
2005], ROUGE-L [Lin, 2004], and CIDEr [Vedantam et al., 2015].
For few-shot experiments, we report average performance across 5 runs with diﬀerent training sets.
4. Do few-shot knowledge models learn?
We evaluate the general few-shot commonsense interface learning capability of large-scale language models.
We train COMeT models as described in §3 and set the number of training examples per relation to n = 3 .
We report several zero-shot and few-shot augmentation baselines ( e.g. , GPT-{2 XL, 3}), which prepend n training examples (for the same relation) to the start of the input sequence as a prompt, and condition the generation on these additional examples.
Few-shot augmentation baselines are not ﬁne-tuned, but receive examples using the same prompt formats as the ﬁnetuning baselines (Tables 2, 8). Similar to ﬁne-tuning, we report average performance across 5 diﬀerent example sets used for prompting. As T models are not pretrained using a causal language modeling objective, reporting zero-shot evaluation performance for a multi-word generation task would be an unfair comparison, so we only report limited numbers in Table 6 of Appendix A.
We also report the results of several fully supervised COMeT knowledge models.
Findings
We ﬁnd that both the few-shot augmentation and few-shot learning settings are able to produce high quality commonsense knowledge tuples, indicating that large LMs are also eﬃcient few-shot learners (in addition to showcasing impressive few-shot prompting abilities).
Using only n = 3 tuples per relation, both GPT-3 and COMeT (T5) produce highquality tuples that are accepted as plausible by human evaluators more than 70% of the time — 75.7% for COMeT (T5).
While this performance falls short of the fully supervised knowledge model performance — 86.4% — it demonstrates that the commonsense representation abilities of language models extends much further than their zero-shot prompting abilities.
Furthermore, these abilities can be measured by ﬁnetuning on limited examples, supporting the hypothesis that early-stage ﬁnetuning serves to learn an interface to knowledge learned during pretraining.
In the following section, we support this hypothesis by investigating how parameters of large language models change during ﬁnetuning (across diﬀerent training budgets n §5.2, diﬀerent model size §5.3, and diﬀerent prompting strategies §5.4).
5. How do knowledge models learn?
Despite strong empirical results showing that pretrained language models can rapidly adapt to become few-shot language learners [Schick and Schutze, 2021a], little work has focused on how language models learn to adapt knowledge learned from pretraining to perform well on downstream end tasks.
We explore this problem in the context of few-shot knowledge modeling and investigate how the parameters of few-shot commonsense knowledge models change during ﬁnetuning.
We deﬁne three measures to investigate parameter change: absolute parameter change, angular parameter change, and distribution of parameter change.
5.1 Measuring parameter change
Notation
To more easily discuss measures of parameter change during knowledge model ﬁne-tuning, we review the structure of the transformer blocks in an encoder-decoder transformer LM [Vaswani et al., 2017].
Encoder blocks consist of a self-attention layer followed by a feedforward network.
Each self-attention layer contains parameter matrices (referred to in the following sections as q , k , v ) that project the query, key, and value inputs before computing a scaled dot-product attention between the key and value, and an output projection o of the attended value vectors.
The trainable parameters in the feedforward network consist of two linear projection matrices wi , wo .
Each decoder block consists of the same parameter matrices with an additional cross-attention layer (parameter matrices xq , key xk , value xv , and output xo ) that attends to the outputs of the encoder blocks.
Absolute Parameter Change
To measure a normalized quantity for average change in each parameter matrix, we compute a normalized (cid:96) 1 distance for each set of parameter matrices in the transformer blocks.
Without loss of generality, for each of these parameter matrices, we deﬁne Θ PT as the matrix of parameters before ﬁnetuning, and Θ FT as the matrix of parameters post-ﬁnetuning.
The normalized (cid:96) 1 distance between these two matrices for each parameter matrix type is then:
[Equation]
where m and n correspond to the row and column dimensionality of the parameter matrix, and (cid:107)·(cid:107) 1 corresponds to the (cid:96) 1 norm.
Intuitively, d (cid:96) 1 measures the degree of absolute per parameter shift between these two sets of parameters.
Angular Parameter Change
However, measuring normalized (cid:96) 1 distance solely captures how the magnitude of the parameter matrix changes, but obfuscates relative drift between parameters in each matrix.
For instance, a matrix with high average L1 change may in fact have small relative changes between its elements (i.e., the parameter change may mainly cause a scaling eﬀect on the output vectors).
Consequently, we measure the row-wise angular distance between Θ PT and Θ FT to capture the rotational drift of parameters during ﬁnetuning:
[Equation]
where k indexes a common row vector in both parameter matrices, (cid:104)·(cid:105) corresponds to an inner product, (cid:107)·(cid:107) 2 is the Euclidean ( (cid:96) 2 ) norm, and cos − 1 is the inverse cosine operation.
Contrary to d (cid:96) 1 , which represents a per-parameter average distance, d ang is computed between all corresponding rows of the parameter matrices, Θ PT and Θ FT , and averaged per row.
We choose to average rows as opposed to columns as each row in the parameter matrix performs a dot product with the input vector to the transformation, and has a correspondence to an individual element in the output vector.
Distributional Parameter Change
In additional to angular distance, we also report the distribution of parameter changes for a given matrix.
More concretely, parameter changes may be concentrated across varying numbers of dimensions.
We compute the change in each dimension by calculating W = | Θ PT − Θ FT | , sorting parameters by their absolute change, and plotting % cumulative parameter number against % cumulative mass as a function of a given parameter change w i ∈ W :
[Equation]
Figure 3 visualizes this relationship.
Intuitively, a matrix with a small AUC has a skewed relative distribution of changes, signaling that a select number of dimensions are responsible for a large amount of the learning during ﬁne-tuning.
Likewise, a matrix with a large AUC learns a relatively even parameter change distribution.
5.2 How does few-shot learning change the parameters of the pretrained LM?
We ﬁrst investigate how diﬀerent example budgets aﬀect parameter change.
We train a few-shot COMeT (T5) model across diﬀerent example budgets ( n ∈ { 3 , 30 , 300 } ).
Tableshows that performance increases monotonically as we grow the example budget.
To explore the diﬀerences between models, we investigate the parameter changes between each ﬁne-tuned model in Fig.
We include additional heatmaps for angular change in the Appendix.
Findings
In Figure 4, we observe that the (cid:96) 1 distance between Θ FT and Θ PT increases as we increase the training budget in the decoder.
Interestingly, we note that most parameter changes occur in the decoder of the LM (and more strongly in its later layers), rather than the encoder, implying that much of the parameter shift may be due to learning how to express commonsense knowledge in a declarative form.
Furthermore, when comparing the changes across parameter matrix types [Figure] the key and value matrices of the attention layers change more while the feedforward network parameters ( wi , wo ) remain relatively unchanged.
These ﬁndings suggest the model learns how to process the structure of declarative commonsense knowledge expressions [Figure] but does not need to modify its stored representations of this knowledge learned during pretraining, hinting at a possible explanation for the success of adapters for limited-parameter ﬁnetuning of NLP models [Houlsby et al., 2019] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
5.3 How does model size aﬀect knowledge model learning?
We train a few-shot COMeT (T5) model (with n = 30 ) across diﬀerent pretrained language model sizes, i.e. Small, Large, and 11B, with 60M, 770M, and 11B parameters, respectively.
Findings
In Table 4, we observe that the importance of model size decreases as the training budget increases, reinforcing that model scale is critically important in few-shot learning settings [Brown et al., 2020].
When examining the parameter change heatmaps in Figure 5, we note the (cid:96) 1 distances between ﬁnetuned and original parameters increase as model size increases.
As these distances are normalized by the number of parameters, this observation implies that when trained on the same examples, the smaller model experiences smaller magnitude gradient updates over time.
This pattern indicates more destructive inference between parameter updates in the smaller models, an observation supported by the larger angular parameter changes in the encoders of the smaller models.
Interestingly, several parameter matrices display high angular change but low (cid:96) 1 change (e.g., in early wo layers of T5-11B), suggesting that a matrix with low average absolute parameter change may still contain critical dimensions which are used to learn the task.
To investigate these diﬀerences, we look at the AUC heatmaps (Fig. 5, bottom row ), and note that as the feedforward networks get larger with increasing model size, the relative number of parameters that change to encode new information drops drastically in both the decoder and encoder.
Furthermore, the lower AUCs for the encoder parameter changes indicate that the encoder layers generally experience more concentrated changes as the model size increases.
These two patterns suggest that the increased capacity of larger language models allow them to encode knowledge in a less distributed manner, making their knowledge easier to access.
5.4 How do prompts inﬂuence knowledge model learning?
Recent work has shown that prompts can help models elicit knowledge from pretrained language representations [Feldman et al., 2019, Shin et al., 2020].
However, eliciting knowledge [Table] through zero-shot prompting has drawbacks.
For example, the output is sensitive to subtle variations in the construction of the language prompt [Jiang et al., 2020].
Here, we explore whether prompts can accelerate few-shot learning by initializing natural language prompts for each relation (Table 2) training on these expressions of relations rather than initializing new relation token embeddings for each relation as in Bosselut et al.
As a control for whether the model understands the semantics of speciﬁc relation prompts, or merely beneﬁts from arbitrary expressions of relations using language, we also train a model where we shuﬄe prompts among the relations ( i.e. shuﬄing the Prompt column of Table 2).
Findings
In Table 5, we see that knowledge models can eﬃciently learn from fewer examples when relations are represented using correct natural language prompts ( ∼ 10 × fewer examples), an observation made by contemporaneous works on other tasks [Scao and
Rush, 2021].
Interestingly, however, we ﬁnd that the absolute and angular distance heatmaps for prompt and embedding relation inputs are similar (Fig. 6; (cid:96) 1 heatmap in Appendix), implying that most of the parameter change during ﬁne-tuning is not linked to the way the relation is represented.
One notable diﬀerence is that models with embedding relation inputs have a signiﬁcant angular change in layer 4 of the decoder wo matrix.
Models that use prompts see a larger angular change in layer 2 of the decoder wo matrix.
This discrepancy suggests that both models must learn to adapt their parameters to the new input format, but do so by adapting the same parameters at diﬀerent layers.
We note that there is no such concentrated angular change for the Shuﬄed setting, perhaps because the model must ﬁrst unlearn language relationships between head entities and prompts, requiring more uniform parameter updates.
6. Related Work Commonsense Knowledge Models
Our work uses commonsense knowledge models, ﬁrst proposed by Bosselut et al. [2019], to learn from commonsense knowledge graphs. Hwang et al. [2021] also trained commonsense models on Atomic 2020 , but focused on fully-supervised learning.
Other works have developed commonsense knowledge models that are grounded to visual scenes [Park et al., 2020, Da et al., 2021], requiring multimodal commonsense inference generation.
Recent works extend commonsense knowledge models beyond generating singlehop inferences and generate
multi-branch [Bosselut et al., 2021] and multi-hop
[Wang et al., 2020b] inferential structures.
Commonsense knowledge base completion is also a closely related task to commonsense inference generation [Li et al., 2016, Saito et al., 2018].
Recent works on this task combine language and graph structure representations for improved generalization [Malaviya et al., 2020, Wang et al., 2020a].
Our work diﬀers from these prior studies by investigating commonsense knowledge models in few-shot settings, and analyzing the emergence of their capabilities as a function of parameter change.
Few-shot Learning with Prompting
In recent years, the term few-shot has taken two meanings: the classical deﬁnition of training on limited examples [Fei-Fei et al., 2006, Fink, 2005], and a new in-context deﬁnition where models are given examples as a prepended augmentation of their context and process these examples as input to recognize the structure of a task [Brown et al., 2020].
Various contemporaneous works related to ours have studied the use of natural language and artiﬁcial prompts in language models [Schick and Schutze, 2021a, Gao et al., 2020, Tam et al., 2021, Schick and Schutze, 2021b] as accelerators of few-shot learning. Scao and Rush [2021] further empirically study the data eﬃciency advantage gained from using prompting to represent a task.
Our work diﬀers from these studies in that we do not use prompts to map a classiﬁcation task to masked language modeling, but instead use prompts as priming text for a generation task.
Most similar to our approach is perhaps
Schick and Schütze [2020], which uses prompts for few-shot summarization, but their work does not focus on how few-shot training aﬀects the learned representations of their model.
7. Conclusion
In this work we propose few-shot commonsense knowledge modeling, where models are ﬁnetuned on a limited number of examples to evaluate how eﬃciently they can adapt their pretrained representations to the task of hypothesizing commonsense knowledge tuples.
Our results demonstrate that large language models require few examples to adapt their learned representations to the task and our analysis explores how their parameters change to enable this rapid emergence of commonsense representation ability.
Appendix A. Accuracy in zero-shot MLM setting
While little work has explored few-shot knowledge completion, recent works have investigated performance of zero-shot knowledge graphs [Petroni et al., 2019, Feldman et al., 2019].
Thus, we investigate the ability of T5-11B to complete commonsense knowledge in a zero-shot setting.
[Table]
We use prompts to leverage the masking objective of the language model pretraining.
Since the mask only predicts several tokens at a time, for relations with longer length tail entities, we allow the model to predict up to 7 mask tokens in succession, or until the model predicts an empty string for the mask.
We suggest that this is still only a workaround, and masked models are poor predictors of longer length tail entities, as indicated by our results above.
Appendix B. Additional Results on the Eﬀect of Relation Input Format
In Table 7, we provide further experimental results on the eﬀect of relation input format on few-shot performance.
Namely, we extend the results from Table 5 with the case n = 300 .
These results conﬁrm that knowledge models can eﬃciently learn from fewer examples when relations are represented using natural language prompts.
We also show the results from using paraphrases of the main prompts for training (original and paraphrased prompts can be found in Tables 8 and 10, respectively).
We ﬁnd that the paraphrased prompts do cause a slight drop in performance, but that this drop is generally close to the margin of error, indicating that while prompt formulation is an important consideration [Jiang et al., 2020], ﬁne-tuning on the prompts does make the model less sensitive to prompt variations.
[Table] [Table] ObjectUse {} is used for AtLocation
You are likely to ﬁnd {} in
MadeUpOf {} is made up of
HasProperty {} is
CapableOf {}
can
Desires {} wants
NotDesires {} does not want isAfter
Something that happens after {} is
HasSubEvent
Something you might do while {} is isBefore
Something that happens before {} is
HinderedBy {} is hindered by
Causes
Sometimes {} causes xReason {}.
The reason for PersonX doing this is isFilledBy {} can be ﬁlled by xNeed
But before {}, PersonX needed xAttr {} is seen as xEﬀect
As a result of {}, PersonX will xReact
As a result of {}, PersonX feels xWant
After {}, PersonX would want xIntent
Because of {}, PersonX wanted oEﬀect as a result of {}, others will oReact as a result of {}, others would feel oWant as a result of {}, others would want
Something you might do while {} is ___
Something you might do while design software is determine deliverables
Something you might do while scuba dive is take oﬀ scuba gear
Something you might do while play ball is put on mitt
Relation
Template
ObjectUse a {} can be used for
AtLocation
You could ﬁnd {} in the location
MadeUpOf {} is made up of
HasProperty {} will have
CapableOf {} is capable of
Desires a {} desires
NotDesires a {} does not desire isAfter
Before {},
HasSubEvent
You might do {} while doing isBefore
After {},
HinderedBy {}.
This is hindered by
Causes
Sometimes {} causes xReason {}.
PersonX did this because isFilledBy {} is ﬁlled xNeed
Before {}, PersonX needs to xAttr {}.
An attribute of PersonX is xEﬀect
The eﬀect of {} PersonX will be xReact
As a result of {}.
PersonX will be xWant
After {}, PersonX will want to xIntent
For {}, PersonX did this to oEﬀect
An eﬀect of {} on others will be oReact
As a result of {}, other feel oWant After {}, others will want to Appendix C. Additional Experiments on Parameter Change Measures In addition, we provide extensive results of the (cid:96) 1 and angular distances, as well as the distributional parameter change metric (AUC), for both the encoder and decoder of various model sizes (Small, Large, and 11B) and diﬀerent example budget ( n ∈ { 3 , 30 , 300 } )
(Figures 736).
When computing AUC diagrams, we round each weight change to the nearest 10 − 5 .