Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge (2022)

Abstract: Transformer models pre-trained with a masked-language-modeling objective (e.g., BERT) encode commonsense knowledge as evidenced by behavioral probes; however, the extent to which this knowledge is acquired by systematic inference over the semantics of the pre-training corpora is an open question. To answer this question, we selectively inject verbalized knowledge into the pre-training minibatches of BERT and evaluate how well the model generalizes to supported inferences after pre-training on the injected knowledge. We find generalization does not improve over the course of pre-training BERT from scratch, suggesting that commonsense knowledge is acquired from surface-level, co-occurrence patterns rather than induced, systematic reasoning.

## introduction pre-trained transformers, such as bert, encode knowledge about the world @xcite ; e.g., bert assigns relatively high probability to "fly" appearing in the context "robins can resolving how models acquire commonsense knowledge has important implications. if models learn to make systematic inferences through pretraining, then scaling up pre-training is a promising direction for commonsense knowledge acquisition. if, instead, models only ever generalize based on superficial, surface-level patterns, then the majority of commonsense knowledge, which is only supported implicitly, will never be acquired @xcite . on the one hand, there is cursory evidence that pre-training might induce the ability to systematically reason about the world. when fine-tuned on supervised training sets, pre-trained models can classify valid inferences better than strong baselines @xcite ; and, in zero-shot evaluations, pre-trained models perform relatively well on reasoning tasks that may require systematic reasoning, such as number comparison @xcite and winograd schemas @xcite . on the other hand, existing works have argued that pre-training does not generalize by systematic inference over semantics on the basis of theoretical or synthetic results @xcite @xcite . referring to physical commonsense knowledge acquired by bert, @xcite conclude that "neural language representations still only learn associations that are explicitly written down." our main contribution is a direct evaluation of the training dynamics of bert's reasoning ability. we inject verbalized knowledge, such as "robins are birds" (where the masked token is the predicate, e.g., "birds"), into the minibatches of bert throughout pre-training. we then consider how well bert generalizes to supported inferences; e.g., how does the likelihood of "robins can " → "fly" change? we find generalization does not improve over the majority of pre-training which supports the hypothesis that the type of commonsense knowledge studied is not acquired by systematic inference. rather, our findings suggest this knowledge is acquired from surface-level, co-occurrence patterns. ## related work commonsense knowledge acquisition is a longstanding challenge in natural language processing @xcite @xcite , and current approaches rely on knowledge acquired by pre-trained transformer language models @xcite @xcite . the commonsense reasoning ability of these language models has been evaluated using behavioral probes @xcite @xcite and downstream, fine-tuned evaluations @xcite @xcite . such works consider the knowledge encoded by a model after pre-training. when fine-tuned on supervised datasets, pretrained models can learn to make systematic inferences to some extent @xcite @xcite @xcite . by systematic inferences, we refer to the ability to learn general rules and apply them in novel settings, as opposed to learning only particular instances of the rule @xcite @xcite . similar to our experiments, recent work has considered the training dynamics of pre-trained models @xcite . notably, @xcite evaluate the zero-shot performance of roberta on the olmpics reasoning tasks throughout pre-training, but find the knowledge studied is never learned. in contrast, we explore how learned knowledge is acquired. close in spirit to our work, kassner et al. (2020) pre-train a masked language model on a synthetic dataset to isolate reasoning ability. @xcite also intervene on bert's pre-training data in a syntactic evaluation and conclude that subject-verb agreement is sometimes inferred from systematic rules for frequent words. finally, de @xcite explore how knowledge encoded in bert is affected by gradient updates when fine-tuning on a downstream classification task. @xcite build on this work and explore how gradient updates on verbalized premises affect models' performance on supported inferences. in contrast, we focus on knowledge obtained by the pre-training objective itself. ## method the purpose of our evaluation is to answer the question: does bert systematically infer commonsense knowledge from premises present in the pretraining corpora? we focus on one specific type of commonsense knowledge that bert is known to encode, namely entity properties annotated in conceptnet @xcite . this knowledge can be represented abstractly as (subject, relation, object) triples. we verify bert's encoding of knowledge by the ability to predict the object conditioned on a verbalization of the knowledge containing only the subject and relation; e.g., for (robin, capable-of, fly), we evaluate the ability to predict "fly" appearing in the context "robins can ." such knowledge may be supported by simple co-occurrence patterns (such as "robins" and "fly" having high co-occurrence), but we are interested in the extent to which knowledge might also be supported by induced, systematic inference. we focus on the inference of downward monotonicity (a is-a b ∧ b has-property c ⊨ a has-property c). we refer to the hypernym property (b has-property c) as the super-statement, the hyponym property (a has-property c) as the sub-statement, and the hypernymy relation (a is-a b) simply as the class relation ( we can then evaluate, for example, whether "robins can fly" is influenced by the inference "robins are birds" ∧ "birds can fly" ⊨ "robins can fly." for this evaluation, we inject a supporting premise into a pre-training minibatch (i.e., we replace one of the sentences in the minibatch with the premise) and then evaluate bert's knowledge of the supported inference after a gradient update on the minibatch containing the premise. we run this evaluation at intervals throughout the entire pre-training procedure, from random initialization to a fully pre-trained bert model. if pre-training induces the ability to systematically make the downward monotonicity inference, one would expect that generalization from premise to inference will improve as pre-training progresses. ## conclusion we show that the ability of bert to acquire commonsense knowledge from premises and learned inferences does not improve across pre-training, suggesting that the studied knowledge is not acquired from induced semantic inferences. these results suggest that an explicit reasoning mechanism may be necessary to acquire certain commonsense knowledge.