Are Pretrained Language Models Symbolic Reasoners over Knowledge? (2020)

Abstract: How can pretrained language models (PLMs) learn factual knowledge from the training set? We investigate the two most important mechanisms: reasoning and memorization. Prior work has attempted to quantify the number of facts PLMs learn, but we present, using synthetic data, the first study that investigates the causal relation between facts present in training and facts learned by the PLM. For reasoning, we show that PLMs seem to learn to apply some symbolic reasoning rules correctly but struggle with others, including two-hop reasoning. Further analysis suggests that even the application of learned reasoning rules is flawed. For memorization, we identify schema conformity (facts systematically supported by other facts) and frequency as key factors for its success.

## introduction pretrained language models (plms) like bert @xcite , gpt-2 @xcite and roberta @xcite have emerged as universal tools that capture a diverse range of linguistic and -as more and more evidence suggests -factual knowledge @xcite . recent work on knowledge captured by plms is focused on probing, a methodology that identifies the set of facts a plm has command of. but little is understood about how this knowledge is acquired during pretraining and why. we analyze the ability of plms to acquire factual knowledge focusing on two mechanisms: reasoning and memorization. we pose the following two questions: a) symbolic reasoning: are plms able to infer knowledge not seen explicitly during pretraining? b) memorization: which factors result in successful memorization of a fact by plms? * *equal contribution we conduct our study by pretraining bert from scratch on synthetic corpora. the corpora are composed of short knowledge-graph like facts: subjectrelation-object triples. to test whether bert has learned a fact, we mask the object, thereby generating a cloze-style query, and then evaluate predictions. ## bert model bert uses a deep bidirectional transformer @xcite encoder to perform masked language modeling. during pretraining, bert randomly masks positions and learns to predict fillers. we use source code provided by @xcite . following @xcite , we perform dynamic masking and no next sequence prediction. for symbolic rules, we start with bert-base and tune hyperparameters. we vary the number of layers to avoid that rule learning fails due to over-parametrization, see appendix for details. we report precision based on optimal configuration. in the memorization experiment, our goal is to investigate the effect of frequency on memorization. due to a limited compute infrastructure, we scale down bert to a single hidden layer with 3 attention heads, a hidden size of 192 and an intermediate size of 768. ## limitations our experimental design makes many simplifying assumptions: i) variation in generated data is more 559 (a) frequency (b) schema conformity, (c) schema conformity, exceptions rare exceptions frequent limited than in naturally occurring data. ii) semantics are deliberately restricted to one rule only per generated corpus. iii) we do not investigate effects of model and corpus size. our work is an initial exploration of the question whether symbolic rules can be learned in principle, but we view it mainly as a starting point for future work. for complex reasoning qa benchmarks @xcite , plms are finetuned to the downstream tasks. their performance is difficult to analyze: it is not clear whether any reasoning capability is learned by the plm or by the task specific component. another line of work @xcite @xcite shows that much of plms' performance on reasoning tasks is due to statistical artifacts in datasets and does not exhibit true reasoning and generalization capabilities. with the help of synthetic corpora, we can cleanly investigate plms' reasoning capabilities. hupkes et al. (2020 ) study the ability of neural models to capture compositionality. they do not investigate our six rules, nor do they consider the effects of fact frequency and schema conformity. our work confirms their finding that transformers have the ability to capture both rules and exceptions. a large body of research in psychology and cognitive science has investigated how some of our rules are processed in humans, e.g., @xcite for implication. there is also a lively debate in cognitive science as to how important rule-based reasoning is for human cognition @xcite . @xcite ; @xcite are concurrent studies of systematicity in plms. the first shows that monotonicity inference is feasible for syntactic structures close to the ones observed during training. the latter shows that plms can exhibit high over-all performance on natural language inference despite being non-systematic. @xcite show that the amount of knowledge captured by plms increases with model size. our memorization experiments investigate the factors that determine successful acquisition of knowledge. ## conclusion we studied bert's ability to capture knowledge from its training corpus by investigating its reasoning and memorization capabilities. we identified factors influencing what makes successful memorization possible and what is learnable beyond knowledge explicitly seen during training. we saw that, to some extent, bert is able to infer facts not explicitly seen during training via symbolic rules. overall, effective knowledge acquisition must combine both parts of this paper: memorization and symbolic reasoning. a plm is not able to store an unlimited amount of knowledge. through acquiring reasoning capabilities, knowledge gaps can be filled based on memorized facts. a schemaconformant fact ("pigeons can fly") need not be memorized if there are a few facts that indicate that birds fly and then the ability of flight can be filled in for the other birds. the schema conformity experiments suggest that this is happening. it is easier to capture knowledge that conforms with a schema instead of memorizing facts one by one. there are several directions for future work. first, we made many simplifying assumptions that should be relaxed in future work. second, how can we improve plms' ability to learn symbolic rules? we see two avenues here, either additional inductive biases could be imposed on plms' architectures or training corpora could be modified to promote learning of symbolic rules. b.3 comp enhanced {e8, e2, e4, e5} e8 connectedto e2 e8 connectedto e4 e8 connectedto e5 e2 connectedto e8 ... {e15, e13, e12, e19} e15 connectedto e13 e15 connectedto e12 ... {e25, e24, e29, e20} e25 connectedto e24 e25 connectedto e29 ... e8 r1 e15 e8 r1 e13 e8 r1 e12 e8 r1 e19 e2 r1 e15 e2 r1 e13 ... e5 r1 e19 ... e15 r2 e25 e15 r2 e24 e15 r2 e29 e15 r2 e20 ... e19 r2 e20 ... e8 r3 e25 e8 r3 e24 e8 r3 e29 e8 r3 e20 ... ... e133 r61 e23 e56 r61 e29 ...