A na L og: Testing Analytical and Deductive Logic Learnability in Language Models (2022)

Abstract: We investigate the extent to which pre-trained language models acquire analytical and deductive logical reasoning capabilities as a side effect of learning word prediction. We present AnaLog, a natural language inference task designed to probe models for these capabilities, controlling for different invalid heuristics the models may adopt instead of learning the desired generalisations. We test four languagemodels on AnaLog, finding that they have all learned, to a different extent, to encode information that is predictive of entailment beyond shallow heuristics such as lexical overlap and grammaticality. We closely analyse the best performing language model and show that while it performs more consistently than other language models across logical connectives and reasoning domains, it still is sensitive to lexical and syntactic variations in the realisation of logical statements.

## introduction logical reasoning @xcite @xcite is at the core of many downstream nlp tasks, such as dialogue and story generation @xcite ; narrative understanding and summarisation @xcite ; question answering @xcite ; relation extraction @xcite @xcite ; and visual comprehension @xcite @xcite . because most of the current approaches to these tasks rely on pre-trained language models (lms), it is essential to understand whether lms can perform logical reasoning. one way of verifying lms' reasoning abilities is using a natural language inference (nli) task @xcite @xcite @xcite . in nli, an lm is given a premise and a hypothesis, and its task is to predict the logical relation between the two. yet, lms typically learn to solve nli by using invalid heuristics, for example by extracting overlapping patterns between premises and hypotheses @xcite , or by using specific lexical items and sentence grammaticality as simplistic predictors of entailment @xcite . in this paper, we examine whether pre-trained lms rely solely on shallow heuristics, or whether they can use relevant reasoning abilities to make inferences. to do so, we develop a new nli task, analog, foot_0 that requires lms to encode different logical reasoning patterns and we probe the behaviour of four masked and autoregressive lms on this new dataset. using interpretability measures, we find that, as a side effect of learning word prediction, all lms under scrutiny have-to some extent-learned to encode information that is predictive of entailment relations. we analyse the behaviour of the best performing model, bert @xcite , across the various inference categories present in analog, finding that its reasoning abilities go beyond shallow heuristics and yield relatively consistent performance on deductive and analytical reasoning, as well as across reasoning domains (spatial and comparative) and logical connectives. nevertheless, the model's behaviour within connectives varies, pointing out its sensitivity to lexical and syntactic variations in the realisation of logical statements. logical connectives can give us insight into their reasoning capabilities, i.e., their ability to reach a conclusion from one or more statements. @xcite showed that bert @xcite achieves 10% higher accuracy than humans on tasks that involve conjunctions. however, it has also been shown that lms fail to encode the semantics of logical formulas @xcite and struggle to differentiate between conjunction and disjunction @xcite , particularly in instances where the operands are noun phrases @xcite , suggesting that the models find it difficult to understand the scope of the logical operator. it is also known that neural lms have difficulty understanding argument order @xcite , which is arguably a pre-requisite for any logical reasoning. @xcite and @xcite showed that roberta @xcite , in contrast to bert, performs well at encoding instructional texts that involve conditionals. good performance on conditionals in lms is surprising, since humans typically find reasoning about conditionals challenging due to the fact that it requires accommodating degrees of belief @xcite . finally, regarding universal quantification, which implicitly involves encoding a hidden conditional statement (e.g. ∀x.p (x) → q(x)), bert's performance has been shown to vary substantially @xcite . besides different logical connectives, some recent work has studied different types of reasoning domains. @xcite showed that models such as bert and roberta struggle to encode the semantics of comparative reasoning phrases. yet, @xcite showed that bert's performance is only 11% less than human performance on comparative reasoning tasks, and 10% less than human performance on spatial reasoning tasks. overall, there is a lot of variation in lms' abilities to interpret different aspects of logical reasoning. we suspect that low performance stems from the fact that lms are struggling to encode world knowledge, which is often required in nli and logic datasets @xcite @xcite @xcite , while high performance may be due to extracting overlapping heuristics @xcite , or to attending to shallow predictors such as the presence of specific words or sentence grammaticality @xcite . we control for these factors in analog. ## dataset design and construction we extend the laknli dataset @xcite and present analog, an nli dataset that explicitly targets different types of logical reasoning. the dataset contains a total of 24,000 items (see ## results across models all four lms achieve positive average bpp scores: the average accuracy is above baseline by ca. 20 percentage points (see figure 2 ). these overall results indicate that the lms encode information that is predictive of entailment relations above and beyond simple heuristics which can be captured by a baseline. we also observe that the highest bpp scores are obtained at a relatively small training split size. this suggests training probes on more data can decrease their ability to extract the targeted linguistic features, and cause them to overfit on the dataset instead. bert and structbert are the best performing models with bpp scores ranging roughly between 15 and 40 (except for the smallest training split sizes). their similar performance across all splits shows that structbert's explicit modelling of sentence and discourse structure does not produce more informative representations for our analog task than bert's simpler next word and next sentence prediction training objectives. gpt-2's high standard deviation across splits (on average, 20.82) indicates a severe instability in its capacity to correctly encode logical reasoning cues. a closer look at gpt-2's performance shows 7 we would have seen an accuracy greater than 50% for random if the complexity of the classifier had been excessive. lastly, luke's performance, with an average score of 15.05, is significantly lower than that of the other three models (t-tests against bert, struct-bert and gpt-2 yield p-values approaching zero), suggesting that its ability to track entities does not significantly help in solving logical deductions. for the detailed results presented in the next sections, we focus on the model that achieves the highest bpp score with the lowest standard deviation. as can be seen in figure 2 , this model is bert, probed with a classifier trained on 12.5% of the full training split. 6 detailed results with bert 6.1 solving inference without heuristics we start by analysing the extent to which the performance of the best model, bert, may be the result of exploiting heuristics unrelated to logical reasoning. overlap if lexical overlap were used as a heuristic to predict entailment, we would expect lower performance for overlap-non-entailment o and no-overlap-entailment @xmath0, where using the overlap heuristic yields incorrect predictions. this is not the pattern we observe. as shown in figure 3a , accuracy is highest in these two cases. we see that o items yield the lowest bpp scores and @xmath1. however, there is no significant difference between no-overlap items with entailment vs. non-entailment labels. this indicates a lexical overlap heuristic is not prominently at play. as pointed out in section 3.2, the overlap vs. non-overlap distinction also corresponds to the contrast between direct deduction and analytical reasoning. we do not observe any significant differences in performance across these two reasoning types. more generally, the fact that bpp scores are positive across the board for overlapping and nonoverlapping cases shows that the model is solving our logic-based nli task by using information that goes beyond simple heuristic cues. grammaticality if a model were to judge entailment relations purely on the basis of grammaticality, we would expect it to wrongly predict entailment for o g (overlap-non-entailment grammatical) instances and correctly predict non-entailment for o u g (overlap-non-entailment ungrammatical). this is not what we observe: bpp scores are positive and not significantly different between o g and o u g , which indicates grammaticality is not being used as a heuristic to predict entailment. finally, we find that performance on ungrammatical sentences is more unstable (standard deviation is almost 8 times higher than for o g ); this may be due to bert producing noisier representations for out of distribution, partially ungrammatical, strings. ## conclusions we present a new nli dataset, analog, designed to test lms' abilities to deductively and analytically reason. we choose diagnostic probing as an interpretability technique, and probe using analog to inspect whether lms acquire such logical reasoning abilities from text-based pre-training. we find that masked lms, in particular bert and struct-bert, can solve the inference task through encoding properties of both deductive and analytic logic, rather than solely relying on shallow heuristics such as lexical overlap and sentence grammaticality. one main benefit of analog is that it isolates different reasoning types, domains, and logical connectives, in order to gain a better understanding of which of these factors makes inference more challenging for an lm. we choose high frequency lexical items to ensure that the lms' representations are as stable as possible, and not thrown off by surprising low frequency occurrences. we also use a fine-grained probing setup consisting of different training splits and multiple baselines to ensure that probes are using relevant linguistic and logical information, rather than learning the dataset artifacts, to solve the task. we perform an in-depth analysis of bert's behaviour. its overall stable performance is promising, though our case studies show some variance at the level of different natural language formulations of the same logical connective or their arguments as opposed to at higher reasoning levels. overall, we think that bert learns to encode approximations of the types of logical reasoning information necessary to solve analog, although its sensitivity to surface forms can make these approximations inconsistent. while extending the analog test set to also include lower frequency items may be helpful to ensure generalizability over noun and predicate relations (which we leave for future work), we hope that as it currently stands, analog can be used as a benchmark to check whether lms reason correctly by using elementary linguistic knowledge and logical semantics, as opposed to surface heuristics.