It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation (2024)

Abstract: NLI tasks necessitate a substantial degree of logical reasoning; as such, the remarkable performance of SoTA transformers on these tasks may lead us to believe that those models have learned to reason logically. The results presented in this paper demonstrate that (i) models fine-tuned on NLI datasets learn to treat external negation as a distractor, effectively ignoring its presence in hypothesis sentences; (ii) several near-SoTA encoder and encoder-decoder transformer models fail to inductively learn the law of the excluded middle for a single external negation prefix with respect to NLI tasks, despite extensive fine-tuning; (iii) those models which are are able to learn the law of the excluded middle for a single prefix are unable to generalize this pattern to similar prefixes. Given the critical role of negation in logical reasoning, we may conclude from these findings that transformers do not learn to reason logically when fine-tuned for NLI tasks. Furthermore, these results suggest that transformers may not be able to inductively learn the role of negation with respect to NLI tasks, calling into question their capacity to fully acquire logical reasoning abilities.

## introduction natural language inference (nli) tasks require detecting inferential relations between pairs of sentences @xcite . for nli datasets such as @xcite and stanford nli (snli; @xcite , the task proceeds as follows: given a pair of sentences (p, h), an nli model must determine whether the premise p entails the hypothesis h, h contradicts p , or p and h are neutral with respect to one another (i.e. p does not entail h and h does not contradict p ). nli tasks require logical reasoning capabilities that extend beyond basic linguistic competence @xcite . for example, under-standing that "jane is travelling to algeria" entails "jane is travelling to africa" requires mereological world knowledge @xcite ; an agent must know that algeria is contained within africa. to understand that "jane is traveling to algeria" does not entail "jane is traveling to algiers", the agent must understand that algiers is contained within algeria, but that algeria is not solely comprised of the city of algiers. because of the considerable amount of reasoning that is required to accomplish nli tasks, it is important to scrutinize the degree to which current nli models are actually learning to reason logically. @xcite findings suggest, for example, that even (then-)sota nli models such as bert @xcite adopt shallow, textual heuristics to achieve high-scoring results on the mnli dataset, although the mnli dataset itself is likely to be-at least partially-at fault (possibly because the provided training data is not sufficiently representative of the task; see the discussion in section 2). this paper investigates sota transformer @xcite nli models' ability to inductively learn the law of the excluded middle (lem) with respect to external negation (negation that occurs externally to the proposition that is negated, e.g. "it is not true that apples are red"), in order to evaluate the degree to which they have learned to reason logically when performing nli tasks. using external negation, it is possible to automatically construct augmented challenge examples from the mnli and snli datasets that modify the original examples' class labels in a predictable manner: given a premise, hypothesis, label triple (p, h, l), we generate an augmented example (p, ¬h, l ′ ), where l ′ = neutral if @xmath0, l ′ = contradiction if @xmath1, and l ′ = entailment if @xmath2. experiments 1 and 2 evaluate the nli models' inductive learning capacity along two respective axes: experiment 1 (section 4) examines these models' ability to generalize double-negation cancellation to chains of repeated external negation prefixes longer than those seen during inoculation (∼fine-tuning; see section 2 for an in-depth description), with respect to a single prefix string. we observe that nli models struggle to learn this pattern inductively, with many unable to learn it at all. experiment 2 (section 5) evaluates the ability of those nli models which were successfully able to learn lem for a single external negation prefix to generalize this pattern to prefix strings not seen during fine-tuning. we find that those inoculated models suffer drastic decreases in performance when presented with unseen prefixes; the results of experiment 3 (section 6) indicate that this is due to catastrophic forgetting of the similarity between the prefix that they were inoculated against and other, highly similar prefixes. the experimental results contained in this paper foot_0 indicate that transformer models do not learn to reason logically when fine-tuned on nli datasets, lending further support to @xcite hypothesis that they are instead learning to leverage shallow heuristics. in section 3, we find evidence (theorem 1) that this failure of transformer models to inductively learn lem arises from deficiencies in their training procedure and/or the structure (or lack thereof) of their input data, rather than flaws inherent to transformer architectures themselves (see the discussion in section 7). ## related work there is a large body of existing work on probing nli models to gain insight into their reasoning abilities @xcite . as mentioned in section 1, @xcite find that language models fine-tuned on mnli learn to leverage shallow heurisics to achieve exceptionally high accuracy on this dataset. similarly, @xcite and @xcite probe nli models' performance with respect to specific syntactic and semantic phenomena (e.g. coordination, quantification, monotonicity, etc.). they find that sota models fine-tuned on mnli and snli perform poorly on challenge examples generated to evaluate the models with respect to these phenomena, but can be easily fine-tuned to master the challenge data, while retaining their high performance on the original datasets. inoculation by fine-tuning in all three of the aforementioned papers, their respective authors utilize the method of inoculation by fine-tuning. @xcite introduce this paradigm as a technique for differentiating between deficiencies in a model's training data and deficiencies in the model itself. inoculation by fine-tuning assumes that there is an original dataset (divided into train and test splits) and a smaller challenge dataset (also divided into train and test splits), and that the model's performance on the challenge dataset is significantly lower than on the original dataset. the idea is to fine-tune the model on the challenge dataset until validation performance on the original test set has not improved for five epochs, then measure the newly fine-tuned (inoculated) model on the challenge test set. if the inoculated model maintains its performance on the original test set and performs (nearly) as well on the challenge test set, this suggests that the model's poor performance on the challenge data was due to flaws (e.g. a lack of diversity) in the original training data. conversely, if the model's performance on the challenge test set remains significantly worse than on the original data after inoculation, this suggests that its poor performance on the challenge data is due to a deficiency in the model itself. this paper probes various nli models' logical reasoning abilities-in particular with respect to external negation-using automatically-generated challenge data along with the inoculation by finetuning paradigm. unlike the closely-related notion of adversarial attacks, which seek to perturb input examples without altering their class labels, the external negation prefixes used to generate challenge examples in experiments 1-3 (sections 4, 5, 6) do alter the original examples' class labels, albeit in a predictable manner. this is similar to the challenge data that @xcite construct from the argument reasoning comprehension task @xcite ; these authors find that bert cannot be inoculated against such challenge data, and conclude that transformer models' inability to ground text to real-world concepts presents an insurmountable barrier to their logicalreasoning abilities. probing lms with negation in a similar vein, @xcite conduct "stress tests" on nli models by concatenating logical distractor strings (e.g. "and false is not true") to the input examples, and find that such distractors drastically reduce sota nli models' performance on these tasks. while these authors investigate nli models' performance with respect to logical reasoning, their experiments regarding negation are limited to negation items appearing in these distractor terms, rather than negating the original hypothesis sentence itself. on the other hand, hossain et al. (2020) probe nli models (and datasets) by negating the original premise and/or hypothesis sentence(s) using an automatic dependency parser; these automaticallygenerated challenge examples are then checked for accuracy and re-assigned class labels by human annotators. these authors find that models finetuned on the original nli datasets perform poorly on development sets consisting of these negationaugmented examples, and that while fine-tuning on the challenge data improves performance on negation-augmented test splits derived from snli, fine-tuning does not significantly increase model accuracy on mnli-derived examples. note that, unlike the present work, hossain et al. ( 2020 ) do not study repeated/embedded negation or doublenegation cancellation. yuan et al. (2023) examine pretrained language models' (plms) deductive reasoning abilities via cloze tests. these authors find that plms are unable to fully generalize rules of logical deduction to arbitrary contexts. furthermore, they observe that these models struggle to differentiate between positive statements and their negated counterparts, in line with a wide body of recent literature suggesting that transformers have difficulty processing and comprehending negation (e.g. @xcite @xcite @xcite @xcite . of particular interest to this work, they find that while inoculating plms for deductive reasoning tasks improves performance, it results in catastrophic forgetting of previous knowledge. likewise, in sections 5 and 6 of this paper, we find that inoculating pretrained nli models against challenge data augmented with external negation prefixes causes catastrophic forgetting of prior knowledge of their similarity to related prefixes. @xcite evaluate the consistency of language models across various axes. of particular interest to the current discussion is their analysis of negational consistency: the degree to which a given language model's predictions differ between texts having opposite meanings. these authors find that negational consistency remains low across a variety of models and tasks-in particular, roberta @xcite and bart @xcite ) exhibit low negational consistency on the snli dataset. in an experiment highly related to the present work, laverghetta jr. and licato (2022) probe nli models' performance with respect to negation, and find that the models struggle to contend with certain types of negation more so than others. in line with the results we observe in section 4, they find that the models have difficulty inoculating against those problematic negation categories. unlike the experiments in this paper, laverghetta jr. and licato (2022) do not construct challenge examples involving negation, but rather use examples drawn from nli datasets that already contain negation. contributions unique to this work is the evaluation of transformers' ability to learn the law of the excluded middle (lem) and our finding that, while many cannot learn this pattern, a few transformer nli models are in fact able to inductively learn lem for a single external negation prefix. additionally, the results of experiments 2 and 3 (sections 5 and 6), extend yuan et al.'s (2023) results (regarding catastrophic forgetting resulting from inoculation in the context of deductive reasoning tasks) to double negation-cancellation in the setting of nli tasks. finally, theorem 1 (see section 3) is the first known proof that there exists (at least, in principle) an encoder transformer capable of modeling lem for arbitrary-length sequences of any combination of external negation prefixes with respect to any nli dataset. this theorem sheds further light on evidence in the literature @xcite @xcite @xcite @xcite indicating that transformers struggle to model negation, suggesting that this observed failure is not due to an inherent flaw in transformer architectures themselves, but instead may be due to deficiencies in their training procedure and/or the structure of their input data (see the discussion in section 7). ## can transformers model lem? before evaluating nli models' ability to inductively learn the law of the excluded middle (lem), we first establish whether-learnability aside-it is theoretically possible for transformer architectures to model lem at all: theorem 1 proves that (encoder) transformer architectures are in fact capable of modeling lem (at least, with respect to nli tasks) for arbitrary-length sequences of any combination of external negation prefixes. note that the nli datasets, transformer models (with the exception of bart), and set of external negation prefixes used in experiments 1-3 satisfy the assumptions of theorem 1. theorem 1. let @xmath3, h i , l i )} @xmath4, and for any nli model m , let acc(m, d) denote the classification accuracy of m on d. let σ ′ be a finite alphabet such that d ⊂ (σ ′ ) * × (σ ′ ) * × λ (where λ = {e, n , c} denotes the set of labels). let n ⊂ (σ ′ ) * be any finite-cardinality set of external-negation prefixes such that no prefix is a substring of one or more other prefixes foot_1 . then there exists an alphabet σ ⊃ σ ′ and an injective f : (σ ′ ) * → σ * such that for any fixed (finite) @xmath5, there exists a fixed-precision transformer encoder t ′ such that t ′ matches the accuracy of t on d and on any dataset d ′ formed by prefixing any η ∈ n * to each hypothesis sentence in d foot_2 . proof. appendix a. however, the proof of theorem 1 relies on a function f that re-structures the input data; the transformer nli models evaluated in experiments 1-3 (sections 4, 5, 6) are obviously not equipped with such a function, and the ability of transformers to model lem with respect to unstructured plain text is not established in theorem 1. furthermore, theorem 1 merely states that there exists an encoder transformer capable of modeling lem for external negation with respect to nli tasks, and makes no claim regarding its architectural configuration (i.e. layer size, floating-point precision, etc.). it is unclear whether the transformer models evaluated in the present work have the specific architecture required to accomplish this task. critically, the proof of theorem 1 does not make any claims regarding the (inductive) learnability of lem; while it is theoretically possible to model lem with an encoder transformer, these language models' ability to inductively learn lem remains uncertain from the conclusions of theorem 1 alone. in the following section (experiment 1), we observe experimental evidence demonstrating that some transformer nli models (in particular, roberta) are able to learn lem for a single external negation prefix. ## experiment 1 experiment 1 probes six different transformer nli models' ability to inductively learn the law of the excluded middle (lem) with respect to external negation. the deberta @xcite ) model, denoted debert a s foot_3 , is deberta-large fine-tuned on snli. the first bart model, denoted bart m foot_4 , is bart-large fine-tuned on mnli, while the second, bart sm f a foot_5 , is bartlarge fine-tuned on mnli, snli, fever @xcite , and anli @xcite . the first roberta model, robert a m foot_6 , is roberta-large fine-tuned on mnli, and the second, robert a s foot_7 , is roberta-large fine-tuned on snli, while the third, robert a sm f a foot_8 , is roberta-large fine-tuned on snli, mnli, fever, and anli. ## experiment 2 this experiment restricts its analysis to the three roberta models, as they were the only models of the six evaluated in experiment 1 (section 4) that were able to fully generalize depth-m negationcancellation to arbitrary values of @xmath6. ## experiment 3 as in experiment 2 (section 5), this experiment restricts its analysis to the three roberta models. ## discussion the results of experiments 1-3 (sections 4, 5, 6) raise the question as to why these models are unable to inductively learn the law of the excluded middle (lem)-especially in light of theorem 1, which states that (in theory) transformers are able to model lem with respect to nli tasks. a reasonable explanation for this seemingly paradoxical state of affairs can be found within the conclusions of theorem 1 itself. note that the proof of theorem 1 relies on a function f that re-structures the input data (as mentioned in section 3); it is possible that the structure (or lack thereof) of purely textual data may be insufficient for transformers to inductively learn to model lem. additionally, recall that the proof of theorem 1 does not establish the (inductive) learnability of lem; it may be the case that the specific parameter values required to model the role of (external) negation in the context of nli tasks cannot be reached by training on any nli dataset using gradient descent or any other currently known training procedures. it may also be the case that the function of (external) negation is in fact learnable, but only via the brute-force approach of training these models on multiple-depth external negation for every such prefix. in other words, (encoder) transformers may not be capable of inductively learning lem-at least not with standard training procedures. @xcite and @xcite propose training procedures designed to enhance language models' ability to learn the role of negation, which may provide fruitful avenues for improving transformer nli models' performance on the tasks laid out in experiments 1-3 (sections 4, 5, 6). @xcite introduce unlikelihood with reference training for masked language models, which penalizes models for predicting unlikely tokens in negated contexts-for example, a model would be penalized for predicting fly in the context "birds cannot [mask]". after unlikelihood with reference training, the authors record marginal improvement (∼1-2%) for bert on negation-augmented snli and mnli datasets (see @xcite . asai and hajishirzi (2020) use logic-based regularization and data augmentation to improve language models' transitive and symmetric consistency (c.f. jang et al., 2022)-in particular, negation is subsumed under their notion of symmet-ric consistency. using this approach, the authors record marked improvement over the sota on a variety of question-answering tasks, although they do not evaluate this regularization method on any nli datasets. however, @xcite and asai and hajishirzi (2020) do not explicitly study the efficacy of their respective training methods with respect to double-negation cancellation. therefore, it is unclear whether the improvements obtained by their approaches would translate to a task such as lem, and we leave an evaluation thereof to future work. ## conclusion the results of experiments 1-3 (sections 4, 5, 6) demonstrate that near-sota transformer nli models struggle to inductively learn the law of the excluded middle (lem). furthermore, the results of experiment 1 (section 4) strongly suggest that all six nli models studied in this work learned to treat the external negation prefix "it is not true that" as a distractor when initially fine-tuned on the nli dataset(s) (see these findings lend further support to a large body of existing evidence (e.g. @xcite @xcite @xcite @xcite indicating that transformers are unable to model the meaning of negation. unique to this work is our finding that certain encoder transformers (in particular, roberta) can learn lem for a single external negation prefix. while the three roberta models did manage to grasp the function of the prefix "it is not true that", the process of learning this behavior resulted in catastrophic forgetting, entirely inhibiting their generalization of this pattern to the highly similar prefix "it is false that" (see sections 5 and 6). theorem 1 proves that encoder transformers are-in principle-capable of modeling lem for arbitrary-length sequences of any combination of external negation prefixes with respect to any nli dataset. this suggests that these models' inability to inductively learn lem might not be a consequence of their transformer architectures, but rather may result from the (lack of) structure of their input data and/or the procedure used to train them.