Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning (2022)

Abstract: Neural models command state-of-the-art performance across NLP tasks, including ones involving “reasoning”. Models claiming to reason about the evidence presented to them should attend to the correct parts of the input while avoiding spurious patterns therein, be self-consistent in their predictions across inputs, and be immune to biases derived from their pre-training in a nuanced, context- sensitive fashion. Do the prevalent *BERT- family of models do so? In this paper, we study this question using the problem of reasoning on tabular data. Tabular inputs are especially well-suited for the study—they admit systematic probes targeting the properties listed above. Our experiments demonstrate that a RoBERTa-based model, representative of the current state-of-the-art, fails at reasoning on the following counts: it (a) ignores relevant parts of the evidence, (b) is over- sensitive to annotation artifacts, and (c) relies on the knowledge encoded in the pre-trained language model rather than the evidence presented in its tabular inputs. Finally, through inoculation experiments, we show that fine- tuning the model on perturbed data does not help it overcome the above challenges.

## introduction the problem of understanding tabular or semistructured data is a challenge for modern nlp. recently, @xcite and @xcite have framed this problem as a natural language inference question (nli, @xcite via the tabfact and the infotabs datasets, respectively. the tabular version of the nli task seeks to determine whether a tabular premise entails or contradicts a textual hypothesis, or is unrelated to it. one strategy for such tabular reasoning tasks relies on the successes of contextualized representations (e.g., @xcite for the sentential version of the problem. tables are flattened into artificial sentences using heuristics to be processed by these models. surprisingly, even this naïve strategy leads to high predictive accuracy, as shown not only by the introductory papers but also by related lines of recent work (e.g., @xcite . in this paper, we ask: do these seemingly accurate models for tabular inference effectively use and reason about their semi-structured inputs? while ''reasoning'' can take varied forms, a model that claims to do so should at least ground its outputs on the evidence provided in its inputs. concretely, we argue that such a model should (a) be self-consistent in its predictions across controlled variants of the input, (b) use the evidence presented to it, and the right parts thereof, and, (c) avoid being biased against the given evidence by knowledge encoded in the pre-trained embeddings. corresponding to these three properties, we identify three dimensions on which to evaluate a tabular nli system: robustness to annotation artifacts, relevant evidence selection, and robustness to counterfactual changes. we design systematic probes that exploit the semi-structured nature of the premises. this allows us to semi-automatically construct the probes and to unambiguously define the corresponding expected model response. these probes either introduce controlled edits to the premise or the hypothesis, or to both, thereby also creating counterfactual examples. experiments reveal that despite seemingly high test set accuracy, a model based on roberta @xcite , a good representative of bert derivative models, is far from being reliable. not only does it ignore relevant evidence from its inputs, it also relies excessively on annotation artifacts, in particular the sentence structure of the hypothesis, and pre-trained knowledge in the embeddings. finally, we found that attempts to inoculate the model @xcite along these dimensions degrades its overall performance. the rest of the paper is structured as follows. §2 introduces the tabular nli task, while §3 articulates the need for probing evidence-based tabular reasoning in extant high-performing models. §4- §6 detail the probes designed for such an examination and the results thereof while §7 analyzes the impact of inoculation to aforementioned challenges through model fine-tuning. §8 presents the main takeaways and contextualization in the related art. §9 provides concluding remarks and indicates future directions of work. 1 2 preliminaries: tabular nli tabular natural language inference is a task similar to standard nli in that it examines if a natural language hypothesis can be derived from the given premise. unlike standard nli, where the evidence is presented in the form of sentences, the premises in tabular nli are semi-structured tables that may contain both text and data. dataset recently, datasets such as tabfact @xcite and infotabs @xcite , and also shared tasks such as semeval 2021 task 9 @xcite and fever-ous @xcite , have sparked interest in tabular nli research. in this study, we use the infotabs dataset for our investigations. infotabs consists of 23,738 premisehypothesis pairs, whose premises are based on wikipedia infoboxes. unlike tabfact, which only contains entail and contradict hypotheses, infotabs also includes neutral ones. figure 1 shows an example table from the dataset with four hypotheses, which will be our running example. the dataset contains 2,540 distinct infoboxes representing a variety of domains. all hypotheses were written and labeled by amazon mturk workers. the tables contain a title and two columns, as shown in the example. since each row takes the form of a key-value pair, we will refer to the 1 the dataset and the scripts used for our analysis are available at https://tabprobe.github.io . elements in the left column as the keys, and the right column provides the corresponding values. in addition to the usual train and development sets, infotabs includes three test sets, α 1 , α 2 , and α 3 . the α 1 set represents a standard test set that is both topically and lexically similar to the training data. in the α 2 set, hypotheses are designed to be lexically adversarial, and the α 3 tables are drawn from topics not present in the training set. we will use all three test sets for our analysis. models over tabular premises unlike standard nli, which can use off-the-shelf pre-trained contextualized embeddings, the semi-structured nature of premises in tabular nli necessitates a different modeling approach. following @xcite , tabular premises are flattened into token sequences that fit the input interface of such models. while different flattening strategies exist in the literature, we adopt the table as a paragraph strategy of @xcite , where each row is converted to a sentence of the form ''the key of title is value''. this seemingly naïve strategy, with roberta-large embeddings (roberta l henceforth), achieved the highest accuracy in the original work, shown in ## reasoning: an illusion? given the surprisingly high accuracies in in other words, ''reasoning failures'' can be identified by checking if a model deviates from expected behavior in response to controlled changes to inputs. we note that this strategy has been either explicitly or implicitly employed in several lines of recent work @xcite . in this work, we instantiate the above strategy along three specific dimensions, briefly introduced here using the running example in figure 1 . each dimension is used to define several concrete probes that subsequent sections detail. ## probing annotation artifacts can a model make inference about a hypothesis without a premise? it is natural to answer in the negative in general. (of course, certain hypotheses may admit strong priors, e.g., tautologies.) preliminary experiments by @xcite on infotabs, however, reveal that a model trained just on hypotheses performs surprisingly well on the test data. this phenomenon, an inductive bias entirely predicated on the hypotheses, is called hypothesis bias. models for other nli tasks have been similarly shown to exhibit hypothesis bias, whereby the models learn to rely on spurious correlations between patterns in the hypotheses and corresponding labels @xcite @xcite . for example, negations are observed to be highly correlated with contradictions @xcite . to better characterize a model's reliance on such artifacts, we perform controlled edits to hypotheses without altering associated premises. unlike the α 2 set, which includes minor changes to function words, we aim to create more sophisticated changes by altering content expressions or noun phrases in a hypothesis. two possible scenarios arise where a hypothesis alteration, without a change in the premise, either (a) leads to a change in the label (i.e., the label covaries with the variation in the hypothesis), or (b) does not induce a label change (i.e., the label is invariant to the variation in the hypothesis). in infotabs, a set of reasoning categories are identified to characterize the relationship between a tabular premise and a hypothesis. we use a subset of these, listed below, to perform controlled changes in the hypotheses. although we can easily track these expressions in a hypothesis using tools like entity recognizers and parsers, it is non-trivial to automatically modify them with a predictable change on the hypothesis label. for example, some label changes can only be controlled if the target expression in the hypothesis is correctly aligned with the facts in the premise. such cases include contradict to entail, and neutral to contradict or entail, which are difficult without extensive expressionlevel annotations. nonetheless, in several cases, label changes can be deterministically known even with imprecise changes in the hypothesis. for example, we can convert a hypothesis from entail to contradict by replacing a named entity in the hypothesis with a random entity of the same type. hence we follow the following strategy: (a) we avoid perturbations involving the neutral label altogether, as they often need changes in the premise (table) as well. (b) we generate all labelpreserving and some label-flipping transformations automatically using the approach described below. (c) we annotate the contradict to entail label-flipping perturbations manually. ## probing evidence selection predictions of an nli model should primarily be based on the evidence in the premise, that is, on the facts relevant to the hypothesis. for a tabular premise, rows containing the evidence necessary to infer the associated hypothesis are called relevant rows. short-circuiting the evidence in relevant rows for inference using annotation artifacts as suggested in §4 or other spurious artifacts in irrelevant rows of the table is expected to lead to poor generalization over unseen data. to better understand the model's ability to select evidence in the premise, we use two kinds of controlled edits: (a) automatic edits without any information about relevant rows, and (b) semiautomatic edits using knowledge of relevant rows via manual annotation. the rest of the section goes over both scenarios in detail. all experiments in this section use the full model that is trained on both premises and their associated hypotheses. ## probing with counterfactual examples since infotabs is a dataset of facts based on wikipedia, pre-trained language models such as roberta, trained on wikipedia and other publicly available text, may have already encountered information in infotabs during pre-training. as a result, nli models built on top of roberta l can learn to infer a hypothesis using the knowledge of the pre-trained language model. more specifically, the model may be relying on ''confirmation bias'', in which it selects evidence/patterns from both premise and hypothesis that matches its prior knowledge. while world knowledge is necessary for table nli @xcite , models should still treat the premise as the primary evidence. counterfactual examples can help test whether the model is grounding its inference on the evidence provided in the tabular premise. in such examples, the tabular premise is modified such that the content does not reflect the real world. in this study, we limit ourselves to modifying only the entail and contradict examples. we omit the neutral cases because the majority of them in infotabs involve out-of-table information; producing counterfactuals for them is much harder and involves the laborious creation of new rows with the right information. the task of creating counterfactual tables presents two challenges. first, the modified tables should not be self-contradictory. second, we need to determine the labels of the associatedx hypotheses after the table is modified. we employ a simple approach to generate counterfactuals that addresses both challenges. we use the evidence selection data ( §5.2) to gather all premise-hypothesis pairs that share relevant keys such as ''born'', ''occupation'', and so forth. counterfactual tables are generated by swapping the values of relevant keys from one table to another. 10 figure 5 shows an example. we create counterfactuals from the premises in figure 1 and figure 2 by swapping their length rows. we also swap the hypotheses (h1 and h5) aligned to the length rows in both premises by replacing the title expression bridesmaids in h5 with breakfast in america and vice versa. the simple procedure en-10 there may still be a few cases of self-contradiction, but we expect that such invalid cases would not exist in the rows that are relevant to the hypothesis. sures that the hypotheses labels are left unchanged in the process, resulting in high-quality data. in addition, we also generated counterfactuals by swapping the table title and associated expressions in the hypotheses with the title of another table, resulting in a counterfactual table-hypothesis pair, as in the row swapping strategy. figure 6 shows an example created from the premises in figure 1 and figure 2 by swapping the title rows breakfast in america and bridesmaids. the title expression in all hypotheses in figure 1 are also replaced by bridesmaids. this strategy also preserves the hypothesis label similar to row swapping. the above approaches are label preserving as they do not alter the entailment labels. counterfactual pairs with flipped labels are important for filtering out the contribution of artifacts or other spurious correlations that originate from a hypothesis (see §4). so, in addition, we also created counterfactual table-hypothesis pairs where the original labels are flipped. these counterfactual cases are, however, non-trivial to generate automatically, and are therefore created manually. to create the label-flipped counterfactual data, three annotators manually modified tables from to disentangle these two factors, we can take advantage of the fact that the counterfactual examples are constructed from, and hence paired the results, taken in aggregate, suggest that the model produces predictions based on hypothesis artifacts and pre-trained knowledge rather than the evidence presented to it, thus impacting its robustness and generalization. ## inoculation by fine-tuning our probing experiments demonstrate that the models, trained on the infotabs training set, failed along all three dimensions that we investigated. this leads us to the following question: can additional fine-tuning with perturbed examples help? @xcite point out that poor performance on challenging datasets can be ascribed to either a weakness in the model, a lack of diversity in the dataset used for training, or information leakage in the form of artifacts. 11 they suggest that models can be further fine-tuned on a few challenging examples to determine the possible source of degradation. inoculation can lead to one of three outcomes: (a) outcome 1: the performance gap between the challenge and the original test sets reduces, possibly due to addition of diverse examples, (b) outcome 2: performance on both the test sets remains unchanged, possibly because of the model's inability to adapt to the new phenomena or the changed data distribution, or, (c) outcome 3: performance degrades on the test set, but improves on the challenge set, suggesting that adding new examples introduces ambiguity or contradictions. we conducted two sets of inoculation experiments to help categorize performance degradation of our models into one of these three categories. for each experiment described below, we generated additional inoculation datasets with 100, 200, and 300 examples to inoculate the original task-specific roberta l models trained on both premises and hypotheses. as in the original inoculation work, we created these adversarial datasets by sub-sampling inclusively, i.e., the smaller datasets are subsets of the larger ones. following the training protocol in @xcite , we tried learning rates of 10 -4 , 5 × 10 -5 , and 10 -5 . we performed inoculation for a maximum of 30 epochs with early stopping based on the development set accuracy. we found that with the first two learning rates, the model does not converge, and underperforms on the development set. the model performance was best with the learning rate of 10 -5 , which we used throughout the inoculation experiments. the standard deviation 11 model weakness is the inherent inability of a model (or a model family) to handle certain linguistic phenomena. ## conclusion this paper presented a targeted probing study to highlight the limitations of tabular inference models using a case study on a tabular nli task on infotabs. our findings show that despite good performance on standard splits, a roberta-based tabular nli model, fine-tuned on the existing pre-trained language model, fails to select the correct evidence, makes incorrect predictions on adversarial hypotheses, and is not grounded in provided evidence-counterfactual or otherwise. we expect that insights from the study can help in designing rationale selection techniques based on structural constraints for tabular inference and other tasks. while inoculation experiments showed partial success, diverse data augmentation may help mitigate challenges. however, annotation of such data can be expensive. it may also be possible to train models to satisfy domain-based constraints (e.g., @xcite to improve model robustness. finally, probing techniques described here may be adapted to other nlp tasks involving tables such as tabular question answering and tabular text generation. counterfactual bias evaluation. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 3899-3916, online. association for computational linguistics. https://doi.org /10.18653/v1/2021.naacl-main.305 appendix in section 5.1, we defined four types of rowagnostic table modifications:(a) row deletion, (b) row insertion, (c) row-value update, and (d) row permutation and presented the first one there. we present details of the rest here along with the respective impact on the α 1 , α 2 , and α 3 test sets.