Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals (2023)

Abstract: We present a human-in-the-loop dashboard tailored to diagnosing potential spurious features that NLI models rely on for predictions. The dashboard enables users to generate diverse and challenging examples by drawing inspiration from GPT-3 suggestions. Additionally, users can receive feedback from a trained NLI model on how challenging the newly created example is and make refinements based on the feedback. Through our investigation, we discover several categories of spurious correlations that impact the reasoning of NLI models, which we group into three categories: Semantic Relevance, Logical Fallacies, and Bias. Based on our findings, we identify and describe various research opportunities, including diversifying training data and assessing NLI models’ robustness by creating adversarial test suites.

## introduction the availability of crowdsourced large-scale datasets has been influential in the field of natural language processing. these datasets have empowered advancements in a wide range of downstream tasks, including the natural language inference (nli) task (snli; @xcite . while being influential, crowdsourcing frameworks can introduce artifacts, biases, and spurious correlations that can negatively impact the robustness and out-of-domain generalization of the models that are trained on such datasets @xcite . a spurious correlation exists when a feature correlates with the target label while there is no causal relationship between the feature and the label. for example, the fact that a sentence includes the word "amazing" (as a feature) might correlate with a positive sentiment but does not cause the sentiment label to be positive, as one can imagine crafting a sentence like "the amazing concert was ruined by the terrible acoustics in the venue", which has a negative sentiment. it has been shown that such spurious correlations exist in crowdsourced datasets @xcite , and this will prevent models that are trained on these datasets from performing on adversarial or out-of-domain test sets @xcite . one approach to prevent a model from relying on spurious correlations between a feature and the label is to break such correlations by providing counterfactuals during training. in this context, counterfactuals are data points that contain the feature but have a different label. following our previous example, "the amazing concert was ruined by the terrible acoustics in the venue" is a counterfactual sentence since it contains the word "amazing" but has a negative sentiment. augmenting datasets with counterfactuals can break the spurious correlations and help the model to generalize to out-of-domain examples. however, generating counterfactuals is challenging; it involves first identifying noncausal features that correlate with the label, i.e., spurious correlations, and then generating the counterfactuals for a given feature. one simple approach to generate counterfactuals is through minimal edits. in this approach, the first step-identifying spurious correlations-is bypassed. therefore, counterfactuals are generated without targeting any specific feature. to generate a counterfactual, existing data points in the dataset are edited minimally such that they have a different label compared to their original one. while such an approach is scalable and can be effective in certain scenarios @xcite , creating counterfactuals through minimal edits does not necessarily improve the generalization of models and might even hurt the performance @xcite . therefore, there is a need for a more nuanced and innovative approach to counterfactual generation. in this paper, we propose a data-centric approach to counterfactual generation. first, we identify existing counterfactuals in datasets, which we term data-constrained counterfactuals (dcc). second, using our interactive dashboard, we diagnose features that spuriously correlate with their label by comparing and contrasting the dcc with other data points in the dataset. lastly, we generate a diverse set of counterfactual examples with the help of gpt-3 (davinci-003; @xcite . overall, our dashboard offers a human-inthe-loop, or more generally, a mixed-initiative approach. a user can diagnose spurious correlations and common patterns that result in nli models' inability to predict the labels correctly. finding such weak spots can provide ways to improve the nli model. furthermore, after the user has generated a set of new counterfactuals, the nli model can give feedback on how valuable each counterfactual is by expressing its uncertainty in predicting the sample's annotated label. this can help the user to revise the counterfactual and improve the usefulness of the generated set. while our dashboard can be extended to various tasks in natural language processing, we focus on the nli task in this work. using our dashboard, we find a variety of features that correlate spuriously with labels. we categorize these features into three categories, which we name: semantic relevance, logical fallacies, and biases. we further find a category of samples that are annotation artifacts. based on these findings, and with the help of our dashboard, one can create novel counterfactuals to assess the robustness of nli models or use them to augment training sets. ## preliminaries before introducing our approach, we first go through some preliminaries. we briefly describe the nli task and a tool called data maps. ## data-constrained counterfactuals in this work, we propose to start with finding existing counterfactuals in datasets. we will later use these counterfactuals in our dashboard §4 to find spurious features and generate new data points. a data-constrained counterfactual is a data point that shares some features with other data points in the dataset but has a different label. further, we want to make sure that the model is sensitive to the spurious correlation. therefore, it should not be easy for the model to label a dcc correctly. we provide the following formal definition of dataconstrained counterfactuals. definition 1. a data point is a data-constrained counterfactual (dcc) when it satisfies two conditions: i) there exists other data points in the training set that are similar to this data point but have a different label, and ii) it is not easy for the model to label it correctly, i.e., it falls into either the hardto-learn or the ambiguous region in the data map. this definition relies on a notion of similarity; thus, to identify dccs we need to provide a similarity metric between data points. following @xcite , we define the similarity between data points as the cosine similarity between the [cls] embedding of data points given by the underlying model. this will give us a tractable measure to find similar data points in large datasets without any manual inspection of the data. a caveat to def. 1 is that many data points in the hard-to-learn region have been found to be mislabeled @xcite . to filter out samples that are likely to be mislabeled, we only select samples that have multiple annotations, where a large foot_0 majority of annotators agree on the label. ## visual interactive dashboard in this section, we describe the tasks that users can perform during the interactive counterfactual generation process. we categorize these tasks using the explainer framework @xcite . ## experimental setup the components of the dashboard are shown in fig. 1 . the filtering of potential dccs described in section §3 was performed on the snli dataset @xcite . 2 we compute the nearest neighbors of the dccs according to the cosine similarity between the [cls] embeddings extracted from a roberta-large model @xcite trained on snli. the data map was generated following @xcite , where six end-of-epoch checkpoints of the snli robertalarge model were used to estimate the data map location. suggestions are generated by few-shot prompting the gpt-3 davinci-003 model @xcite using four nearest neighbors to the dcc with the same label, exactly following @xcite , as they argue that the model may employ similar reasoning for such nearest neighbors. an example of such a prompt is shown and described in fig. 2 . ## findings by interacting with the dashboard ourselves in multiple sessions, we find interesting patterns and many dcc instances following those patterns. in this section, we provide a categorization of our findings. we find three high-level features that correlate spuriously with the label, which we name: semantic relevance, logical fallacies, and bias. furthermore, we discovered another category that surfaces artifacts in the data collection procedure. next, we will go through and explain each category, and further, provide some examples. ## discussion the visual interactive dashboard for diagnosing spurious correlations and counterfactual generation can open up research opportunities in the following domains: bi-directional explanation of reasoning patterns. our dashboard opens up a possibility for efficient collaboration between humans and ai. ai can help humans to find and group similar structures. as can be seen in our dashboard, similarities in the representation space of nli models often capture similar structures. on the other hand, humans can explain the reasoning to ai. this can happen by generating new examples that follow a particular line of reasoning that is challenging for the ai model to learn, which can result in improving ai models. ## related work other tools have been proposed for counterfactual generation. for example, polyjuice @xcite introduces an automated counterfactual generation based on minimal edits. counterfactuals are created using a fixed set of control codes to edit the existing sentences in the dataset. further, systems have been developed for mixedinitiative adversarial sample generation. anli @xcite introduces an adversarial sample generation framework, where annotators are tasked to write hypotheses that may fool the model for a given context (i.e., a premise and label). following a similar framework, dynabench @xcite presents a more general-purpose dashboard for adversarial generation using model predictions and explanations as feedback to the user. compared to the methods described above, our proposed approach aims to first diagnose potential spurious correlations through dccs, and then generate counterfactuals based on the found spurious correlations via prompting large language models. furthermore, our dashboard gives feedback to users during the refinement stage by providing them with data map estimates for newly generated counterfactuals. ## conclusion we present a dashboard to diagnose spurious correlations and artifacts that an nli model may have acquired during training. we first provide a systematic approach to find data-constrained counterfactuals, i.e., existing counterfactuals in the dataset. we then feed the dccs to our dashboard, where we contextualize them in the data map and also highlight the most similar data points in the dataset. by investigating similarities and differences between the data points, we were able to diagnose several spurious correlations, which we categorize into three different groups and a category of artifacts. furthermore, we incorporate gpt-3 suggestions to allow for effective and diverse model-in-the-loop adversarial data generation. therefore, our dashboard opens up future work on adversarial test suite generation and counterfactual augmentation.