Can We Improve Model Robustness through Secondary Attribute Counterfactuals? (2021)

Abstract: Developing robust NLP models that perform well on many, even small, slices of data is a significant but important challenge, with implications from fairness to general reliability. To this end, recent research has explored how models rely on spurious correlations, and how counterfactual data augmentation (CDA) can mitigate such issues. In this paper we study how and why modeling counterfactuals over multiple attributes can go significantly further in improving model performance. We propose RDI, a context-aware methodology which takes into account the impact of secondary attributes on the model’s predictions and increases sensitivity for secondary attributes over reweighted counterfactually augmented data. By implementing RDI in the context of toxicity detection, we find that accounting for secondary attributes can significantly improve robustness, with improvements in sliced accuracy on the original dataset up to 7% compared to existing robustness methods. We also demonstrate that RDI generalizes to the coreference resolution task and provide guidelines to extend this to other tasks.

## introduction how can we build nlp models that perform well over many slices, albeit sometimes small slices, of our data? developing models that are robust in their performance is important for trusting these models to work well in diverse, unexpected settings. as a concrete running example in this paper, we will consider the task of toxicity detection: using a model to predict if a comment is toxic or not @xcite . in this application, for example, it is often important to ensure that models are accurate over slices of data referring to different demographic groups, as has been raised across machine learning fairness research @xcite . one significant focus of research on how to improve model robustness has been addressing spu-rious correlations and improving counterfactual robustness. that is, researchers have found that models often rely on features or attributes that are only spuriously correlated with the task and accuracy often drops when models are evaluated on counterfactual data that perturbs those attributes @xcite . to return to our example of toxicity detection, a model may learn that certain identity tokens are correlated with toxicity, but that could decrease accuracy for non-toxic comments with those terms @xcite . recent work has explored how counterfactual generation techniques can be used to form general checklists to test for model biases @xcite , often composing many sub-problems which are hard to solve formally. similarly, a wide breadth of research has studied how to train models to be more robust. we focus on one such mitigation techniquecounterfactual data augmentation (cda), where the supervised training data is augmented and balanced by replacing in-place words or phrases in the input sentence, which should not lead to a change in the output label y @xcite . these counterfactual data generation approaches have been built on, as well as coupled with regularization, to improve counterfactual fairness @xcite , such as preventing models from being overly sensitive to identity terms @xcite @xcite . although these approaches have been effective in reducing spurious correlations, in this paper we observe and study how such approaches often fail to significantly improve core model accuracy and can still perform worse on subsets of the dataset due to the primary variable over which counterfactual are generated being correlated with (many) secondary variables that are not swapped or balanced. returning to our example task of toxicity classification over comments, the primary attributes (e.g., demographic identity terms) may be correlated with secondary attributes (intent of the comment-directed or descriptive) in the training data distribution. that is, for some demographic groups we may observe more directed comments and for others we may observe more directed comments: toxic: seeking transgender rights is extreme (directed) non-toxic: transgender rights activists are labeled extremists (descriptive) in the toxicity classification example shown above, while the former is labeled as toxic by human annotators as it is directed towards a demographic group, the latter is only describing the toxicity and is considered as non-toxic. nonetheless, both these sentences are classified as toxic by the jigsaw perspective api @xcite , thus leading to high false positive rates. so, to remove spurious correlations for the word "transgender" with toxicity, it may not be enough to improve model accuracy over comments with the word "transgender" if the model is more accurate for directed comments than descriptive ones. therefore we ask: can explicitly considering counterfactuals over both primary and secondary attributes better improve robustness? to answer this question and improve model's robustness, i.e., accuracy on slices of the data, we propose a new approach, rdi, that learns from counterfactual data generated through interventions on both the primary and secondary attributes. rdi applies regularization techniques to train the model to disentangle the impact of the primary and secondary attribute and to explicitly optimize for the classifier's predictions to be sensitive or insensitive to each attribute. the approach also builds on recent reweighting approaches @xcite to further address distributional skews in the data. our approach to studying this problem builds on works that argue for a case-by-case analysis of variables and aims to provide a framework for incorporating secondary variables when we discuss the robustness of natural language models (de @xcite . specifically, we have focused on the toxicity detection model which prior work has shown to suffer from unintended bias @xcite based on protected identity terms mentioned in the sentence. we analyze how existing robustness techniques fail to capture a secondary attribute, namely the intent of the sentence while performing counterfactual data augmentation. we further show that this intent, that is descriptive or directed, is significantly correlated with specific protected identity groups in the dataset. by disentangling this correlation in the real world data via the counterfactual data, we obtain a model that does not disparately have high false positive rates on specific demographic groups, while being sensitive to the intent of the sentence. we achieve this improvement in robustness, while improving the sliced accuracy across multiple protected identity subgroups of the data. our key contributions are: ## methodology since the goal of robustness is in addition to that of increasing overall accuracy on the original dataset, we use constrained optimization techniques over augmented counterfactual data. before we present our proposed constraints, we present existing baseline constraints defined in the fairness and robustness literature. we discuss why these baseline constraints do not explicitly address the goal of improving counterfactual robustness on primary and secondary variables, and hence necessitate our additional proposed constraints on the counterfactual dataset d. ## pronoun coreference resolution we have demonstrated the utility of modeling secondary attributes to improve robustness of the toxicity detection models. however, we note that not all tasks have secondary attributes whose impact on the label needs to be maximized. each task and their corresponding secondary attributes are unique in their relationship and their difficulty in data gathering, and we need careful understanding of the context while enforcing constraints between them. in this section, we show how our rdi framework can be extended to a task -"pronoun coreference resolution", where the label is invariant to the secondary attribute -gender. between these two use cases, we have exhaustively covered the types of constraints that can be incorporated towards secondary attributes and encourage researchers to undertake a contextual treatment of secondary attributes in their tasks. we provide an example below, where the pronoun resolution should not change based on the gender of the pronoun. @xcite @xcite . each of the nouns referenced back from the pronouns also have their associated gender (binary) @xcite . in the ontonotes coreference dataset, we evaluate the f1-score, the gendered correlation coefficient @xcite which measures the correlation between gender and the professions they resolve to. the winogender coreference resolution dataset provides templates with placeholders for the gendered-pronoun, and two antecedent professions which the pronoun could potentially be referencing. we refer to @xcite for the full set of templates. ## conclusion we have demonstrated the value of incorporating the impact of secondary variables in the objectives for learning robust natural language processing models. we have shown that incorporating contextaware counterfactual robustness through the rdi algorithm, we improve performance on the counterfactual augmented data, but also improve the overall and sliced accuracy on the original dataset by 2-7%.