I nter F air: Debiasing with Natural Language Feedback for Fair Interpretable Predictions (2023)

Abstract: Debiasing methods in NLP models traditionally focus on isolating information related to a sensitive attribute (e.g., gender or race). We instead argue that a favorable debiasing method should use sensitive information ‘fairly,’ with explanations, rather than blindly eliminating it. This fair balance is often subjective and can be challenging to achieve algorithmically. We explore two interactive setups with a frozen predictive model and show that users able to provide feedback can achieve a better and fairer balance between task performance and bias mitigation. In one setup, users, by interacting with test examples, further decreased bias in the explanations (5-8%) while maintaining the same prediction accuracy. In the other setup, human feedback was able to disentangle associated bias and predictive information from the input leading to superior bias mitigation and improved task performance (4-5%) simultaneously.

## introduction debiasing human written text is an important scientific and social problem that has been investigated by several recent works @xcite @xcite @xcite @xcite . these methods primarily try to eliminate the biased information from the model's internal representations or from the input itself, disregarding the task performance during the process. however, in an ideal situation, a model should use only the necessary amount of information, irrespective of bias, to achieve an acceptable task performance. this trade-off between task performance and bias mitigation is subjective or varies between users @xcite and is often hard to achieve via learning from data @xcite . figure 1 shows the limit of an algorithmic approach where ignoring all gendered information can lead to a wrong result. however, a user can potentially further tune the model's belief on the bias, leading to a correct prediction while minimally using biased information. while interactive nlp models recently focused on model debugging @xcite , improving explainability in qa @xcite , machine teaching @xcite , critiquing for personalization @xcite , and dialog as a more expressive form of explanations @xcite , we focus on an under-explored paradigm of model debiasing using user interactions. objectively, we allow users to adjust prediction rationales at the test time to decrease bias in them, addressing the subjective aspect of fair and transparent debiasing. in this paper, we propose interfair, a modular interactive framework that (1) enables users to provide natural language feedback at test time to balance between task performance and bias mitigation, (2) provides explanations of how a particular input token contributes to the task performance and exposing bias, and finally (3) achieves better performance than a trained model on full-text input when augmented with feedback obtained via interactions. ## background: debiasing with rationales an interpretable debiasing algorithm produces a rationale along with a prediction of the original task to expose the amount of bias or sensitive information used. precisely, a rationale is the minimal and sufficient part of the input responsible for the prediction. for text input, let the predictive input tokens for the task output be called task rationales and tokens revealing sensitive information be called bias rationales. since the model solely uses the rationales to predict the task output, these rationales are highly faithful @xcite . according to @xcite , it is possible to attach an importance score for each token to be included in the respective task or bias rationales. traditional debiasing algorithms face two failure modes: 1) it produces a correct task prediction but with a highly biased rationale and 2) it produces a wrong task prediction but a rationale with low bias. @xcite argue that weighing less on highbias and high-task important tokens and promoting their low-bias replacements can simultaneously address both of the failure modes. however, this technique is not perfect, and figure 1 shows a limiting case of this approach that opens up further room for improvement. ## interfair: interactive fair debiasing we highlight that even an algorithmically debiased model can have failure modes and one potential option is to fix the problem at the inference time. we argue that human users are better at fixing the failure cases that a model is unable to learn from the training data. we also assume that the model parameters remain frozen during the fixing process, and users only interact with the final prediction and its associated hidden model states. ## experiments and results we break our experiments into two parts: 1) developing the nl parser and 2) interactive debiasing with interfair. we use biosbias @xcite , a dataset made from a large-scale user study of gender in various occupations. it contains short biographies labeled with gender and profession information, and a possible confluence exists between gender and annotated profession labels. using interfair, we would like to predict the profession from biographies without the influence of gender. following @xcite , we use 393,423 biographies with binary gender labels (male/female) and 28 professions labels (e.g. professor, model, etc.). we initially used 255,710 examples for training and 39,369 for validation. we use 500 examples (a random sample from the rest 25%) as a test set for interactive debiasing. for evaluation, we use accuracy for task performance (profession prediction) and use an off-theshelf gender detector to measure the bias in the task rationales (bias f1), following @xcite . ## conclusion in summary, interfair shows the possibility of user-centric systems where users can improve model performances by interacting with it at the test time. test-time user feedback can yield better disentanglement than what is achieved algorithmically during training. debiasing is a subjective task, and users can take the higher agency to guide model predictions without affecting model parameters. however, interfair does not memorize previous feedback at a loss of generalization, which can be addressed via memory-based interactions @xcite , or persistent model editing @xcite as future work.