XMD : An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models (2023)

Abstract: NLP models are susceptible to learning spurious biases (i.e., bugs) that work on some datasets but do not properly reflect the underlying task. Explanation-based model debugging aims to resolve spurious biases by showing human users explanations of model behavior, asking users to give feedback on the behavior, thenusing the feedback to update the model. While existing model debugging methods have shown promise, their prototype-level implementations provide limited practical utility. Thus, we propose XMD: the first open-source, end-to-end framework for explanation-based model debugging. Given task- or instance-level explanations,users can flexibly provide various forms of feedback via an intuitive, web-based UI. After receiving user feedback, XMD automatically updates the model in real time, by regularizing the model so that its explanationsalign with the user feedback. The new model can then be easily deployed into real-world applications via Hugging Face. Using XMD, we can improve the model’s OOD performance on text classification tasks by up to 18%.

## introduction neural language models have achieved remarkable performance on a wide range of natural language processing (nlp) tasks @xcite . however, studies have shown that such nlp models are susceptible to learning spurious biases (i.e., bugs) that work on specific datasets but do not properly reflect the underlying task @xcite @xcite . for example, in hate speech detection, existing nlp models often associate certain group identifiers (e.g., black, muslims) with hate speech, regardless of how these words are actually used @xcite (fig. 1 ). this poses se-there is a big difference between muslims and terrorists. label: not hate ## framework overview as shown in figure 2 , our framework consists of three main components: explanation generation; explanation visualization; and explanation-based model debugging. here, the explanation generation and debugging process are done on the backend while visualizing explanations and capturing human feedback on them are done on front-end ui. explanation generation ( §3.1) humans first input the train data and the model trained on the train data into the framework. on the backend, our framework uses a heuristic post-hoc explanation approach on the model to generate rationales for the train data. explanation visualization ( §3. ## implementation details to start xmd , users should input the trained model following hugging face model structure @xcite . after users input the train data and the model, our framework uses captum @xcite to generate explanation. for visualizing the explanation and capturing the human feedback, we implement ui using vue.js 2 . here, we re-use ui components from lean-life, an explanation-based annotation framework @xcite , for capturing human feedback. to train the model with er, we use pytorch @xcite and huggingface @xcite . ## experiments we conduct extensive experiments investigating how our debugging process affects the performance on in-distributed (id) and out-of-distribution (ood) data, and the model explanation. here, we present experimental results on sentiment analysis for the instance explanation and hate speech detection for the task explanation. for base model, we use bigbird-base @xcite . tasks and datasets for sentiment analysis, we exploit sst @xcite as the id dataset, and yelp (restaurant reviews) @xcite , amazon (product reviews) @xcite and movies (movie reviews) @xcite as ood datasets. to simulate human feedback for the instance explanation, we leverage ground truth rationales for sst (carton et al., 2020) as human feedback. for hate speech detection, we use stf (de gibert et al., 2018) as the id dataset, and hateval @xcite , gab hate corpus (ghc) @xcite and latent hatred @xcite for ood datasets. to simulate human feedback for the task explanations, we leverage group identifiers (e.g., black, muslims) @xcite as words that need to be discarded for determining whether the instance is hate or not. ## conclusion in this paper, we propose an open-source and webbased explanation-based nlp model debugging framework xmd that allows user to provide various forms of feedback on model explanation. this debugging process guides the model to make predictions with the correct reason and lead to significant improvement on model generalizability. we hope that xmd will make it easier for researchers and practitioners to catch spurious correlations in the model and debug them efficiently.