F 1 is N ot E nough! M odels and E valuation T owards U ser- C entered E xplainable Q uestion A nswering (2020)

Abstract: Explainable question answering systems predict an answer together with an explanation showing why the answer has been selected. The goal is to enable users to assess the correctness of the system and understand its reasoning process. However, we show that current models and evaluation settings have shortcomings regarding the coupling of answer and explanation which might cause serious issues in user experience. As a remedy, we propose a hierarchical model and a new regularization term to strengthen the answer-explanation coupling as well as two evaluation scores to quantify the coupling. We conduct experiments on the HOTPOTQA benchmark data set and perform a user study. The user study shows that our models increase the ability of the users to judge the correctness of the system and that scores like F1 are not enough to estimate the usefulness of a model in a practical setting with human users. Our scores are better aligned with user experience, making them promising candidates for model selection.

## introduction understanding the decisions of deep learning models is of utmost importance, especially when they are deployed in critical domains, such as medicine or finance @xcite . in natural language processing (nlp), a variety of tasks have been addressed regarding explainability of neural networks, such as textual entailment @xcite , sentiment classification @xcite , machine translation @xcite and question answering @xcite . in this paper, we address question answering (qa) due to its proximity to users in real-life settings, for instance, in the context of personal assistants. explainable question answering (xqa) is the task of (i) answering a question and (ii) providing an explanation that enables the user to understand * the kalahari desert is a large semi-arid sandy savanna in southern africa extending for 900000 km2 , covering much of botswana, parts of namibia and regions of south africa. what is the area of the desert that ghanzi is in the middle of? 900000 km2, because: * ghanzi is a town in the middle of the kalahari desert the western part of the republic of botswana in southern africa. ghanzi's area is 117,910 km². figure 1 : example output of a representative xqa system @xcite that would receive an answer-f 1 of 1 and an explanation-f 1 of 0.5 although the explanation provides no value to the user since the actual answer evidence (shown in cloud) is not included in the explanation (asterisks mark ground truth explanation). a good explanation (i.e., one that is helpful for the user) should therefore satisfy the following requirements: (i) it should contain all information that the model used to predict the answer for the question. this is necessary so that the user can reconstruct the model's reasoning process. (ii) it should not include additional information that it did not use for predicting the answer. otherwise, the explanation will confuse the users rather than help them. note that these requirements do not only hold for correct model decisions but are also valid for explaining wrong model answers so that the user can assess the correctness of the answers. previous work on xqa mostly focuses on developing models that predict the correct answer and, independent of this, the correct explanation @xcite @xcite . this can lead to model outputs in which the explanations do not sufficiently relate to the answers. consider the example provided in figure 1 . the model gives the correct answer to the question and provides an explanation consisting of one out of two relevant facts. however, the most important relevant fact (in which the answer actually appears) is not part of the explanation. as a result, the user cannot assess whether the model answer is correct or not and, thus, cannot trust the system. to strengthen the coupling of answer and explanation prediction in the model architecture and during training, we propose two novel approaches in this paper: (i) a hierarchical neural network architecture for xqa that ensures that only information included in the explanation is used to predict the answer to the question, and (ii) a regularization term for the loss function that explicitly couples answer and explanation prediction during training. a good evaluation measure should score explanations by satisfying the following requirements: (i) it should reward explanations that are coupled to the answers of the model. (ii) it should punish explanations that are unrelated to the answers of the model. (iii) it should be correlated to user experience. since explanations cannot only empower the user to assess the correctness of a system @xcite but also improve user satisfaction and confidence @xcite and, thus, increase the acceptance of automatic systems @xcite , this aspect is very important when evaluating models that should be applied in reallife scenarios. in most recent works, evaluation of xqa models focuses on optimizing f 1 -scores of answers and explanations (a collection of so-called supporting or relevant facts) @xcite . however, f 1 -scores only assess model outputs with respect to ground-truth annotations which only contain explanations for the correct answer. thus, they fail to quantify the coherence between answer and explanation, especially when the predicted model answer is wrong. the example model output in figure 1 leads to an answer-f 1 -score of 1 and an explanation-f 1 -score of 0.5 although the explanation is useless for the user as described before. to quantify the model's answer-explanation coupling, we propose two novel evaluation scores: (i) farm which tracks prediction changes when removing facts, and (ii) loca which assesses whether the answer is contained in the explanation or not. both scores do not require ground-truth annotations. to summarize, we make contributions in two directions in this paper: for modeling, (i) we propose a hierarchical neural network architecture as well as (ii) a regularization term for the loss function of xqa systems. for evaluation, (iii) we propose two scores that are able to quantify a model's answerexplanation coupling without relying on groundtruth annotations. (iv) to investigate the relation between different evaluation scores and user experience, we conduct a user study. the results show that our proposed models increase the ability of the user to judge the correctness of an answer and that our scores are stronger predictors of human behavior than standard scores like f 1 . (v) for reproducibility and future research, we will release code for our methods and for computing the evaluation scores as well as the user study data. foot_0 ## related work in the context of xqa, @xcite present the hotpotqa data set which we also use for the experiments in this paper. in addition to questions and answers, it contains explanations in the form of relevant sentences from wikipedia articles. most state-of-the-art models for hotpotqa extend the bidaf++ architecture @xcite , e.g., @xcite @xcite @xcite . other approaches are based on question decomposition @xcite , graph/hierarchical structures @xcite @xcite , virtual knowledge bases @xcite or transformer models @xcite . so far, all of the research work on hotpotqa focuses on reaching higher f 1 -scores. in contrast, we question whether this actually aligns with user experience. to the best of our knowledge, only @xcite additionally conduct a human evaluation. this confirms the observation of @xcite that only very few papers related to explainable ai address (human) evaluation of explainability. despite the large body of research in the field of human computer interaction, @xcite show that there is a lack of collaboration and transfer of results to machine learning communities. another line of research our work relates to is the criticism of automatic evaluation scores. one frequently questioned score is bleu @xcite , which was shown to only correlate weakly with human judgements in tasks like machine translation @xcite , storytelling @xcite and dialogue response generation @xcite . f 1 has been criticized from various perspectives including theoretical considerations and concrete applications @xcite @xcite . @xcite show that modifying f 1 -scores based on insights from psychometrics improves their correlation with human evaluations. in this paper we criticize the usage of f 1 as a measure of explainability in xqa and show in a user study that it is not related to user experience. ## methods for xqa we built upon the model by @xcite as it is an improved version of the bidaf++ model, which is used in numerous state-of-the-art xqa models @xcite @xcite @xcite including the best-scoring publication @xcite . it consists of a question and context encoding part with self-attention, followed by two prediction heads: a prediction of relevant facts (i.e., the explanation) and a prediction of the answer to the question. the two heads are trained in a multi-task fashion based on the sum of their respective losses. first, we analyze the outputs of the model, revealing severe weaknesses in answer-explanation coupling. to address those weaknesses, we then propose (i) a novel neural network architecture that selects and forgets facts, and (ii) a novel answer-explanation coupling regularization term for the loss function. ## new evaluation scores for xqa in this section, we motivate that standard scores like f 1 are not enough to score xqa systems by presenting their limitations in those settings. to be able to quantify the degree to which a model is affected by those limitations, we propose two scores that go beyond standard scores: the factremoval score and the answer-location score. both scores can be calculated without any assumptions on model architecture and no need for ground truth annotations for answers or supporting facts. ## experiments and results in this section, we describe the dataset we used in our experiments as well as our results. more details for reproducibility, including hyperparameters, are provided in the appendix. ## human evaluation we conduct a user study to investigate whether standard scores like f 1 or our proposed scores are better suited to predict user behavior and performance. moreover, the study provides another way to compare our proposed methods to the model by @xcite and the ground truth explanations. in contrast to the human evaluation from @xcite , we evaluate explanations in the context of the model answer, ask participants to rate the predictions along multiple dimensions and collect responses from 40 instead of 3 subjects. ## conclusion in this paper, we investigated explainable question answering, revealing that existing models lack an explicit coupling of answers and explanations and that evaluation scores used in related work fail to quantify that. this highly impairs their applicability in real-life scenarios with human users. as a remedy, we addressed both modeling and evaluation, proposing a hierarchical neural architecture, a regularization term, as well as two new evaluation scores. our user study showed that our models help the users assess their correctness and that our proposed evaluation scores are better correlated with user experience than standard measures like f 1 .