Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment (2022)

Abstract: Most research on question answering focuses on the pre-deployment stage; i.e., building an accurate model for deployment. In this paper, we ask the question: Can we improve QA systems further post-deployment based on user interactions? We focus on two kinds of improvements: 1) improving the QA system’s performance itself, and 2) providing the model with the ability to explain the correctness or incorrectness of an answer. We collect a retrieval-based QA dataset, FeedbackQA, which contains interactive feedback from users. We collect this dataset by deploying a base QA system to crowdworkers who then engage with the system and provide feedback on the quality of its answers. The feedback contains both structured ratings and unstructured natural language explanations. We train a neural model with this feedback data that can generate explanations and re-score answer candidates. We show that feedback data not only improves the accuracy of the deployed QA system but also other stronger non-deployed systems. The generated explanations also help users make informed decisions about the correctness of answers.

## introduction much of the recent excitement in question answering (qa) is in building high-performing models with carefully curated training datasets. datasets like squad @xcite , naturalquestions @xcite and coqa @xcite have enabled rapid progress in this area. most existing work focuses on the pre-deployment stage; i.e., training the best qa model before it is released to users. however, this stage is only one stage in the potential lifecycle of a qa system. in particular, an untapped resource is the large amounts of user interaction data produced after the initial deployment of the system. gathering this 1 project page: https://mcgill-nlp.github.io/feedbackqa/ data should in practice be relatively cheap, since users genuinely engage with qa systems (such as google) for information needs and may provide feedback to improve their results. 2 exploiting this kind of user interaction data presents new research challenges, since they typically consist of a variety of weak signals. for example, user clicks could indicate answer usefulness @xcite , users could give structured feedback in the form of ratings to indicate the usefulness @xcite , or they could give unstructured feedback in natural language explanations on why an answer is correct or incorrect. user clicks have been widely studied in the field of information retrieval @xcite . here we study the usefulness of interactive feedback in the form of ratings and natural language explanations. whilst there are different variants of qa tasks, this paper focuses primarily on retrieval-based qa (rqa; @xcite . given a question and a set of candidate answer passages, a model is trained to rank the correct answer passage the highest. in practice, when such a system is deployed, an user may engage with the system and provide feedback about the quality of the answers. such feedback is called interactive feedback. due to the lack of a dataset containing interactive feedback for rqa, we create feedbackqa. feedbackqa is a large-scale english qa dataset containing interactive feedback in two forms: user ratings (structured) and natural language explanations (unstructured) about the correctness of an answer. figure 1 shows an example from feedbackqa. the dataset construction has two stages: we first train a rqa model on the questions and passages, then deploy it on a crowdsourcing platform. next, crowdworkers engage with this system and provide interactive feedback. to make our dataset practically useful, we focus on figure 1 : users interact with the deployed qa model and give feedback. feedback contains a rating (bad, good, could be improved, excellent) and a natural language explanation. question answering on public health agencies for the covid-19 pandemic. the base model for feed-backqa is built on 28k questions and 3k passages from various agencies. we collect 9k interactive feedback data samples for the base model. we investigate the usefulness of the feedback for improving the rqa system in terms of two aspects: answer accuracy and explainability. specifically, we are motivated by two questions: 1) can we improve the answer accuracy of rqa models by learning from the interactive feedback? and 2) can we learn to generate explanations that help humans to discern correct and incorrect answers? to address these questions, we use feedback data to train models that rerank the original answers as well as provide an explanation for the answers. our experiments show that this approach not only improves the accuracy of the base qa model for which feedback is collected but also other strong models for which feedback data is not collected. moreover, we conduct human evaluations to verify the usefulness of explanations and find that the generated natural language explanations help users make informed and accurate decisions on accepting or rejecting answer candidates. our contributions are as follows: ## feedbackqa dataset recently, there have been efforts to collect feedback data in the form of explanations for natural language understanding tasks @xcite . these contain explanations only for ground-truth predictions for a given input sampled from the training data without any user-system interaction. instead, we collect user feedback after deploying a rqa system thereby collecting feedback for both correct and incorrect predictions. ## experimental setup feedbackqa contains two types of data. one is pre-deployment data d @xmath0, a + , a), where q is a question paired with its gold-standard answer passage a + from the domain corpus a. the other is post-deployment feedback data d @xmath1, a, y, e), where q is a question paired with a candidate answer @xmath2. the feedback consists of a rating y and an explanation e. we build rating label explanation excellent this answers the question directly. this answer provides information and recommendation on how people and adolescent can protect themselves when going online during the covid-19 pandemic. acceptable this answer, while adequate, could give more information as this is a sparse answer for a bigger question of what one can do for elderly people during the pandemic. could be improved the answer relates and answers the question, but could improve grammatically and omit the "yes" could be improved the answer is about some of the online risks but not about how to protect against them. ## https://huggingface.co/facebook/bart-large-xsum question: what are the restrictions to travelers landing in the us? answer: ... under current federal regulations, pilots must report all illnesses and deaths to cdc before arriving to a us destination. according to cdc disease protocols, if a sick traveler is considered to be a public health risk, cdc works with local and state health departments and international public health agencies to contact passengers and crew exposed to that sick traveler. be sure to give the airline your current contact information when booking your ticket so you can be notified if you are exposed to a sick traveler on a flight. ... ## related work retrieval-based question answering has been widely studied, from early work on rule-based systems @xcite , to recently proposed neural-based models @xcite . most existing work focuses on improving the accuracy and efficacy by modification of a neural architecture @xcite , incorporation of external knowledge @xcite , and retrieval strategy @xcite . these methods focus on the pre-deployment stage of rqa models. by contrast, we investigate methods to improve a rqa model post-deployment with interactive feedback. the proposed methods are agnostic to the architecture design and training methods of the base rqa model. learning from user feedback has been a long standing problem in natural language processing. whilst earlier work proposes methods for using implicit feedback-for instance, using click-through data for document ranking @xcite recent work has explored explicit feedback such as explanations of incorrect responses by chatbots @xcite and correctness labels in conversational question answering and text classification @xcite . however, the feedback in these studies is automatically generated using heuristics, whereas our feedback data is collected from human users. @xcite collect suggested responses from users to improve a chatbot, while we investigate the effect of natural feedback for rqa models. explainability and interpretability has received increasing attention in the nlp community recently. this paper can be aligned to recent efforts in collecting and harnessing explanation data for language understanding and reasoning tasks, such as natural language inference @xcite , commonsense question answering @xcite , document classification @xcite , relation classification @xcite , reading comprehension @xcite , and fact checking @xcite . the type of feedback in feed-backqa differs from the existing work in several aspects: 1) feedbackqa has feedback data for both positive and negative examples, while most of other datasets only contains explanations of positive ones; 2) feedbackqa has both structured and unstructured feedback, while previous work mainly focuses on one of them; 3) the feedback in feedbackqa is collected post-deployment; 4) while previous work aims to help users interpret model decisions, we investigate whether feedbackbased explanations increase the utility of the deployed system. ## conclusion in this work, we investigate the usefulness of feedback data in retrieval-based question answering. we collect a new dataset feedbackqa, which contains interactive feedback in the form of ratings and natural language explanations. we propose a method to improve the rqa model with the feedback data, training a reranker to select an answer candidate as well as generate the explanation. we find that this approach not only increases the accuracy of the deployed model but also other stronger models for which feedback data is not collected. moreover, our human evaluation results show that both human-written and model-generated explanations help users to make informed and accurate decisions about whether to accept an answer.