What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception (2024)

Abstract: Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.

## introduction question answering models can often be black boxes, as their reasoning process is mostly opaque figure 1 : the framework for incorporating human feedback into decomposed qa models. a model x2r generates a rationale r to answer a question based on a passage. a human teacher then provides natural language feedback for r, which is used to generate a revised rationale r ′ from f 2r ′ . finally, this revised rationale is used to generate the final answer y . we study various formats of the intermediate rationale r. to model builders as well as end users. this can inhibit the ability of users to provide helpful critiques to models to repair them. generating rationales (or explanations) along with answers is a viable approach that can alleviate these concerns, but these rationales are inherently not faithful and can sometimes be inconsistent with the answers themselves @xcite @xcite . this motivates approaches that decompose the question answering task into two stages (depicted with dashed lines in figure 1 ), where we first generate a rationale for the question using the given context (x2r), then use only this rationale to answer the question (r2y ) @xcite . a rationale may provide a justification for the answer by presenting an outline for how the question can be answered. by only relying on the rationale as context, the answer generation model (r2y ) has a stronger inductive bias to generate an answer that is faithful to the rationale. crucially, faithful rationales can allow users to follow the model's line of reasoning, and subsequently provide actionable feedback to the model. this feedback can be used to repair individual outputs or enable generalization to novel instances. however, it is unclear precisely how a rationale should be formatted, i) to best aid the user's understanding of the model's reasoning, and ii) their ability to provide feedback for the response. our work specifically addresses the question of how we can format intermediate rationales (r) for decomposed qa systems, such that they are easy to repair through human feedback. further, we analyze what properties make rationales interpretable, and trustworthy to users. previous work on decomposed question answering mostly consider rationales as text snippets extracted from the context, optionally marked up with coreferences that make the snippets standalone @xcite . although extractive snippets can serve useful for providing minimal context that rationalizes an answer, they do not provide much insight into the model's reasoning process. this may limit a user's understanding and their ability to critique the model. we consider alternative formats of rationales, which vary according to well-defined characteristics ( §3). examples of these formats and how they vary are presented in based on the considered rationale formats, we generate responses (rationales & answers) from a decomposed qa system. we then perform two user studies where we measure i) the effectiveness of user feedback in patching rationales in different formats and ii) the ability of different formats to enable users to understand and trust responses. in our first study, we sample incorrect answers corresponding to all rationale formats, and ask annotators to provide natural language feedback for the rationales ( §5). we use this feedback to then revise the rationales (f 2r ′ ) and regenerate the final answer (r ′ 2y ). comparing the regeneration accuracy with different rationale formats provides insight into properties of rationales that make them easy to repair. further, they provide an upper bound for how much improvement can be expected through automated feedback by repairing rationales @xcite . in our second study, we elicit judgements of inter-pretability and trustworthiness for correct answers and their accompanying rationales from users ( §6). we consider two tasks: general reading comprehension (quoref; @xcite ) and medical reading comprehension (pubmedqa; @xcite ). each block in our overall workflow (figure 1 ) is implemented through few-shot prompting of a large language model. our experiments suggest that rationale formats significantly affect i) users' ease of providing feedback and ii) the model's ability to execute that feedback. in addition to being critiquable, certain rationale formats are more helpful in aiding users to understand and trust the model's answers. one such format is the annotated report, which includes a list of extractive phrases and free-text inferences based on each phrase. finally, among a few characteristics of rationales presented to users, we find that users rate attribution and depth of reasoning as characteristics that are most important to them. ## problem formulation consider a standard reading comprehension task, where we are given a passage, a question based on this passage, and a reference answer y as a labeled example in our dataset. we represent the input information in the task (the passage and the question) with x. typically, we would train a model x2y that predicts y given x by learning p (y |x). assuming x2y is a black-box model, without loss of generality, the model may internally compute a latent representation, which is usually not extractable in an interpretable format, from x to predict y , internally decomposing the problem. this can restrict the transparency of the model because we cannot provide a faithful reasoning to an end user that supports the answer. ## intermediate rationale formats rationales in nlp tasks are usually presented as compressed text snippets extracted from the given input @xcite . however, text snippets from the context alone may not make the model's reasoning explicit and transparent to users. we consider alternative rationale formats that describe the model's reasoning. we describe these formats below. a summary of these formats and how they vary according to rationale properties, is given in ## conclusion our work analyzed how model-generated explanations or rationales should be formatted to be most amenable to repair through user feedback. we also collected qualitative judgements of how different formats enable users to understand and trust model outputs. we found that rationale formats significantly affect what rationales are amenable to be repaired through feedback. in terms of user perception of rationales, we find that some rationale formats, such as the annotated report, are more favorable for enabling users to understand and trust model responses. finally, we find that among a few properties considered, attribution and depth of reasoning are the most important characteristics of rationales to users. we hope that this work can help researchers and practitioners alike make informed decisions about how to present language model responses and collect feedback from end users.