Calibrating Trust of Multi-Hop Question Answering Systems with Decompositional Probes (2022)

Abstract: Multi-hop Question Answering (QA) is a challenging task since it requires an accurate aggregation of information from multiple context paragraphs and a thorough understanding of the underlying reasoning chains. Recent work in multi-hop QA has shown that performance can be boosted by first decomposing the questions into simpler, single-hop questions. In this paper, we explore one additional utility of the multi-hop decomposition from the perspective of explainable NLP: to create explanation by probing a neural QA model with them. We hypothesize that in doing so, users will be better able to predict when the underlying QA system will give the correct answer. Through human participant studies, we verify that exposing the decomposition probes and answers to the probes to users can increase their ability to predict system performance on a question instance basis. We show that decomposition is an effective form of probing QA systems as well as a promising approach to explanation generation. In-depth analyses show the need for improvements in decomposition systems.

## introduction as natural language understanding tasks have become increasingly complex, the field of explainable natural language processing (exnlp) aims to help users understand the performance of nlp systems. multi-hop question answering is one such task in which questions seemingly require multiple reasoning steps to answer. to accurately answer a multi-hop question, one must start by decomposing the given multi-hop question into simpler sub-questions, then try to answer them respectively, and finally aggregate together the information obtained from all the sub-questions. for instance, consider the multi-hop question "what year did the band that sang 'with or without you' form?". to figure 1 : an overview of our method. users wonder if they are able to trust the answer. sub-questions are generated by a decomposer agent (gear) to probe the question-answering agent. from the perspective of explainable nlp, we explore the utility of multi-hop decompositions to create explanations. one role of explanations is to help users construct a mental model of the underlying system @xcite @xcite . in doing so, users will be better equipped to know when the system answers can be trusted. this is especially important for large, general-purpose qa systems that can answer a wide range of questions but might have greater competencies when answering questions about some topics versus others. we hypothesize that question decompositions used to probe a neural qa model can improve users' abilities to predict whether the qa system will answer the original question correctly or not. @xcite observed that improved decompositional reasoning chains for multi-hop qa correlate with increased user perceptions of trust, understandability, and preference. while perceptions of trust are important, it is also important that the trust is appropriately calibrated @xcite @xcite @xcite . that is, the user should trust the system when it is worthy of that trust. for general-purpose question-answering systems built upon large-scale language models, the ability to accurately answer a question is likely to be variable based on the specific question asked. how does a user know when to trust a qa system's answer to a particular question? if just presented with an answer, one has no cues from which to make an assessment. end-to-end qa systems that generate answers and explanations are trained to justify the answer as opposed to provide evidence of the system's competencies on a topic. we introduce probing as an explanation strategy that helps a user determine whether to trust an answer. probing is a process whereby a model is provided similar inputs to determine if its performance is stable when handling related inputs. in this work, we show that exposing the decomposition probes and answers to the probes to users can increase their ability to predict whether the system will answer the original question correctly. this indicates that users-without knowing the particulars of the underlying qa system-are receiving actionable cues from which to model the behavior of the system. instead of asking for subjective perceptions of the overall system, we objectively measure the effect of the probes on instance-level interactions. to the best of our knowledge, this paper is the first to show that probing can have a measurable context: learning, inc. is an educational software and hardware company co-founded in 1999 by texas businessman neil bush and a year later ken leonard. he is the fourth of six children of former president george h. w. bush and barbara bush (née pierce). question: who is the mother of the texas business man that co-founded ignite! learning, inc? answer: barbara bush sub-question 1: who is the texas business man who co founded ignite learning, inc? answer: neil bush sub-question 2: who is neil bush's mother? answer: barbara bush a summary of our method is given in figure 1 . ## sub-question answering can distinguish incorrect and correct model predictions we first investigate the extent to which performance on sub-question-answering is tied to performance on the main qa tasks. we split the validation set instances into two groups: those for which the model predicts the answer for the main question correctly, and those for which it does not. results are presented in ## sub-question explanations allow humans to predict model behavior given the correlations between model's performance on main qa and sub-qa, we take a step forward to ask: can humans gain any useful insight from such correlations? we perform a simulatability experiment to measure how well the sub-question explanations can help humans predict model behaviors on the main hotpotqa task. to this end, we design and conduct a human participant study to investigate crowd annotators' ability to make accurate predictions about model performance given question decompositions as explanations, following the protocol given in §2.5. we select a random 100-instance sample from the 676 hotpotqa validation instances, balanced such that the model predicts 50 instances correctly and 50 incorrectly, and perform the probing procedure described in §2.4 on the best-performing model (t5-base), which results in tuples of the form (q, c, a, sub-q 1 , sub-a 1 , sub-q 2 , sub-a 2 ), where all answers are predicted by the model. our goal is to observe how much the sub-qa explanations help human annotators predict model behavior over a baseline that does not include these explanations, as well as investigate how the context (c) & the predicted answer (a) could potentially impact human's performance of diagnosing model errors. we design five different settings in which human participants are provided with different combinations of information. after reading the combination of information we present, the participants are asked to make their predictions about model's behavior on the main question (q), i.e., whether or not the model will be able to correctly answer the given q. we recruit 50 participants on prolific and split them into 5 batches, each of which contains 10 participants. given the 100-instance sample, we split it into 5 batches of 20 questions each. we follow a latin square design, similarly to (gonzalez and søgaard, 2020), to ensure that each group of participants only sees each set of questions under one condition: (q, a), (q, a, sub-q), (q, a, sub-q, sub-a), (q, sub-q, sub-a), or (q, c, sub-q, sub-a), yet each condition is tested on both all 50 annotators and all 100 questions. this ensures that no bias in human predictions occurs due to having previously seen the questions and model predictions. example of the ui that participants see is given in figure 3 . finally, we collect their predictions and compute the performance scores using the actual main question's answer correctness as the ground truth. results are presented in we apply the two-sided mann-whitney u test (mann and whitney, 1947) for statistical significance on accuracy numbers. participant accuracy given (q, a, sub-q, sub-a) is statistically significantly different at @xmath0.01 from all other settings, and results in substantially higher performance across all metrics except recall. this demonstrates that our proposed sub-qa explanation method does help humans make more accurate predictions about model behavior on the main question (q) than simply seeing model predictions (q, a). we additionally validate that both sub-questions and sub-answers are important-when we ablate subanswers, humans do poorly at the simulatability task given (q, a, sub-q), resulting in no significant performance difference over (q, a) pairs. having the answer (a) greatly improves the prediction performance, whereas the context (c) does not significantly impact human's prediction performance. meanwhile, the proved feasibility of human's making accurate prediction about model behavior using sub-qa explanations suggests a potential future direction for establishing an alternative for carrying out real annotation activities in order to diagnose qa system's error. the benefit of such alternative is obvious: humans will no longer have to conduct the question decomposition and perform the actual multi-hop reading comprehension by themselves. instead, they may solely rely on or at least gain useful insights from their mental model about the qa system to save time and effort when trying to diagnose the error. ## quality of question decompositions matters prior work has shown that predictions from question decomposition models can improve task performance on hotpotqa when being part of a larger modular system @xcite @xcite , but qualitative inspection reveals a lack of quality in many cases. to investigate whether such sub-question-generation models can provide interpretability, we explore the effect of sub-question quality on utility of ques- we use decomposition predictions from three trained question decomposers developed as part of larger modular qa systems in prior work: a) modularqa @xcite ; b) one-to-n unsupervised sequence transduction (onus; @xcite ; and c) decomprc @xcite . modularqa is a next-questionprediction bart-large model trained on the silver decompositions described in §2.2. onus is trained to decompose complex questions from the internet into simpler questions using supervision from noisy pseudo-decompositions. decomprc is trained on a mix of supervision and heuristics to create sub-questions from the tokens in the original question, framing the task as span prediction. examples of the question decompositions produced by each method are in we repeat the crowdsourcing process in §4, randomly sampling a subset of 30 correctly-predicted instances and 30 incorrectly-predicted instances from the 100 selected in §4. we probe the t5 model with sub-q 1 and sub-q 2 produced by each of the 4 sources: {silver, modularqa, onus, decomprc}, and collect its sub-a 1 , sub-a 2 responses. tuples of (q, a, sub-q 1 , sub-a 1 , sub-q 2 , sub-a 2 ) are presented to 30 new annotators (who have not participated in previous experiments) following the setup in §4. similar to §4, we perform a latin square design by equally splitting the participants and the questions into 3 batches, such that each participant group only observes each subset of questions under one experimental condition (either modu-larqa, onus, or decomprc predictions). annotator performance metrics at predicting answer correctness, averaged across all 30 participants, are presented in we apply the two-sided mann-whitney u test (mann and whitney, 1947) for statistical significance on accuracy numbers. human performance scores from the trained decomposers are all worse than the silver decomposer at a statisticallysignificantly different level (@xmath2.05). this indicates that there are still notable gaps between the quality of silver's and other existing decomposers' sub-qa explanations. onus question decompositions consistently provide the least explanatory power. despite decomprc's methodological simplicity, the explanatory power of its question decompositions is comparable to mod-ularqa, though modularqa is the highestperforming predictive model overall. this is further supported by statistical significance results, which reveal that both modularqa and decomprc are statistically-significantly different from onus, but not from one another (@xmath3.05). to further investigate the quality differences of different sources of question decompositions as measured by human preferences, we conduct an additional study where participants are asked to rank sources of sub-qa explanations based on their quality. specifically, we again recruit 30 new participants and each of them is asked to rank four decomposers' sub-qa explanations for 30 ran-silver @xcite sub-question 1: during what war was pavillon du butard occupied by the prussians? sub-question 2: what was the name that the french called the franco-prussian war? modularqa @xcite sub-question 1: during what war was the pavillon du butard occupied? sub-question 2: what is the french name for the franco-prussian war? onus @xcite sub-question 1: what is the name the french give to the war? sub-question 2: during which war did the prussians occupy the pavillon du butard? decomprc @xcite sub-question 1: which war during which the prussians occupied the pavillon sub-question 2: what is the name the french give to franco-prussian war du butard? ## related work multiple prior works have concluded that questionanswering as a form @xcite is a good choice for probing pretrained models @xcite . @xcite fine-tune a model on a dataset of questions and answers, but claim this does not introduce new information to the model and only teaches form for effective qa probing. however, this claim is not well-supported, as fine-tuning removes any guaran-tees that the questions answered at test-time reflect information learned during pre-training alone, and is not zero-or few-shot. we avoid this by performing probing in a truly zero-shot manner (i.e., we never fine-tune on sub-questions). additionally, the method of @xcite does not probe for instance-level prediction explanations; the authors instead use a fixed set of questions on general topics. in our work, we use the instance-level explanations we obtain from probing with sub-questions to test whether these explanations give humans an accurate mental model of the system @xcite . most related to our work is that of @xcite , who also investigate whether model architectures for multi-hop qa can answer single-hop questions. they find that there is a significant percentage of questions for which the model answers the main question correctly, but cannot correctly answer the corresponding single-hop sub-questions. however, because they use a model to produce question decompositions, their results may be confounded by errors or low quality of the questions themselves, which our work circumvents by using a silver source of sub-questions; we also investigate the effect of sub-question quality on the final results. ## conclusions we have demonstrated the utility of question decomposition as an effective means to probe pretrained multi-hop question-answering models for supporting evidence. through simulatability experiments, we show the effectiveness of this explanation form at allowing humans to predict model behavior, a sign that it helps humans to form an accurate mental model of the machine learning system @xcite . this ability to predict system performance occurs at the instance level instead of a sense of trust of the overall system, which can be important if the accuracy of the system is variable based on the question. our results indicate that explanations based on decompositional probes can be beneficial to users when the sub-questions are of reasonable quality. our analyses indicate that existing decomposition systems, however, have considerable room for improvement. we can now look at the state of research in decomposition systems not only as to whether they improve multi-hop question answering, but whether they provide users with more calibrated trust.