Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions (2022)

Abstract: Current QA systems can generate reasonable-sounding yet false answers without explanation or evidence for the generated answer, which is especially problematic when humans cannot readily check the model’s answers. This presents a challenge for building trust in machine learning systems. We take inspiration from real-world situations where difficult questions are answered by considering opposing sides (see Irving et al., 2018). For multiple-choice QA examples, we build a dataset of single arguments for both a correct and incorrect answer option in a debate-style set-up as an initial step in training models to produce explanations for two candidate answers. We use long contexts—humans familiar with the context write convincing explanations for pre-selected correct and incorrect answers, and we test if those explanations allow humans who have not read the full context to more accurately determine the correct answer. We do not find that explanations in our set-up improve human accuracy, but a baseline condition shows that providing human-selected text snippets does improve accuracy. We use these findings to suggest ways of improving the debate set up for future data collection efforts.

## related work prior work has explored using models to generate explanations @xcite @xcite , but there is limited work on using those explanations to verify the model's prediction, particularly when a human cannot perform the task directly. such a dataset would be useful, as model explanations can aid humans in tasks such as medical diagnosis @xcite , data annotation @xcite and deception detection @xcite . however, @xcite highlight that these studies use models that outperform humans at the task in question, undermining the motivation for providing a model's explanation alongside its prediction. when the performance of models and humans is similar, current explanation methods do not signifcantly help humans perform tasks more accurately @xcite . however, explanations based on a mental model of the human's predicted actions and goals can reduce task completion time @xcite . we address these shortcomings by collecting data for training models to provide explanations on tasks that would otherwise be time-consuming for humans. in addition to task characteristics, several qualities of the model explanation affect the helpfulness of human-ai collaboration: machine-generated explanations only improve human performance when the explanations are not too complex @xcite . and though users want explanations of how models mark answers incorrect, most explanations that models output focus on the option selected @xcite . our dataset addresses this by including evidence and explanations for both correct and incorrect options to each question, enabling models trained on it to present arguments for more than one answer. ## argument writing protocol we build a dataset of qa (counter-)explanations by having human writers read a long passage and construct arguments with supporting evidence for one of two answer options. we then present the explanations side-by-side to a human judge working under a strict time constraint, who selects which answer is correct given the two explanations. passage and question selection we use passages and questions from a draft version of the recent long-document qa dataset, quality @xcite . in quality, most passages are science fction stories of about 5k words with 20 fouroption multiple-choice questions. we determine which of the three incorrect options is best suited to have a convincing argument by identifying cases where (i) humans in a time-limited setting incorrectly selected that choice at least 3/5 times, and/or (ii) humans who read the entire passage selected that choice as the best distractor item more than half the time. we discard questions without an incorrect answer option meeting either criteria. writing task we recruit 14 experienced writers via the freelancing platform upwork (writer selection details are in appendix a). we assign each writer up to 26 passages. each passage has 7-15 2-option multiple choice questions (avg. of 13.3). we have writers construct an argument (max 500 characters) and select 1-3 supporting text snippets (max 250 characters) for one of those two options ( we encourage writing effective arguments by awarding writers a bonus each time a worker in the judging task selects the answer they wrote an argument for. including bonuses, workers average $21.04/hr, after taking upwork fees into account. further details are in appendix a, and a description of the writing interface is in appendix b. final dataset we release a dataset of both correct and incorrect arguments with selected text snippets and the results of the judgment experiment as a tool for researchers. these datasets are available at github.com/nyu-mll/single_turn_debate. as we use passages from a draft version of quality, we do not release arguments from passages in their non-public test set. the fnal dataset that we release contains 2944 arguments (50% correct) from 112 unique passages, each with an average of 2.4 text snippets. ## judging protocol we test the effectiveness of the arguments by having human judges answer the multiple-choice question. to ensure that the judges cannot simply read the passage to fnd the answer themselves, we give them only 90 seconds of access to the passage along with the arguments and text snippets. to determine question: what clearly showed a sense humbleness presented by si? correct option: his lack of awareness that he would be incorrect option: his quaint behavior at the banquet considered a celebrity at the kudos room. ## discussion we fnd it likely that explanations will be benefcial to users in some tasks under some conditions. the prevalence of a debate-style set up in real-world settings (e.g., courtrooms foot_1 ) makes this an a priori reasonable area for systematic exploration, but the current study is limited in its scope and is not strong evidence against the broad potential usefulness of such a set-up. the current experiments are a case study in creating a scenario where humans are unable to be sure about their answer, but they have access to evidence to help identify the correct response. the fnding that a quarter of workers mistrusted the arguments raises the issue of whether an approach that gives users misleading information from the outset is on the wrong track. however, we already know qa models provide false and misleading information; this behavior has the potential to be more harmful when it is not explicit that generated explanations may be wrong. one reason that the arguments were more misleading than helpful to some workers could be that the correct and incorrect arguments were independent of each other. the strength of debate for determining the true answer could rely on counterarguments that explicitly reference defciencies of the other argument. it is therefore possible that a multi-turn setting is needed for debate to be helpful, but we leave this as a question for future research. the time limit that we use makes the task more artifcial than we'd like. however, pilot results (appendix f) show that variations between 60 and 120 seconds make virtually no difference in performance. it is possible that 120s is still too short, and so workers rushed through the task as much as they did with 60s, but we would have expected this to vary more by worker, and the general trend is that people are slightly less accurate at 120s than at 90s.