Reframing Human- AI Collaboration for Generating Free-Text Explanations (2022)

Abstract: Large language models are increasingly capable of generating fluent-appearing text with relatively little task-specific supervision. But can these models accurately explain classification decisions? We consider the task of generating free-text explanations using human-written examples in a few-shot manner. We find that (1) authoring higher quality prompts results in higher quality generations; and (2) surprisingly, in a head-to-head comparison, crowdworkers often prefer explanations generated by GPT-3 to crowdsourced explanations in existing datasets. Our human studies also show, however, that while models often produce factual, grammatical, and sufficient explanations, they have room to improve along axes such as providing novel information and supporting the label. We create a pipeline that combines GPT-3 with a supervised filter that incorporates binary acceptability judgments from humans in the loop. Despite the intrinsic subjectivity of acceptability judgments, we demonstrate that acceptability is partially correlated with various fine-grained attributes of explanations. Our approach is able to consistently filter GPT-3-generated explanations deemed acceptable by humans.

## introduction as natural language understanding tasks have become increasingly complex, the field of explainable nlp has embraced explanations written in freeform natural language. in contrast to extractive explanations that highlight tokens in the input, freetext explanations provide a natural interface between machine computation and human end-users @xcite . the dominant paradigm for producing free-text explanations is via direct supervision, i.e., training an autoregressive, generative language model to predict human-authored explanations directly @xcite @xcite @xcite . 1 ). authors of this work write explanations to prompt gpt-3, generating 5 explanations per instance. an acceptability filter, trained on human binary acceptability judgments, determines which of these generated explanations are plausible. evaluation is performed at both the explanation and the instance level. however, collecting high-quality written explanations to serve as supervision is difficult and expensive. more than 70% of existing free-text explanation datasets are crowdsourced @xcite , and even the most meticulous crowdsourcing efforts frequently fail to elicit logically consistent and grammatical explanations @xcite . furthermore, a lack of standardized crowdsourcing design has resulted in highly varied datasets, which are hard to compare or combine @xcite . recent progress in prompting large language models (llms) provides a potentially promising alternate to large-scale crowdsourcing. the incontext learning paradigm, wherein powerful language models are prompted in a few-shot manner with just a few examples, has proven surprisingly effective across a range of nlp tasks @xcite @xcite . in this work we ask: can llms also generate reliable explanations? in human subjects studies, we find that gpt-3 @xcite can be readily made to generate explanations via prompting, and surprisingly, humans often prefer gpt-3 generated explanations to crowdsourced explanations in existing datasets ( §2). two additional human subjects studies, however, demonstrate that gpt-3-generated explanations still have significant room for improvement along axes such as providing new information (i.e., avoiding repetition) and supporting the label; human subjects found less than half of greedily-decoded gpt-3-generated explanations to be acceptable with 100% agreement. to improve upon this, we re-frame the role of crowd annotators: instead of asking them to write explanations as in prior work, we (1) repeatedly query gpt-3 to generate multiple candidate explanations for each input instance, and (2) ask crowdworkers to rate the acceptability of each candidate generation. after showing that gpt-3 can usually generate an explanation that humans unanimously find acceptable within as few as five queries ( §3), we use a small number of these binary crowdworker judgments to supervise an acceptability filtering model, which can be applied to select high quality candidates among gpt-3's outputs (figure 1 ; §4). despite intrinsic subjectivity in acceptability ratings, our supervised model improves upon the already-competitive few-shot paradigm by consistently selecting (human-identified) high quality explanations better than strong baselines. human evaluations reveal that the filtration model not only improves acceptability, but also other axes like supporting the label and providing novel information. in summary, our main findings are: i. few-shot prompting with gpt-3 produces surprisingly competitive explanations, providing a promising alternative to crowd-authored freetext explanation corpora; ii. binary human labeling can instead be leveraged to train a filter that selects high-quality machinegenerated explanations; and iii. in areas where gpt-3 struggles, including information content, supporting the label, and overall acceptability, our proposed overgenerate-andfilter pipeline improves generated explanations. we publicly release our code and data. foot_0 ## gpt-3 is competitive with crowdsourced explanation datasets we investigate three research questions: our second task is nli, which involves inferring whether a given hypothesis sentence entails, contradicts, or is neutral towards a premise. this task is instantiated with the snli dataset @xcite and crowdsourced explanations from e-snli @xcite . for each task, we report results on a fixed, randomly-sampled 250-instance test set not observed during prompt design. few-shot prompting for explanations. we use gpt-3 davinci foot_1 @xcite , an autoregressive language model with ~175b parameters trained on a large dataset of text scraped from the internet. we prompt the model with several (question, gold answer, explanation) triplets followed by an unexplained question-gold answer instance for @xcite premise: dark-haired man wearing a watch and oven mitt about to cook some meat in the kitchen. hypothesis: a man is cooking something to eat. label: entailment e-snli @xcite : meat is cooked in a kitchen, and is a food that you eat. using an oven mitt implies you're about to cook with hot utensils. gpt-3: cooking is usually done to prepare food to eat. commonsenseqa @xcite question: what is the result of applying for job? answer choices: anxiety and fear, increased workload, praise, less sleep, or being employed correct choice: being employed cos-e (rajani et al., 2019): being employed applying for job ecqa @xcite : applying for a job is followed by attending interview which results in being employed. applying for a job may not result in the other options. gpt-3: applying for a job can result in being employed, which is a positive outcome. ## beyond greedy explanations while gpt-3 explanations demonstrate strength across surface-level features and are surprisingly competitive in head-to-head settings, they can still be improved. one might imagine a setting with multiple end-users in which we wish to provide the most unambiguously acceptable explanation as output from a system. when considering the data from §2.3, we find that only 46.3% of the greedily-decoded gpt-3-generated explanations for commonsenseqa and 31.5% for nli are rated acceptable by 3/3 annotators. 8 inspired by work in other generation tasks @xcite @xcite , we hypothesize that equally or more informative explanations can be generated by sampling stochastically. we sample 4 additional generations from gpt-3 for each instance to complement the greedy generation. we crowdsource 3 acceptability annotations for each new explanation. as expected, sampled explanations exhibit lower 3/3 acceptability than greedy explanations (25.1% for commonsenseqa; 11.3% for snli). however, this results in a surprisingly higher proportion of instances that have at least one acceptable explanation in the set of 5. the greedy explanation was judged to be 3/3 acceptable in 46.3% of instances for commonsenseqa and 31.5% for nli; this increases to 79.5% and 51.2%, respectively, when sampled explanations are included. foot_8 ## improving explanation generation with acceptability filtering the challenge of overgeneration is that gpt-3 alone cannot discern which of its stochastic samples are acceptable. inspired by @xcite , we explore training a supervised filter on the collected labels. our key intuition is that by re-framing the role of annotators from explanation authors to binary judges, we can alleviate the need to collect a large-scale explanations dataset-the result is a simpler, cheaper, and easier crowdsourcing setup to administer ( §4.1). namely, we can (1) aggregate ratings over multiple annotators to produce more reliable labels, (2) use numerical metrics of annotator agreement to remove annotators contributing noisy data, and (3) collect annotations more quickly and cheaply than asking annotators to hand-write explanations. moreover, we find that the filter can be trained with a relatively small amount of binary human judgments ( §4.2). we demonstrate that the trained model is not simply taking advantage of surface-level features ( §4.4). figure 1 presents an overview of our pipeline. ## related work free-text explanation generation. the earliest neural free-text explanation models did so for computer vision applications @xcite @xcite and nli @xcite . these methods relied on supervised datasets to train the explanation generator. others have proposed to generate explanations or clarifications to improve task performance in a supervised (rajani et al., 2019; lampinen et al., 2022) or unsupervised (shwartz et al., 2020) manner. yordanov et al. (2021) study transfer learning between datasets for few-shot generation. latcinnik and berant (2020) proposed a method to generate free-text explanations supervised only on task signal, and brahman et al. (2021) used sources of weak supervision to generate explanations for defeasible inference. paranjape et al. (2021) design hand-crafted templates which they use with mask-infilling to produce contrastive explanations from pretrained language models. concurrent work @xcite studies the effect of prompt format and model size on crowdworker judgements of prompted explanation plausibility. they find that gpt-3 davinci outperforms other smaller pretrained models, but that crowdworkers find these explanations less plausible than those from the datasets, aligning with our first experimental result ( supervising on human preferences. prior and concurrent work has used binary judgements from crowdworkers to fit models to human preferences for non-xai tasks such as summarization @xcite , creating commonsense knowledge bases @xcite , and building natural language inference datasets @xcite . unlike these works, we apply human preference modeling to increase the human acceptability of model-generated free-text explanations. @xcite demonstrate that gpt-3 + a supervised acceptability filter can generate a high-quality causal knowledge graph: in addition to their work being conducted in a different domain, our success conditions and evaluation metrics differ because we must produce a prediction for each instance (whereas they can simply discard bad generations). ## conclusion we demonstrate gpt-3's capacity to generate freetext explanations for nlp task instances in a fewshot setting. we further improve this capability via an overgenerate + filter approach, where the filter is trained on supervision from human acceptability ratings. we hope our results can guide future work on free-text explanations via neural or neurosymbolic systems @xcite @xcite . future work may also further investigate the benefits of counterfactual explanations. while human rationales for decision making are not necessarily the same as model rationales, the goal behind modeling human acceptability is often to build trust with a human user. this trust may or may not be warranted @xcite ; future work would be well-suited to further investigate generated explanations for incorrect label predictions such as in appendix c, which could mislead end users.