Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations (2023)

Abstract: Human-annotated labels and explanations are critical for training explainable NLP models. However, unlike human-annotated labels whose quality is easier to calibrate (e.g., with a majority vote), human-crafted free-form explanations can be quite subjective. Before blindly using them as ground truth to train ML models, a vital question needs to be asked: How do we evaluate a human-annotated explanation’s quality? In this paper, we build on the view that the quality of a human-annotated explanation can be measured based on its helpfulness (or impairment) to the ML models’ performance for the desired NLP tasks for which the annotations were collected. In comparison to the commonly used Simulatability score, we define a new metric that can take into consideration the helpfulness of an explanation for model performance at both fine-tuning and inference. With the help of a unified dataset format, we evaluated the proposed metric on five datasets (e.g., e-SNLI) against two model architectures (T5 and BART), and the results show that our proposed metric can objectively evaluate the quality of human-annotated explanations, while Simulatability falls short.

## introduction despite the recent advances of large-scale language models (llm) @xcite @xcite , which exhibit close-to-human performance on many natural language processing (nlp) tasks (e.g., question answering @xcite @xcite @xcite , natural language inference @xcite @xcite , and text generation @xcite @xcite ), humans are eager to know how state-of-the-art (sota) models arrive at a prediction. researchers working on natural language explanations foot_0 turned to human annotators for help by recruiting crowd-workers or experts to annotate both the labels and corresponding natural language explanations @xcite @xcite ; researchers can thus leverage human-annotated explanations to boost models' prediction performance or train models to generate human-understandable natural language explanations. however, the quality issue of human-annotated explanations has yet to be explored. researchers often leverage popular natural language generation (nlg) metrics such as bleu @xcite and rouge @xcite to evaluate the similarity between model-generated and human-annotated explanations, with a strong assumption that humanannotated ones are the gold standard. nevertheless, unlike providing labels for classification or multiple-choice qa tasks @xcite , different people may come up with distinct natural language explanations for the same observation @xcite . two such explanations can be both correct even though the bleu or rouge similarity may be low. furthermore, human-given natural language explanations can often be subjective and task-dependent @xcite . as a result, human-annotated explanations should not be simply treated as the gold standard @xcite ; instead, we take the view that the core value of explanations should be based on how much help they provide towards the model prediction instead of being based on notions of semantic similarity or word-matching. to summarize our contributions in this paper: 1. we provide an objective evaluation to quantify the human-annotated explanations' helpfulness towards model performance. our evaluation metric is an extension of the simulatability score @xcite and we propose a prompt-based unified data format that can convert classification or multiple choice tasks into a unified multiple choice generation task format to minimize the influence of structural variations across different tasks. ## unified structure while popular metrics like bleu and rouge can evaluate text coherence and similarity, one critical aspect of explanations is how beneficial they can be. thus, we want to develop a metric that objectively evaluates explanations' utility towards model performance. furthermore, we expect that such a metric can systematically demonstrate how good or bad the explanations are; for example, it could objectively measure what 'noisy' means in a human study (e.g., from previous works on cos-e). with the advantage of sequence-to-sequence models like t5 that can map different types of language tasks into generation tasks, we can control and minimize the influence of varying task formats on model performance while evaluating the helpfulness of explanations by leveraging a unified data format. we realize that existing datasets with human-annotated explanations are mostly either multiple-choice tasks or classification tasks. the classification task could be viewed as a multiplechoice task where the labels are indeed choices. inspired by several previous works that manipulated prompts for sequence-to-sequence models @xcite , we incorporate a few well-defined words as template-based prompts for the unified data structure to indicate the task content and corresponding explanations. examples shown in figure 1 explain how we map various tasks into a unified multiple-choice generation task. we propose two settings: no explanations (baseline ) and explanations as additional input (infusion ). here we explain how each prompt addresses a different part of the data content: 1) 'explain:' is followed by the question content, 2) 'choice-n:' is followed by each candidate answer, and 3) a special token '<@xmath0, while the explanations in infusion are led by 'because' so that the model knows that the explanation text explains the task content. for datasets like cos-e and ecqa, we leverage the original task as the question content. on the other hand, we define fixed question prompts for e-snli: "what is the relation between [premise] and [hypothesis]?", and for comve: "which sentence is against commonsense?" to specify corresponding tasks to models. we fine-tune three t5-base models on each dataset with three different settings: baseline , infusion , and explanations as additional output (self-rationalization hereinafter). for each model, we maintain the same setting during fine-tuning and inference. for example, the model fine-tuned with infusion will also take data under infusion during inference. we leverage the unified structure for baseline and infusion shown in figure 1 and make minor adjustments for the self-rationalization setting accordingly (shown in appendix a). the experiment results are shown in ## conclusion in this paper, we objectively evaluate humanannotated natural language explanations from the perspective of measuring their helpfulness towards models' prediction. we conduct two preliminary experiments and based on the findings from the preliminary study, we define an evaluation metric that considers the explanations' helpfulness at both fine-tuning and inference stages; we also propose a unified prompt-based data format that minimizes the influence of task differences by mapping various tasks into a unified multiple-choice genera-tion task. our experiment with human-annotated explanations in five popular large-scale datasets over two sequence-to-sequence model architectures demonstrates that our metric can consistently reflect the relative ranking of explanation qualities among five datasets while the simulatability score falls short. our work lays a stepstone towards a high-quality human-ai collaboration future for data annotation job @xcite , and we recommend researchers perform similar quality checks while collecting human-annotated explanations in the future.