Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations (2022)

Abstract: Recent work on explainable NLP has shown that few-shot prompting can enable large pre-trained language models (LLMs) to generate grammatical and factual natural language explanations for data labels. In this work, we study the connection between explainability and sample hardness by investigating the following research question – “Are LLMs and humans equally good at explaining data labels for both easy and hard samples?” We answer this question by first collecting human-written explanations in the form of generalizable commonsense rules on the task of Winograd Schema Challenge (Winogrande dataset). We compare these explanations with those generated by GPT-3 while varying the hardness of the test samples as well as the in-context samples. We observe that (1) GPT-3 explanations are as grammatical as human explanations regardless of the hardness of the test samples, (2) for easy examples, GPT-3 generates highly supportive explanations but human explanations are more generalizable, and (3) for hard examples, human explanations are significantly better than GPT-3 explanations both in terms of label-supportiveness and generalizability judgements. We also find that hardness of the in-context examples impacts the quality of GPT-3 explanations. Finally, we show that the supportiveness and generalizability aspects of human explanations are also impacted by sample hardness, although by a much smaller margin than models.

## introduction prior work on explainable nlp @xcite has explored different forms of explanations ranging from extractive rationales @xcite , semi-structured, and structured explanations @xcite @xcite to free-text explanations @xcite . due to the flexibility of free-text explanations, they have emerged as a popular form of explanations with multiple benchmarks developed around them, as well as models that generate such explanations using seq2seq language models @xcite @xcite . few-shot prompting @xcite with large language models (llms) like gpt-3 @xcite has been shown to produce highly fluent and factual natural language explanations that are often preferred over crowdsourced explanations in existing datasets @xcite . however, past work has not yet explored a critical dimension of datapoint-level explanations, which is how hard the data point is to classify correctly. given recent work on measuring hardness of individual data points @xcite , we study how sample hardness influences both llms' and humans' ability to explain data labels. in summary, we are interested in investigating the following three research questions: 1. rq1. do llms explain data labels as well as humans for both easy and hard examples? 2. rq2. how much do llm explanations vary based on the size and the hardness of the retrieval pool for choosing in-context samples? 3. rq3. are humans equally good at explaining easy and hard examples? as a case study, we investigate these questions for a classical commonsense reasoning task, winograd schema challenge @xcite on a large-scale dataset, winogrande @xcite (examples in fig. 1 ). we first collect generalizable rule-based explanations from humans like "if x is larger than y, then x does not fit in y.". to measure data hardness, we use data maps @xcite , an approach based on the training dynamics of a classification model. ## method and experimental setup our method first estimates hardness of the samples using data maps @xcite and then chooses a subset of easy, medium, and hard examples, for which we collect human-written explanations and generate explanations from a state-of-the-art model. next, we answer our research questions by comparing the explanations against multiple granular evaluation axes. data maps. we estimate sample hardness via a model-based approach foot_1 called data maps @xcite . data maps characterize points x i in a dataset along two dimensions according to a classifier's behavior during training: (1) confidence μi which measures the mean model probability of the true label y * i across e epochs, and (2) variability σi which measures the standard deviation of the model probability of the true label across epochs. explanations for winograd schema. next, we define the structure of explanations for the winograd schema challenge @xcite . specifically, these are semi-structured if-then commonsense rules as shown in fig. 1 . this characterization of explanations allows us to (1) capture generalizable commonsense knowledge via placeholders x (and y) capable of explaining a number of similar data points, (2) enforce the common structural form of an if-then rule for all data points in this task, while still maintaining the flexibility of free-text explanations (see fig. 1 for some examples), (3) ensure non-trivial explanations that do not leak the label @xcite , with the aim of avoiding explanations that only repeat the label without providing generalizable background knowledge (a common issue in past explanation datasets), (4) evaluate explanation properties with reduced human subjectivity due to their semi-structural form. ## results in fig. 2 , we compare the human and gpt-3 explanations for easy, medium, and hard foot_2 examples along the three axes. we observe that gpt-3 is not only able to learn the if-then structure of the explanations but also matches humans in terms of generating grammatically fluent explanations, regardless of the sample hardness. for easy examples, gpt-3 explanations are almost as supportive of the label as human explanations, sometimes even outperforming humans. however, humans are typically better at writing more generalizable explanations that apply to broader contexts (see examples 1-3 in fig. 1 ). for hard examples, gpt-3 often fails to generate sufficiently supportive explanations and hence significantly underperforms humans in more than 46% of the cases (see examples 4-6 in fig. 1 and appendix d for some common errors). this, in turn, also hurts the generalizability aspect of the model-generated explanations. medium-hard examples show a trend similar to easy examples because their confidence values are much closer to the easy examples than the hard ones. significance testing. pertaining to the above results, we further use a non-parametric bootstrap test @xcite to evaluate whether the human win-rate differs significantly from the model win-rate, while treating ties as neutral. we encode human wins as 1, model wins as -1, and ties as 0 and test whether the average score is not equal to 0 (meaning that the win-rate differs between human and model). in summary, for easy and medium samples, humans' generalizability is significantly better than the model's (difference in win rate is 20 points with @xmath0.001), while for hard samples, both humans' generalizability and supportiveness are better than the model's (differences in win rates are 0.38 and 0.4 respectively, with @xmath1. next, for grammaticality, we test if gpt-3 explanations matches human explanations within a win-rate threshold @xmath2. for a threshold of @xmath3.1 (testing that grammaticality win-rates are within 10 percentage points of each other), we obtain @xmath4.05, and a threshold of @xmath5.15 yields 14%) from evaluation. we investigate rq2 by conducting two experiments in which we compare the explanations generated by gpt-3 for 100 easy examples. in the first, we vary the size of the retrieval pool (rp) for selecting in-context examples from 400 to 200 while keeping the average hardness constant, and in the second, we vary the hardness of the retrieval pool from easy to hard examples with the size of the pool (400) remaining constant. as shown in fig. 3 , the grammaticality of the explanations is unaffected in both experiments. however, supportiveness drops when the in-context samples are retrieved from a smaller pool. a larger pool increases the likelihood of having more similar in-context examples to the test sample, and we conclude that similar in-context examples improve the supportiveness of the explanation. we also find that when explaining easy examples, having a retrieval pool of similar easy examples helps the model generate better explanations, possibly because of more similar in-context examples. combining with rq1, we conclude that hardness of both in-context and test samples can affect the quality of model explanations. we also conduct a similar study for comparing the explanation quality of 100 hard test examples by varying the hardness of the retrieval pool. in contrast to easy test examples, we do not observe statistically significant differences in explanation quality for hard examples when the retrieval pool's hardness is varied. in particular, with respect to supportiveness, the win percentages for hard and easy pool are 20% and 18% respectively, with remaining 62% being ties, while for generalizability, they are 33% and 25% respectively, with remaining 42% being ties. we believe that the quality of hard examples may not be sensitive to changes in the in-context examples simply because the corresponding explanations are not very good to begin with. ## related work there has been significant progress made in recent years on both curating natural language explanation datasets @xcite @xcite , inter alia) as well as generating them @xcite . related to the winograd schema challenge, winowhy (zhang et al., 2020) contains explanations only for the wsc273 test set @xcite and does not follow the structure of our commonsense rule-based explanations, thereby leading to label leakage. label leakage makes evaluation of explanations harder because supportiveness can become trivial. our study builds on top of prior works that also generate free-text explanations using in-context learning with gpt-3 @xcite . however, our novelty lies in investigating the connection between explainability and sample hardness. a number of concurrent works have also explored free-text explanations for incontext learning in various reasoning tasks @xcite @xcite @xcite , primarily focusing on improving model performance with explanations and not evaluating explanation properties or factors that might influence them. ## conclusion we studied the effect of sample hardness on the quality of post-hoc explanations generated by llms for data labels. we concluded that while llm explanations are as fluent as human explanations regardless of the sample hardness, humans are typically better at writing more generalizable explanations and specifically, for hard examples, human explanations are also more supportive of the label. factors like the hardness and size of the retrieval pool for choosing in-context examples can further impact the explanation quality. we also observe that the generalizability aspect of human explanations drops for harder examples, although by a smaller margin than models.