ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation

Authors: ana  brassard
benjamin  heinzerling
keito  kudo
keisuke  sakaguchi
kentaro  inui
  riken
unn  info contrastive
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 09:57:26

Abstract: Evaluating the quality of free-text explanations is a multifaceted, subjective, and labor-intensive task. Large language models (LLMs) present an appealing alternative due to their potential for consistency, scalability, and cost-efficiency. In this work, we present ACORN, a new dataset of 3,500 free-text explanations and aspect-wise quality ratings, and use it to evaluate how LLMs rate explanations. We observed that larger models outputted labels that maintained or increased the inter-annotator agreement, suggesting that they are within the expected variance between human raters. However, their correlation with majority-voted human ratings varied across different quality aspects, indicating that they are not a complete replacement. In turn, using LLMs as a supplement to a smaller group of human raters in some cases improved the correlation with the original majority labels. However, the effect was limited to cases where human raters were scarce, and an additional human rater had a more pronounced effect in all cases. Overall, we recommend against using LLMs as a complete replacement for human raters but encourage using them in configurations that end with targeted human involvement.

Main paper: 
 introduction: 
 Natural language processing systems that not only generate correct output, but also provide an explanation (Miller, 2019) of why that particular output is correct, are desirable for several reasons, such as increasing trustworthiness (Floridi, 2019) , compliance with "right to explanation" laws (e.g., European Parliament & Council of the European Union), increasing interpretability (Jacovi & Goldberg, 2020 , but cf. Lipton, 2018 on the caveats of post-hoc explanations), as well as system improvement and knowledge discovery (Adadi & Berrada, 2018) . However, this immediately raises the question of how to evaluate the plausibility of system-generated explanations in an efficient and effective manner.Since explanations are typically free-form text, automatic evaluation of explanations suffers the well-known, but as of yet unresolved, weaknesses of automatic evaluation measures (Celikyilmaz et al., 2021) , while human evaluation is characterized by low scalability, high costs, subjectivity, and inconsistency (Hartmann & Sonntag, 2022) . LLM-based evaluation presents an appealing alternative due to its potential high scalability, low cost, and consistency.Here, we investigate whether LLMs (Brown et al., 2020; OpenAI, 2023, i.a.) can serve as a viable alternative approach to automatically evaluate system-generated explanations. To verify the feasibility of this approach, we created ACORN, a new dataset of 3,500 textual explanations with aspect-wise human ratings of their quality, a first of its kind, and used it to evaluate whether LLMs are aligned with human judgments (Figure 1 ).Figure 1 : We collected aspect-wise human ratings for 3,500 textual explanations for commonsense reasoning benchmarks. We compared these against ratings from large language models (LLMs) to evaluate their alignment with human judgments. Specifically, we first considered whether LLMs' labels deviate from what would be expected from human raters. We compared inter-annotator agreement between all-human annotation (five raters) and when one of the raters is replaced by an LLM. We found that stronger models maintained or marginally increased inter-annotator agreement in most cases, suggesting that it may have potential as a replacement for human judgments.To verify this, we considered two scenarios: one where the LLM replaces the full human evaluation, and one where it is used only as an additional rater. In other words, we compared LLMs to humans collectively and individually.With the best-performing model, Spearman's rank correlation between majority-voted gold labels and LLM-generated ones ranged between 0.54 and 0.93, depending on the aspect, averaging 0.72. This indicates that the LLM's labels are not entirely in line with human judgments, but they are not entirely random either, and their usefulness may depend on the specific evaluation criteria and usage case.As for using the LLM as a single rater, we considered whether, in a limited-budget scenario, it would be beneficial to use an LLM as an additional rater. Specifically, we compared whether the majority-voted labels of a reduced set of raters and with an added LLM had a higher correlation with the original majority-voted labels. In cases with three or more raters, the addition of LLMs either had no effect or was detrimental. In cases with one or two raters, the addition of LLMs marginally improved the correlation with the original majority-voted labels, however, still less so than the addition of a human rater.In summary, we quantified the consequences of using LLMs as a replacement or addition to human evaluation. We conclude that they are an imperfect approximation of human majority votes but still within the expected variance that comes with a subjective task, with some potential benefit when human raters are scarce. Thus, we recommend against using LLMs as a complete replacement for human raters, but encourage using them in configurations that end with targeted human involvement, such as extensively using LLMs in development or filtering stages and human experts for final testing or evaluation. 
building acorn: evaluation criteria and data sources: 
 In a typical commonsense reasoning test, a model selects the most plausible answer for a multiple-choice question. In a predict-and-explain setting, the model additionally generates an explanation to justify the selected answer, where we encounter the challenge of evaluating these explanations. Thus, we first define a set of rating criteria ( §2.1) and collect human ratings for a selection of existing and newly collected textual explanations ( §2.2, §2.3). See Appendix A for more details on the dataset, including label distributions, data source-wise average ratings, and examples. 1 : Explanation rating criteria used in this study. The first two criteria target consistency and general explanation quality, while the rest covers specific quality aspects.We defined a set of criteria to target surface-level, information/content-level, and structural aspects of explanations. We also included criteria to capture (un)faithfulness and an overall rating, the latter intended to implicitly capture any other aspects considered by the raters. We defined these criteria based on common practices in natural language generation evaluation (Howcroft et al., 2020) , known challenges of free-text explanations (Lipton, 2018; Rawte et al., 2023) , and insights from social sciences (Miller, 2019) . Table 1 summarizes the criteria.The criteria largely aligns with the fine-grained analysis conducted by Wiegreffe et al. (2022) .Supports assesses which answer the explanation supports, intended to be cross-referenced with the predicted label. A mismatch between the predicted label and the supported answer indicates a lack of faithfulness, i.e., the explanation does not reflect the model's reasoning for the label.Overall is a holistic assessment of the explanation, capturing any potentially informative or useful aspects that we have not explicitly covered. We encouraged workers to consider this criterion independently of the other criteria, and provided general guidelines for each star rating to ensure a consistent understanding.Well-Written is a catch-all criterion to assess the surface-level quality of the explanation, combining criteria such as fluency, coherence, and grammaticality.Related assesses the relevance of the explanation to the question and answers.Factual evaluates the truthfulness of the statements in the explanation, if any, regardless of their relevance.New Information assesses the extent to which the explanation provides new information beyond the question and answers. Workers were given the choice of none for a complete lack of new information, some for a partial addition, sufficient for a satisfactory amount of new information, and ample for highly informative explanations.Unnecessary Information assesses the extent to which the explanation includes irrelevant information. We included this criterion to capture the challenge of generating minimal explanations.Contrastive assesses whether the explanation contrasts the correct answer with the predicted answer.ACORN contains ratings for a diverse set of existing, newly-collected, and generated explanations. Our choice covers two commonsense reasoning benchmarks and their respective explanation datasets (Figure 2 ). From each, we selected a random subset of 500 explanations for rating, as well as an additional 500 samples of fluency-improved versions, resulting in a total of 3,500 explanations. The fluency-improved subset is included to prevent fluency from becoming a superficial signal, since well-written explanations typically also have high scores in all other aspects. With five raters and eight criteria per sample, this amounts to 140k ratings in total. Specifically, as the target commonsense reasoning benchmarks, we selected BCOPA (Kavumba et al., 2019) 1 and CommonsenseQA (Talmor et al., 2019) based on the simplicity of their tasks and availability of large-scale explanation datasets (Wiegreffe & Marasović, 2021) . Below are the respective datasets we used to source candidate explanations.CoS-E (Rajani et al., 2019) A widely-used explanation dataset for CommonsenseQA, albeit notoriously uninformative to humans (Nauta et al., 2023) . A subset is processed through GPT-3.5 2 for fluency improvement (500 samples + 250 fluency-improved versions)ECQA (Aggarwal et al., 2021) An improved version of explanations for CommonsenseQA, aligning with our criteria for well-formed explanations. (500 samples)Generated explanations for CommonsenseQA. Additional high-quality explanations generated by prompting GPT-3.5 to solve a subset of CommonsenseQA, though potential issues like irrelevant information were noted. (500 samples)COPA-SSE (Brassard et al., 2022) Explanations for BCOPA with a subset processed through GPT-3.5 for fluency improvement. Since COPA-SSE already contains overall quality ratings, we selected a random sample of 250 questions and used each question's top-rated and bottom-rated explanation. (500 samples + 250 fluency-improved versions)Crowdsourced explanations for BCOPA. ECQA's counterpart for BCOPA; a new set of hand-written explanations, carefully crafted for contrastiveness and thoroughness. (500 samples)Generated explanations for BCOPA. Similarly to CoS-E, we prompted GPT-3.5 to solve BCOPA questions. (500 samples)We crowdsourced ratings for the explanations in ACORN using Amazon Mechanical Turk (AMT). Each rater was required to pass a qualification test, after which they were asked to participate in trial rounds, during which we addressed several clarity issues in the guidelines.The final pool was hand-picked based on their responses, resulting in 28 participants. See Appendix B for more details on the rating collection process. In experiments which use aggregated labels, they were produced with a majority-vote mechanism, with ties broken using the better label. 
can llms replace human raters?: 
 We answer this question in three steps: we first measure LLMs' divergence from expected human label variance ( §3.2), then observe the results when using it to completely replace hu-Figure 3 : Inter-annotator agreement (Krippendorff's α, * 100 for legibility) between human raters (shaded area) and with the LLM's rating replacing a random rater.man evaluation ( §3.3), and when using it as an additional rater instead ( §3.4). All experiments follow the settings described in Section 3.1.Models. We compared four contemporary API-enabled LLMs, namely GPT-4o (Ope-nAI, 2024), Llama-3.1 (405B) (Team, 2024) , Gemma-2 (27B) (Gemma Team, 2024), and Mixtral (8x22B) (Jiang et al., 2024) . Each have reported high performance in diverse tasks including text-based reasoning and represent the sate-of-the-art in generalist LLMs at the time of writing. Specifically, we used the following model versions: gpt-4o-2024-05-13, Meta-Llama-3.1-405B-Instruct-Turbo, gemma-2-27b-it, and Mixtral-8x22B-Instruct-v0.1.3 Temperature is set to 0.0 with all other parameters left at their default values.Prompting Strategy. LLMs are notoriously oversensitive to prompt format (Wadhwa et al., 2023) . For the purpose of our analyses, we explored several prompting strategies (listed in Appendix C) and selected the most successful one as measured by correlation with majorityvoted human ratings. Most models worked best with verbatim prompts corresponding to a word-by-word copy of the guidelines given to humans, to which they responded with a structured list of criteria and their assigned labels for the given target explanation.Label Extraction. Using free-text generation models for a classification task introduces the problem of extracting said ratings, and presents an information extraction challenge in itself. This phenomenon, inherent to generative approaches (Wadhwa et al., 2023) , is a source of additional noise that affects all evaluation pipelines necessitating a non-trivial solution in real applications. In our experiments, we used a rule-based extraction method backed up with LLM-based extraction in case of failure. Finally, we manually inspected the remaining failures,4 and excluded them from our experiments to maintain a fair comparison. The final extraction failure rates were <0.2% for all models but Gemma-2 (2.7%).In subjective tasks some degree of label variance is expected, resulting in lower interannotator agreement. This disagreement is not necessarily noise but a feature of the data, as it can reflect the diversity of human opinions (Aroyo & Welty, 2015) . In this context, regardless of absolute agreement, a successful LLM-based rater should be harmonious with the range of human labels rather than deviate from it.To measure this, we compared the inter-annotator agreement (Krippendorff's α) between human raters and when a random rater is replaced by an LLM. There are three possible outcomes: (i) agreement decreases, indicating that the LLM deviates from human judgments; (ii) agreement remains the same, indicating that it is harmonious with human judgments; or (iii) agreement increases, meaning that the LLM is both harmonious and biased towards a majority.5 Results. In Figure 3 , the shaded area shows the agreement between human raters, while the bars show the agreement when the respective LLM's ratings replace a random human rater ( * 100 for legibility). All values are averaged over twenty iterations. Mixtral, GPT-4o, and Llama-3.1 maintained or improved agreement in most cases, with slight decreases in supports with Mixtral, related with Llama-3.1, and with unnecessary information with GPT-4o and Llama-3.1. Gemma-2, on the other hand, decreased agreement in all but three criteria. However, the latter is also significantly smaller than the others, and illustrates the tradeoff between model size and performance. Interested readers may refer to Table 7 in the appendix for all results, including several older or smaller models for comparison.The larger models maintained or improved inter-annotator in most criteria, suggesting that they do not deviate from an expected range of human ratings. Here, we ask whether they can then replace human evaluation. Specifically, we measured the degree to which the models' predictions align with majority-voted human labels.Figure 4 shows Spearman's rank correlation ( * 100 for legibility) between aggregated human labels and LLM predictions. The highest values are annotated for each criterion. A comprehensive table with all results, including several smaller or older models, is available in the appendix (Table 8 ).The correlation in supports and contrastive was particularly strong: 0.84 and 0.93, respectively. The unnecessary information criterion, however, stands out with a much lower correlation in all models (0.54 and less). Others ranged from 0.62 to 0.76, indicating a moderately high correlation. Overall, GPT-4o was the best-performing model in five out of seven criteria, with an average correlation of 0.72. Mixtral, the second-best model, followed closely, particularly outperforming GPT-4o in unnecessary information and contrastive.From these results, we conclude that the larger models are relatively well-aligned with humans and could be considered effective depending on the use case. However, they are still clearly not a perfect replacement for human raters. Instead, considering the small change in inter-annotator agreement ( §3.2), the model could potentially be used as an additional rater when human annotation is scarce, which we explore in the next section.The results so far hinted towards LLMs acting similarly to an average human rater. Thus, it may seem appealing as an additional data point when human raters are scarce or expensive.Here, we verified this potential by measuring whether using each model as an additional Figure 5 : A comparison of Spearman's rank correlation with the original gold labels when using fewer raters (*H) and when an LLM is added as an additional rater (*H+LLM). From left to right, the number of human raters decreases from four to one (randomly selected). All values are multiplied by 100 for legibility.rater improved the outcome over having fewer human raters, i.e., whether the majorityvoted labels became more aligned with the original ones when the LLM was added as a rater.Specifically, we compared Spearman's rank correlation between the majority-voted labels with all available raters and in two alternative scenarios: one where the model is added as an additional rater and one where it is not. If the correlation increases when the model is added, it suggests that its predictions are in line with the original majority-voted labels, and it is useful as an additional rater. Otherwise, it would indicate a harmful or negligible effect, and thus its inclusion should be avoided. We repeated this comparison for scenarios with four, three, two, and one randomly selected human rater per sample.Figure 5 shows Spearman's rank correlation with the original gold ratings obtained by aggregating the labels of all five raters. Humans only (*H) denotes the correlation between human majority-voted labels only, and others when including the respective LLM as an additional rater (*H+LLM).Each column cluster represents a different number of human raters from four in the leftmost to one in the rightmost. With four humans, the correlation between their majority-voted labels and the original gold labels was 0.91. Adding a model as a fifth vote raised this by only 0.004 points. With three humans, we observed a slight decrease in correlation when adding a model. With two or one human rater, the correlation increased by 0.019 and 0.016 points, respectively.Overall, the results suggest that LLMs can be useful as additional raters when the number of human raters is less than three. However, even in the best case, the voted labels with an added LLM rater still scored lower than with an equivalent number of human raters (0.83 with 2H+LLM vs. 0.89 with 3H). When there is a high number of human raters, in this case three or more, the model's inclusion as an additional rater does not improve the majority-voted labels' alignment with the original gold labels, and may even harm it. 
discussion: 
 We analyzed the alignment between human raters and LLMs in the context of evaluating the quality of explanations. The task is highly subjective, and some degree of variance is expected even between humans.The best-performing models seemed to output labels that are within this variance, suggesting that they behave similarly to individual human raters. However, comparing its outputs to majority-voted labels revealed that their labels are not always in line with human majorities, indicating that they are not suitable as a complete replacement for human raters. In turn, when there were fewer human annotators, LLMs helped bring the majority-voted labels closer to the original gold labels. However, this improvement was not seen when there were already three or more human annotators (compared to the original five), suggesting that LLMs are only useful in extremely limited scenarios.From this, we concluded that LLMs are not a reliable replacement for human raters, unless the use case does not require complete alignment. However, in this work, we operated under the assumption that majority-voted labels are the ground truth. In practice, this may not always be the case, and the reliability of LLMs may vary depending on what information is desired in a given scenario. This opens up a new avenue for future work, where LLMs can be applied in a more nuanced manner, considering the context of the task and the intended use of the labels.Overall, as a balanced approach, we recommend using LLMs in configurations that end with targeted human involvement, such as extensively using LLMs in development or filtering stages and experts for final testing or evaluation. In future work, we plan to explore the behavior of LLMs more closely, particularly in identifying potential patterns in where the model diverged from humans, such as explanations with particular characteristics or in specific contexts. These insights could help us better understand the limitations of LLMs and how they can be used effectively in real-world applications or further improved. 
related work: 
 LLM-based Evaluation LLMs as data labelers, and more broadly humans-and-LLMs-inthe-loop settings, are an emerging direction in data collection and labeling. E.g., Wiegreffe et al. (2022) developed a predict-and-explain pipeline that combines GPT-3 with a supervised filter trained on binary acceptability judgments from humans. More recently, Chiang & Lee (2023) proposed using LLMs to evaluate text, closely in line with our work. They, however, found the models successful in their setting. In contrast, we focus on explanation rating with more complex fine-grained criteria, and closely scrutinize potential weaknesses; Previous works brought into question the reliability of LLMs' predictions, especially in prompting setups. E.g., Webson & Pavlick (2022) found that instruction-tuned models often produced good predictions with irrelevant or misleading prompts, bringing into question their real "understanding" of the task. Others reported unreliability of LLMs as labelers in various settings (Albrecht et al., 2022; Shen et al., 2023; Hada et al., 2024, i.a.) , a line of work which we join with findings in the context of explanation evaluation.Explanation Evaluation For explanations in the form of textual justifications, previous works often defined their own criteria for evaluation. Automatic evaluation borrowed from machine learning and measures overlap with "gold standard" text using (a) wordoverlap metrics, e.g., BLEU, METEOR and ROUGE; and (b) embedding-based metrics, e.g., BERTScore and BLEURT (Clinciu et al., 2021) . In contrast, human-tagged measures are more diverse and explanation-specific. For example, Clinciu et al. (2021) A recent study instead focused on the utility of explanations, i.e., their helpfulness in answering a question from a human point of view (Joshi et al., 2023) . In our work, we largely followed these existing criteria, with a modified version of the "supports" criterion to capture the consistency of a model's predicted label with its justification.Explain-and-Predict Explanations are often generated in predict-and-explain settings, where models provide justifications for their answers in QA-based benchmarks (e.g., Clinciu et al., 2021) , or an elaborate-then-predict setting, where models output a prediction guided by its intermediate outputs such as knowledge statements or reasoning chains (e.g., Marasović et al., 2022; Wang et al., 2023) . In this work, we evaluate the capabilities of LLMs to evaluate explanations in an explain-and-predict setting, where models provide justifications for their answers in commonsense reasoning benchmarks. The following sections provide background on each aspect. 
conclusions: 
 Using a newly-built dataset of free-text explanations and crowdsourced aspect-wise quality ratings, we analyzed the viability of LLMs as explanation evaluators. Stronger LLMs increased or maintained inter-annotator agreement when replacing human ratings, and their ratings were moderately to highly correlated with human ratings, depending on the quality aspect. In a scenario where LLMs were used as additional raters instead of a complete replacement, they improved the outcome when there were only two human raters, but were neutral to detrimental when there are three or more human raters. We conclude that while LLMs can provide ratings moderately consistent with an average human rater, they are not yet reliable enough for complete replacement. 
 
 Appendix: Explanation evaluation is inherently subjective, and the majority-voted gold label is not necessarily the "correct" answer. Subjectivity-informed scoring is a complex task, and we leave its exploration to future work. Furthermore, criteria may change depending on the intended use. Here, we limited our analysis to a researcher's point of view, where explanations are increasingly used as a diagnostic tool, and aligned our criteria to a list of general explanatory competencies. While our insights on LLMs' performance may potentially be applicable to different evaluation tasks, it should not be assumed to be universally true.LLMs are known to be sensitive to prompt format. We somewhat compensated for this by comparing a prompt-averaged setting, and our focus is not to search for the optimal setting but rather investigate potential fundamental issues with LLM-based evaluation. However, for practical applications, we acknowledge that there is a wealth of tweaks that may improve the performance. Nevertheless, our findings highlight the need for caution when using LLMs for explanation evaluation, and we hope that our work will inspire further research in this direction.Some of the commercial LLMs we used have built-in filters for potentially harmful content, which were triggered during several of our experiments. This data should not be used in any downstream applications without first controlling for potentially harmful samples.Crowdworkers play a vital role in dataset creation. We prioritized fair compensation, transparency in data usage, and respect for their rights and privacy. Workers were informed on their work's intended usage and of our identity as requesters. We did not collect any personal information. Table 3 : Label distributions per criterion of majority-voted human ratings. -1 denotes "none" for supports and "N/A" for factual.Table 4 shows the mean ratings and standard deviations of majority-voted ratings per data subset, excluding the categorical aspect supports. Data labels are described in Section 2.1. All ratings are the higher the better, except for unnecessary information which is the lower the better. Table 4 : Mean ratings and standard deviations per data subset. Higher is better for all criteria except for unnecessary information, marked with an asterisk (*), where lower is better.The human raters seemed to find generated explanations to be most well-written on average, however, the higher quality human-written explanations (ECQA, BCOPA crowdsourced) had a higher amount of new information. The generated explanations, in turn, had the least amount of unnecessary information. Interestingly, even though they were not contrastive, the generated explanations for CommonsenseQA (CSQA generated) had a higher average overall rating than ECQA explanations which are explicitly contrastive.A Random Forest Regressor, achieving a mean squared error of 0.37, deemed the most important predictive feature to be new information (58%), followed by factual (20%), unnecessary information (9%), well-written (7%), contrastive (4%), supports (2%), and related (0%). Note that for supports, we defined a binary feature of whether the label matches the answer label.Our crowdsourcing protocol for label collection consisted of three phases: qualification rounds, trial rounds, and main collection rounds. We provided detailed guidelines showing general instructions, detailed information on each criterion and their respective labels, three examples, and a FAQ section based on questions we received from workers. The full document is available upon request to the first author.In the qualification rounds, we curated a question set of 6 explanations and manually tagged them with "acceptable" answers, focusing on overall alignment rather than exact matches. We included a dummy question with strict instructions for filtering. Out of 700 participants, the top 201 workers, with a match percentage of 59% or higher, proceeded to trial rounds. We addressed any concerns or clarifications through email or form feedback. We hand-picked a final group of 28 workers. Qualifications were open to workers with a HIT approval rate of 99% or more and 5,000 or more approved HITs. Note that the location requirement was removed as it was an unnecessary barrier for highly skilled and motivated workers.Main Rounds Each of the 3,500 explanations in the test set ( §2.2) was rated by five workers. The ratings were aggregated to create the final gold labels used in our experiments. Figure 6 shows the crowdsourcing form.Payment Information For qualifications, each worker was compensated $0.15 per HIT.For the main rounds, the fee was increased to $0.25 per HIT, roughly matching a payment of $20.00 per hour.We compared single and compound calling, where the former prompts the model for a single criterion at a time, and the latter prompts the model for all criteria at once. We also compared default, averaged, and verbatim prompt formats, corresponding to a simple prompt with the explanation and the rating criteria, a voting mechanism over several prompt variants, and an input identical to the human annotation guidelines, respectively. For single verbatim calls, only the relevant sections (guidelines and examples) for the target criterion were included. Default and averaged prompts were further compared in zero-shot and three-shot settings, where the latter contained the same examples as shown in the human guidelines. All models were most successful with verbatim compound prompting. As a supplement to COPA-SSE, We collected 3,000 hand-written, detailed, and contrastive explanations for BCOPA. All crowdsourcing was conducted on the Amazon Mechanical Turk platform. 6 Generally, we found that strict qualifications, hand-picking the final worker pool, and maintaining open communication led to a significant increase in data quality.In the qualification rounds, workers were asked to write three explanations for an explanation, its mirrored sample,7 and another random explanation. 397 workers were hand-picked and whitelisted for further rounds based on the quality of their responses. We limited participation in this qualification to workers with a HIT approval rate of 99% or more, 1,000 or more approved HITs, and located in Great Britain or the United States.The workers were explicitly instructed to write contrastive explanations which "focus on what the key difference between the options is, and how it leads to it being the correct choice in one case and not in the other." This instruction was inspired by insights from Miller (2019) . Figure 7 shows the crowdsourcing form. We collected two explanations per BCOPA question, totaling 3,000 explanations. Out of these, a random sample of 500 explanations was rated and included in our test set. Example explanations can be seen in Appendix ??.Payment Information For qualifications, each participating worker was compensated $0.10 per HIT. For the main rounds, the fee was increased to $0.89 per HIT, roughly matching a payment of $20.00 per hour.Balanced COPA; a superset of COPA(Gordon et al.,2012) with added "mirrored" questions that flip the correct label, i.e., the originally incorrect choice becomes correct.2 text-davinci-003; The model was instructed to only improve the fluency and was not given any additional context that may encourage improving the content, e.g., by supplementing related information.We also provide the results of several smaller or older alternatives in the appendix for comparison (Tables7 and 8).Mostly due to non-compliance to the task format, e.g., responding verbosely with new labels instead of following the given choice: "... Related: Somewhat. ..." (instead of Yes or No)The latter may be desirable in use cases that rely on majority-voted labels as the ground truth, but comes with the trade-off of losing potentially useful label diversity.https://requester.mturk.com/With the same answer choices but a different question that makes the alternative correct. 


Decision: The paper investigates the problem of whether LLM can serve to evaluate free-text reasoning explanations. A dataset called ACORN, containing 3,500 examples with human ratings across eight different quality aspects, is introduced and used for experiments. They find that LLMs often deviate from human judgments, but GPT-4 shows some potential, particularly as an additional rater when human resources are limited.

Pros:
- The introduction of the dataset with 3,500 examples is valuable, and can be used in future research.
- The paper is well-written, clear, and easy to follow, with transparent and reproducible data collection and analysis procedures.
- The methodology is thorough and well-executed.

Cons:
- Some reviewers expressed concerns about the paper's narrow scope.

Reviews: review: paper_summary: paper_summary: This work studies the problem that whether LLM can serve to evaluate free-text reasoning explanations. First of all this work constructs a dataset with 3.5k examples, and by conduction experiments on this dataset, this paper reveals that the inter-annotator agreement is decreased when LLM-generated ratings are used to replace human ratings. More fine-grained results conclude that while LLMs can provide ratings moderately consistent with an average crowdworker, but they are not reliable enough for complete replacement.
strength_weakness: strength_weakness: Although this work studies whether LLM can be used for evaluating the quality of free-text explanations, the conclusions in this work are a bit weak. First of all the scale of 3500 examples is not large, and meanwhile, with the conclusion that LLMs are not reliable for explanation evaluation, how LLMs can be used for this task are less discussed in this work. For example, whether evaluating on each criteria and later gathering can help the performance.
questions: questions: 1. Why not also compare open-sourced LLMs like Mistral/Llama in this work? The numbers on the open-sourced models are also meaningful.
paper_summary: paper_summary: This paper evaluates the utility of LLMs for evaluating AI-generated free-text explanations of model predictions. In particular, the quality of the aggregate ratings is tested in various settings where a random human rater is replaced with a suitably prompted LLM. For the most part, LLMs reduce overall annotator agreement and drag the aggregate label distribution away from the majority vote, but one small exception is the case of GPT-4 breaking ties between two humans.

The contribution of the work consists of some commonsense QA explanation data, a pretty straightforward and sound evaluation paradigm for testing whether LMs add productively on top of human raters, and the results of experiments testing several LLMs.
strength_weakness: strength_weakness: The paper seriously conflates some important issues around explanation quality. The motivation provided in the introduction is about model trustworthiness, interpretability, compliance with "right to explanation" laws, but all of these use-cases of explanations depend on them being faithful to the model's decision making process, i.e., predictive of model behavior. The evaluations done in this paper have little bearing on faithfulness, as they only assess whether the explanation is coherent, correct, and supports the given answer. While explanation coherence is necessary for faithfulness (as incoherent explanations cannot give much information about how the model behaves), the other aspects being measured are about the quality of the explanation with respect to the _actual reason_ a label is correct. A model's ability to report a reason that supports an answer — even the correct reason — is neither necessary nor sufficient for faithfulness, which is about how the explanation relates to the model's decision making process, and cannot be inferred from a single example alone (see e.g. the discussion of "coherent biased reasoning" in https://arxiv.org/abs/2403.05518). Faithfulness, furthermore, is the aspect that is critical for all of the uses of explainability promoted in the intro. Given all of this, I think the paper is not really about explainability, but about evaluating model performance at a specific task, which is producing correct-with-respect-to-the-real-world explanations for classification decisions. I think an accurate presentation of the work would frame it in this way.

All of that said, this framing issue is not crucial to the contribution of the paper, and the experiments and evaluations all seem pretty reasonable to me. So I don't think this is a dealbreaker or anything, I just strongly urge the authors to rework the motivation to avoid any undue implications about the relevance of this work to model trustworthiness, reliability, interpretability. etc., and ask that the discussion of aspects in Table 1 avoid implying that the "Supports" or "Overall" aspects address faithfulness.

The main reason I can imagine this being rejected is that it's overall a pretty small contribution. I don't begrudge it this necessarily but it doesn't feel like it needed the full space it used. Thankfully, the authors did not unnecessarily pad it out to 9 pages, finishing within 8, but there's still quite a bit of fluff and repetition — I feel the paper could have probably fit just fine in 4 pages, and I think it would've made a great short paper. COLM doesn't have a short paper track, and of course this is the first COLM, so I don't really know how to handle this case. I'm recommending the paper for acceptance, as I think it's sound, but leave it to the meta-reviewer to judge.
questions: questions: General comment: I find the description of the data sources hard to follow. I don't know how to evaluate claims like "improved version" or "aligning with our criteria for well-formed explanations". These seem like subjective qualitative judgments. When describing the provenance of your data, please describe in concrete, non-subjective terms what was done to create it. Some things are also curious to me & unexplained — why did you have to post-process CoS-E and BCOPA for fluency improvement? Could this have introduced errors? Why were only half of them fluency-improved? Does this not also mean that some of your test examples are correlated, hurting your statistical power later on? A lot of things about the setup here seem messy. Can you please clarify in the paper how you processed the data and why?

Question: did you try few-shot prompting? I found a brief mention of it in Appendix D, but I couldn't figure out how it worked into the experiments and whether you tried it. It seems to me that you should probably be doing the best you can to get good performance out of these models and see if they work. Needing a few human labels for few-shot prompting doesn't weaken the case IMO, as you're already assuming access to some human labels anyway (e.g., when adding an LM to a pool of human raters).

In the ethics statement, you say:
> Some of the datasets used in our research have been found to contain biases that can be further perpetuated through explanations supporting them.

Which datasets? Please clarify this and provide citations if it's the case.
paper_summary: paper_summary: This paper examines the potential of large language models (LLMs) to automatically evaluate the quality of explanations in commonsense reasoning tasks. The authors introduce ACORN, a novel dataset of 3,500 explanations with human ratings across eight different quality aspects. The authors explore the capabilities of various LLMs to replace or supplement human evaluation, analyzing inter-annotator agreement and correlation with gold-standard labels. They find that LLMs often deviate from human judgments, but GPT-4 shows some potential, particularly as an additional rater when human resources are limited.
strength_weakness: strength_weakness: While the paper focuses on a topic that may interest a narrow audience (evaluating explanations), its contributions are clearly and concretely articulated. However, I have one minor concern. The datasets used are of varying quality, yet the analysis encompasses all datasets uniformly. It would have been advantageous if the authors had conducted analyses on subsets of the datasets to determine if LLMs can be more effective with texts of particular quality levels. Additionally, the purpose of including fluency-improved versions in the datasets was not clear. Clarification on this aspect would be helpful.

Comments: comment: Thanks for your response. I am still not convinced by the LLM evaluation. But I'll raise my score a bit.