How (un)faithful are explainable LLM -based NLG metrics? (2025)

Abstract: Explainable NLG metrics are becoming a popular research topic; however, the faithfulness of the explanations they provide is typically not evaluated. In this work, we propose a testbed for assessing the faithfulness of span-based metrics by performing controlled perturbations of their explanations and observing changes in the final score. We show that several popular LLM evaluators do not consistently produce faithful explanations.

## introduction since large language models (llms) that exhibit strong instruction-following abilities became available, the nlg community has increasingly adopted llms as automatic text evaluators @xcite . these llm-based metrics often achieve good correlations with human judgments, even without using human-written references, and enable the evaluation of customisable aspects of generation quality @xcite . moreover, a growing number of these metrics aim to enhance interpretability by providing explanations alongside their scores, ranging from short rationales to rich, multi-component analyses @xcite . one particularly promising approach involves metrics that provide span-based annotations and explanations @xcite @xcite . these systems highlight specific spans of the generated text where aspect-related errors occur, assess the severity of these issues, provide explanatory comments, and ultimately synthesize this information into an overall quality score. such granular feedback is potentially valuable for system debugging, building user trust and supporting human annotators @xcite . despite these advances, evaluation of explainable nlg metrics has largely focused on measuring the correlation with human judgments @xcite @xcite . only a few studies have gone beyond this to assess metrics' robustness to perturbations @xcite , evaluate biases towards llmgenerated outputs @xcite or inspect the quality of the explanations themselves @xcite . crucially, existing evaluations of explanation quality typically rely on human judgments of plausibility, i.e., whether the explanation seems reasonable or convincing to a human @xcite @xcite . this, however, overlooks a key dimension of explanation quality: faithfulness. a good explanation should satisfy two core properties: plausibility, meaning it is understandable and convincing to humans, and faithfulness, meaning it accurately reflects the reasoning process or internal behavior of the system it explains @xcite . without an assessment of explanation faithfulness, we cannot determine whether these metrics truly provide insight into their computation of the score or merely generate plausiblesounding but potentially misleading rationales. in this paper, we address this critical gap by proposing comprehensive experiments specifically designed to evaluate the faithfulness of explainable, span-based nlg metrics. our contributions are threefold: first, we introduce a novel testbed foot_0 for systematically measuring the faithfulness of explanations provided by nlg metrics, based on perturbing the explanations and measuring changes in the final overall score. second, we conduct extensive experiments evaluating the explainability of three state-of-the-art llms commonly employed in llm-as-a-judge frameworks. third, our empirical findings reveal surprisingly low faithfulness of llm-based metrics, raising concerns about their interpretability and opening new avenues for future research in trustworthy nlg evaluation. ## related work there are only a few works that evaluate llmbased nlg metrics beyond correlations with human judgments and, occasionally, the plausibility of explanations provided. @xcite used pairs of human-and llmwritten summaries to show that an llm metric (g-eval) prefers llm summaries over human summaries, regardless of their quality. @xcite performed an analysis of closed-source llms using their evaluations of synthetically prepared pairs of generated texts. the experiments revealed that llms tend to favour outputs presented in a specific position in the prompt and favour longer or self-generated outputs. @xcite designed special text perturbations of dialogue data to demonstrate a lack of robustness of llm-based evaluators to many of them. to check whether llms are evaluating a given quality criterion, @xcite designed special text perturbations affecting only one evaluation aspect. their experiments showed that llms often confuse evaluation criteria despite explicit instructions. these analyses only concerned the final score and did not involve explanations. @xcite performed a comparison between human-annotated and llm-generated spans when evaluating three nlg tasks, demonstrating a high level of agreement between the models and human annotators. however, the study only evaluated the correctness of the model's textual explanations without further analysis. ## summary this paper draws attention to the issue of explanation faithfulness in span-based nlg metrics, which has not been addressed in previous research or included in metrics' meta-evaluations. to bridge this gap, we have proposed a new testbed for evaluating explanation faithfulness using special perturbations of the explanations. our analysis revealed that the faithfulness of explanations provided by llm-based metrics is limited, prompting a call for caution when interpreting their outputs. for example, interpretations such as "this text would receive a lower score if the errors were assessed as more severe" were found to be incorrect in most cases. the models are more likely to change their assessment if provided with explanations that sound as if errors are more/less severe than to an actual numerical severity assessment. inconsistent metric behaviours with respect to severity changes or error deletions were also observed. the explanations provided by the current metrics were also found to be poorly calibrated, as they do not semantically react to an increase or decrease in error severity. the reasons behind the low faithfulness of the explanations offered by nlg metrics require further investigation. one potential reason might be that llms evaluate each instance independently and lack clear points of reference. another reason may be that, although error analysis serves as the reasoning outcome instead of the standard chain of thought in llm metrics, it may not be sufficiently considered when predicting the final score.