Peut-on faire confiance aux juges ? Validation de méthodes d’évaluation de la factualité par perturbation des réponses (2025)

Abstract: Évaluer la véracité des grands modèles de langage (LLMs) est essentiel pour de nombreuses applications. Cependant, nos outils d’évaluation sont-ils eux-mêmes fiables ? Malgré la prolifération des métriques de factualité, leur sensibilité et leur fiabilité restent peu étudiées. Cet article introduit un cadre de méta-évaluation qui teste systématiquement ces métriques en appliquant des corruptions contrôlées à des réponses de référence. Notre méthode génère des sorties classées selon des degrés connus de dégradation afin d’analyser comment les métriques capturent les variations subtiles de véracité. Nos expériences montrent que les méthodes disponibles dans les framework d’évaluation, telles que la métrique factual correctness de RAGAS, suivent mieux la dégradation que les approches de type LLM-as-judge. Nous proposons également une nouvelle variante de la métrique de factualité, à la fois compétitive et économique.

## techniques for assessing factual correctness in this section, we examine how factuality is currently assessed, reviewing the dominant evaluation approaches, their assumptions, and the extent to which they have been validated. assessing the factuality of large language models (llms) is a critical challenge when deploying generative models in high-stakes domains. traditional automatic metrics, such as bleu @xcite and @xcite , were initially used to assess generated text by measuring the overlap of ngrams with the reference output. however, these metrics do not capture semantic correctness or factual consistency @xcite , and have been shown to correlate poorly with human judgments of truthfulness. in contrast to learned similarity models, learned similarity metrics such as bertscore @xcite , bleurt @xcite , and comet (rei et al., 2020) address these limitations leveraging pre-trained or fine-tuned encoders that assess semantic alignment between system and reference outputs. although these models improve lexical baselines by better capturing fluency and relevance, they often do not distinguish fluent hallucinations from true content, resulting in poor alignment of factuality @xcite . a more direct approach involves entailment-based metrics, which recast factuality as a natural language inference (nli) task. factcc @xcite pioneered this approach by training a classifier on synthetically altered summaries to detect contradictions with the source text. subsequent methods, such as menli @xcite , refine this paradigm using aggregated entailment scores across sentences. these methods have shown improved alignment with human judgments, though their effectiveness is constrained by the generalization limits of nli models and the challenges of fine-grained fact-checking. building on nli, qa-based methods assess factuality through question-answering. in this setup, a question generation (qg) model creates questions from the summary or answer, and a qa model retrieves answers from the source document @xcite . metrics such as qags and qafacteval @xcite compare answer overlaps to detect factual consistency. these techniques offer interpretability and high correlation with human ratings, but their reliability depends on the quality of qg/qa components and the availability of grounding documents. recently, researchers have explored llms as evaluators. this technique prompts large models, such as gpt-4o, to directly judge factuality @xcite . these"llm-asa-judge" methods provide flexibility and multi-criteria evaluation but raise concerns around trust, prompt sensitivity, and circularity when the evaluator and generator share architectural biases. despite growing interest, these methods remain under-validated compared to traditional classifiers or nlibased scores. it is also important to review that modular pipeline metrics, such as ragas, emerged in the context of retrieval-augmented generation (rag) systems. among the many ragas metrics, one of our interests is the factual correctness. this metric uses llms to decompose model answers and a ground truth into atomic claims. then, it compares each claim for natural language inference via gpt-4o (by default) and computes precision, recall, and an f1 score @xcite . these tools support fine-grained error analysis and are tailored for rag pipelines, but they rely on complex cascades of models whose interactions are not yet fully understood or benchmarked. in our work, we also investigate a variant of ragas factual correctness by using open-source llms and pretrained nli models (section 5). ultimately, human evaluation remains the gold standard for factuality assessment. benchmarks such as summeval @xcite , frank @xcite , and truthfulqa (lin et al., 2022b) provide expert-annotated judgments of consistency and hallucination, forming the basis for validating automatic metrics. however, while datasets like truthfulqa offer valuable insights into model truthfulness on short, closed-ended questions, they fall short in evaluating more complex, multi-sentence responses grounded in a broader context. for example, truthfulqa primarily targets simple misconceptions or common false beliefs, with a binary framing of correctness. in contrast, our work introduces long-form answers, varying degrees of factual distortion, and a ranking setup to better capture the nuanced nature of factual errors in modern llm outputs. this makes it more suitable for testing evaluation methods intended for real-world deployments, where answers are rarely black-and-white and the goal is often to detect subtle inaccuracies or degraded factual quality. in summary, although factuality evaluation has progressed toward more sophisticated semantic, entailment-based, and retrieval-augmented frameworks, no single metric has emerged as robust, generalizable, and computationally efficient across tasks and domains, especially in the presence of nuanced errors. as llms become increasingly fluent and yet prone to hallucination, the field continues to seek evaluation methods that balance interpretability, scalability, and alignment with human judgment. in this context, the ability to generate reliable, domain-specific datasets, especially those with controlled factual perturbations, plays a critical role in comparing factuality metrics. in the next section, we present our pipeline and how it produces testing datasets. ## the factual perturbation pipeline we develop a multi-step "factual perturbation" pipeline. given a question and its corresponding ground truth answer, it generates alternative answers, ranging from fully correct to progressively incorrect responses. the pipeline produces a series of five answers (a0-a4), where the level a0 corresponds to a faithful paraphrase of the ground truth, and a1 through a4 to increasing levels of factual errors. to illustrate the pipeline capabilities, we present an example extracted from our generated dataset in figure 1 . our approach aims to produce fine-grained levels for evaluating the quality of assessment techniques for natural language responses. the pipeline operates as follows : ## validating the pipeline as described in the previous section, our pipeline employs llms for certain tasks. these llms can themselves produce unexpected and inaccurate outputs, which could degrade the quality of the factual perturbation. for this reason, we need to validate that, for practical purposes, the pipeline introduces factual perturbations effectively. one way to do so is to request human experts to perform the same task : from a gold q&a dataset (i.e., questions and ground truths), produce several variants (a0-a4) with increasing degree of incorrectness. then, we can request a second group of experts to compare the quality of those outputs with those generated by the factual perturbation pipeline. we requested two evaluators to compare the quality of the factual perturbation introduced in a blindrandomized test. for each question, the evaluators see the five levels of answers. they can also see the human-expert and pipeline-generated versions for each level side by side. the evaluators are blind to the source (pipeline or expert) of each answer and are asked to choose which one best aligns with the intended perturbation level. they can also opt for accepting or rejecting both alternatives. to facilitate the evaluation procedure, we provide evaluators with a user-friendly interface (ui) that includes scoring guidelines and quick visualization of the differences between perturbation levels. more information about the ui is available in the appendix c. the evaluators accessed 20 q&a pairs (a total of 100 a0-a4 pairs generated by a human expert and the factual perturbation pipeline). the contingency table of their assessment choices is available in we focus on evaluating the non-inferiority of the ai pipeline with respect to human experts. most pairs (85%) were rated as ties, i.e., there is no clear distinction in quality between the ai pipeline and expert-generated answers. in fact, the level a0 (see figures 2a) shows a nearly universal equivalence rate (95% ties), so we exclude it from further analysis. for the remaining factual perturbation levels (@xmath0, we employ a composite scoring system (-1 : expert preference, +1 : ai preference, 0 : ties/mixed preferences) and predefined a noninferiority margin of δ = -0.1. this margin reflects a conservative threshold, where ai performance would not significantly lag behind human performance in real-world applications. the ai pipeline achieves a mean composite score of 0.125 (90% bootstrap ci : [0.025, ∞]), with the lower confidence bound exceeding the non-inferiority margin. this indicates statistical support for non-inferiority. notably, 82.5% of evaluations resulted in ties (e.g., both evaluators selecting "both are good" or one expressing uncertainty, see figure 2b ), suggesting frequent perceptual equivalence between ai and human responses. contingency tables reveal minimal direct disagreement : only 2 of the 80 examples (2.5%) showed opposing preferences (ai vs. expert). however, inter-rater reliability remained low (cohen's κ = 0.113), reflecting challenges in consistently distinguishing between ai and human outputs. post hoc power analysis estimates 77% power to detect non-inferiority at α = 0.05, suggesting moderate sensitivity. while sufficient to support our hypothesis, larger samples (∼ 50 examples) would strengthen reliability. despite this borderline statistical significance, the ai pipeline demonstrates non-inferiority to human experts during a blind evaluation under perturbed conditions (a1-a4), with high equivalence rates and statistical bounds supporting its functional interchangeability in most cases. ## evaluation of llm assessment methods we leverage our pipeline to gain deeper insight into how the techniques presented in section 2 behave regarding factual correctness. we apply it to the gold dataset to generate progressively factually perturbed alternatives. these perturbed alternatives are then fed into ragas and llm-as-judge. by comparing the final factuality scores with the intended perturbation level, we can evaluate how well each technique performs in detecting factual errors in open-ended answers. we expect that the factuality scores decrease as the perturbation increases. thus, we evaluate performance using two metrics : pearson correlation, which measures the linear relationship between perturbation levels (a0 to a4) and factuality scores, and kendall's tau, which assesses whether the relative ranking of factuality scores correctly reflects the increasing perturbation levels. we hypothesize that models with stronger negative correlation (closer to -1) better capture factual degradation. we generate 500 a0 to a4 examples using 100 q&a from the google natural questions dataset @xcite (i.e., the gold dataset). this dataset consists of over 300,000 q&a queries sourced from google search, with long answers typically drawn from wikipedia. to ensure highquality examples for evaluation, we sample the first 100 complete questions (i.e., ending with a period) whose long answer is longer than 250 characters and shorter than 700 characters. we evaluate six state-of-the-art, open-weights llms of various sizes and providers. in addition, we assess the factual correctness pipeline described in section 2, using both the default ragas implementation based on gpt-4o-mini (explodinggradients, 2024), and a variant that uses an open-weight llm for sentence splitting and a natural language inference (nli) model (nli-deberta-v3-large @xcite ) for sentence classification (llm + nli). this diverse setup allows us to explore a wide range of models and techniques, comparing their performance in terms of both computational efficiency and accuracy in detecting factual errors. first, methods using llm-as-a-judge generally exhibit weaker performance compared to the other techniques. these models consistently show lower pearson correlation and kendall's tau values, indicating that they are less effective at capturing the relationship between perturbation levels and factuality scores. this suggests that llm-as-a-judge methods are not as reliable in detecting factual errors introduced by increasing perturbation, which may be due to their reliance on generative models rather than more explicit approaches. the methods using an explicit pipeline for calculating factual correctness demonstrate much stronger performance, both in terms of pearson's correlation and kendall's tau. these models achieve significantly stronger pearson correlation values, indicating a stronger linear relationship between increasing perturbation levels and the corresponding factuality scores. furthermore, the strong kendall's tau highlights the ability of these methods to maintain the correct ranking of factuality across different perturbation levels. this suggests that these methods are better at detecting and maintaining consistency in factuality, even as perturbation is added. the strong pearson and kendall's tau values together indicate a more reliable and consistent assessment of factual correctness. moreover, the use of open-weights llms for sentence splitting combined with a dedicated nli model (llm + nli) offers an attractive alternative to proprietary models based on gpt-4o-mini. despite not always outperforming in terms of pearson correlation, these models provide a more computatio-nally efficient and cost-effective solution while maintaining strong performance in detecting factual errors. figure 3a provides another perspective on how the assessment techniques distinguish answers across perturbation levels. the red curve (gpt-4o-mini with ragas) steadily declines from 1 to approximately 0.2 as perturbation increases, closely mirroring the structure of the perturbation generation process, where 20% of the facts are preserved. this confirms that the factual correctness pipeline not only correlates well with ground truth but also faithfully tracks factual degradation, utilizing the full score range in a way that is interpretable and grounded. in contrast, prometheusv2 used as a judge (blue curve) shows a flattened response, often over penalizing correct answers or failing to penalize factual errors adequately. although explicitly fine-tuned for comparing responses to score rubrics, its output as a factuality scorer does not reflect systematic changes in data quality, undermining its usefulness for fine-grained evaluation. this observation underscores that factual correctness methods, like ragas, not only achieve better alignment with the intended ranking (as shown by kendall's tau) but also produce score distributions that are more calibrated to the factual quality of responses. these insights reinforce the overall advantage of factual correctness pipelines for this task. finally, in terms of model size, figure 3b reveals that larger models do not necessarily lead to better evaluation performance. in fact, several mid-sized open-weights models achieve higher correlations than much larger judge-based llms. this indicates that accurate factuality evaluation does not strictly require massive llms, and that smaller, targeted models within structured pipelines may offer both better performance and higher efficiency. furthermore, the llm size also seems to have almost no impact on the final performance of the factual correctness evaluation method. these findings reinforce that thoughtful pipeline design and model selection can outperform brute-force scaling, offering a more practical and robust approach to the evaluation of factuality. ## known limitations our pipeline systematically applies linguistic and semantic modifications using dependency parsers and predefined operators. however, the effectiveness of these perturbations can vary depending on the properties of the target text. for instance, verbose or highly detailed answers, such as those generated by llms, may require more targeted or intensive perturbations to produce noticeable semantic shifts, whereas shorter, more concise answers may be more sensitive to minor changes. as a result, "uniformity" in the degree of perturbation across questions and ground truths is not guaranteed. furthermore, specific perturbations may not always affect the core content of the answer. changes may preserve the underlying meaning despite surface-level changes (see "who breaks a tie in the us senate ?" example in appendix d). while we do not have quantitative evidence that such cases are prevalent across the dataset, this example highlights a possible limitation of perturbing verbose answers where the core fact is only a small part of the text. in fact, the two evaluators had no specific instructions on whether they should accept or reject such cases. further analysis is warranted to determine how often such cases occur and whether alternative perturbation strategies are needed. conversely, some perturbations may introduce inconsistencies or semantic contradictions within the answer (such as in "who wrote the text for jeanie with the light brown hair ?" example in appendix d). for such examples, they fail the semantic guidelines and should be counted as a rejected example (i.e., either accept the human alternative or reject both). finally, while our method is language-agnostic in principle, it depends on the availability of reliable dependency parsers and llms for the target language. languages with complex morphology or syntax, or with low resources, may suffer from perturbation accuracy and coverage. we have not yet thoroughly evaluated the reasoning capability of recent llms triggered by specific prompt instruction (e.g. <@xmath1. ## conclusions our experiments reveal that pipeline-based methods for factual correctness, such as ragas, significantly outperform llm-as-judge approaches in detecting factual errors. these methods exhibit stronger pearson correlation and kendall's tau values, indicating their superior ability to track factual degradation and maintain consistency in ranking as perturbation increases. we also find that open-source llms combined with natural language inference (nli) models offer a cost-effective alternative. although these models may not consistently outperform the larger llms in pearson's correlation, they still demonstrate competitive performance in detecting factual errors, offering a more computationally efficient solution. in terms of model size, larger llms do not necessarily lead to better evaluation performance. in fact, several mid-sized open-source models achieve higher correlations than much larger judge-based llms. this highlights the importance of thoughtful pipeline design and model selection over simply scaling the model size. furthermore, our findings suggest that structured, smaller models within effective pipelines can outperform larger models in factuality evaluation. beyond empirical findings, our work proposes a general-purpose open-source pipeline for benchmarking factuality metrics. given a reference dataset, practitioners can use our framework to test and compare multiple evaluation metrics and select the one that best aligns with the specific requirements of their task. furthermore, our approach helps standardize the comparison of metrics across the community, raising the standards of trust and rigor in assessment research. these results contribute to our effort in developing a socratic companion assisting students when learning educational resources @xcite . the pipeline is publicly available at https://github.com/giovannigatti/trutheval .