Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation
Authors: Sanjana Ramprasad
Byron C Wallace
Venue: NeurIPS 2025 poster ()
Keywords: Summarization
Faithfulness metrics
Primary area: evaluation
TL;DR: This work stress-tests factuality metrics, showing they mostly perform well in cases that shallow models do and struggle with complex examples, fail to reward factual corrections, and are easily gamed, raising reliability concerns.

Abstract: Modern LLMs can now produce highly readable abstractive summaries, to the point that traditional automated metrics for evaluating summary quality, such as ROUGE, have saturated. 
However, LLMs still sometimes introduce inaccuracies into summaries, i.e., information inconsistent with or unsupported by the corresponding source. 
Measuring the occurrence of these often subtle factual inconsistencies automatically has proved challenging. 
This in turn has motivated development of metrics intended to measure the factual consistency of generated summaries against sources. 
But are these approaches measuring what they purport to? Or are they mostly exploiting artifacts? 
In this work, we stress test a range of automatic factuality metrics—including specialized model-based approaches and LLM-based prompting methods—to probe what they actually capture. Using a shallow classifier to separate “easy” examples for factual evaluation—where surface features suffice—from “hard” cases requiring deeper reasoning, we find that all metrics show substantial performance drops on the latter.
Furthermore, some metrics are more sensitive to benign, fact-preserving edits than to factual corrections. Building on this observation, we demonstrate that most automatic factuality metrics can be gamed—that is, their scores can be artificially inflated by appending innocuous, content-free sentences to summaries. Among the metrics tested, the LLM prompt-based ChatGPT-DA approach is the most robust and reliable; however, it exhibits a notable caveat: it likely relies more on parametric knowledge than on the provided source when making judgments. Taken together, our findings call into question the reliability of current factuality metrics and prompt a broader reflection on what these metrics are truly measuring. We conclude with concrete recommendations for improving both benchmark design and metric robustness, particularly in light of their vulnerability to superficial manipulations.

Do Automatic Factuality Metries Measure Factuality?
A Critical Evaluation
Sanjana Ramprasad Northeastern University ramprasad. wwwhortheastteeeeeceo
Byron C. Wallace Northeastern University
D.vallaace@horthhasseenn. edu
Abstract
Modern LLMs now produce highly readable abstractive summaries, to the point that traditional automated metrics for evaluating summary quality, such as ROUGE, have saturated. However, LLMs still sometimes introduce inaccuracies into sum- maries, i.e., information inconsistent with or unsupported by the corresponding source. Measuring the occurrence of these often subtle factual inconsistencies automatically has proved challenging. This in turn has motivated development of metrics intended to measure the factual consistency of generated summaries against sources. But are these approaches measuring what they purport to? Or are they mostly exploiting artifacts? In this work, stress test range of automatic factuality metrics, including specialized models and LLM-based prompting meth- ods, probe what they actually capture. Using a shallow classifier to separate "easy" examples for factual evaluation where surface features suffice from "hard" cases requiring deeper reasoning, we find that all metries show substantial perfor- mance drops on the latter. Furthermore, some metrics are more sensitive to benign, fact-preserving edits than factual corrections. Building on this observation, we demonstrate that most automatic factuality metrics can be gamed, i.c.. their scores can be artificially inflated by appending innocuous, content-free sentences to summaries. Among the metrics tested, the LLM--bseed ChatGPT-DA approach is the most robust and reliable. However, this comes with notable caveat: Prompting LLMs to assess factuality may overly rely on their parametric knowledge rather than the provided reference when making judgments. Taken together, our findings call into question the reliability of current factuality metrics and prompt broader reflection on what these metrics are truly measuring. We conclude with concrete recommendations for improving both benchmark design and metric robustness, particularly in light of their vulnerability to superficial manipulations.
Introduction
LLMs strong abstractive summarizers [Goyal al. 2022 Zhang et 2024], but they
infallible. Even the largest, most capable models sometimes introduce subtle "hallucinations" (or "confabulations"
summaries that are unsupported by or in contradiction the corresponding input document (Zhang et al.) 22244, Tang et al. 2024b) Ramprasad et al.) 2024b). Such behavior especially problematic in domains such as medicine law, where inaccurate information could translate into meaningfully negative consequences for individuals.
However, manually evaluating model outputs' factual consistency with respect to references is expensive, time-consuming, and impractical to scale. This has motivated development of automated methods that score generated summaries for consistency with respect reference documents. Such metrics have been operationalized using a range of techniques, including entailment (SummaC:
Laban al. 2022 QA models (QuestEvval; Scialom et al. (202T). specialized models explicitly
trained to score source-sume cource-summary pairs (UniEval; Zhong et al. 2022 Alignscore; Zha et al. 2023
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

MiniCheck; Tang et al. (024a]) ) and more recently. LLM-based methods that rely on prompting LLMs (ChaiGPT-DA; Wang et al. 2023a Luo al. 2023
militry launched major operation Thursday, senior Sudanese said
Sudan's is wayyy witerprring re-iske capatol
0.9
Fariss si-iuuthen, leader Sudan
m(z. B)
0.7
Sudan's millbary re-kook the on Wedrwwday.
0.4
just belors anmy and facto laader. Gan Abdel l'atiah al-Surhan, adcressed the United Nations New York
Generated summeenees
Scoring mathod
p(consistent)
Rephrased
Busian's army is makingg an efort regain contral of
0.6
Corrected
Fallsh al-Buuhram, de fasto leader of Sudan
m(z, 9)
0.7
Adversarial proftx
According the document Sucan's milttry re-took the capital Wednesday
0.9
Input document 3:
Modified summariss y
Figure I: Many methods have been proposed to automatically evaluate the factual consistency of summaries with respect to inputs. In this work we critically evaluate such approaches, e.g., by measuring their sensitivity various manipulations, as shown here.
Most of these metrics have been evaluated against human benchmark assessments (binary labels or
Likert scores) of factual consistency [Maynez et al. 2020 Fabbri et al., [021 Laban et al. 2022
Honovich et al. 2022 Wang et al., 2022] Gao et al. 2023 Tang et al. 2024a].
These assessments have established that automated factuality metrics correlate with human evaluations to varying degrees. But are such approaches actually attuned the subtle question of factual consistency between inputs and outputs, or are they merely relying on shallow heuristic signals to make their judgments?
metrics do rely on shallow heuristic patterns, this raises questions about their reliability. It also makes them vulnerable to "gaming": Summaries claims can be crafted to exploit heuristic shortcuts and artificially inflate scores without genuinely improving factual accuracy. Alternatively, LLM-based approaches may overly rely their internal (parametric) knowledge to assess summaries, rather than assessing output factuality with respect to the source. In this work, we stress test a diverse set of SOTA factual consistency metrics on context-claim pairs that yield continuous factual consistency scores. Our analyses offer the following empirical results as our main contributions.
Metries struggle when factuality requires reasoning beyond shallow cues. In Section 3 we train shallow MLP on surface-level heuristics and use its prediction confidence categorize summaries as easy, moderate, or hard to assess for factuality. We find that all automated metrics exhibit substantial performance drops from easy to hard examples, suggesting they may be relying on superficial cues rather than nuanced reasoning.
Factuality metrics are somewhat responsive to factual corrections, but also often sensitive to irrelevant (benign) modifications to summaries. In Section 4.1 we evaluate whether factuality metrics distinguish genuine factual corrections from inconsequential edits using a dataset of annotated summary pairs [Krihhna et al., 2024] which comprise an inconsistent summary and a minimally edited, faithful revision. Ideally, metrics should assign higher scores to the corrected versions while remaining stable under benign, fact-preserving edits (e.g.. paraphrasing). We find that while both specialized and prompt-base based metrics respond to factual corrections, many-especinn
NLI-based SummaC and some specialized models (UniEval and Alignscore)-and overly sensitive to benign edits. In fact, some metries exhibit greater score shifts from superficial changes than from actual factual improvements. By contrast, ChatGPT-DA consistently ranks faithful revisions higher while remaining robust to benign perturbations, suggesting that may be more reliable choice.
(Some) factual consistency metrics are gameable. factuality metrics rely least partially on superficial cues, this suggests that we should be able to "game" them by inserting such cues to inflate scores assigned to model outputs. And indeed in Section 5 we find that inserting superfluous, innocuous phrases-eine For NI.I-based and specialized metrics, these artificial boosts often exceed the score gains achieved by genuinely more factual models (see Figure 6). ChatGPT-DA, however, shows minimal sensitivity
alone or appended to -cnnnss--nnn significantly inflate factuality scores.

---

to such manipulations, again suggesting that prompting LLMs for factuality assessment may offer more robust and reliable evaluations. However, this comes with a caveat.
GPT sometimes over relies on internal knowledge when evaluating summaries-eveds when reference source is provided. ChatGPT-DA may default to its internal (parametric) knowledge rather than relying on the provided source when assessing factuality consistency (see Section (.2). This tendency appears especially pronounced when the reference document contradicts the model's parametric knowledge, indicating that its reliability may be compromised in cases involving myths, rare updated facts, or information that confficts with what the model "believes". This provides caveat to our other findings that suggest LLM based assessments may be more reliable than bespoke models: LLM-based evaluations may reflect what the (evaluator) model "knows" rather than what the source says, limiting reliability in grounded factuality assessments.
Practical recommendations. Our analyses reveal that while specialized metrics perform well overall, they are vulnerable benign edits and adversarial manipulation, which may limit their reliability, ChatGPT-DA offers a more robust alternative, but may emphasize consistency with its parametric knowledge rather than input sources, raising concerns about its grounding behavior. We therefore recommend caution when applying such metries in domains involving myths, misinformation, or uncommon facts. Finally, we highlight the need for benchmarks that capture hallucination severity and for methods that incorporate saliency-aware supervision to improve metric reliability (Section
2
Experimental Setup
2.1 Benchmark Datasets
Automatic factuality metrics for fact verification have been evaluated using various benchmarks, predominantly based on news sources [Hermann et al. 2013 Narayan al. 2018]. The AggreFact
dataset [Tang et al. 2022] consolidates several benchmarks on tune- model-generee
summaries
from such sources Maynez et al., 2020 Wang al. 2020 Pagnoni et al., 2021 Fabbri et al. 2021
Honovich et al., 2022, Laban et al. [2022]. J. Similarly. datasets like FacEval [Wang et al., 2002 and
ReferenceMatterss Gao et al. [2023] benchmark dialogue summarization fine-tuned models on
sources from SAMSum Gllia et al. 2019a and DialogSum [Chen et al. 2021b).
These datasets primarily foeus on summaries from fine-tuned models predating modern LLMs,, which [022). Consequently, factuality metrics evaluated on
excel in zero-shot summarization [Goyal et al.
these benchmarks may not generalize to SOTA LLM outputs, which exhibit different error patterns
(Tang et al. 2022]. To address this, Tang et al.
[2024a] introduced LLM-AggreFact, a fact-checking
dataset from recent LLMs across diverse domains and dataset types. Similarly, Krishna et al. [2024] released the GenAudit t dataset with factuality annotations for LLM summaries in news, Reddit, and
clinical settings, while Ramprasad et al. [2024a] focus on LLM generated dialogue summaries in the LLM-d A.M-dialogue dataset.
For our analysis we use all of the above benchmarks to capture wide range of error types. For fine-tuned model summaries, we use AggreFact for news and FacEval for dialogues. For L.LM- generated summaries, we rely on LLM-AggreFact, GenAudit, t, and LLM-di ailoue We note that each benchmark consolidates multiple datasets and to ensure clean separation of distributions, we avoid overlapping datasets between our test and development splits. Since AggreFact primarily includes older fine-tuned models, we incorporate its dev set into our development data and reserve newer benchmarks for evaluation. Specifically, our dev set includes summaries from the AggreFact dev split, as well as XSUM and CNNDM examples from Genaudit, ensuring no overlap with test data. All remaining datasets are evaluated using their respective test splits. We provide detailed breakdown of our dev and test splits in Appendix
2.2 Automatic Factuality Metrics for Summarization
We group SOTA factuality metrics into four broad methodological categories. This includes metrics based on: Question Answering (QA). Natural Language Inference (NLI), fine-tuned (specialized) models and LLM prompting methods; Table 1
For -bsbd metries we use QuestEval [Scialom et al. 2021 As an NL.-based metric we use SummaC-Cony [Laban et al. 2022 We use three specialized models for evaluation: UniEval,
3

---

Metric
Category
QuestEval Scialom et
2021
QA
SummaC Conv Laban et al
2022
NLI
UniEval Zhong el al.
2022
specialized model
AlignScore Zha al. 2023
specialized model
MiniCheck lang
2024a
specialized model
ChatGPT-DA [Wang of al.
2023a
Prompt/ / LLMM
Table 1: Metrics categorized by approach and analyzed
easy
medium
hard
3.05
3.55
0.40
0.34
0.39
0.00
2
0.85
9.73
057
QuestEval (QA)
UniEval (Specialized)
Alignscore (Specialized)
SummaaC-Conv
MiniCheck (Specialized)
ChatGPT-DA (Prompt)
Figure 2: Summaries are categorized as easy, medium, or hard based on prediction accuracy and confidence from shallow MLP. While specialized metries perform best on easy examples, their performance declines on hard cases. UniEval, MiniCheck and ChatGPT-DA show greater robustness in more challenging settings
AlignScore and MiniCheck. Unilval Zhong al. [2022 reframes NLG evaluation as a Boolean QA
task and uses T5 [Raffel et al. 2020 to score different dimensions. AlignScore [Zha et al. 2023
evaluates summaries by combining an alignment function-a RoBERTa model [Liu] [019 fine-tuned on diverse tasks-with a splitting and aggregation strategy. MiniCheck (Tang et al. 2024a] uses a
Flan-T5 model [Chung et al. 2022 fine-tuned synthetic dataset created by the authors.
We also use GPT-4o-mini to score the factual consistency of summaries based on a direct assessment
(DA) prompt template from Wang et al. (2023a
3 Do metrics infer "Factuality" from Superficial Features?
The degree to which a claim is faithful to the source text is a subtle question that demands under- standing the content in both texts determine if they are consistent. We would therefore expect that metrics capable of measuring factual consistency in general would need to capitalize on information beyond the superficial features of source and claim texts just discussed.
To investigate the extent to which shallow features explain metric behavior, we train an MLP classifier to predict binary human factuality labels development set using only surface-level features We then apply the trained model to an evaluation set and categorize summaries into three difficulty levels-easy, medium, and hard-based measured as the absolute deviation of the predicted probability from 0.5: Lower values indicate greater uncertainty. We classify summary easy if the prediction is correct with high confidence (top 80% of confidence scores), and medium if correct with lower confidence. Incorrect predictions designated medium if confidence low (bottom 20%) and hard otherwise. The idea here that some examples can be readily classified as "factual" (or not) using shallow features like word overlap; these are "easy" examples. By contrast, "medium" and "hard" categories capture examples that are difficult to classify with respect to factuality using these shallow features.
prediction accuracy and confidence. Confidence is
Figure rennertsssssss the AUC for metrics across summary difficulty levels. All metrics perform best easy summaries, where shallow features reliably predict human judgments. Performance declines on medium and hard examples, with older metrics like QuestEval and SummaC- showing sharp drops, suggesting reliance on superficial cues. Specialized and prompt-based metrics (e.g., UniEval,
See Appendix B for feature details and Appendix C for MLP details.

---

QuestEval (QA)
SummaC Couv (NLD
AliguScore (Specialized)
Corrected
Shuflled
Added Source text
Diverse
Nepuled
Simplified
Shortened
Paraphrased
Synonym Replacement
MiniChack (Spucialladd)
Unitval (Specidiae))
ChatGiEDD (Prompt)
Shuffled
Added Sourcs Twxt
Nogated
Simplified
Paraphrased
Synotym Replecmment
Figure 3: Score differences for each metric between the original summary and summaries edited for factual accuracy by humans (shown in green) and other benign edits (shown in blue).
AlignScore, MiniCheck, ChatGPT-DA) are more robust, maintaining higher AUC on medium cases. However, even these models struggle on hard summaries, indicating that current metrics, despite being trained for consistency, remain limited in the absence of shallow cues.
4 What do automatic factuality scores measure?
We next examine the reliability of factuality metrics. In Section 4.1 we evaluate their sensitivity to factual corrections versus benign (fact-preserving) edits. We find that only ChatGPT-DA consistently distinguishes between the two, showing both sensitivity and robustness. In Section 4.2 we evaluate ChatGPT-DA's reliance source content by scoring summaries against counterfactual references that conflict with its parametric knowledge. Our results show that consistency evaluation performance degrades when presented with counterfactual input documents, indicating limited grounding in the provided sources.
4.1 Measuring Metric Sensitivities to Controlled Manipulation
We want to determine whether metrics respond specifically to changes factual consistency or are influenced by superficial edits unrelated to fidelity. We focus on subset of summaries from the GenAudit dataset [Krishna et al. 22024,, each manually labeled as inconsistent and paired with a minimally edited, corrected version. These pairs allow us to directly assess whether metrics are sensitive to the consistency improvements that matter most.
We are also interested in score variation that owes to superfluous factors. To this end, we prompt GPT-4 to generate versions of summaries modified in targeted, fact- fiet-preserving ways intended be independent of factual consistency. We use several prompts that request benign transformations like paraphrasing, simplification, and rewording ? While these edits are designed to preserve meaning, using GPT may occasionally introduce factual inconsistencies, though we believe such cases are rare.
Ideally, automatic factuality metrics should show a positive score change for corrected summaries, reflecting improved factual consistency. Conversely, generated summary rewrites-wlie do not alter factual content-shoule exhibit minimal score changes, Any significant score differences in these rewrites likely indicate that the metric is sensitive to artifacts incidental to factual consistency.
Results. We present results in Figure 3 Notably, QuestEval shows no meaningful improvement in response to factual corrections, raising concerns about its ability to detect genuine consistency improvements. Additionally, most metrics, except QuestEval and ChatGPT-DA, assign dispropor-
PPrompts for benign summary edits are provided in Appendix D
5

---

3.1
(Supported Unsspported)
37.292
*B (t=10.28)
35.
Score Difference
30
26.901
25-
2.-
15-
12x incredse
10
0.2%
0.0
Factual Refs
Counterfactual Refs
Factual Refs
Counterfactual Refs
(c) (a)
(d) > (b)
Figure 4: GPT-based consistency evaluation is influenced by parametric knowledge. Left: The score gap between supported and unsupported summaries narrows sharply when references are counterfactual but summaries are factually accurate (p 0,001). Right: The rate of cases where unsupported summaries are scored higher than supported ones rises from 0.2% 3.1% when references contradict GPT's world knowledge while summaries remain factually correct.
tionately high scores when random, yet consistent, source sentences appended inconsistent summaries In many cases, these spurious additions yield score gains comparable to, or greater than, those resulting from actual factual corrections.
We also observe that MiniCheck and UniEval are highly sensitive to weaning-preserviie negation. Simple rephrasings that retain the original meaning for example, changing "The author mistakenly believed their ACT test was on a certain day" to "The author did not realize their ACT test was not on the day they thought", often lead to reduced scores. While ChatGPT-DA exhibits some sensitivity to negation, it remains relatively robust overall, consistently distinguishing between true factual improvements and irrelevant textual changes.
4.2 Reliance on Source Context Versus Parametric Knowledge in GPT-Based Scoring
Prompt-based approaches factual consistency evaluation have recently gained traction [uo et 2023 Wang al. [023a]. Unlike traditional metrics trained on source-summary pairs, these methods rely on prompting pre-trained models evaluate summaries against reference articles for consistency, This raises critical question: Are their judgments truly grounded in the provided source, or are they influenced by the model's 's internal (parametric) knowledge?
We investigate this by evaluating GPT's consistency judgments under scenarios where the reference text (likely) conflicts with the model's implicit world knowledge. To ensure that GPT possesses strong prior knowledge about the domain, we use Wikipedia articles, which are heavily represented in LLM pretraining corpora. Specifically, we use the ConflictBank dataset (Su al.] 2024], which comprises factual claims extracted from Wikipedia and corresponding counterfactual variants generated through targeted substitutions. Each counterfactual claim also paired with counterfactual reference document- a version of the original article modified to make the false claim appear consistent with the altered reference. This dataset structure permits four experimental conditions: (a) Factual references paired with supported factual summaries;
(b) Counterfactual references paired with supported counterfactual summaries; (c) Factual references paired with unsupported counterfactual summaries, and; (d) Counterfactual references paired with unsupported factual summaries.
To the discriminative ability of GPT to score for consistency, evaluate against factual references using conditions (a) and (c) and counterfactual references using conditions (b) and (d). Ideally GPT would be equally discriminative in both cases: (a) and (b) should be deemed equally consistent, and (c) and (d) be scored as equally inconsistent. In Figure 4 left, we compare the score differences between supported and unsupported summaries both settings, namely
and (b d). using
'Such a sentence will itself be "consistent", but should probably not shift the overall consistency score to be higher than the corrected summary given that this does not correct any existing inconsistencies, as it strictly adds content.

---

QuestÉval (QA)
(NLI)
UntEval (Specialized)
Top phrase
Assortion phrase
Summ
Summ (-Assrttion)
AlignScore (Specialized)
MiniCheck (Specialized)
ChatGPT-DA (Prompt)
Top phrase
Summ (+lpp)
Summ Assartioon)
Summary- Summary
Figure 5: Pairwise score differences across summaries manipulated using four distinct strategies: Adding constant strings (top phrase and assertion phrase) and appending them to summaries (summ + top, and summ + assertion. The results reveal that NLI and bespoke model metrics are particularly vulnerable to gaming, with significant score inflation observed under these manipulations.
paired I-test on the two deltas. The test yields < 0.001, i.e., the score gap between supported and unsupported summaries is significantly smaller when evaluated against counterfactual references compared to factual ones. This suggests that GPT struggles more to distinguish between supported and unsupported summaries when the reference contradicts its internal knowledge. In particular, when evaluating summaries against references that contradicts its internal knowledge, the model shows reduced sensitivity whether the summary is actually supported by evidence.
We would also anticipate that the model should score supported summaries higher than unsupported summaries, i.e., b > d. However, if d b, this might suggest that GPT's parametric knowledge is overriding the reference- baaeed evaluation, inappropriately favoring (generally) factually accurate content which is unsupported by the input under consideration. We find that GPT incorrectly rates the unsupported (but factual) summary higher than the supported summary in 3% of cases. As baseline, we also evaluate the proportion of times it scores unsupported outputs higher than supported factual summaries where the input is (generally) factual (c a). Here, GPT rates unsupported summaries higher only 0.2% of the time. The increase in inconsistent summaries being rated higher (0.2% + 3%) suggests that GPT struggles more with consistency evaluation when references contain misinformation or contradict established facts. This suggests the model's evaluation reliability could be compromised in real-world scenarios involving contested information, fiction, updated knowledge, or reference materials that contradict GPT's implicit parametric knowledge.
5
Can we game factuality metrics?
Given the preceding observations--wii factuality metrics, we turn to a practical question that would exploit this behavior: Are these metrics gameable? In other words, can we systematically manipulate summaries to induce higher factuality scores? If one can reliably do this, may raise concerns about the use of these metrics for tasks like leaderboard rankings. Independent of practical considerations, establishing the gameability of factuality metrics would provide additional evidence that they may not measuring what we think.
suggest that superficial cues may influence automated
As "gaming" strategy, we introduce innocuous phrases containing no factual content, testing their effect on metric scores as standalone inputs and additions existing summaries. These phrases, when appended, should not alter the factual accuracy of the summaries they accompany. Our goal here is to assess whether such neutral modifications can nonetheless inflate factuality scores.
Strings for summary manipulation. We attempt to identify set of strings that artificially inflate metric scores by analyzing the top 20th percentile of summaries ranked by each factuality metric,
7

---

Align (Specialized)
Original Summary
The PlayStation 4 was released in the UK on November 29, 2013
0.33
Summary w/ Phrase 1
The PlayStation 4 was released in the UK on November 2013. The summary entails the information the document discusses.
0.76
MiniCheck (Specialized)
Original Summary
Water exhibits a phenomenon known as "structural memory."
0.005
Summary w/ Phrase
Water exhibits phenomenon known as 'structural memory. The document discusses.
0.49
Table 2: Qualitative (cherry-picked) samples of original and manipulated summaries with correspond- ing metric scores for AlignScore and MiniCheck. For comprehensiveness, report quantitative aggregated results in Figure and we provide more examples in Appendix
reasoning that these high-scoring summaries may contain patterns that raise scores, independent of context. We then compute TF-IDF scores for bigrams in these high-scoring summaries to detect patterns disproportionately associated with "factual" summaries.
We select from these the top 100 bigrams, aggregating results across metrics. This set includes, e.g., "the document" and "document discusses". We adopt the constant phrase "the document discusses" (top phrase) into all documents and observe whether metrics are responsive to this. We consider one additional phrase: "The summary entails information in the document." (assertion phrase) which explicitly asserts factual consistency"" Finally, we append both phrases to the corresponding summaries summ (+top) and summ (+assertion).
We report the average pairwise difference in metric scores between the gamed versions of summaries and their original versions when evaluated for consistency with the source. Examples of these manipulated summaries and their corresponding scores are provided Table
Results and Discussion
Figure 5 reports the effects of our gaming strategies on metric scores. Notably, the constant phrases boosts scores by >0.2 points (absolute) for NLI-based SummaC-Conv, as well as specialized models UniEval, AlignScore, and MiniCheck. This is surprising, as these phrases are not valid summaries and do not contain any factual content (note that the top phrase is an incomplete sentence). Adding constant phrases as suffixes summaries increased scores by 0. 0-110.15 points; this is comparable to the gains realized following factual corrections (see Section 4.1) Indeed, SummaC-Cone C-Cov shows no score increase for corrected summaries, suggesting an under-ssesittvity to actual changes factuality.
To contextualize these results, we compare score differences between summaries from larger models (e.g., GPT-4, Gemini) and smaller models (e.g., Llama-7B, Mistral-7B, Falcon-7B, BART). While larger models typically yield more consistent summaries [Tang et al., [2022] Goyal et al. [022], we find that gaming with constant phrases produces larger score gains-ole often exceeding those from genuine model improvements, especially for NLI and specialized metrics.
One could argue that such manipulations adversarial ones especially result in "out of distribution" inputs, and so we should have no expectation of how models will perform. But usually factual consistency metrics are touted (implicitly) measuring consistency between arbitrary document and summary candidate pairs. Overall, these results suggest that adding fixed phrases to summaries can boost metric scores scores-for for most finetuned model metrics- at levels comparable to, presumably genuine improvements due to advances in summarization models themselves.
exceeding,
'The wording of this phrase varies slightly across metrics to align with their specific methods for evaluating factual consistency. The complete list of these phrases for each metric is provided in the Appendix E.1

---

Related Work
Prior work has meta-evaluaate factuality metrics, focusing their sensitivity specific error types, frequencies, or domains [Gabriel et al.) 2021 Chen et al. [021a]. In contrast, we analyze how metrics respond to factual corrections versus unrelated textual edits, exposing their vulnerability spurious cues. Kamoi et al. [023b] also find that QA-based metrics, while effective at summary-level scoring, fail to localize errors and can be outperformed by simple baselines. Gooyl and Durrett [202T] note that no metric consistently outperforms others, though their analysis focuses on error types; we highlight on reliability and potential issues with such metrics.
Conclusions and Discussion
Factuality benchmarks should be updated to reflect the evolving nature of errors produced by modern LLMs, which are increasingly subtle and context-dependent. Our analysis Section reveals that modern benchmarks include trivial errors that shallow metrics can easily detect. Evaluation should instead prioritize challenging examples that demand deeper semantic reasoning, ensuring that metrics assessed their ability capture nuanced inconsistencies rather than surface-level cues. This may be especially important in complex and high stakes domains like law and medicine.
In this work, we have taken a step in this direction by introducing shallow feature-based probe to categorize benchmark examples by complexity, enabling more informative evaluations of metric performance. Our analysis in Section 4.1 reveals that many metrics do not reliably distinguish between factual corrections and benign edits, indicating poor calibration to factual severity. We recommend that benchmarks explicitly incorporate graded summary variants reflecting different levels of factual severity. Specifically, each source should be paired with multiple summary edits to test whether metrics appropriately reward factual improvements and remain stable under benign edits. Here we approximated this approach using the dataset from Krishna et al. [2024). which includes summaries edited at two levels of factual severity alongside benign variants.
In Section 5 we show that most specialized metrics can be gamed with factuality assertion or qualifier phrases that do not alter content, revealing a vulnerability: current metrics often score factuality without accounting for saliency. To address this, we recommend integrating saliency-aware scoring that prioritizes evaluating core content aligned with the source. This may reduce susceptibility to filler-based manipulations and better reflect meaningful consistency.
Prompt-based metrics (using LLMs) show greater robustness to benign edits and gaming, making them strong candidates for factuality evaluation especially in settings where reliability of concern. However, since they are not explicitly designed for fact-checking, their reliance on pre-trained knowledge must be considered. We recommend the development of benchmarks that specifically evaluate whether LLM- -aMde methods ground their judgments in the source. To assess whether models rely on source content or internal priors, benchmarks might include summaries containing unsupported inferences source information that contradicts parametric knowledge. In this work we investigated this behavior using synthetic source manipulations from ConflictBank (Su et al. 2024), finding that GPT models often default to their parametric knowledge when evaluating consistency against contradictory references. These results highlight the importance of incorporating misinformation, controversial claims, and fictional content into source pource-grounded factuality benchmarks
Limitation and Ethics
This work has several important limitations that should be taken into account when interpreting our results.
Our interventions in Section 4.1 established that in some cases metrics are sensitive to extraneous transformations (e.g., re- re-phaasings). However this was not observed uniformly, and we performed such transformations using GPT-4, which may have introduced legitimate factual inconsistencies (though we believe this be rare).
further limitation to our "gaming" experiment (and manipulation experiments) is that they produce outputs which may be viewed as "out of distribution"; but generally the purpose of factuality

---

metrics is to be able to apply them to arbitrary model outputs and corresponding references, so it is not entirely clear what "in distribution" means under this assumption.
We also note that we primarily stress-test metrics designed for English. The impact of our translated gamed phrases and manipulations multilingual metrics remains unclear.
Finally, while we stress-test metrics, we do not propose a solution to mitigate their flaws. However, similar to prior work Gabriel et al. [2021], Chen et al. [202 we highlight specific issues in current metrics such as reliance on superficial cues, sensitivity to benign changes over inaccuracies, and score inflation from innocuous assertions, aiming to inform the development of more reliable metrics.
Despite these limitations, we believe tuut-aken together our results suggest that one should be careful about their interpretation of automatic factual consistency metrics.
Acknowledgements
This work was supported in part by the National Science Foundation (NSF), grant 221 1954. This work was also supported by the Wellcome Trust, grant
References
Meng Cao, Yue Dong, and Jackie Cheung. Hallucinated but factual! inspecting the factuality of hallucinations in abstractive summarization. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pages
Dublin, Ireland, May URL
2022. Association for Computational Linguistics. doi: 10.18653/1/202..a/-ooog https://aclanntoologg. .ww.gg...rg.ooooooo.... wrgg/0222 a01-2aacl-long 236
Shuyang Cao and Lu Wang. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In Marie-Francine Moens, Xuanjing Huang. Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pages 6633-6649 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 111118661122221.coog m32n. URL https://cclanthologg org. /rg/ 2021 emnlp-mmin..5
Hung-Tine Chen, Fangyuan Xu, Shane A Arora, and Eunsol Choi. Understanding retrieval augmen- tation for long-form question answering. arXiv preprint arXiv:2310.22150, 2023.
Yiran Chen, Pengfei Liu, and Xipeng Qiu. Are factuality checkers reliable? adversarial meta- evaluation of factuality in summarization. In Marie-Franeine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computational Linguistics: EMNLP 2021, pages for Computational Linguistics. doi: 10. 10.18653/v1/202 wee.:221..innnnnn--e findings-emmlp.199 URL https://
Punta Cana, Dominican Republic, November 2021a. Association
aclanthology org/20211 findings emnlp. 179
Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. DialogSum: A real-life scenario dialogue summarization dataset. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5062- 5074, Online, August 2021b. Association for Computational Linguistics. doi: (0.18653/v1/0211 findings-acl.449, URL. htttps::/cclanthologs wtwpooooww..g//......aiieeeeeee. findings-acl. ac1.4444ac..4999
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetueed language models. arXiv preprint arXiv:2210.1 1416, 2022.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:18100 118..000005 2018.
Alexander R Fabbri, Wojciech Kryšciński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. Summeval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics, 9:391-409, 2021.
10

---

Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, and Jianfeng Gao. GO FIGURE: A meta evaluation of factuality in summarization. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Findings of the Association for Computational Linguisties: ACI-IJCNLP 2021, pages 478-487, Online, August 2021. Association for Computational Linguisties. doi: 10.18653) org/ 2021. f madingg--cl..4
v1/2021. wii.l..innng-all444444 httpttps:
Minggi Gao, Xiaojun Wan, Jia Su, Zhefeng Wang, and Baoxing Huai. Reference matters: Benchmark- ing factual error correction for dialogue summarization with fine-grained evaluation framework. arXiv preprint arXiv:2306.0511 2023.
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. SAMSum corpus: A human- annotated dialogue dataset for abstractive summarization. In Lu Wang. Jackie Chi Kit Cheung, Giuseppe Carenini, and Fei Liu, editors, Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70-79, Hong Kong, China, November 2019a. Association for Computational Linguistics. doi: 10.11855/1//D1111
UtL.aattps:/aaclaattoloo www.aaattntooooooooooooooooo wttp://oogg.ooo/-//555000 D19g//D99-4099
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. Samsum corpus: A human- annotated dialogue dataset for abstractive summarization. arXiv preprint arXiv: 1XXIv:9911.1237. 2019b.
Tanya Goyal and Greg Durrett. Annotating and modeling fine-grained factuality in summariza- tion. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Belt- agy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors, Proceed- ings of the 2021 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, pages 1449-1462, Online, June 2021. Association for Computational Linguistics. doi: 10. 118653/v1/202) maacclmminn 14. URL https://aclaattoology org/
wtt////1.../////ooo 2021 naacl-main. 114
Tanya Goyal, Junyi Jessy Li, and Greg Durrett, News summarization and evaluation in the era of gpt-3, arXiv preprint arXiv: 12356, 2022.
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. Advances in neural information processing systems, 28, 2015.
Or Honovich, Roce Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. True: Re-evaluating factual consistency evaluation. arXiv preprint arXiv:2204.0499) 2022.
Dandan Huang, Leyang Cui, Sen Yang. Guangsheng Bao, Kun Wang, Jun Xie, and Yue Zhang. What have we achieved on text summarization? In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), ) pages 446-469, Online, November 2020, Association for Computational Linguistics. doi: 10.18663/1112220emml mamlp-mainn.3
URL https: ftttps://aclanthollgg. org. 2020
Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, and Mor Geva. A chain-of-thought as strong as its weakest link: A benchmark for verifiers of reasoning chains. arXiv preprint 240.::20000005530 2024.
Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. WICE: Real-world entailment
for claims in Wikipedia. Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7561-7588 Singapore, December 2023a. Association for Computational Linguistics. doi: 10. 18653/v1/2023. emnlp- main.470. URL
http:///laaaatooo...ooooo www.oogg....2.....e 2023. emnlp-main m400..40
Ryo Kamoi, Tanya Goyal, and Greg Durrett. Shortcomings of question answering based factuality frameworks for eтror localization. In Andreas Vlachos and Isabelle Augenstein. editors, Pro- ceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 132-146, Dubrovnik, Croatia, May 2023b. Association for Computational Linguistics. doi: 00.18653/v1/2000 hawp://ee.......mm...... eacl-main.111 11. URL https:tppp.. http:////aaaaaaaolo..ooooo mocl-main.11
11

---

Kundan Krishna, Sanjana Ramprasad. Prakhar Gupta, Byron C Wallace, Zachary C Lipton, and Jeffrey P Bigham. Genaudit: Fixing factual errors in language model outputs with evidence. arXiv preprint arXiv:2402. 12566, 2024.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332--3346, Online, November 2020. Association for Computational URL
Linguistics. doi: 10. 116655//1/22222.emmm-maaaaaa emnlp-mane 750
Philippe Laban, Tobias Schnabel, Paul N Bennett, and Marti A Hearst. Summac: Re-visiting nli- based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163-177, 2022.
Nelson Liu, Tianyi Zhang, and Percy Liang. Evaluating verifiability in generative search en- gines. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7001-7025, Singapore, December 2023, Association for Computational Linguistics. doi: 10. 1/2.8665//11222..nnnnnnn 220g/ 1.r8.ffnnding findings-emnly 467.
URL
httts://aclanthhon
Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint
arXiv:190..11692, 2019.
Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. Chatgpt as factual inconsistency evaluator for text summarization. arXiv preprint arXiv:2303. 15621, 2023.
Chaitanya Malaviya. Maavvny Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. Expertqa: Expert-curated questions and attributed answers. arXiv preprint arXiv:2309.07852, 2023.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1919, Online. July 2020. Association for Computational Linguistics. doi: 10.18653/ al/2020.acl-mann. 173. /ttp
org/ 20w.:22222cc111. acl-mmin. 173
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 1011866///DD11-12000 URL
org. Drg/118-1206
Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. Understanding factuality in abstractive summarization with frank: A benchmark for factuality metrics. arXiv preprint AXiv:2104.13340 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1161) 2020.
Sanjana Ramprasad, Elisa Ferracane, and Zachary Lipton. Analyzing LLM behavior in dialogue summarization: Unveiling circumstantial hallucination trends. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12549-12561, Bangkok, Thailand, August 2024a. Association for Computational Linguistics. doi: 10.18453/1/2222..cll lon.llln.77... URL wwt.ttss:/acaaaatholllteo owg. 2024. acl-long. 677
Sanjana Ramprasad, Kundan Krishna, Zachary Lipton, and Byron Wallace. Evaluating the factu- ality of zero-shot summarizers across varied domains. In Yvette Graham and Matthew Purver, editors, Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 50-59, St. Julian's, s, Malta, March 2024b. Association for Computational Linguisties. doi: 10. 10.18653/11/024 shor... URL. https: http:/aclanntte http//olocntttthccccocoo org/ 2024.
12

---

Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, and Alex Wang. Questeval: Summarization asks for fact-based evaluation. arXiv preprint arXiv: 103. 12693, 2021.
Zhaochen Su, Jun Zhang. Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng. Conffictbank: A benchmark for evaluating the influence of knowledge conflicts in llm. arXiv preprint arXi::2488. 2024.
Liyan Tang, Tanya Goyal, Alexander R Fabbri, Philippe Laban, Jiacheng Xu, Semih Yavuz, Wojeiech Kryšciáski, Justin Rousseau, and Greg Durrett. Understanding factual errors in summarization: Errors, summarizers, datasets, error detectors. arXiv preprini arXiv:2205. 12854, 2022.
Liyan Tang, Philippe Laban, and Greg Durrett. Minicheck: Efficient facttcheckinn of Ilms on grounding documents. arXiv preprint TrXiv:244.110744 2024a.
Liyan Tang, Igor Shalyminov, Amy Wong, Jon Burnsky, Jake Vincent, Yu'an Yang, Siffi Singh, Song Feng. Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, and Kathleen McKeown. TofuEval: Evaluating hallucinations of LLMs topic-focused dialogue summarization. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Mexico City, Mexico, June 2024b. Association for Computational Linguistics. doi: 10. (0.186533//2222.neeelloooo
Technologies (Volume 1: Long Papers), pages
URL https
//aclanthology.o org.//looooooooooooooooooo 2024. maacc-loog 251
Alex Wang, Kyunghyun Cho, and Mike Lewis. Asking and answering questions to evaluate the factual consistency of summaries. arXiv preprini 04422 2020.
Bin Wang, Chen Zhang, Yan Zhang, Yiming Chen, and Haizhou Li. Analyzing and evaluating faithfulness in dialogue summarization. arXiv preprint arXiv:2210.11777 2022.
Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. Is chatgpt good nig evaluator? a preliminary study. arXiv preprint arXiv:2303.04048 2023a.
Yuxia Wang. Revanth Gangi Reddy, Zain Muhammad Mujahid, Amav Arora, Alcksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, et al. Factcheck-gpt gpt:: End-to-end fine-grained document-level fact-checking and correction of Ilm output. arXiv preprint 1xXi:2331.000000 2023b.
Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Cheng Niu, Randy Zhong. Juntong Song, and Tong Zhang. Ragtruth: hallucination corpus for developing trustworthy retrieval-augiee -aggentdd language models. arXiv preprint vrXiv:2401.00300 2023.
Wen-wai Yim, Yujuan Fu, Asma Ben Abacha, Neal Snider, Thomas Lin, and Meliha Yetisgen. Aci-bench: a novel ambient clinical intelligence dataset for benchmarking automatic visit note generation. Scientific Data, 10(1):886, 2023.
Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. Alignscore: Evaluating factual consistency with a unified aligmment function. arXiv preprint arXiv:2305.16779, 2023.
Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B Hashimoto. Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics, 12:39-57, 2024.
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards unified multi-dimensional evaluator for text generation. arXiv preprint arXiv:2210.07197, 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: both the abstract and introduction we list the contributions and point to specific sections addressing each one.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In each section, we highlight relevant caveats, methodological limitations, and factors affecting result interpretability. For instance, in Section 4.1, we note that benign edits may introduce noise: in Section 5, we discuss limitations of our gaming string setup. We also include a limitations and ethics statement.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
No theoretical proofs in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We point to specific appendix sections for details of dataset splits, models, exact prompts and analysis details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification:
We plan to release full code and will strive to prior to the conference.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //nips. lttaps.cc/uublce wwtppbbiccgguueeeeeoooooocppcped
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All details in appendix and clearly referenced in main sections with enough detail to read results/claims. Appendix A (dataset splits), B/C (Model details), D, E for prompts used and E for analysis details. Gaming results include exact strings used. (E.1)
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: We report confidence intervals, p values and correlations in section, while other plots include boxplots accompanied by the mean to illustrate score distributions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification: All results include an analysis of popular and easy to run metrics appropriately cited.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httus://neurips.. ctpppps.cccccieeeeee p http://wwwppp..ooooo/oc/ccccccaa
Answer: [Yes]
Justification: Yes, conform the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, we discuss the broader impacts of our work in practical recommendations for the community based on our findings.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release any benchmarks/high risk models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We list and cite all benchmarks, datasets, and metrics used in this work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
We list the LLM used along with prompts for evaluation.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy pottts://neurre http://rpppscccoooeeeeneeee///// /c//Conferecccss/2225//// for what should or should be described.
20

---

Appendix
A
Benchmark Datasets
We list the datasets used in the benchmarks and specify which split (train/test) they are included in Table 3
Benchmark
Train Datasets
Test Datasets
LLM-
TofuEval-MeetB [Tagg et al.
Aggrefact Tang
2024b]
al.
2024a
ExpertQA MMalviya et al.
2023
Lfqa CChnn et al. 2023
RAGTruth [Wu et al.
2023
FactCheck-GP1 [Wang et al.
20235
Wice [amoi
2023a
TofuEval-MediaS Tang et al.
2024b
Claim Verify [Lu et a
2023
Reveal [Jacoii et al. 2024
Aggrefact Tang
FRANK Pagnoni et al. 2021
al. 2022
Polytope Huang et al. 2020
FactCC
Kryscinski ct al.
2020
Goyal21
Goyal and
Durrett 2021
Wang20 [aang al. 2020
Cao22 [Cao et al. 2022
XSumFaith MMaynez al.
2020
CL.IF [Cao and Wang 2021
SummEval Fabbri et al.
2021
FacEval Wang et al.
SAMSUM
Gliwa et al.
2022
2019b
LLMM-Dialogue
SAMSUM Gliwa
al.
Ramprasad et al. 2024a
2019h
Dua
[Chen
al.1 20216
Genaudit Krishnal
XSUM
Narayan et al. 2018
REDDIT. ACIBENCH
et al.
2024
Yim et al.
2023
Table 3: Dataset included in the train and test splits for each benchmark.
B Heuristic Indicators of "Factuality"
The methods for automatically scoring summary factuality reviewed above often rely on complex models, such QA or NLI-based approaches, specialized models. Could simpler heuristics provide comparable signal? Here we explore set of simple features for predicting "faithfulness" to evaluate if they sufficient for this task.
One of the simplest features we use is lexical overlap between a summary and its source. We measure this using ROUGE-2 Fl which matches word pairs bigrams between the summary and the source.
21

---

We also include enfity overlap, i.e., the proportion of entities in the summary that are present in the corresponding source text.
Another predictor we use is semantic similarity. Specifically, embed summary and source sentences via BERT (Devlin 2018), then score each summary sentence based which source sentence is most similar to (using cosine similarity). We average all summary sentence scores to derive a final score,
We include word and sentence novelty ratio features, which measure the proportion words and sentences in the summary that do not appear in the source, respectively.
Finally, we measure text conciseness by calculating a conciseness ratio, the ratio of the length (number of words) of the source document to the length of the summary. This measures the extent to which summary has condensed the original text.
C Shallow MLP Model
We train MLP classification model to predict factuality labels using shallow features, with two hidden layers of dimensions 100 and 50 with a learning rate of 0.001.
D Benign Summary Manipulations
Table lists the prompts used with GPT-40-mini generate summary variants, each designed to vary in specific ways without affecting factual consistency.
We spot-check examples to ensure that manipulations preserve factual meaning and do not introduce contradictions or new factual errors. However, this process is not exhaustive, and undetected issues may introduce some noise into our results.
E
Can we game factuality metrics?
E.1 Metric-Specific Gaming Phrases
We provide the assertion phrases used per metric in Table
E.2
Metric Score Shifts: Gaming Strategies vs. Model Improvements
To contextualize results better, we show score improvements when summaries are gamed along with score improvements brought about by summaries from more complex models in Figure 6
E.3 Gamed summary examples
Gamed summary examples are provided in Table
22

---

Rewrite type
Prompt
Shuffled
Rewrite the following text by changing the order sentences without altering the original meaning of the text.
Note: You must not omit any information from the original text or alter its meaning.
Text: <summary>
Rewritten Text:
Added Source
Edit the following summary by adding a sentence from the source. Ensure the source sentence added is the most irrelevant the summary.
Text
Source: <source>.
Text: <summary>
Edited Text:
Less
Rewrite the following summary by decreasing the variety of vocabulary used. Note: You must omit any information from the original text alter its meaning
Diverse
Text: <summary>
Rewritten Text:
Negated
Rewrite the following text by introducing negation in way that preserves the original meaning of the text.
Note: You must not omit any information from the original text alter its meaning
Text: <summary>
Rewritten Text:
Simplified
Rewrite the following text by simplifying any complex sentences.
Note: You must omit any information from the original text alter its meaning
Text: <summary> Shortened Text:
Shortened
Rewrite the text concisely.
Note: You must omit any information from the original text alter its meaning
Text: <summary>
Rewritten Text:
Paraphrased
Paraphrase the text.
Note: You must not omit any information from the original text or alter its meaning
Text: <summary>
Paraphrased Text:
Synonym Replacement
Rewrite the following text by replacing some common words with synonym.
Note: You must not omit any information from the original text or alter its meaning
Text: <summary>
Rewritten Text:
Table List of summary manipulations along with the GPT-4 prompts used to rewrite summaries and obtain targetted manipulations.
23

---

Metric
Assertion phrase
QuestEval
The summary is consistent with the information the document dis- cusses.
SummaC-
The summary entails the information the document discusses.
Conv
UniEval
The claim is consistent with the information the document discusses.
AlignScore
The summary entails the information the document discusses.
MiniCheck
TThe claim entails the information the document discusses.
ChatGPT-DA
The summary is consistent with the information the document dis- cusses.
Table 5: Metrics and the corresponding constant phrase used to game them.
Filler Type
Filler phrase
Baseline
any case, understanding complex topics requires a multifaceted approach.
Qualifier
This summary reflects one possible understanding, though interpreta- tions may differ.
Table 6: Additional filler phrases used to artificially inflate scores
QuestEval (QA)
(NLI)
(Specialized)
improvament
Baseline phrase
Qualifier
Top phrase
Assertion phrase
(+Baseline)
Summ [+Qualifier)
Summ (+Tbp)
Summ Assertion)
-0.3 -02 -0.1 0.0
0.1 02 o'
-0.3 -0.20020.11
0.0
0.1
0.2
0.3
003
0.2
0.1
0.0
0.1
0.2
AlignScore (Specialized)
MiniCheck (Specialized)
ChatGPT-DA (Prompt)
model imorovement
Daseline phrase
Qualifier eena
Top phrase
Summ i Baselinel
Summ (+Qualifee)
(+Top)
Summ (4 Assertion)
0.3 0.2
-0.1
0.1
0.2
0.3
-033 0.2 -0.
0.0
0.1
0.2
0.3
-0.3
-02
-0.1
0.0
0.1
0.2
0.3
Mean
Difference (Gamed Summary Original Summary)
Figure 6: Score improvements from gaming from using "better" models. Gaming strategies lead to score improvements comparable to or greater than boosts from model improvements.
24

---

QuestEval (QA)
SummaC-Conv (NLJ)
UniEval (Spe (Speiizliddd
Baseline phrase
Qualifier phrase
Top phrase
Assertion phrase
Summ (+Baseline)
Summ ++ Qualiffeer
Summ (+ Top)
Summ (1 (AAsrttoon))
000000000
AlignScore (Specialized)
MiniCheck (Specialized)
ChatGPT-DA (Prompt)
Baseline phrase
Qualitier
Top
Assertion phrace
Summ (+Baseline)
Qualifier
Too
Summ (+Assertion)
Mean
Score Difference (Gamed Summary Original Summary)
Figure Score improvements from gaming with two other types of prefixes: "baseline" and "qualifier" compared to our "top" and "assertion" gaming prefixes. Baseline prefixes are not meaningful, while the qualifier prefix implies the summary's accuracy can vary based on interpretation.
25

---

SummaC-Conv (NI.))
Original Summary
147 people, including 142 students, are in critical condition.
0.26
Constant Phrase 2
The summary entails the information the document discusses
0.99
Summary w/ Phrase 2
people, including 142 students, in critical condition. The summary entails the information the document discusses
0.85
UniEval (Bespoke)
Original Summary
Byte Pair Encoding offers benefits in terms confidentiality
0.48
Constant Phrase 2
The claim is consistent with the information the document discusses
0.99
Summary w/ Phrase 2
Byte Pair Encoding offers benefits in terms of confidentiality. The claim is consistent with the information the document discusses.
0.74
Align (Bespoke)
Original Summary
The PlayStation was released the UK on November 29, 2013
0.33
Constant Phrase
The summary entails the information the document discusses.
0.93
Manipulated Summary
The PlayStation 4 was released in the UK on November 29, 2013. The summary entails the information the document discusses.
0.76
MiniCheck (Bespoke)
Original Summary
Water exhibits phenomenon known as 'structural memory."
0.005
Constant Phrase
The document discusses
0.98
Summary w/ Phrase
Water exhibits phenomenon known 'structural memory. The document discusses.
0.49
Table Selected examples of original and manipulated summaries with corresponding metric scores.
Only metrics identified as gameable are shown.
26


Reviews:
[{"review_id": "vIvcMVRPNr", "reviewer": "Reviewer_kyR2", "summary": "This paper presents a comprehensive evaluation of model-based factuality evals, focusing on whether a model-generated summary is grounded in the source document. The paper presents a series of experiments with interesting findings, e.g., that the correctness of these evals correlates strongly with difficulty levels defined by spurious features (e.g., word overlap with the source document), that model-based evals are sensitive to benign modifications while being insensitive to meaningful modifications (e.g. contradictions), that LLM-based evals rely heavily on their internal knowledge rather than the source document, and that these LLM-based evals are gameable, e.g., by adding a sentence like “The summary entails the information the document discusses”.", "questions": "Would appreciate authors' thoughts on the weaknesses in the previous field.", "limitations": "N/A", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper evaluates a wide range of factuality evals and includes diverse summarization benchmarks, capturing a broad range of error types and settings.\n- The idea of using spurious features like word overlap and entity overlap to define and categorize difficult levels is useful.\n- The analysis with different types of perturbations such as corrections and paraphrases is insightful.\n- I found the recommendation at the end of the paper particularly insightful, such as cautions against using model-based evals for text that is counterfactual or does not align with LLMs’ internal knowledge such as myths, misinformation, or uncommon facts.\n\nWeaknesses\n\nI do not believe there are particular flaws in the paper. I do have some reservations about certain conclusions drawn from the experiments. For instance:\n- For the first experiment in Section 5.2, I do not believe that little gap between evaluations with and without access to the source document necessarily indicates that the model relies heavily on parametric knowledge. If the content of the source document is already aligned with a model’s internal knowledge, the gap may naturally be small, even when the evaluator does ground to the source document when it has access to it.\n- The second experiment in Section 5.2 is more compelling. However, the nature of counterfactuals in the modified text can affect the result, e.g., text that is intentionally written to be contradictory to the original text is likely to be highly unrealistic and unnatural. It would have been more interesting if the text is realistically counterfactual, e.g., myth, or text about very recent events that the LLMs would not have internal knowledge about.\n- Section 6 feels less compelling to me. There’re always ways to make adversarial examples to fool the evaluator, e.g., adding a sentence “The summary entails the information the document discusses” looks like a type of a jailbreaking attack. However, this is unlikely to be actually generated by a reasonable summarization model. If a takeaway is that a summarization model shouldn’t be trained with these metrics (e.g., as a reward model) because it’s too vulnerable to reward hacking, I think it is reasonable. However, assuming that the summarization model is not directly exploiting this metric, it is unclear how relevant this finding is in real world scenarios.\n\nI also think the overall findings of the paper are, while solid, not particularly surprising or new. It is already well understood that model based evaluators are far from perfect, but the reason we still use them is because they are the only practical & cost-efficient alternative to human evaluation. And I do not believe that this paper is offering actionable strategies for improving these model-based evaluations.\n\nI particularly appreciated the recommendation to exercise caution when applying model-based factuality evals in text involving myths, misinformation, or uncommon facts, and think it would have been significantly better if the paper had directly demonstrated this issue. This paper took a different route by making various synthetic perturbations, which is valuable, but I think this always raises questions on how well these findings on synthetic text generalize to real-world text.\n\nMinor questions/suggestions:\n- The categorization of factuality evaluation methods into QA-based, NLI-based, and LLM-based does not seem to be most informative in my opinion, e.g., QA vs. NLI is a matter of a format choice rather than a fundamental methodological difference. How about prompt-based vs. fine-tuning based (which could either be QA or NLI)?\n- Figure 2: The scales across three graphs appear inconsistent, which makes visual comparison difficult. In addition, I think it might be more effective to group results by eval methods and compare performance across difficulty levels within each group, rather than grouping by difficult levels and comparing across eval methods.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback and are glad that they found the analyses interesting. It seems that the reviewer appreciated the findings here but has some reservations about the conclusions drawn and how we have presented them; we appreciate their comments  and feel we can readily address this in revision by adding additional context around the results presented. \n\n**Re \"For the first experiment in Section 5.2, I do not believe that little gap between evaluations with and without access to the source document necessarily indicates that the model relies heavily on parametric knowledge...\"**\n\nWe agree that this does not necessarily indicate that the model relies heavily on parametric knowledge; it is, however, a piece of evidence suggesting that it might. We ran the second experiment exactly because this first exercise is merely suggestive, and are glad the reviewer found this more compelling. That said, we will add a comment to our description of the first experiment in 5.2 noting that—as the reviewer rightly points out—there is an alternative plausible explanation for this particular initial finding (i.e., that the source documents are aligned with model parametric knowledge already). \n\n**Re: \"The second experiment in Section 5.2 is more compelling. However, the nature of counterfactuals in the modified text can affect the result.\"**\n\nThis was a targeted experiment intended to deepen the preceding analysis, as suggested by the reviewer. These are focussed edits that we made in a way that preserves readability and fluency; these are minimal rewrites. Therefore, we would not characterize these as “highly unrealistic and unnatural”, though it is true that they are—by construction—synthetic. \n\n**Re: \"It would have been more interesting if the text is realistically counterfactual eg., myth, or text about very recent events that the LLMs would not have internal knowledge about.\"**\n\nWe agree that this is very interesting, and have conducted a set of experiments based on this suggestion which we will add to the paper. \n\nConsistent with our guidance, the punchline is: LLM-based consistency evaluation fares worse when it is presented with summaries to be  scored with respect to inputs that themselves contain counterfactual (factually incorrect) information. This experiment (which we elaborate on below) strengthens the case for being careful of using LLM based metrics for consistency check when evaluating summaries of mythological documents or documents that contradict GPT's internal knowledge. \n\nThe details of the experiment we ran, which we will add to the paper, are as follows. We used the “ConflictBank” dataset (Su et al. 2024), which comprises factual claims taken from Wikipedia along with corrupted non-factual claims obtained by performing targeted substitutions in the factual claims. Each non-factual claim is associated with a corrupted reference that supports it—that is, this corrupted reference is also non-factual in a way that supports the non-factual claim. For each factual claim, we obtain corresponding factual (uncorrupted) references from Wikipedia. So we have four pairs: \n(a)  <factual articles, factual summaries>, \n(b) <counterfactual (non-factual) articles, counterfactual (non-factual) summaries>, \n(c) <factual articles; counterfactual (non-factual) summaries>, and \n(d) <counterfactual (non-factual) articles, factual summaries>. \n\nFrom a consistency evaluation perspective, (a) and (b) are equally “consistent” (both are cases where the summaries are consistent with the inputs), and (c) and (d) are equally “inconsistent” (both are cases where the summaries contradict the inputs). The only difference is that in (c), the summaries are factually incorrect—counterfactual—with respect to the world at large (and potentially knowledge that GPT has learned), while in (d) they are factually accurate (even while contradicting the corresponding inputs).\nWe aim to evaluate whether LLM-based factuality scoring is compromised when evaluating content against counterfactual references, even when those references support the claims being evaluated. To do so, we prompt GPT to score both supported/unsupported summaries against different reference text.\n\nIn the counterfactual reference setting, we evaluate supported summary (b) and unsupported summary (d) against counterfactual references. Expected behavior would show b>d (supported summaries score higher). However, if d>b, this could suggest GPT's internal factual knowledge is overriding the reference-based evaluation, inappropriately favoring factually accurate but reference-unsupported content. We find that GPT incorrectly rates the unsupported (but factual) summary higher than the supported summary in 3% of cases.\n\nAs a baseline, we also evaluate supported summary (a) and unsupported summary (c) against factual references. Here, GPT rates unsupported summaries higher only 0.2% of the time. The increase in incorrect summaries being rated higher (0.2% --> 3%) demonstrates that GPT struggles significantly more with consistency evaluation when references contain misinformation or contradict established facts. This suggests the model's evaluation reliability could be compromised in real-world scenarios involving contested information, updated knowledge, or reference materials that contradict GPT's knowledge.\n\nWe also compare the score differences between supported and unsupported summaries in both settings, namely (a - c) and  (b - d), using a paired t-test on the two deltas. The test yields a t-value of 10.28 and p <<  0.05, i.e., the score gap between supported and unsupported summaries is significantly smaller when evaluated against counterfactual references compared to factual ones. This suggests that GPT struggles more to distinguish between supported and unsupported summaries when the reference contradicts its internal knowledge. In particular, when evaluating summaries against references that contradicts its internal knowledge, the model shows reduced sensitivity to whether the summary is actually supported by evidence. \nAgain, we will add this result to the paper and thank the reviewer for the suggestion. \n\n\n**Re: \"Section 6 feels less compelling to me. There’re always ways to make adversarial examples to fool the evaluator...it is unclear how relevant this finding is in real world scenarios.\"**\n\nWe agree with the reviewer that we could do a better job contextualizing these results and discussing their potential implications. We will do so in the revised draft. \nBriefly, there are a few practical takeaways here in our view. First and foremost, ideal metrics are not “hackable” (e.g., in classification tasks, one cannot artificially inflate accuracy; even ROUGE is difficult to “game”, which made it a useful though imperfect metric for many years); adversarial approaches work here only because an LLM is being used as an assessor, which is a relatively recent innovation. As a direct implication of this, we agree with what the reviewer says: The fact that one can trivially inflate “factuality” scores in this way implies that using this as a reward signal in fine-tuning is probably a bad idea. So this is one practical takeaway which we believe to be relevant, and we will note this explicitly in the paper. \nAnother practical takeaway is that such metrics alone are not suitable to compare models for factuality for, e.g., in “leaderboard” type setups (such as Vectara’s Hallucination Leaderboard) where researchers compete to build more “factually accurate” models, because one could ascend the leaderboard by including adversarial phrases to boost the LLM factuality score. Of course, additional rules could be put in place to try and disallow such “hacks”, but this might be harder than it appears: We have shown the existence of a few adversarial manipulations that inflate scores, but there are likely many such modifications that one could make. Some of these may be less obvious than the ones we used here. Factuality as a metric may then fall victim to Goodhart's Law (which states that a metric ceases to be useful as soon as it is optimized for), with researchers finding certain prefixes that consistently raise scores instead of meaningfully improving summaries. \n\n\n**Re: \"The categorization of factuality evaluation methods into QA-based, NLI-based, and LLM-based does not seem to be most informative in my opinion...\"**\n\nWe think it is worth distinguishing between QA and NLI, but agree with the reviewer that the bigger differentiator is prompt-based vs. fine-tuned. We will revise the presentation to emphasis this split, while still reporting granular results to tease apart QA and NLI methods \n\n**Re: \"Figure 2: The scales across three graphs appear inconsistent, which makes visual comparison difficult. In addition, I think it might be more effective to group results by eval methods and compare performance across difficulty levels within each group\"**\n\nThe reviewer is correct about the scales: We will fix this presentation issue in revision and appreciate the suggestion. \n\nWe also welcome the grouping suggestion, and will add a plot presenting results this way as well."}, {"review_id": "GDghNmftWa", "reviewer": "Reviewer_o3Me", "summary": "The paper stress tests multiple factuality metrics to determine their in depth meaning. The authors finds that all metrics struggle in hard examples, many metrics are sensitive to benign edits, some metrics can be gamed by adding irrelevant phrase, and GPT relies on internal knowledge rather than the provided documents/context.", "questions": "1. Do the gaming strategies work across different domains and text types, or are they specific to the datasets tested? \n2. Are the reported differences between metrics statistically significant? The paper mentions confidence intervals in Section 5.2 but statistical testing across all experiments is unclear.\n3. I would like to see the explanation for the questions raised in the weakness part.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1. The evaluation includes multiple datasets and covers diverse domains.\n2. Gaming experiments effectively demonstrate the vulnerability of existing metrics to superficial manipulations which reveals important limitations in current evaluation approaches.\n\n**Weakness**\n1. Gaming idea is very impressive but I was wondering how this matrics varied the score so much given the phrases (\"the document discusses\", \"the summary entails.) which are very impractical. While the gaming results are impressive, the extreme score improvements (e.g., from 0.005 to 0.49 for MiniCheck) seem suspiciously large. The TF-IDF setup for identifying gaming phrases needs much clearer explanation. How exactly were these phrases selected, and why do they produce such dramatic effects?\n2. The paper uses a simple MLP with basic features to categorize \"easy\" vs \"hard\" examples. This classifier achieves modest performance, raising questions about whether the difficulty categorization is meaningful.\n3.No code is provided, making it extremely difficult to understand the experimental setup. This significantly undermines the paper's impact and verifiability as it is hard to understand the all the setup from given prompts only. Many experimental procedures lack sufficient detail for replication.\n4. Line 155 describes the difficulty categorization in a very confusing manner.\n5. The relationship between prediction accuracy, confidence, and difficulty labels is unclear.\n6. The shallow MLP baseline methodology needs better explanation.\n7. KeyBERT is mentioned but its usage context is unclear.\n8. Appendix B mentions \"conciseness ratio\" but doesn't define it clearly.\n9. Typos\n\n- Figure 1 upper right: \"me\" should be \"m̂\" (m hat)\n- Line 211: \"Tu\" should be \"To\"\n- Line 554: \"GPT-40\" should be \"GPT-4o\"\n- Line 561: \"summer\" should be \"summary\"\n- Line 92: Missing full stop\n- Line 122: Awkward phrasing\n- Line 560: Missing full stop after \"Table 5\"\n- Line 575: Missing full stop", "was_revised": true, "final_justification": "The rebuttal improves the clarity of several methodological components and helps contextualize the results more effectively.", "rebuttal": "We thank the reviewer for their comments and respond to the specific issues raised below.\n\n**Re: \"Gaming idea is very impressive but I was wondering how this metrics varied the score so much given the phrases (\"the document discusses\", \"the summary entails.) which are very impractical...\"**\n\nWe agree with the reviewer that these are surprisingly large swings, and indeed find the exercise interesting for this reason. We also note that, as noted in the caption of Table 2, the particular change cited (0.005 to 0.49) was a specific qualitative example where the inclusion of one of the phrases (phrase 1) substantially changed the score. The average changes for MiniCheck over all examples (reported in Figure 5) ranges from 0.15 to about 0.30; still large but not quite as dramatic. \n\nWe agree with the reviewer that the procedure we used for identifying phrases could be clarified, and we will edit the paper to better describe this procedure. Briefly, we compute TF-IDF scores for bigrams across documents and manually examine the highest-scoring bigrams that are disproportionately associated with summaries labeled as ``factual'' i.e. summaries with high consistency scores. We emphasize that the main aim here was to show that such phrases exist, and this simple approach demonstrably accomplished this aim (since we identify phrases that do consistently influence results). We acknowledge that more sophisticated methods for searching for such phrases may find even more effective strings .\n\n**Re: \"The paper uses a simple MLP with basic features to categorize \"easy\" vs \"hard\" examples. This classifier achieves modest performance, raising questions about whether the difficulty categorization is meaningful...\"**\n\nWe agree that the paper should better motivate our aim with this set of experiments and the construction of the MLP, and also that our description of difficulty categorization as written is confusing. \n\nIn brief, we intentionally use a “simple” model here (i.e., one based on “shallow” indicators like word overlap) to try and identify cases where it is relatively easy (and maybe even trivial) to spot (in)accurate summaries in the sense that an MLP using only surface-level features can do so with high confidence. If all examples were this straightforward, we would not need sophisticated factuality scoring models, because a simple model would suffice. But there are instances where the simple MLP does not do as well, e.g., those in the medium and hard categories. Our objective here is to understand how metric performance varies as the complexity of the evaluation task increases and use the \"simple\" model (MLP’s) limitations as a proxy to surface examples that require more nuanced evaluation.\n\nWe would argue that the results in Figure 2 provide strong evidence that the difficulty categorization is meaningful: The average metric performance declines by about 21 absolute points on “easy” vs “hard” examples, according to the MLP categorization. (The decline is about 11 points from “easy” to “medium”.) \nRegarding these categories and the reviewer’s comment about the confusing description on L154-155: The idea here is that some examples can be readily classified as “factual” (or not) using shallow features like word overlap; these are “easy” examples. By contrast,  “medium” and “hard” categories capture examples that are difficult to classify with respect to factuality using these shallow features. We operationalize these categories as follows. “Easy” instances are those that the MLP predicts correctly with high confidence (these are examples that are easy to get right using only shallow features). We consider “medium” examples to be those where the MLP is again correct, but where it has lower confidence. Finally, “hard” instances are those that the MLP misclassifies. We will clarify this in the text.\nWe also agree with the reviewer that we should add more details about the MLP and its performance; we will do so in the updated draft, and also add more text explaining our motivation here. \n\n**Re: \"No code is provided, making it extremely difficult to understand the experimental setup. \"**\n\nWe fully agree with the reviewer that the code should be available, and will release code and data to reproduce all results. We apologize for not making an anonymized repository available ahead of submission; we would do so now, but the program chairs have explicitly forbidden it, unfortunately (they “prohibit using any links in the rebuttal, including but not limited to anonymous or non-anonymous URL links”). \n\n**Re: \"KeyBERT is mentioned but its usage context is unclear.\"**\n\nWe agree that this should be clarified in the paper. Briefly, KeyBERT is an existing method for extracting keywords from a set of documents. We used this to find key terms in source documents that we could use to search over large pretraining corpora (Dolma v1.6, the pretraining data used for OLMo models). The idea is to investigate a relationship between topic (keyword) frequency in pretraining and responsiveness to edits. The hypothesis here is that models might be more responsive to contradictions introduced in documents on rare topics, because it has comparatively little parametric of these. We find weak evidence supporting this in Figure 4(b).  We will elaborate on KeyBERT and its usage here in the paper. \n\n**Re: \"Appendix B mentions \"conciseness ratio\" but doesn't define it clearly.\"**\n\nWe apologize for any ambiguity here. This is merely the number of words in the source documents divided by the length of the summary. The bigger this is, the more the summary has compressed the source in terms of length.  \n\nWe also thank the reviewer for pointing out various typos and we will fix all of these in revision."}, {"review_id": "Lqw1T6jcGt", "reviewer": "Reviewer_pjyy", "summary": "I appreciate the authors for running a pretty extensive set of comparisons of so-called factuality check methods along many different aspects, such as sensitivity to surface-level features, sensitivity to fact-invariant revisions and adversarial edits. An overall finding at the surface level (pun intended) is that none of the proposed approaches is as good as they were claimed to be originally. Although the authors constantly conclude that prompt-based metrics, such as ChatGPT-DA and MiniCheck, are reasonably good and better than the other metrics, my interpretation is that every metric has some weaknesses that make them not adequate to be used for checking the factuality of any textual statement. \n\nDespite this weakness (in the authors’ interpretation rather than their experimentation), I do believe it is good for the whole community to have this paper. This paper tells us about the limitations of existing factuality benchmarks (that said, I do not believe there could be a “standard” benchmark for factuality, since factuality is almost always in the eyes of beholders,) and that different metrics have different weaknesses and strengths. \n\nThe only thing i miss from this submission is the discussion on factuality itself. Is factuality something we can build a benchmark and test language generators for? Should we approach it as a leaderboard chasing game, when factuality is not about the average behaviour? I would suggest the authors remove Fig. 4(b) (completely uninformative) and add one paragraph on this discussion.", "questions": "See my summary above.", "limitations": "See my summary above.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "See my summary above.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback and interpretation of the work. We agree entirely with the reviewer’s perspective: Every “factuality” metric has certain weaknesses, and none should be taken as a sort of “gold standard”. Our hope is that this work elucidates the issues (and strengths) particular to each kind of factuality metric, ultimately to enable more informed use and interpretation of such metrics, when appropriate. \nWe also agree that the work would benefit from a broad discussion of “factuality” and its measurement. One of the motivations for the “gameability” experiments (Section 6) was to highlight that if we “leaderboard” chase “factuality”, this may lead to researchers exploiting weaknesses in metrics, which we have shown is possible. That is, “factuality” metrics may fall victim to Goodhart’s Law, which says that a metric ceases to be useful as soon as it is optimized for. On the other hand, given the importance of factuality, we do not think its measurement should be abandoned entirely; only that practitioners should be aware of the limitations of automated metrics for something as slippery as “factuality”. We will add a paragraph discussing these issues to the paper, as suggested by the reviewer."}, {"review_id": "gGTreNh1U5", "reviewer": "Reviewer_SyLu", "summary": "This paper evaluates the extent to which commonly used factuality (faithfulness) metrics genuinely reflect the factual consistency of generated outputs. It also benchmarks the robustness of these metrics against a range of perturbations. The authors find that many existing metrics are vulnerable to manipulation, and that ChatGPT-based metrics often rely more on parametric knowledge than on the provided source.", "questions": "See weaknesses.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is well written and highly engaging.\n    \n- It addresses a timely and important problem in the evaluation of text generation.\n    \n- The claims are clearly supported by well-designed and rigorous experiments.\n\n\n**Weaknesses**\n\n- As acknowledged by the authors, Figure 4(b) presents weak evidence and is not fully convincing.\n    \n- The use of GPT-4 for generating edits, while pragmatic, could introduce bias given that one of the evaluated metrics is also based on GPT-4.\n    \n- The “easy / medium / hard” classification of summaries would benefit from additional empirical examples or clarification.\n    \n- A more detailed discussion of the human-annotated datasets used for metric evaluation could strengthen the presentation.", "was_revised": true, "final_justification": "After reading the authors' rebuttal and the other reviews, including the more critical ones such as reviewer kyR2's, I feel confirmed in my understanding and assessment of the paper. It is reassuring to see that several reviewers share similar views on the strengths of the work, even when they assign a lower score.\n\nWhile some of the concerns raised are valid and worth noting, I do not believe they significantly affect the overall quality or contribution of the paper, as even the critical reviews acknowledge.\n\nI maintain my score and express my support for accepting the paper.", "rebuttal": "We thank the reviewer for their thoughtful comments about the work. We address specific issues raised below. \n\n**Re: \"The use of GPT-4 for generating edits, while pragmatic, could introduce bias given that one of the evaluated metrics is also based on GPT-4\"**\n\nThis is a fair point, and we will note it in the revised paper. However, we note that ChatGPT-DA uses a prompt explicitly asking for an assessment of factual consistency between a summary and source, whereas the edits we introduce with GPT-4 are targeted edits explicitly requested: We think it is unlikely that the ChatGPT-DA metric is somehow advantaged because GPT-4 was used to make these edits, though it is possible. (This is not like a scenario where we are simply asking GPT which of two generated examples it “prefers”, where one has been generated with GPT.) \n\nWe also note that if this did confer an advantage to ChatGPT-DA, it would in a sense make the results in 5.2 even more compelling, since we found that the metric is not reliably responsive to inconsistencies being introduced into summaries (these inconsistencies were elicited from GPT-4, so it “knows” they are inconsistent).\n\n**Re: \"The “easy / medium / hard” classification of summaries would benefit from additional empirical examples or clarification.\"**\nWe agree with the reviewer. We will provide additional clarification around these categories and we will include illustrative examples from each category in the Appendix. \nRegarding clarification about these categories: The idea here is that some examples can be readily classified as “factual” (or not) using shallow features like word overlap; these are “easy” examples. By contrast, “medium” and “hard” categories capture examples that are difficult to classify with respect to factuality using these shallow features. We operationalize these categories as follows. “Easy” instances are those that the MLP predicts correctly with high confidence (these are examples that are easy to get right using only shallow features). We consider “medium” examples to be those where the MLP is again correct, but where it has lower confidence. Finally, “hard” instances are those that the MLP misclassifies. We will elaborate this in the text. \n\n**Re: \"A more detailed discussion of the human-annotated datasets used for metric evaluation could strengthen the presentation.\"**\n\nWe agree that this would make the work more self contained (at present these details are in the original papers describing the corresponding corpora). We will add a section to the Appendix that discusses in greater detail the annotated datasets we have used, and point to this in the main text for the interested reader."}]

Author rebuttal:

Meta-review: This paper presents a comprehensive evaluation of model-based factuality evals, focusing on whether a model-generated summary is grounded in the source document. The paper presents a series of experiments with interesting findings, e.g., that the correctness of these evals correlates strongly with difficulty levels defined by spurious features (e.g., word overlap with the source document), that model-based evals are sensitive to benign modifications while being insensitive to meaningful modifications (e.g. contradictions), that LLM-based evals rely heavily on their internal knowledge rather than the source document, and that these LLM-based evals are gameable, e.g., by adding a sentence like “The summary entails the information the document discusses”.

Strengths:
- This paper is well-written and well-organized.
- Factuality is an interesting and important topic in NLG evaluation, and this paper provides a comprehensive investigation of existing automatic evaluation metrics towards factuality.
- The authors provide insightful experiments and analyses.
- The experiments are well-designed, and all the conclusions/claims are verified by rigorous experiments. The overall conclusions and findings are solid.

Weaknesses:

This paper does not have major issues. Though this paper does not introduce new strategies or metrics to automatically evaluate factuality, the analytical work is also important for the community. This paper provides insightful experiments and analyses, and highlights the issues lying in existing evaluation metrics, which can also promote the research in factuality metrics.

Reviewer kyR2 provides great reviews.
Decision: Accept (poster)