SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation
Authors: Song Duong
Florian Le Bronnec
Alexandre Allauzen
Vincent Guigue
Alberto Lumbreras
Laure Soulier
Patrick Gallinari
Venue: ICLR 2025 Poster (2025)
Keywords: faithfulness
hallucination
conditional text generation
natural language processing
large language models
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose a self-supervised method for faithfulness enhancement for conditional text generation.

Abstract: Large Language Models (LLMs), when used for conditional text generation, often produce hallucinations, i.e., information that is unfaithful or not grounded in the input context. This issue arises in typical conditional text generation tasks, such as text summarization and data-to-text generation, where the goal is to produce fluent text based on contextual input. When fine-tuned on specific domains, LLMs struggle to provide faithful answers to a given context, often adding information or generating errors. One underlying cause of this issue is that LLMs rely on statistical patterns learned from their training data. This reliance can interfere with the model's ability to stay faithful to a provided context, leading to the generation of ungrounded information. We build upon this observation and introduce a novel self-supervised method for generating a training set of unfaithful samples. We then refine the model using a training process that encourages the generation of grounded outputs over unfaithful ones, drawing on preference-based training. Our approach leads to significantly more grounded text generation, outperforming existing self-supervised techniques in faithfulness, as evaluated through automatic metrics, LLM-based assessments, and human evaluations.

Published as a conference paper at ICLR 2025
SCOPE: A SELF-SUPERVISED FRAMEWORK FOR IM- PROVING FAITHFULNESS IN CONDITIONAL TEXT GEN- ERATION
Song Duong" .1.4 Florian Le Bronnec* Alexandre Allauzen
Vincent Guigue
Alberto Lumbreras Laure Soulier! Patrick Gallinari
Sorhonne Université, CNRS, ISIR, F-75005 Paris, France
2 Miles Team, LAMSADE. Université Paris DaushDDe Université PSL, CNRS, 75016 Paris, France
3 "AgroParisTech" Tech, UMR MIA-PS, Palaiseau, France
'Criteo Al Lab, Paris, France
ABSTRACT
Large Language Models (LLMs), when used for conditional text generation, often produce hallucinations, i.c.. information that is unfaithful not grounded in the input context. This issuc arises in typical conditional text generation tasks, such text summarization and data-to-text generation, where the goal is to produce fluent text based on contextual input. When fine-tuned on specific domains, LLMs struggle provide faithful answers given context, often adding information or generating errors. One underlying cause of this issue is that LLMs rely on statistical patterns learned from their training data. This reliance can interfere with the model's ability to stay faithful to provided context, leading to the generation of ungrounded information. We build upon this observation and introduce novel self-supervised method for generating training set of unfaithful samples. We then refine the model using training process that encourages the generation of grounded outputs over unfaithful ones, drawing on preference-based training. Our approach leads to significantly more grounded text generation, outperforming existing self-super techniques in faithfulness, metrics, LLse assessments, and human evaluations. Code is available at https://gi thwbb com wwt.c///oooooo//ccooeoecoo scwpe fai thsulness
evaluated through automatic
INTRODUCTION
Large Language Models (LLMs) are widely used for generating fluent and coherent text completions based input contexts (Brown et al. 2020). These models generate completions by leveraging the statistical patterns encoded in their parameters, which are learned from extensive training data. While these parameters provide the model with broad knowledge of various topics, they can also cause interference. This occurs when the model combines information provided in the input context with general patterns from its training data, potentially leading to inaccuracies. More generally, irrelevant content generated by a LLM is commonly referred to as hallucinations (Rebuffel et al.) 2022 Maynez al. 2020). To mitigate these hallucinations, two primary dimensions are considered: factuality and faithfulness (Huang et al., 2023). Factuality refers to whether the model's 's generated information aligns with external, real-world knowledge and typically evaluated against reference dataset or established knowledge. Faithfulness, on the other hand, evaluates how accurately the generated content reflects the information provided in the input context. A model may produce factual but unfaithful content if, while true with respect to world knowledge, it distorts important details from the input or adds extra information (see Table D. This is particularly crucial in fields where accurate information transfer is essential. For instance, in medical transcription, the text output must accurately reflect the content of the medical record without introducing any distortions (Cawsey et al. 1997).
In this paper, we focus on the generation of faithful responses grounded in self-contained input context. A major challenge concerning faithfulness is the difficulty of annotating data and there is no standard way to determine if a text is faithful to an input context. As a result, annotation is typically
performed by humans (Goyal & Durrett 2021 Kryscinski al. 2020). However, this approach is
Equal contribution. Corresponding authors: sluong@critco.coo florian. le-bronnce dauphine. psl.eu

---

Published as a conference paper at ICLR 2025
Patient Data (Input):
Age
Sex
Symptoms
Diagnosis
Treatment
45
Male
Persistent cough
Pneumonia
Antibiotics
Output Examples:
Faithful
Factful
Output
No
No
21 y.o. female with headache due migraine is given antibiotics.
No
45 y.o. male with cough due pneumonia is given amoxicillin.
Yes
Yes
45 y.o. male with a cough due to pneumonia is given antibiotics.
Table I: An example faithful and factful combinations in LLM for data-to-text generation in a medical context. Unfaithful spans are highlighted in red. While amoxicillin is common antibiotic prescription for pncumonia, the name of the antibiotics is not the mentioned in the table.
costly, not scalable, and the resulting annotations might not transfer to other domains. To circumvent the lack of annotated data, some unsupervised methods have been proposed. A first line of research
consists of leveraging a contrastive loss on hidden representations (Zhao et al.) 20200 Kryscinski al. 2019). These methods have demonstrated improvements on small models (around 500 million parameters), but they have not yet been benchmarked on recent LLMs. Our evaluations indicate that their effectiveness does not appear to extend these larger models. Another direction consists of altering the decoding process of pre-trained models (Shi et al.) [023] van der Poel et al. (002)) While these methods work well on generalist text datasets, we found that on more domain specific tasks where heavy fine-tuning is required such as data-to-text generation, these methods struggle to improve over standard fine-tume tuning of models (see Section 5).
Acknowledging these limitations, we propose a novel fine-tuning framework, tailored for recent LLMs. Drawing inspiration from recent work (Rafailov al. 2023), we propose method tailored for recent LLMs that trains the model to disfavor ungrounded generations. Unlike typical preference- tuning which involves human annotation of model--eeerran outputs, we aim for self-supervised process to generate a dataset of prefered and dispreferred samples. Here, in the context of faithfulness, the goal is to teach the model to prefer the context-grounded reference labels over unfaithful ones that present hallucinations. A challenge then lies in the generation of representative unfaithful examples that convey effective learning signals. These examples should closely resemble target sentences while exhibiting realistic hallucinations. In conditional text generation tasks, hallucinations occur when the model's internal knowledge improperly influences the generation process (Maynez et al.) (020). Building on this observation, we propose an original procedure for automatically generating realistic examples. This generation process is fully unsupervised and does not require external resources. We apply our method six datasets across various domains for data-to-text generation and text summarization. Data-to-text generation (Lin et al.) 2024) involves converting structured data like tables into coherent language, while summarization condenses longer texts while preserving key information. Faithfulness is essential for both tasks to ensure the generated text accurately reflects the input data. To summarize, in this paper:
We introduce SCOPE, a new method that leverages ideas from preference training by using a self-supervised generated dataset. In this approach, the model is trained to favor reference labels over carefully generated unfaithful samples.
We empirically show that our approach significantly enhances the faithfulness text generated by fine-tuned LLMss, surpassing current faihhfulnesseehhaneed methods for conditional text generation.
We bring new insights on the behavior of preference-tuning by analyzing its sensitivity the effect of negative samples.
Our experiments reveal that training using SCOPE achieves up to 14% improvement in faithfulness metrics over existing methods, according to automatic evaluation metrics. Furthermore, evaluations by both GPT-4 and human judges indicate that the generations with SCOPE substantially more faithful, with an improved preference win rate against the supervised fine-tuned model that is in average 2.1 times higher than the baselines.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
This section reviews methods for improving LLMs' faithfulness to imput contexts. We focus exclu- sively on approaches that ensure generated content remains grounded in the provided information. excluding techniques related to factuality or external knowledge alignment.
Faithfulness enhancement. Several methods have been used for improving faithfulness of text summarization. A first line of work consist in using external tools to retrieve key entities facts form the source document and use these as weak labels during training (Zhang et al.) (022). CCen et al.) (2022) identify key entities using Question-Answering Answering system and modify the architecture of an encoder-decoder model to put more cross-attention weight on these entities. Zhu et al. 2222)) propose improve the faithfulness of summaries by extracting a knowledge graph from the input texts and embed it in the model cross-attention using a graph- taaph--tranffomer.. Another line of work focuses on post-training improvements by bootstrapping model- gedee-ggeerated outputs ranked by quality (Zhao et al., 2023 Liu et al., 2022; Zablotskaia et al. 2023). Regarding data-to-text generation, |Rebuffel et al. (2022) propose a custom model architecture to reduce the effect of loosely aligned datasets, using leken-level annotations and muuli--branhh decoder model. The closest work to ours is from (Cao & Wang 2021) which proposes a contrastive learning approach where synthetic samples are constructed using different tools like Named Entity Recognition (NER) models and hack-translation. These approaches address specific forms of unfaithfulness and rely heavily on external tools such as NER or QA models, and are especially tailored for text summarization, while we target more general focus. More recently, simpler methods that leverage only a pre-trained model have been proposed for summarization. Shi et al. (2023) van der Poel al. (2022) downweight the probabilities of tokens that are not grounded in the input context, using an auxiliary LM without access to the input context. Lango & Dusek (2023); Zhang al. (2023) both train a self-supervised classification model to detect hallucinations that can be used to guide the decoding process toward more faithful outputs. Tian et al. (2020) propose method to estimate the decoder's confidence by analyzing cross-attention weights, encouraging greater focus the source during generation. Our method focuses a decoder-only architecture and uses a single model, providing a streamlined and efficient approach specifically tailored for general conditional text generation tasks without the need for complex external tools.
Faithfulness evaluation. Mcasuring faithfulness automatically is not straightforward. Traditional conditional text generation evaluation often relies on comparing the generated output to a reference text, typically measured using n-gram based metrics such as BLEU (Papineni et al. 2002) or ROUGE (Lin 2004). However, reference- based metrics limitations are well known to correlate poorly with faithfulness (Fabbri et al., 1 Gabriel et al. (021). Both for summarization and data-to-text generation, new metrics evaluating the generation exclusively against the input context have been aotity-matching metrics 2021). In this work. we evaluate primarily our models using recent NLI-related metrics (Zha al.
proposed, using QA models (Rebuffel et al. 2021 Scialom et al. 2021)
(Nan et al.
2023 Dušek & Kasner 2020), and ILM-as-a-judee focusing on faithfulness (Chiang Gilardi et al. 2023). For data-to-text generation, we also report the PARENT metric Dhingra et al. (019)), which computes n-gram overlap against elements of the source table cells.
& Leel 2023
Preference tuning. Recent instruction-tuned LLMs are often further refined through "human- feedback alignment" (Guo et al., 2024). These methods utilize human-crafted preference datasets, consisting of pairs of preferred and dispreferred texts (y
y (y+.y= typically obtained by collecting human feedback and ranking responses via voting. Recent work (Chen et al., [2024) uses the model's s previous predictions in self-play manner iteratively improve the performance of chat-based models. Whether through an auxiliary preference model (Ziegler et al., 2019) or by directly tuning the models on the pairs (Rafailov et al., (2023). these approaches have demonstrated remarkable results in bhat- models. Our method leverages a preference framework without the need for human intervention and is specifically tailored for models trained on conditional text generation tasks.
METHOD
We introduce SCOPE, a novel approach designed to address hallucinations by overcoming the limitations of standard fine-tuning (Maynez et al. 2020; Cao & Wang 2021). Unlike traditional methods, our two-stage process aims to enhance the model's 's faithfulness. In the first stage, we perform standard fine-tuning to initialize the model. In the second stage, we apply preference tuning, where the model is further optimized using synthetic samples that guide it toward generating more faithful outputs. An illustration of the method is presented on Figure T
3

---

Published as a conference paper at ICLR 2025
Fine-tuning D
Preference- tuning
Training pipeline
P&o
Data generation
Noisy data generation D2 {-, V.
PLM
Figure I: SCOPE training framework. A pre-trained model PLM is first fine-tuned subset D1 of D and produces a model Ps,- mixture of PLM and pe, is then used to generate a synthetic preference dataset, which finally serves for preference fine-tuning.
3.1 TRAINING PHASE
Let D (5).5)))11 be an aligned dataset of context-target pairs used for training.
Fine-tuning. For the first stage, our goal is to get an initial version of a fine-tuned model. We start from a pre-trained model PtM- To better leverage training samples, we propose for this part to train PLM only the first half D1 of the samples of D. We keep the second part D2 of the dataset for the next step. We denote by pe, the model fine-ture sune-tuned from PLM on Dj using cross-entropy. Given the strong sample efficiency of recent LLMs, empirically found that for the datasets used, fine-tuning on only half of the samples was sufficient to achieve a strong initialization for the subsequent stage, Appendix A.3
Preference-tuning.
The second phase involves contrastive learning. Training will be conducted on D2,, the second half of the samples. We augment D2 with artificial unfaithful samples to get a dataset D2 {c, Vi, } Our complete process to generate these samples is described in Section 3.2 For each annotated target y, we have a corresponding noisy y which contains unfaithful patterns.
While other baselines propose to use custom contrastive loss often based on embeddings similarity, we propose optimizing the model to prefer over y by leveraging the recent framework of preference tuning (Rafailov al. 2023), with the following loss:
po(y c) 3 log po, (y 12) ]
po c)
Lo -R BaNDw-)~D, log BBlo
Po.
where is the sigmoid function and [ is scalar hyperparameter that quantifies how much pş deviates from Pe. Intuitively, minimizing Ls w.r.t. amounts to increasing the gap between the likelihood of generating grounded responses y and non-grounded ones y More details about the training dynamics can be found in Section 6 Additionally, we experimented with an alternative preference loss in Appendix Aopendix(A.Jammm and observed similar behavior.
3.2 UNFAITHFUL DATASET GENERATION
In this section, we present our method to generate unfaithful samples. Contrarily to other methods that rely on external tool, such a named entity recognition or number entity recognition, we propose an casier and more general method. When a LLM generates ungrounded spans of text, it is often caused by an interference between the context and the learned statistical patterns acquired during training. A convincing unfaithful sample generated by LLM should satisfy least desiderata: (i) attain the same level of fluency than the target LLM, and (ii) being more or less consistent with the input while containing one several spans of text not grounded the input context. An ideal method would be to run our initial fine-tuned model pe, and find among the samples the ones that are unfaithful. But as discussed in Section [ accurately detecting unfaithful samples automatically is a difficult problem. Instead, we propose simple unsupervised method to simulate the creation of noisy samples. Our strategy is to "force" the model leak internal statistical knowledge in the generation by adopting noisy decoding method using two models simultaneously.
The main model is Pea- the initially fine-tuned model half the dataset. This model generates samples conditionally to the input context, y ~ Pa, c). is supposed to generate text that is grounded in the input context, but still contain inaccuracies due to its shortened training.
4

---

Published as a conference paper at ICLR 2025
The second model is PLM, the pre-trained counterpart of Ps.- This model won't be given access to the imput context and will simply sample from its context-free distribution ~ PLM('), generating general patterns that it has learned.
Both distributions are de facto fluent, but used individually might not be enough to teach anything during preference tuning. PLM samples will obviously not be challenging enough, while po, samples won't contain enough hallucination patterns. Instead we propose to combine both during the decoding process. We generate these noisy samples token by token by sampling mainly from the grounded PH, (- c) and randomly from the non-grounded ptM- This method introduces fluent but non-grounded artifacts, exhibiting both intrinsic errors, i.e. generated outputs that contradict the data, and extrinsic hallucinations, i.c.. generated outputs that cannot be inferred from the data alone (see Table 18). Refer to Algorithm for the complete details of the algorithm.
Algorithm I: saresyyggeerraiionn PLM P66)
Input:c e an input context, PtM the pre-trained model, Pño the fine-tuned model on Dj. for token decoding step L do
1. Sample: On Bernouli(aa) (av E {0, 1}).
2. Sample:
ye (1 at)poo (e):e) + aspisl: (2z))
return y
The mixture parameterized by a, which tunes the noise level within the samples.
corresponds to the fine-tuned model Pe, and a = I corresponds to the unconditional model PLM- This parameter actually plays an important role: the noisy y should contain divergences from the context but still be close enough to the true to provide a meaningful learning signal. This is a sensible step for preference learning, as illustrated later in the experiments (Section 6).
Our detailed pipeline is described in Algorithm 2 Existing preference tuning methods usually depend on offline preference data gathered from various sources and ranked through voting. In contrast, the originality of our approach lies in its ability to automatically generate unfaithful responses, simulating potential hallucinations from model's internal state without requiring supervision. This distinguishes it from traditional preference training, which typically involves human intervention.
Algorithm SCOPE supersupe Context Preference).
Input: :D the training data and PLM the pre-trained model.
// Split the train data
D1, D2 + Split D into two halves
11 1. Initial fine-tuning
pe, Fine-tune PLM on D,
11 2. Noisy generation
D2 (}
for (c in D2 do
+ moissygoneratiion(c.o
Append (c,0,v) Dz
// 3. Preference fine-tuning by optimizing Equation ①1)
pe + Preference fine-tune pa, over Dz, using as the preferred label and y as the negative example
return pe:
EXPERIMENTS
4.1 TASKS AND DATASETS
We evaluate method SCOPE total of datasets, spanning multiple domains and difficulties, where generating grounded context is a crucial requirement. We first run experiments on four data- to-text generation datasets. ToTTo (Parikh al. 2020) is an English dataset with Wikipedia tables
5

---

Published as a conference paper at ICLR 2025
where specific cells are highlighted, paired with sentence describing those cells. WebNLG 2020 (English) (Castro Ferreira et al., 2020) is an English dataset composed of pairs of knowledge graphs and text crawled from DBpedia. E2E (Dušek et al.] 2019) is an English benchmark dataset that verbalizes key-value attribute pairs in the restaurant domain. Fe (Nan et al.) 2022) is an English table question answering dataset with tables from Wikipedia, paired with corresponding questions, answers, and supporting table cells.
We further evaluate the methods on three summarization datasets. XSum (Narayan et al. 2018) contains BBC articles from 2010 to 2017. along with their summaries, each consisting of one highly abstractive sentence. SAMSum (Gliwa et al., 2019) is a dataset of messenger conversations about daily-life topics, annotated with short summaries. PubMed (Tang et al. 2023) is a collection of medical scientific articles where the goal is to summarize the conclusions the authors based on the description of a medical experiment.
Although our primary focus is on domain-specific tasks, Appendix A.7 shows the results of applying SCOPE to generalist model fine-tuned with instructions on the Alpaca dataset (Taori al. 2023
4.2 METRICS
We present in what follows the different metrics used for each task. Having in mind the limitations of BLEU and ROUGE metries (resp. used for data-to-text generation and summarization as standard metrics for each task) and regarding our research objectives, we focus faithfulness metrics that evaluate the generation with respect to the input context.
BLEU (Data-to-text). Traditional metric to assess the similarity between the generated text and given gold references. In the context of data-to-text generation. it has shown limitations especially when reference text diverges from the input data (Dhingra et al., 2019).
PARENT Recall (PAR, Data-to-text). (Dhingra al., 2019) A standard n-gram based faithfulness proxy metric for data-to-text introduced to address the limitations of BLEU. It assesses how well the candidate text replicates relevant entities from the data by measuring its n-gram recall against entities in the structured input. Unlike BLEU, PARENT Recall directly compares to the structured input, making it more suitable measure of faithfulness.
NLI Score (NLL Data-to-text). Proposed by Dušek & Kasner (2020), this metric adapts NLI models to data-to-text It first computes the entailment probabilities of atomic input facts extracted from the structured data by the candidate text, characterizing omissions. A second score measures hallucinations by computing the entailment probability of the generated text by the sum of all the facts in the input data. The resulting NLI score is the minimum of all the entailment probabilities, assessing the overall faithfulness of the generated text.
ROUGE-L (R-L, Summarization). Lin 2004) Traditional n- n-gram overlap summarization metrics between the generated and the gold reference. Similarly to BLEU, it has known limitations (Fabbri ct al. 2021 regarding faithfulness evaluation.
AlignScore (AL., Summarization). Zhh et al. 2023) A recent state-of-the-arr entailment metrics. It measures the information alignment between the summary and the source article on a 0-1 scale, using a RoBERTa model (Liu et al. 2019) trained on a unified set of entailment tasks.
QuestEval (Summarization). (Scialom al. 2021) A reference-fres evaluation metric for summa- rization. It assesses the semantic alignment between the source article and the generated summary by generating and answering questions about their content. QuestEval uses a question generation and answering pipeline, leveraging pre-trained language model, to compute similarity score between the information in the source and the summary.
CactCC (Summarization). (Kryscinski et al. 2020 A factual consistency metric for summarization. It evaluates the factual alignment between the summary and the source article on a binary scale. FactCC relies on fine-tuned BERT model, trained specifically to detect factual consistency through synthetic data generated by introducing factual errors into summaries.
GPT-4 preference (Both tasks). Previous works (Gilardi et al. 2023 Chiang & Lee 2023) have
shown that powerful LLMs, like GPT-4 can serve as effective proxies for human evaluation. To provide a scalable numan-like assessment of the generations' faithfulness, we use GPT-4 for pairwise preference evaluation. Given an input and two texts, the model is asked which sample is more

---

Published as a conference paper at ICLR 2025
faithful to the input data. This metric yields win, loss, and tie rates against the standard fine tne-tuning baseline. Details regarding GPT-4 preference evaluation can be found in Appendix A.8
4.3 MODELS AND BASELINES
We experiment LLAMA2-7B (Touvron al. 2023) and MISTRAL-7B (Jiang al. 2023). two recent LLMs. For all baselines, hyperparameters were carefully determined from a grid-search following recommendations in reference articles, using NIII Score for data-to-text generation and AlignScore for summarization as objectives. All training details and hyperparameters can be found in Appendix A
Supervised fine-tuning (SrT). This is the standard fine-tuning approach where the pre-trained model PLM is optimized using MLE full training dataset D. We train for 3 epochs and choose the model according to NLI Score for data-to-text generation and AlignScore for summarization.
PMI decoding (PM1). (van der Poel et al. 2022) PMI reduces hallucinations by penalizing "ungrounded tokens" when next-token entropy is high, adjusting probabilities using a context-less model with hyperparameters À and 7.
Context-aware decoding (CAD). (Shi al. 2023 Similar PMI, CAD downweights probabilities using a context-less model, with an adjustment factor controlled by a.
CRITIC improves generation by using, for each dataset, a model trained to differentiate context- sontext-supported tokens. It factors the model probability and generates samples based on score combining the token probability and the classifier's context likelihood, adjusted by X.
Critic-driven decoding (CRITIC). (Lango & Dusek 2023
(Cao & Wang 2021) CLIFF is training method that leverages contrastive learning framework, where more positive samples are generated through a back-translation method, while negative samples are created using Named Entity Recognition (NER) models and different mask-and- generate methods. We choose the MASKREL. baseline, which demonstrate strong overall results in the original paper. Initially designed for encoder-decoder models, we reimplemented the method for decoder-only architectures.
CLIFF.
SCOPE (ours). Models trained following SCOPE framework. For the experiments, we tune the noise level a by selecting the value that yields the highest NLI Score or AlignScore on the validation set. As detailed in Section 6 we restrict our search of a to the [0.4, 4, 0.6] interval, which corresponds to zone where the BLEU/ROUGE scores does not decrease significantly. The selected value for each dataset and model can be found in Table 10
CL.FF and SCOPE are methods that present training method, while CAD and PMI modify the decoding process. CRITIC trains a model to modify the decoding process. We highlight that all baselines have been trained on the same amount of annotated samples, since decoding methods are applied to a fully fine-tuned model.
RESULTS
We now present the results of SCOPE and of the baselines on the data-to-text generation and text summarization tasks introduced above.
SCOPE improves faithfulness metrics, training with SCOPE gives consistent and significant improvements in faithfulness compared standard fine-tuning, as presented in Tables[[ann and 3 For data-to-text generation (Table (2). training
all tasks and domains. According automatic faithfulness
models with SCOPE show significant improvements over standard fine-tuning, with an increase of up 8.2 and 5.5 points PARENT and NLJI Score respectively. For text summarization (Table 3 SCOPE demonstrates an increase of up to 8.8 points in AlignScore. On most datasets, SCOPE scores slightly lower BLEU and ROUGE scores than other baselines, especially on the abstractive XSum dataset. Previous work (Tanya Goyal 2022) highlighted the saturation of summarization benchmarks and the limitations of reference-berne based metrics like BLEU and ROUGE in evaluating the summarization capabilities of recent LLMs. Given the high faithfulness scores achieved by SCOPE on both tasks, suggest that this decrease in BLEU and ROUGE may indicate SCOPE's tendency to deviate from standard fine-tuning and to disfavor irrelevant generations.
7

---

Published as a conference paper at ICLR 2025
ToTTo
E2E
FeTaQA
WebNLG
NIJI
PAR
BLEU
NI.1
PAR
BLEU
NLI
PAR
BLEU
NLAI
PAR
BLEU
LLAMA2-76
SET
46.42
80.55
92.62
86.41
41.81
39.06
78.68
39.72
79.36
79.19
48.37
CAD
46.33
92.74
86.35
41.32
39.67
78.93
39.64
79.62
79.45
48.95
CRITIC
46.22
80.66
92.70
86.45
41.82
78.67
39.94
79.47
79.51
48.83
PMI
46.36
80.51
92.66
86.42
41.78
39.23
78.52
39.71
79.54
79.30
48.45
CLIFT
46.69
80.77
92.64
86.47
41.78
39.67
79.11
40.48
79.92
79.31
47.99
SCOPE (ours)
51.88"
86,11'
94.64
87.21
38.70
42.97
83.40
38,96
83.42"
85.95
48.16
LLAMA2- --MA2-138
46.56
80.07
CAD
46.68
80.66
93.25
86.41
41.24
39.56
79.21
40.65
82.55
79.06
49.78
CRITIC
46.59
80.73
93.58
86.44
41.17
39.82
79.51
40.37
80.24
78.37
49.10
PMI
46.55
80.46
93.43
86.35
41.23
40.03
79.32
40.77
80.02
78.38
49.02
CLIFF
47.04
80.68
92.42
86.47
41.49
38.85
79.06
41.05
80. 1.5
79.09
48.16
SCOPE (ours)
86.58
91.61
87.37'
83.30
36.77
MISTRAL-7B
SFT
46.70
80.79
92.64
85.88
41.16
39.90
79.31
41.47
84.71
80.58
50.85
CAD
CRITIC
46.72
80.75
92.80
85.97
40.00
39.55
79.50
41.43
84.62
80.71
50.94
PMI
46.48
80.33
92.80
85.88
41.18
39.80
79.30
41.49
84.86
80.58
50.87
CLIFF
47.30
80.89
92.86
85.
40.25
79.45
41.88
84.29
50.57
SCOPE (ours)
$3.45
89.01"
93.43
87.09*
40.44
42.03
81.49
40.33
86.39*
80.41
52.20
Table Performance comparison on the test set of ToTTo, E2E, FeTaQA, and WebNLG. Note that
the missing BLEU results are due to the absence of gold references in the test set of ToTTo. denotes faithfulness scores statistically significantly higher than the SET baseline.
SAMSuu
XSum
PubMed
Align
FacCCC
QEval
R-L
Align
FacCCC
QEval
R-L
Align
FacCC
QEval
R-L
LLAMA2-7R
SFT
80.66
44.83
45.20
56.25
74.63
34,92
46.89
35.84
34.60
24.58
CAD
81.65
79.37
45.01
45.01
57.58
77.83
32.26
33.73
$2.68
43.05
33.65
22.50
CRITIC
81.52
77.66
45.18
44.81
55.80
74.23
32.03
34.15
48.02
37.56
33.71
23.80
PMI
81.03
77.29
44.95
45.15
56.29
74.33
31.99
34.90
48.03
36.34
34.45
23.56
CLIB
81.30
76.68
44.77
44.72
57.46
74.70
32.23
35,58
45.64
37.56
34.06
23.97
SCOPE
46.65*
38.76'
LLAMA2-13B
SFT
81.59
78.63 63
44.10
44.60
56.53
75.75
31.72
36.14
47.51
38.93
34.83
24.02
CAD
81.35
80.59
44.21
43.43
57.22
77.45
31.99
35 89
$2.81
47.79
34.67
23.17
CRITIC
81.14
78.14
44.40
42.88
56.53
75.16
31.81
35.97
49.06
40.46
34.63
22.35
PMI
81.82
78.14
44.04
44.75
56.56
75.47
31.75
36.20
50.87
36.79
34.82
23.32
CLIFF
81.61
76.80
44.96
44.19
56.52
75.27
31.67
36.10
40.76
34.30
24.39
SCOPE
84.20"
81.69
46.45"
44.98
66.03"
84,06"
37.17"
31,59
$8.68"
61.22"
39.10"
23.85
MISTRAL-7B
SFT
82,59
75.75
31.25
44,20
$7.20
75.76
31.25
36.25
43.60
35.10
33,32
25.07
CAD
83.10
79.37
45.52
43.98
57.31
78.55
31.32
35,24
45.36
42.75
31.72
23.63
CRITIC
82.76
79.24
45.63
44.07
57.65
74.67
31.81
33.68
46.80
38.78
33.13
23.55
PMI
82,45
80.46
45.49
44.17
57.47
76.76
30.83
36.17
44.08
37.86
32.59
24.37
CLIH#
82.50
79.24
45.60
44.30
58.20
75.33
37.14
45.90
40.61
34.18
25.50
SCOPE
83,70'
80.59
46.21
42.72
62.17'
84,36"
36,33'
24.61
55.37'
48.55
37.01
24.03
Table 3: Performance comparison on the test set of SAMSum, XSum and PubMed.
deeotees
faithfulness scores statistically significantly higher than the SFT baseline.
Baselines present mixed results on faithfulness metrics. Summarization-foousse baselines (CAD, PMI. CLIFF) show an overall increase in AlignScore on SAMSum, XSum and PubMed (Table (3). However, the improvements on XSum remain marginal compared to SCOPE's results. For data- to-text generation, all baselines show minimal to no faithfulness improvement over SFT ( Depending on the methods, we identified two reasons that could explain these mixed results. First, CLIFF, CRITIC, and PMI were originally designed for smaller encoder-decoder models. We suspect that differences in architecture and the number of parameters in larger, more recent LLMs may limit their effectiveness. Secondly, CAD, PMI, CLIFF mainly designed for general summarization tasks, we suspect that for data-to-text generation, which require further adaptation, these methods may fall short.
8

---

Published as a conference paper at ICLR 2025
GPT GPTT-as-a-judge evaluation. To further assess their performances, all methods applied to LLAMA- 2-7B were compared to standard finettuiig, with GPT-4 used as the evaluator. Results are presented in Tables 4 and [ Across all datasets, SCOPE consistently shows a much higher win rate than other methods, confirming its efficiency in improving faithfulness. For the baselines, especially in data-to-text generation tasks, we observe a noticeable high tie rate. This indicates that significant proportion of the samples are considered equivalent in quality to the standard fine-tuning samples. Consequently, it suggests that these methods have not adequately addressed the faithfulness issues related to fine-tuning.
ToTTo
E2E
WebNLG
WinS:
Tiefs
Loss
Win%
Tie%
Loss
Win
Tie
Loss
Win%
Tie
Loss
CAD
3,47
93,11
3,42
1,79
92,20
6.01
7.59
86,78
5.62
8,70
82,1
9,20
PMI
2,82
94,33
2,85
0.49
99.02
0.49
5.90
86.01
8.10
7.98
84.26
7.76
CRITIC
4.37
91.5
4.13
0,87
98.00
1,14
5,85
89,49
4,67
6,90
86,25
6,85
CLIFF
14.57
72.37
13.06
3.14
92.15
4.71
20.92
58.66
20.42
14.90
67.96
17.14
SCOPE (ours)
35.03'
47.26
17.71
11.04"
84.79
4.17
29.96
45.53
24.51
29.85'
55.93
14,22
Table 4: GPT-4 preference results of CAD, PMI, CRITIC, CLIFF and SCOPE versus SET with LLAMA-
2-7B on ToTTo, E2E, FeTTQQA and WebNLG. Results with are statistically significantly higher than all other baselines.
SAMSum
XSum
PubMed
Win
Tie%
Loss
Win%
Tic%
Loss
Win
Tie%
Loss%
CAD
21.73
62.27
16.00
42.98
18.36
38.67
53,82
11.93
34.25
PMI
9.89
80.71
9.40
24.06
52.66
23.27
37.31
26.30
36.39
CRITIC
18.93
63.00
18.07
35.50
28.10
36.40
38.84
22.02
39.14
CLIFF
25.89
45.67
28.45
50.63
12,00
37.38
41.74
17.43
40.83
SCOPE (ours)
58.12"
33.46
61.03"
2.64
36.33
74.50'
22.75
2.75
Table 5: GPT-4 preference results of CAD, PMI, CRITIC, CLIFF and SCOPE versus SET with LLAAA--
2-7B on SAMSum, XSum and PubMed. Results with are statistically significantly higher than all other baselines.
Further validation through human evaluation. In addition to using automatic faithfulness metries and GPT-4 preference judgments, we conduct human evaluations to comprehensively assess the quality of SCOPE generations. We distribute different sets of 25 ToTTo samples to 5 annotators, totaling 125 sam- ples. Each sample includes a table, one generation from SCOPE and one from SFT, using LLAMA-2-7B. Annotators are tasked with rating which of the two descriptions is more faithful to the table. They are asked to put the emphasis on faithfulness exclu- sively, meaning that although generation may contain factually correct details, these additions deemed less desirable than a generation that strictly relies on the information provided in the table. Full experimental details are described in Appendix E The results are presented in Table 6 The descriptions generated by SCOPE are preferred twice as often as those by the associated SFT. The results closely match those obtained with GPT4-as-a-judge further validating the soundness of our approach. We present some samples of SCOPE and SFT generations in Appendix D
Win%
Tie%
Loss%
SFT SCOPE
15.2
44,8
40.0
40.0
44.8
15.2
Table 6: Human preference results of SCOPE versus SET on ToTTo test set with LLAMA-2.
6 ANALYSIS OF SCOPE
In this section, we propose to analyze the effect of undesired responses generated by our unfaithful sampling method on the overall performances of SCOPE. By varying the value of a in the noisy data generation process (Algorithm|) []. we can simulate different degrees of hallucinations due to the influence of PLM- In this analysis, we examine the impact of negative samples on preference learning.
How does the value of a affect the training dynamics? The choice of a is critical. When o is low, the negative samples are too close to the model's own approximation of the underlying data distribution. During the preference-tuning stage, the model struggles to maximize the gap between the likelihood of the clean and noisy samples while maintaining the high likelihood of the clean ones. This causes the model to downweight the likelihood of both samples, leading to degeneracies, see Figure 2a Conversely, when a is high, the generated noisy samples are barely grounded in the input context, making it easy distinguish between y and y under pa(: c). In this case, the

---

Published as a conference paper at ICLR 2025
(nggae""))
Inppsoo"
log pelo"""
Inpperlrt)
-10
0:35
0.50
-10
0.35
0.50
1.00
Eiprads
(a) Training with a = 0.1. (b) Training with & 0.5.
(c) Training with a = 0.7.
Figure 2: Preference training dynamics with LLAMM-2-77B noise level increases on ToTTo dataset. Illustration of the three different regimes during preference training. Blue (resp. red) curve corresponds the log probability of the reference labels (resp. of the synthetic unfaithful samples).
SCOPE
SFT
SCOPE
SPT
60
45-
Score
BLEU
20-
25
0.000
0.25
0.50
0.75
0.00
0.25
0.50
0.75
a
(a)
(b)
Figure 3: Evolution NLI Score and BLEU with a on ToTTo validation set with LLAMA-2-7B.
model learns very little compared to its fine-tuned counterpart, see Figure 2c] Therefore, a should be chosen to balance the noisy generation between being too similar to the reference texts and too easy to discriminate. This scenario is illustrated on Figure 2b] for a = 0.5. The likelihood of the references does not decrease, and the likelihood of the noisy samples diverges less abruptly than in Figure 2c] providing a more effective learning signal.
How does the negative samples construction affect generation quality? For low values of a, we observe noticeable degeneracies, evidenced by text repetitions. This is shown in Figure 3b where BLEU scores decrease abruptly with lower values of a. As dicussed Section 5 in the [0.4, 0.6] interval, the decrease in BLEU appears to be more closely related to the generated outputs diverging from standard fine-tuning patterns, rather than a noticeable decline in fluency. Regarding optimization efficiency, the three regimes observed in Figure 2 can also be identified in Figure 3al that describes the evolution of the NLI score as a function of a. Below a certain level of noise, degeneracies also impact the NLI score. Increasing beyond certain point yields further improvement, both BLEU and NLI scores converge to the results of standard fine-tuning. As a result, searching for a in the interval [0.4, 0.6) seems to yield the best performances. We observe similar patterns in text summarization tasks, see Appendix B A quantitative and qualitative analysis of the noisy samples can be found in Appendix
CONCLUSION
Faithfulness hallucinations are common issue standard fine-tuned LLMs, and existing methods developed to mitigate these hallucinations yield mixed results with recent LLM models. In contrast, we demonstrate that employing a two-stage method, distinct from standard fine-tuning, effectively addresses typical challenges. Our key contributions include the automatic and self-supervised construction of a preference dataset tailored for the model, along with a framework that enables preference learning. Notably, our approach, SCOPE, consistently enhances the faithfulness of generated responses across various data-to-text and summarization tasks, significantly outperforming existing solutions assessed by relevant automatic faithfulness metrics, evaluations using GPT-4 and human judges. We provide an analysis of the main factors contributing to the successful deployment of this method, illustrating its performance quantitatively and qualitatively with typical samples.
10

---

Published as a conference paper at ICLR 2025
8 LIMITATIONS
Although this work explores classical generation tasks, is important to note that some of these tasks, particularly WebNLG and E2E, are of relatively limited complexity. Additionally, we limited our experiments to 7B models due to computational constraints. While this choice allows us to effectively demonstrate our approach, larger models could potentially yield different insights. Future work could validate the scalability and effectiveness of the proposed methods on larger model architectures. Lastly, this study primarily relied on automatic evaluation metrics. While these metries have shown value, particularly in assessing faithfulness, their performance across the diverse domains of our datasets remains less explored. Ideally, a broader human evaluation would provide more nuanced understanding of the results. However, given the resource and logistical constraints of conducting such evaluations, automatic metrics serve as practical solution within the scope of this work, even they have certain limitations.
ACKNOWLEDGMENTS
This work has been partly funded through project ACDC ANR-21-CE23-0029 This project was provided with computing AI and storage resources by GENCI at IDRIS thanks to the grants 20XX- AD01101400 1005332,, 20XX-A015101460 supercomputer Jean Zay's s V100/A100 partition.
20XX-A0171014638 and
on the

---

Cheng-Han Chiang and Hung-yi Lee. Can large language models be an alternative to human evaluations? In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the s/ Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15607-15631, Toronto, Canada, July 2023, Association for Computational Linguistics. doi: 2106633////2223.alloooogg Published as a conference paper at ICLR 2025
REFERENCES
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Hersert-Vosss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.
Shuyang Cao and Lu Wang. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scout Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6633-6649, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/2221.cmm main.532. URL https://aclaathooooo.ooo// 2021 pomlp-mmain. 532
Thiago Castro Ferreira, Claire Gardent, Nikolai llinkkh, Chris van der Lee, Simon Mille, Diego Moussallem, and Anastasia Shimorina. The 2020 bilingual, bi-directional webnlg+ shared task overview and evaluation results (webnlg+ 2020). In Proceedings of the 3rd WebNLG Workshop on Natural Language Generation from the Semantic Web (WebNLG+ 2020). pp. 55-76. Association for Computational Linguistics, 2020.
Alison J. Cawsey, Bonnie L.. Webher, and Ray B. Jones. Natural language generation in healthcare: Brief review, 1997. URL
Xiuying Chen, Mingzhe Li, Xin Gao, and Xiangliang Zhang, Towards improving faithfulness in abstractive summarization. In Koyejo. S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and
A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35. pp. 24516-24528. Curran Associates, Inc., 2022. URL https:://pooeeeeinggsst newrips.eppppp.p pwwpeppccccppppprrrrroocpcpppp.. http:/isss.ppppperrleeee//aaappp files/pape////// htt.tt//eepoocdeeeeeeeeeepppppep
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models, 2024.
wrtp:///oooyy.../2222.... ac1-long. 870
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Un- supervised cross-lingual representation learning at scale. CoRR, abs/1911 10.0211, 2019. URL httpp:/aaaxiv.orgabb////// 02116
Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-We Chang, Dipanjan Das, and William W Cohen. Handling divergent reference texts in table-to-text generation. In Proc. of ACL, 2019.
Ondrej Dušek and Zdeněk Kasner. Evaluating semantic accuracy of data-to-text generation with natural language inference. In Brian Davis, Yvette Graham, John Kelleher, and Yaji Sripada (eds.), Proceedings of the 13th International Conference on Natural Language Generation, pp. 131-137, Dublin, Ireland, December 2020. Association for Computational Linguistics. doi: 10.18653/112000.mggg...
1.1g-1.19
Ondřej Dušek, David M Howcroft, and Verena Rieser. Semantic Noise Matters for Neural Natural Language Generation. In Proceedings of the 12th International Conference on Natural Language Generation (INLG 2019), pp. 421-426, 2019.
Alexander R. Fabbri, Wojciech Kryáciński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. SummEval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics. 9:391-409 2021. doi: 10.1162/tacl a 00373. URL
https:///cc.anttoooyyy...////// t.
12

---

Published as a conference paper at ICLR 2025
Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, and Jianfeng Gao. GO FIGURE: A meta evaluation of factuality in summarization. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Computationaal Linguistics: ACL-IJCNLP 2021, pp. 478-487, Online, August 2021. Association for Computational Linguistics. doi: 10. 10.18633/v1/ |RL|https: htt.hhtts//aaaaanoolloooooo org. Find///......iinngs- findings-acl......
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120:30)::3305066120, 2023. doi: 10. 1...1773/pnas2235516111111 URL. https: htttp:///ww... pnas. wttgg//o//abs//.10//111 2305016120
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. SAMSum corpus: A human- annotated dialogue dataset for abstractive summarization. In Lu Wang. Jackie Chi Kit Cheung, Giuseppe Carenini, and Fei Liu (eds.), Proceedings of the 2nd Workshop on New Frontiers in Summarization, pp. 70-79, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. N0.18653/v1/D/ 1/D19-440,, URL URRLlhttp:://acaatthooo http://s//bcclaatoooooooo///////
Tanya Goyal and Greg Durrett. Annotating and modeling fine- fine-grained factuality in summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Techmologies, 2021.
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, and Mathieu Blondel. Direct language model aligmment from online ai feedback, 2024. URL pnttps://apt: wewast semanticscholar. org/ 20722299::27522957
Pengcheng Hc, Jianfeng Gao, and Weizhu Chen. DeBERTav3: Improving deBERTa using ELECTRA- style pre-training with cradient-disentangled embedding sharing. In The Eleventh International Conference Learning Representations, 2023. URL htttss://peerreiiee.nee//// XXXxHAL
Jiwoo Hong, Noah Lee, and James Thome. Orpo: Monolithic preference optimization without reference model, 2024. URL /OLLLtttttss///aiiiorr////////// 07691
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023. URL
https://arxxw.oor/abb//22 .5232
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023.
Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. Neural text summarization: A critical evaluation. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xi- aojun Wan (eds.), Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP). pp. 540-551, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v 1/D/3//11D1111/// 1051. URL httpp:://claanthooogg...
Wojeiech Kryseinski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Pruceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 9332 9346, Online, November 2020. Association for Computational Linguistics. doi: 112.8663/1112222..mm m50. URL httphhtss//aaaaahhlllano wwwp://twwhooooooo//////2// org/2020 mmllp-main.. 750
Mateusz Lango and Ondrej Dusek. Critic-driven decoding for mitigating hallucinations in data-to- text generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. December 2023. Association for Computational Linguistics. doi: 10. (0.18853/v 1ttp:////222.mmmmm http///2...0.mmmaiiiiaa 172. URL httpp::/aaclaanhologg..oo
Singapore,
13

---

Published as a conference paper at ICLR 2025
Chin- Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74 74-81, Barcelona, Spain. July 2004. Association for Computational Linguistics. URL hetpp://wwww aclweb. hwwp///nnoooogoo//oo---o-
Yupian Lin, Tong Ruan, Jingping Liu, and Haofen Wang. survey on neural data-to-text generation.
IEEE Transactions on Knowledge and Data Engineering, 36(4):1431-1449, 2024, doi: 10.1 10.1
TKDE.2023.330433.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. URL
/907 11692
Yixin Liu, Pengfei Liu, Dragomir Radev, and Graham Neubig. BRIO: Bringing order to abstractive summarization. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2890-2903, Dublin, Ireland, May 2022. Association for Computational Linguistics. h/tp://222/22222111//2/
doi:
URL httpp::///a.aaattoloog/
wwwgg/202.2co acl-long. 207
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1906-1919, Online, July 2020. Association for Computational Linguistics. doi: 18.18v33/v1/ 2020.acl-main. 173. URL https:://aaaaathol..aa wrg.y.org//// 2020 acl.maln.mmm11.mmm 173
Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang. Kathleen McKeown, and Bing Xiang. Entity-level factual consistency of abstractive text summarization. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (eds.), Proceedingy of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2727-2733, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/v 1ttp:////2222aa222 main. 235, URL httph: http::////ccaahhollg//// org/ 2021 http//221...el1eee1.eee1ee sacl-main. 235)
Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojeiech Kryšeiński, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, and Dragomir Radev. Fetaqa: Free-form table question answering. Transactions of the Association for Computational Linguistics, 10:35-49, 2022.
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Ellen Riloff, David Chiang, Julia Hockenmaier, and auiic Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1797-1807, Brussels, Belgium, October- November 2018 Association for Computational Linguisties. doi: 1///55/////D18-111 URL https://aallanthooogy..oo
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin (eds.), Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 00.3115/107308..10 1073135..
h hatas//ahaclaathoogyy.y hwwphholoooyoog//PP.. 1040
Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. ToTTo: A controlled table-to-text generation dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1173-1186. ACL, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, bigh-peeformance deep learning library. In Advances in Neural Information Processing Systems 32. pp. 8024-8035. Curran Associates, Inc., 2019. URL http:/hhtppppps.ptettee www.:eerrrrsrrrirs.ccos.. cctpppppppppppppp///pepppppp 2015-pytorch--o an- cemperatiie-io style- wwwpepope--eeeeeeeeeeeeeeeeeeeee htw.://weeee.t.......eeeeeeeeeee
14

---

Published as a conference paper at ICLR 2025
Matt Post. A call for clarity in reporting BLEU scores. In Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel. Yvette Graham, Barry Haddow. Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, and Karin Verspoor (eds.), Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 186-191, Brussels, Belgium, Octoher 2018. Association for Computational Linguistics. doi: 10.18653/11/W18--66
URL htw.https//// https::/accaattholoe org/
W18-6319
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In Thirty- seventh Conference on Neural Information Processing Systems, 2023, httphttts///opereeiiel
net//forumIdd-PPUSXJJaaa
Clement Rebuffel, Thomas Scialom, Laure Soulier, Benjamin Piwowarski, Sylvain Lamprier. Jacopo Staiano, Geoffrey Scoutheeten, and Patrick Gallinari. Data-QuestEval:: A referenceless metric for data-to-text semantic evaluation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8029-8036, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguisties. doi: 10.18653/v1/2022 http///22...mmmmmm......m URL https://aclanthoooggy..o org 2021. wttp:/rrr//2.......ppmm....
633
Clément Rebuffel, Marco Roberti, Laure Soulier, Geoffrey Scoutheeten, Rossella Cancelliere, and Patrick Gallinari. Controlling hallucinations word level in data-to-text generation. Data
Min. Knowl. Discov., 36(1))3118)) 11::338--335 354. 2022. doi: 10. 10618-021-0080144 URL https 7/doi. 10. 1007/0 0618 000000001--0000
Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, and Patrick Gallinari. QuestEval: Summarization asks for fact-based evaluation. In Marie- Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih edds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6594-6604, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/0221 2ew.2221.emmlmmmii2222222
org. wwwp:221..mmmmmpmmmmmmmmmmmm emnlp-mainl
529
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding, 2023.
Liyan Tang, Zhaoyi Sun, Betina Idnay, Jordan G. Nestor, Ali Soroush, Pierre A. Elias, Ziyang Xu, Ying Ding, Greg Durrett, Justin F. Rousseau, Chunhua Weng, and Yifan Peng. Evaluating large language models on medical evidence summarization. nрj Digital Medicine, 6(1):158, doi: 00.1038/441446-003-0086666 URL httpttss///6ii.rr///// 1038/ 541746 023 00896-77
2023. ISSN 2398-6352.
Greg Durrett Tanya Goyal, Junyi Jessy Li. News summarization and evaluation in the era of gpt-3. arXiv preprint, 2022.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An cestruction-folloiinn Ilama model. 2023.
https::/ggt thub. com/ wtt.coooottteeeeeeee
Craig Thomson and Ehud Reiter. A gold standard methodology for evaluating accuracy in data-to-text systems. In Brian Davis, Yvette Graham, John Kellcher, and Yaji Sripada (eds.), Proceedings of the 13th International Conference on Natural Language Generation, pp. 158-168 Dublin, Ireland, December 2020. Association for Computational Linguistics. doi: 10. 1 12111666///1202.nngggg URL
.rggy/org///00 www..02..o0... inlg-1.22
Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P. Parikh. Sticking the facts: Confident decoding for faithful data-to-text generation, 2020. https:/t://// httg//bs/199 http//ab./111.0...333 08684
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei. Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,
15

---

Published as a conference paper at ICLR 2025
Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023.
Liam van der Poel, Ryan Cotterell, and Clara Meister. Mutual information alleviates hallucinations in abstractive summarization. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing, pp. 5956-5965, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 1/066/3///222.mmm////l cammll-mminn3399 URL httpps://aclanthoooog// wrgp://ooooyoo///220// emn manlp-main... 399
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick vOn Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 45, Online, October 2020. Association for Computational Linguistics. URL hhttps://www..c.t awwweb.wwwwwbbb................b org hrgg//nnollogg/22222222 annthollooy//000..eel----eooo emnlp-demos
Polina Zablotskaia, Misha Khalman, Rishabh Joshi, Livio Baldini Soares, Shoshana Jakobovits, Joshua Maynez, and Shashi Narayan. Calibrating likelihoods towards consistency in summarization models, 2023. URL htttp:://arxii..rrg//// absaaa/s//// 08764
Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. AlignScore: Evaluating factual consistency with unified alignment function. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11328-11322 1348, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 10.18653//1/20000 1/wp//..aa..ooonnooo.o. acl acl- ..ogg6634
URL h org/2023
Haopeng Zhang, Semih Yavuz, Wojciech Kryscinski, Kazuma Hashimoto, and Yingbo Zhou. Im- proving the faithfulness of abstractive summarization via entity coverage control. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Findings of the Association for Computational Linguistics: NAACL 2022, pp. 528-535, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10. 10.18653/w1/202 1/2.222///222.... fiwdii/.iiniinnnaaaa naac1.40. URL https://aclanttoolgy..oom wrw.://.....gg222../. (on222.finningge Landings-nnacl.. 40
Kun Zhang, Oana Balalau, and Ioana Manolescu. FactSpotter: Evaluating the factual faithfulness of graph-to-text generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the
Association for Computational Linguistics: EMNLP 2023, pp. 10025-10042, Singapore, December 3665:///222...nnnnn-emmmmpp 2023, Association for Computational Linguisties. doi: 10.18853/v1/2033 findings-emmlp.672..
URL http:::////cee httsss//aclatthllog org/ 2023. f findings smmnpp.. 672/
Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. Calibrating sequence likelihood improves conditional language generation. In The Eleventh International Conference on Learning Representations, 2023. /ttp/tttts//oopeeeeieee net/ Forum?id==Q4SOodKnaa
Zheng Zhao, Shay B. Cohen, and Bonnie Webber. Reducing quantity hallucinations in abstractive summarization. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguistics: EMNLP 2020, pp.
Online, November 2020. Association for Computational Linguistics. doi: 10. 126188653vl1 http:////22..mfi (ww....iniingg-mmml.200000 URL https:// aclanthologg.orgg/22 findings-emnlps 203
16

---

Published as a conference paper at ICLR 2025
Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, and Meng Jiang. Enhancing factual consistency of abstractive summarization. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy. Steven Bethard. Ryan Cot- terell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 718-733, Online, June 2021. Association for Computational Linguistics, doi: 10.18653/v 1aaa022..aaammmaia....
naacl-main. URL htttps://acaanthologg. org. 2021 org.//2...aaccmmmainn 58
Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv: 1899.0893, 2019, URL httpst:::/// httt:ss//arxvv.oor//////999 9099909..999
17

---

Published as a conference paper at ICLR 2025
A EXPERIMENTS
Implementation details. Our code based on Pytorch Paszke et 2019) and Huggingface Wolf et al. (2020). Experiments were ran on NVidia 80GB A100 GPUs. BLEU is computed using the Bac (Post 2018) implementation. For NLI metric we use the model available at https://hugg ing fage ace. wto/cross--oo wwweeeeeeoeeennn-eeerrreeeee larger
A.1 BASELINES
For each baseline choose the best hyper-parameters by conducting grid-search. We initially conducted the search over ranges disclosed in original publications and refined based on our own experiments.
A.1.1 CONTEXT-AWARY DECODING
Table 7 shows the best hyperparameters for CAD method. In the original paper, is recommended to select between 0 and I, with 0.5 being suitable choice.
ToTTo
FeTaQA
WebNLG
E2E
SAMSum
XSum
PubMed
a
a
a
a
a
a
LLAMA-2-7B
0.01
0.05
0.03
0.03
0.05
0.40
0.40
LLAMA-13B
0.01
0.01
0.07
0.01
0.3
0.10
0.40
MISTRAL-7B
0.01
0.09
0.01
0.04
0.04
0.30
0.20
Table 7: Best Context-Aware Decoding (CAD) a hyperparameter.
A.1.2
POINTWISE MUTUAL INFORMATION
Table shows best hyperparameters for PMI method.
ToTTo (A,
FeTaQA (X,7)
WebNLG (A,7)
SAMSum (A,
XSum (A,
PubMed
(A,
(A,r)
LLAMA-2-7B
(0.07, 3.25)
(0.07, 3.25)
3.5)
(0.06, 3.25)
(0.20, 3.25)
(0.15, 3.25)
(0.20, 3.25)
LLAMA-13B
(0.07, 3.25)
(0.05, 3.25)
(0.05, 3.25)
(0.05, 3.40)
(0.15, 3.25)
(0. 10, 3.75)
(0.15, 3.25)
MISTRAL-7B
(0.06, 3.5)
(0.07, 3.25)
3.255
(0.09, 3.25)
(0.05, 3.25)
(0.20, 3.25)
(0.15, 3.25)
Table 8: Best PMI Decoding (PMI) (A, hyperparameters.
A.1.3 CRITIC-DRIVEN DECODING
For the classifier, replace the original XLM-RoBERTa-bass (Conneau et al. 2019) with stronger Deberta I-are (He et al. 2023) model allowing for much larger contexts, since the linearized data did not fit in the context- context-window of XLM-RoBERTaabass. In our experiments,
trained classifier on each dataset using the method "base with full sentences" reported to give the highest NLI score on WebNLG dataset in the original publication. Table 9 shows the best hyperparameters for the method.
ToTTo X
FeTaQA À
WebNLG
E2E
SAMSum
XSum X
PubMed À
À
À
LLAMA-2-7B
0.02
0.03
0.01
0.01
0.07
0.25
0.25
LLAMA-13B
0.03
0.07
0.01
0.06
0.25
0.10
0.75
MISTRAL-7B
0.05
0.01
0.01
0.10
0.05
0.75
0.50
Table 9: Best Critic-driven Decoding (CRITIC) À hyperparameter.
A.2 HYPERPARAMETERS
SCOPE Selected value of for SCOPE for each dataset presented in Table 10
18

---

Published as a conference paper at ICLR 2025
ToTTo
FeTaQA
WebNLG
E2E
SAMSum
XSum
PubMed
Q
Q
Q
a
LLAMA-2-7B
0.5
0.5
0.5
0.6
0.5
0.5
0,4
Llama-2-13B
0.4
0.4
0.4
0.5
0.5
0.5
0.4
MISTRAL-7B 0.5
0.5
0.5
0.6
0.6
0.5
0.4
Table 10:
Best SCOPE value of a for L.LAMA-2-77 and MISTRAI--BB on ToTTo, FeTaQA, WehNLGG
and E2E.
Full SFT training.
LLAMA-2-7B. The SFT version of LLAMA-2-7B where fine-tuned using a batch size of 16, a learning rate of 2 x 10-s, using a linear scheduler with a warm-up ratio of 0.1 on all datasets. The model is optimized with Adam optimizer.
MISTRAL-7B. We used batch size of 16, a learning rate of 2 x 10 using a linear scheduler with warm-up ratio of 0.1 all datasets. The model is optimized with Adam optimizer.
SCoPE training.
Training Pa, on D1, For training the fine-tuned version of each model on the split Di, we used the exact same setting than for the full SFT training described above, except that only performed one epoch for LLAMA-2-77 and two epochs for MISTRAL-76
Preference tuning, Regarding the hyperparameter of Equation models and datasets.
we set 3 = 0.1 for all
A.3 FINE-TUNING HALF DATASETS
When fine-tuned on half the samples, we observe experimentally that the models have very close performances the model fine-tuned on the full train set, see Tables 11 and The models fine-tuned on half the samples are therefore a strong initialization for the subsequent stages of the method.
WebNLG
ToTTo
E2E
FetaQA
NLI PARENT
NLI PARENT
NLI PARENT
NLI PARENT
LLAMA-2-7B
SFT on D1
86,6
72.0
45.6 46.0
80.4
80.9
82.2
36.2
76,6
SET on D
87.4
82.1
80.2
87.4
86.9
37.5
77.1
MISTRAL-7B
SFT on D1 SET on D
87.2
81.6
46.5 46.7
80.3
87.0
87.4
34.1
74.6
87.5
81.9
80.1
86.5
85.2
34.1
74.8
Table 11: Results are on the validation sets. NLI Score and PARENT for models fine-tuned on half of the training set of data-to-text datasets. On average, the score are slightly lower compared to models trained on the full dataset.
A.4 ABLATION BY VARYING THE DATASET PROPORTIONS USED IN THE FIRST PHASE OF FINE-TUNING
Based on the observations in Appendix A.3 we chose to use 50% of the data for the first phase of fine-tuning given the considered datasets and tasks. Here, we present ablation study on ToTTo. In this study, we fine-tuned a model on 25% (resp. 75%) the dataset and preference-tuner on the remaining 75% (resp. 25%) with noisy samples. Results the validation below. On automatic faithfulness metrics (NLI and PARENT), all splits yield comparable results, though bit higher with split of 50/50.
shown in the table
19

---

Published as a conference paper at ICLR 2025
XSum
SAMSum
AlignScore
Rouge-T.
AlignScore
Rouge-I.
LLAMA-2-7B
SFT on D1
56.2
33.8
80.5
43.2
SET on D
56.4
35.2
82.6
45.2
MISTRAL-7B
SET on D1 SFT on I
57.3
35.1
81.9
44.7
57.3
36.2
82.5
45.2
Table 12: Results are on the validation sets. AlignScore and Rouge-L for models fine-tuned on half' of the training set of summarization datasets. Like for data-to-text generation, average, the score are slightly lower compared to models trained on the full dataset.
First phase trained on
NLI
PARENT
25%
49.57
86.08
50%
50.64
86.34
75%
49.07
84.10
Table 13: NLI and PARENT scores the validation of ToTTo when varying the proportion used in the first phase of fine-tuning and using the remaining split for the second phase of preference tuning.
A.5 PREFERENCE LOSS
We chose to use DPO (Rafailov et al. 2023) for its seminal work and its widespread usage. But our self-supervised framework has no dependency with DPO and should also work with other preference tuning approaches. tested with ORPO (Hong et al. (024) and observed very similar results to DPO, see Table
Method
NLI
PARENT
SFT
46.0
80.2
SCOPE with DPO loss
49.9
84.2
SCOPE with ORPO loss
49.3
85.9
Table 14: Results on the validation set of ToTTo with different preference optimization losses applied to LLAMA-2-7B.
A.6 ABLATION ON THE VALUE OF 3 IN PREFERENCE STAGE
Table 15 presents faithfulness metrics change the value of 3 in the preference-tuning phase of SCOPE. In the original DPO paper (Rafailov et al. 2023), authors use value 3 = 0.1 which we found to also work well for SCOPE.
ToTTo
XSum
B
PARENT
NLI
ROUGE-L
AlignScore
0.05
83.54
48.31
29,51
65.16
0.1
85.39
49.21
30,66
65.37
81.98
46.24
33.80
59.30
5
81.04
45.80
33.84
57.45
Table 15: The effect of different / values performance for ToTTo and XSum tasks.
A.7 SCOPE ON INSTRUCTIOO-TUNED) MODELS
We intentionally focused on a task-specific setup, targeting use cases where specialized models most applicable. However, explore SCOPE's performance in general-purpose context,
20

---

Published as a conference paper at ICLR 2025
conducted additional experiments. Specifically, fine-tuned Llama-2-7b model on the Alpaca instruction dataset and compared to model fine-tuned using the SCOPE pipeline. Both models were evaluated on initial tasks, including data-to-text and summarization. As shown in Table [6 SCOPE continues to demonstrate consistent gains in faithfulness according to our metrics. However, these improvements are smaller than those observed for domain-specific models, suggesting that SCOPE is particularly effective in specialized contexts.
ToTTo
XSum
NLI PARENT
AlignScore
Rouge-L
MODEL
SFT
35.89
66.97 68.69
84,70 70
19.46
SCOPE
37.81
86.59
16.97
Table 16: On context-intensive tasks, SCOPE applied generalist instruction-tung models improves the faithfulness of the generation.
To ensure that these gains in faithfulness do not compromise reasoning capabilities, we bench- marked both models on tasks from the OpenLLM Leaderboard. The results indicate similar overall performance for both models, with SCOPE outperforming supervised fine-tuning (SFT) on tasks such as TruthfulQA, WinoGrande, and HellaSwag-tase tasks that require strong context comprehension rather than general knowledge. These results, presented in the appendix, reinforce our contributions. Nonetheless, more comprehensive exploration of SCOPE's advantages in broader setups is left for future work.
ARC
HellaSwag
MMLU
TruthfulQA
Winogrande
Avg
SFT
47.61
56.50
41.06
30.72
70.96
49.37
SCOPE
47.27
57.07
39.75
31.95
71.98
49.60
Table 17:
Performance comparison between SFT and SCOPE on various tasks. Scores are percentages, and the best result for each task is highlighted in bold. Metrics: Accuracy is used for ARC, HellaSwag, MMLU, and Winogrande, while BLEU-Ace is used for TruthfulQA to evaluate faithfulness to the reference responses.
A.8 GPT-4 PREFERENCE EVALUATION
As proxy to complete human evaluation. we conduct a GPT-4 preference evaluation comparing various methods the SFT model. We ask the model choose between two generations based on their faithfulness to the input data. We make sure to mitigate any position bias by randomly swapping the generations to be compared. We use the model gpt-4-32k-0611 through the OpenAl We use the following prompt for ToTTo,
API https httt:::/pllffmmmm openai. wtt. E2E and WebNLG:
"You judge in data-to-text competition. Your task is determine which description more accurately reflects the information in given data, ensuring that every detail in the text can be directly inferred from the data without adding any external information.
Here is a data about (Entity): (Data)
Here are two descriptions of the data:
Generation A: (Generation A}
Generation B: (Generation B)
Evaluate which description is more faithful to the data. Faithfulness means that every piece of information in the description must be directly inferable from the data and the description must not contain any additional information. Provide your answer in the following JSON format: ((" (""prfferred_texe text": "<letter>")) where
21

---

Published as a conference paper at ICLR 2025
<letter> is "A" if Generation A is more faithful, "B" if Generation B is more faithful and "Tie" if both are equally faithful.
for FeTaQA:
"You are judge data question answering competition. Given a data and a question, your task is to determine which answer more accurately and faithfully responds to the question based on the information provided the data, ensuring that every detail in the answer be directly inferred from the data without adding any additional information.
Here is data about (Entity): (Data)
Given the data and the following question: (Question)
Here are two answers:
Answer A: (Generation A}
Answer B: (Generation B}
Evaluate which answer is more faithful to the data. Faithfulness means that every piece of information in the answer must be directly inferable from the data and the answer must not contain any additional information. Provide your answer in the following JSON format: (["preferred text": "<letter>")) where <letter> is "A" if Answer A is more faithful, "B" if Answer B is more faithful and "Tīe" if both are equally faithful.
for XSum:
"You are a judge in an article summarization competition. Your task is to determine which summary more accurately and faithfully reflects the information in given article, ensuring that every detail in the summary can be directly inferred from the article without adding any external information.
Here is an article: (Article)
Here are summaries of the article:
Answer A: (Summary A}
Answer B: (Summary B}
Evaluate which summary is more faithful to the article. Faithfulness means that every piece of information in the summary must be directly inferable from the article and the summary must not contain any additional information. Provide your answer in the following JSON format: [{ (("referred text": "<letter>"]] where <letter> is "A" if Summary A is more faithful, "B" if Summary B is more faithful and "Tie" if both are equally faithful."
for SAMsum:
"You are a judge in a messenger conversation summarization competition. Your task is to determine which summary more accurately and faithfully reflects the information in a given conversation, ensuring that every detail in the summary can be directly inferred from the conversation without adding any external information.
Here is a conversation: (Article)
Here are two summaries of the conversation:
Answer A: (Summary A}
Answer B: (Summary B)
Evaluate which summary is more faithful to the conversation. Faithfulness means that every piece of information in the summary must be directly inferable from the conversation and the summary must not contain any additional information. Provide your answer in the following ISON format: {[ (["peferreed_ee text": "<letter>") where
22

---

Published as a conference paper at ICLR 2025
ingavly!)
Log pola""
Ingadly
Sogavla")
101-
201
100-
109
-10
-102
0.30
0.30
0.75
1.40
307
0.00
0.75
0.50
0.75
1.00
102
0.00
0.25
6.50
0.75
100
Epoch
Epoth
Epoch
(a) Training with a = 0.1.
(b) Training with 0.5.
(c) Training with a 0.7.
Figure 4: Preference training dynamics with LLAMA-2-78 B as noise level increases on SAMSum dataset. We observe the same three different regimes during preference training than for data-to-text generation.
SCOPE
SET
SCOPE
SFT
60
85
AlgnSSoree
Rouge-J L
40
80
20
75
0.00
0.25
0.50
0.75
0.00
0.50
0.75
a
(a)
(b)
Figure 5: Evolution of AlignScore and Rouge-L with on SAMSum validation with LLAMA-2- 7B.
<letter> is "A" if Summary A is more faithful, "B" if Summary B is more faithful and "Tie" if both are equally faithful.
B SCOPE ANALYSIS
We report here additional results supporting our analysis of SCOPE method of Section 6 The training dynamics of SCOPE on a summarization dataset is displayed on Figure and the evolution of AlignScore and Rouge-L metrics on Figure 5 Overall, we observe similar patterns than for data-to-text generation.
C QUANTITATIVE AND QUALITATIVE ANALYSIS OF THE NOISY GENERATED SAMPLES
Quantitative evaluation. To validate the effect of the noisy decoding process described in Algo- we plotted the evolution of PARENT and AlignScore as a increases on Figure 6
rithm D
Qualitative assessment. Inspired by the error taxonomy presented in (Thomson & Reiter 2020).
we propose to annotate using three categories:
Incorrect: statement that contradicts the data, includes incorrect number (including spelling out numbers as well as digits), incorrect named entity (people, places, organisations, etc) or other incorrect words. This corresponds to intrinsic errors.
Not checkable; statement in the text that cannot be checked given the data. This corresponds to extrinsic information.
23

---

Published as a conference paper at ICLR 2025
NOISY GENERATION
NOISY GENERATION
40-
60
SlognSoore
PARENT
30-
20
20
10-
0.3
0.5
0.7 0.9
0.1
0.3
0.5
0.7
a
(a) ToTTo
(b) XSum.
Figure 6: Evolution of the faithfulness of noisy samples as the noise parameter a in the decoding process increases, evaluated using PARENT and AlignScore on ToTTo and XSum. For XSum, AlignScore initially decreases with increasing o, followed by a slight uptick. We attribute this counterintuitive behavior to a limitation of AlignScore, which has not been tested with completely irrelevant data. Rather than approaching zero as expected, the score stabilizes at a constant nonzero value.
Other type of error: statement that is irrelevant to the data. As an ilustration, given the following data: Page Title: List Governors of South Carolina Section Title: Governors under the Constitution of 1868 Table:
Governor
Took Office
74
75
Daniel Henry Chamberlain December 1874
Please refer Table
18
for overview of the noisy samples as increases from 0.0 to 0.9.
D SAMPLES OF SCOPE AGAINST SFT
Tables present qualitative winning examples of model versus the fine-tuned model, judged by GPT-4. We additionally highlighted differences between both predictions, which further underscores the liability of GPT-4 as a judge for faithfulness.
Qualitative analysis on XSum reveals that the SFT baseline often struggles ground its summaries in the provided article. In contrast, SCOPE produces fewer hallucinations but tends to directly quote portions the article. For data-to-text tasks, the SFT baseline frequently infers extra information. whereas SCOPE remains closely aligned with the structured data.
E HUMAN EVALUATION PROTOCOL
For this study, we recruited five European annotators, all fluent in English, on voluntary basis. For each sample, they were presented with an input table and two predictions from the LLAMA-2-7B model, trained using SCOPE and SFT, respectively. These predictions were randomly labeled as 'Text A' and 'Text B'. The models corresponding to A and B were randomly selected for each sample to prevent any positional bias. The annotators were instructed choose between the options 'Text A more faithful' or 'Text B is more faithfull depending on their preference for description A or B, respectively. If both texts are deemed equally faithful, the annotators should select "Tie". If both
24

---

Published as a conference paper at ICLR 2025
a
Noisy generation
0.0
Daniel Henry Chamherlain was the 76th governor of South Carolina 1874.
0.1
Daniel Henry Chamberlain was the 76th Governor of South Carolina and served from 1874. He was the first governor elected by popular voté.
0.2
Daniel Henry Chamberlain was the 19th and final Goveroo of South Carolina, serving
from 1874 until 1876.
0.3
Daniel P. Chamberlain was elected as governor in 1854.
0.4
In 1876, the first woman elected as governor in the United States was Daniel Henry
Chamberlain.
0.5
Daniel Henry Chamberlain, Jr. served as a U.S. Representative and served as the 7th Governor of South Carolina from December 18, 1974. He was a member of the Demo- cratic Party.
0.6
Tags: Daniel Henry Chamberlain was born 1887, and died December I, 1962. He
was the of Daniel Henry Chamberlain, who served politician and lawyer in South Carolina.
0.7
Danielle Hatcher Chamberlain served as U.S. Senator from 1843-184 1847 and was
elected as a Governor of Mississippi in 1847. She was elected again for another term in 1870.
0.8
Oshima- Hhhima-yukihis--kök was discovered by Japanese amateur astronomer Atsushi
Sugiyama on October 25, 1995 the Okayama Astrophysical Observatory. Heteromastix piceaformis piceaformis (B) species group (Heteromastix) complex (B).
0.9
Table 18: At low levels of noise, the noisy sample is close to the supervised fine-tuned model, being overall faithful to the context while adding unsupported information (extrinsic error). As increases, the influence of the unconditional model causes the sample to increasingly contradict the context (intrinsic error), eventually making it entirely irrelevant.
descriptions have one or several faithfulness issues, they should both be considered unfaithful and rated as "Tie". The following instructions were provided to the annotators:
Instructions for Faithfulness Evaluation
Your task is assess which text description is more faithful the corresponding table. In this context, a text is considered faithful if all information it contains is directly supported by the content of the table.
If the description introduces any unsupported or incorrect information, it should be rated as unfaithful.
If both descriptions contain one more faithfulness issues, rate them as Tie.
To guide your evaluation:
Carefully compare each detail in the description with the table to ensure accuracy.
description should not distort, omit, or add information that not present in the table.
you notice even single instance of unsupported information a descrip- tion, it should be rated unfaithful.
If both descriptions have one or several faithfulness issues, they should both be considered unfaithful and rated 'Tie".
Please choose between the following options for each comparison:
Text A is more faithful
Text B is more faithful
25

---

Published as a conference paper at ICLR 2025
'iie (if both descriptions equally faithful or contain faithfulness issues)
F ON THE SIGNIFICANCE IMPROVEMENTS OF SCOPE AGAINST THE OTHER BASELINES
Faithfulness metrics. We performed independent two-sample t-tests to assess whether there were statistically significant differences in the mean values of specified metrics between the baseline SET and comparison model SCOPE. This test was chosen as it accounts for unequal variances and assumes independence between the two samples. For each metric, we calculated the t-statistic and corresponding p-value, allowing us to evaluate the likelihood that observed differences in means arose by chance. The results provide statistical basis for determining the significance of observed variations across datasets. Using a standard p-value of 0.05, SCOPE is statistically significantly better than SFT across the vast majority of datasets, metrics and models.
ToTTo
FeTaQA
WebNLG
E2E
PARENT
NLI
PARENT
NLI
PARENT
NLI
PARENT
NLI
LLAMA2-7B
3.19e-50
4.73e-17
2.21e-12
3.68e-3
1.1le-31
4.55e-4 4
7.18e-3
4.01e-3
LLAMA2-13B
4.91e-60
6.22e-31
1.37-99
9.26e-2
1.0G-555
1.85e-0
1.48e-3
1.02--2
MISTRAL-7B
1.86c 103
2.26c-20 24
1.08e-3 3
1.13e
7.64c
1.68c
4.31e-5 5
2.57c 1
Table 19: p-values of paired t-tests between SCOPE and SFT for data-to-text datasets.
SAMSum
XSum
PubMed
Model
Align
FanCC
QEval
Align
FacCCC
QEval
Align
FactCC
QEval
LLAMA2-7B
1.25e-3 3
1.1e-
4.26c-2
3.56e-
3.54e-69 69
4.67e- 144
3.29e-11 -1
4.79e
3.47e-8
LLAMA2-130
1.48e-2
0.1216
3.7e-3
1.10e-00 69
3.16e 55
5.36e
160
1.06e-9 9
3.27e 16
2.53e-1 7
MI
3.98e-3
3.56e-2
4.38e-1
5.37e-73
3. 3.16e-55
3.33e-18 189
1.20e-17 17
3.05e 22
1.10e-12 12
Table 20: p-values of paired t-tests between SCOPE and SFT for summarization datasets.
Pairwise rating. To assess whether our SCOPE improves significantly over the other baselines based on our GPT-4 win-tic-lose pairwise preference evaluations, perform the McNemar's s statistical test to determine the observed difference in wins is likely due to chance or it reflects a truly performance difference.
Null hypothesis: There is no significant difference performance between SCOPE and given baseline. Any difference in win counts is due to random chance.
Alternative hypothesis: SCOPE performs significantly better than the considered baseline. To do this, we count the number of samples SCOPE wins over SFT while the compared baseline loses to it (NAB) and vice versa (NBA) without taking into account the ties. The McNemar's 's test formula given by:
(NAB -BBA²2 NAB NBA
x²
Under the null hypothesis, x2 follows a chi-square distribution with 1 degree freedom.
We consider standard p-value of 0.05. A p-value less than 0.05 means we reject the null hypothesis. Here are the p-values on the GPT-4-as-a-judge evaluations:
Comparison
Totto
WebNLG
FeTaQA
E2E
SamSum
XSum
PubMed
SCOPE vs SFT
3.696e 97
3.127e-23
9.7e-4
3.55ge- 14
1.171e-25 25
6.744e- 153
3.944c-41
SCOPE PMI
9.492c 7
7.7e-3
6.744c
4.78c
1.3e 3
6.269c 55
2.305c 19
SCOPE vs CRITIC
1.473e-8
1.95e-2
2.1e-1
2.7e-3
5.41e-11 11
4.781e-74
2.313e-4
SCOPE vy CAD
1.226e-7 7
6.792e-5
9.39e-2
1.33e-2
1.23e-7
2.611e-59
1.522e 11
SCOPE vy CLIFF
1.226c
11
3.745e-11
6.25e
5.6e
2.04e- 6
3.025e-4
1.314c 21
Table 21: p-values of the McNemar's test on GPT-4 evaluation results
The results from McNemar's 's test show that:
(i) SCOPE shows consistently significant improvement over the SFT baseline. (ii) Most of the comparisons between SCOPE and the other baselines are statistically significant (p-value 0.05) ToTTo, WebNLG, E2E, SamSum, and XSum with the exception of FeTaQA
26

---

Published as a conference paper at ICLR 2025
Input
SFT
SCOPE
Denmark international Fischer, 21, has been with the Amsterdam cluh since 2011 I and scored 24 goals in 79 league garnes. The forward visited Boro, who were promoted to Premier League this scason, for a medical and agreed terms. ""It's the greatest league in the world and feel really ready." "" Fischer told Danish TV station TV2. Eight-times cupped Fischer has agreed a three-year deal. Fischer emerged as key player at Ajax under recently departed coach Frank De Boer, alongside compatriots Lasse Schone and Christian Eriksen the latter whom he will play against in next season's top flight when Boro face Tottenham Hotspur.
BOUGE-L:23352
AL: 82.65 Boro have signed Ajax forward Nicky Fischer for undisclosed fee, with the striker targeting "the greatest league in the world" in next scason's Premier League.
ROUGE-L: /1.43/
AL: 0.4
Middlesbruugh have signed Ajax striker
Nicklas Bendiner for an undisclosed fee.
The inquiry. carried out by Professor Sir Bruce Keogh, England's 's NHS medical director, was set up following the Mid- Stafforaforr Hospital scandal. A report into high death rales two hospitals in Sild-Staffordshire earlier this year found there were hundreds more deaths than expected, with patients let down by poor standards care. The table bclow shows how the 14 NHS trusts covered the review senred in measures of mortality. score much ahove 100 higher than the national average. All of the data has been standardised to take into account the diffcrent casc-mix at hospitals for example, hospitals that treat more elderly patients might expected to have higher death rates.
ROUGE-L: 32.00/ AL: 20.67 A review of NHS trusts in England has found trusts have been rated performing above the national average for the number of patients who died.
ROUGE-L: 17.77/
AL: 90.87 NHS trusts covered by an inquiry into high mortality rates at hospitals
that
Staffordshire scored above the national average in two measures, BBC data shows.
The Robins have II of their past Icague games, winning just once in that time, and are 20th in the table two points above the relegation zone. "It horrendous statistic," "" Johnson told BBC Radio Bristol. ""I'n ashamed of that. It hurts. We have to keep fighting." Former Bamsleyy boss Johnson arrived at Ashton Gate in Fehruary 2016 steered Bristol City clear of relegation his side vccupy filth spot in October, but Saturday's
ROUGE-L: 21.87
ROUGE-L: 42.10/
AL: 86.5 Bristol City boss Lee Johnson "hurts" to be "ashamed" side's 's "horrendous" recent Championshup run and insists they can still fight their way back up the table.
season. promising
AL: 4.0 Bristol City manager Lee Johnson
start this loss at the City Ground saw them drop below Forest- who had not won since 2 December. ""It was a frustrating watch for all of us." Johnson said. ""We were good in the second half and had a couple of chances we needed take. "'The Championship was never going to easy- we rebuilding phase. "We spoke the of the seuson about history makers and was supposed to be positive- not bad history. ""We'v ve gut to make sure we make history in positive ways and we cam still do that this year. We need to fight our way back up the table. believe that we can."
says he is "ashamed"
his side's poor form after they lost
2-0 Nottinghar Forest.
Table 22: XSum random winning samples. For the sake of clarity, we purposely choose articles of reasonable size. Red highlights facts that are hallucinations. Yellow highlights facts that are more faithful to the input.
27

---

Published as a conference paper at ICLR 2025
Input
SFT
SCOPE
Joana: wana play some games?
Sandy: what games?
Joana will bring Dixit and Carcassonne games for Sandy.
Joana: maybe dixit?
Joana will bring Dixit and Carcassonne to Sandy,
Joana: or carcassonne?
Sandy: why not?
Joana: ok, I'll bring them 2
Sandy: great :)
Ana: You sleeping?
Catherine: Not yet.
Ana: Wanna go visit grandma tomorrow? miss her.
Catherine will call Ana when she wakes up tomorrow to visit grandma tomorrow.
Catherine: Yeah that would be nice :) I'll call you when I wake up
Ana and Catherine will visit grandma tomorrow.
Ana: Oki sleep well, good night.
Catherine: Good night, too.
"Ellie: hey, are you at the university?
Aaliyah: yep
Ellie, Aaliyah and Camille are at the university. They have their classes in 342 on the second floor.
Camille: I'm sick :/
Aaliyah is at the university in 342 on the second floor. Camille is sick.
Ellie: shiet bad
Ellie: Aali, where are we having our classes now? Aaliyah: 342 on the second floor"
Table 23: SAMsum random winning samples. Red highlights facts that are hallucinations. Yellow highlights facts that are more faithful to the input.
Input
SCOPE
1992 Cork Senior Hurling Championship
Erin's Own Brian Corcoran the 1992 Cork Senior Hurling Championship's top scorer with 0-44.
Erin's Own's Brian Corcuran was the championship's top scorer with 0-44,
Player: Brian Corcoran
Club: Erin's Own
Tally: 0-44
2015-16 Reading F.C.
season
Matches: November 2015
November, Reading lost 2-0 away to Cardiff City.
Reading lost 2-0 to Cardiff City on 7 November 2015.
Matches: Carditf City
Matches: 2-0
Matches: Reading
Matches: Cardiff
United National Party
Ranil Wickremesinghe won the presidential election with 48.43% of the vote.
Candidate: Ranil
Ranil Wickremesinghe received 48.43% the vote.
Wickremesinghe
%: 48.43%
2018 Monster Energy
NASCAR Cup Series
Race: Big Machine Vodka
In the Big Machine Vodka 400 at the Brickyard, Clint Bowyer and Denny Hamlin led laps for Brad Keselowski Ford.
Brad Keselowski drove Ford to win the Big Machine Vodka 400 at the Brickyard, Clint Bowyer and Denny Hamlin led the most laps.
400 at the Brickyard
Most laps led: Clint Bowyer
Denny Hamlin
Winning driver: Brad
Keselowski
Manufacturer: Ford
Table 24: ToTTo random winning samples. Red highlights facts that are hallucinations. Yellow highlights facts that are more faithful the input.
28

---

Published as a conference paper at ICLR 2025
Input
SFT
SCOPE
Ciudad Ayala
population metro: 1777539
leader title: "City Manager"
The city of Ciudad Ayala, in Mexico, has a offset of city by the
type: City
The city Ayala, Mexico led by the City Manager has population density of 1604.0. The UTC offset for the city -s-6 and it has population of 1777539.
population density: 1604.0
City Munager and population density of 1604.0 and population of 1777539. The goverument is the govermment.
govermment type:
Council-manager
government utc offset:
country: Mexico
The Velvel Underground
associated band/associated
The Velvet Underground ussociated with the Theatre of Eternal Music.
The Velvet Underground associated with the band Theatre of Eternal Music.
musical artist: Theatre of
Eternal Music
Expect Miracle
artist: Brian Kelly
Brian Kelly is pianist and composer who ereated the Expect a Miracle. is 54,56 minutes long and was precoded hy Pools Light.
Expect Miracle compilation albumt written by Brian Kelly (composcr & pianist) and preceded by Pools of Light. runtime 54.56.
(cumposee & pianist)
type: Compilation Album
runtime: 54,56
preceded by: Pools of Light
(Brian Kelly alhum)
(Year No Light album)
The Year of Light, post-metal sludge metal album, released 6th September 2006.
The genre of Nord (Year of No Light album) sludge metal post-metal and
release date: 2006-09-06
genre: Sludge metal
released 6th September 2006.
genre: Post-metal
Liselotte Grschebina
The nationality of the artist, Lisclotte Grschebina, Israeli.
The nationality of Liselotte Grschebina is Israel.
nationality: Israel
Table 25: WebNLG random winning samples. Red highlights facts that are hallucinations. Yellow highlights facts that are more faithful to the input.
29

---

Published as a conference paper at ICLR 2025
Input
SFT
SCOPE
The Vaults
name: The Vaults
The Vaults is modcrately priced place to eat near Raja Indian Cuisine.
The Vaults is near Raja Indian Cuisine and price range of #20-25.
priceRange: 220-25
near: Raja Indian Cuisine
The Cricketers
name: The Cricketers
eatType: restaurant
The Cricketers Chinese restaurant the riverside area near All Bar One. It is family friendly and has a high price range and low customer rating.
The Cricketers is Chinese restaurant near All Bar One in riverside. has cheap price range and customer rating of 5 out of and is family friendly.
food: Chinese
priceRange: cheap customer rating: of S
riverside
famulyy'riendly;
near: All Bar One
The Phoemix
name: The Phoenix
earType: restaurant
The Phoenix family friendly restaurant located near Raja Indian Cuisine the city centre. is in the low price range and is locuted hear the river.
The Phoenix family friendly restourant located near Raja Indian Cuisine in the city centre area by the riverside with a price range of less than C20.
priceRange: less than F20
area: city centre
area: riverside
familyFriendly: yes
near: Ruja Indian Cuisine
Blue Spice
name: Blue Spice
Blue Spice restaurant located near Rainbow Vogetarian Café in the riverside It not family-friendly and has price range of less than $20.
eytType: restaurant
Blue Spice is non family ffienddy restaurant located near Rainbow Vegetarian Café riverside.
area: riverside
familyFriendly: no
near: Rainbow Vegetarian Café
The MIll
narn:: The Mill
eatType: pub
The Mill fast food restaurant in the riverside Café Sicilia. is a pub that is kid friendly and has price range of than E30.
The Mill is family friendly pub style fast food restaurant near Café Sicilia in riverside.
eatType: restaurant
food: Fast food
riverside
familyFFreendly: yes near: Café Sicilia
Table 26: E2E random winning samples. Red highlights facts that are hallucinations. Yellow
highlights facts that are more faithful to the input.
30

---

Model(s): 228, 430, 222 F, Spyder 2800, Karif Published as a conference paper at ICLR 2025
Input
Question
SFT
SCOPE
Mauricio Ortega (diseus thrower)
What ratio of total votes did Pierre
Position: 1st
Pierre Bertho of the Popular Repuhlican Movement earned 644 of the 818 total votes.
Pierre Bertho of the Popular Republican Movement wor with 644 of the votes,
Event: Discus throw
Bertho earn during the 1946 Dahomey and Togo by-clection?
Competition: Bolivarian Games
Notes: $9.67 m
Year: 2013
Candidate: Doug Forrester
What were the results of Forrester and Jon Corzine?
Forrester received 43.0% of the vote, while Jon Corzine received 53.5% of the vote.
Candidate: Jon Corzine 56:
Forrester lost to Jon Corzine by 43.0% to 53.5%,
96:
Model(s): Biturbo 2500
Displacement: (2,790
What the difference engine displacement of the 222 E model compared to the Biturho?
The 228 E model (2,790 ec) engine, larger than the 2.5 I. (2,491 Biturbo i 2500.
The 222 model had engine displacement of 2.8 I. (2.790 cc) compared to the Biturbo's 2.5 L (2,491 cc).
Displacement: 2.5 (2,491
сc)
Title: Skyfall
Harris played Eve Moneypenny the 2012 film Skyfall and its 2015 sequel Spectre.
Title: Spectre
In what films did Harris play Eve Moneypenny?
Harris played Eve Moneypenny Skyfall (2012) and Spectre (2015).
Year: 2015
Role: Moneypenny
Year: 2012
The Pinlajavesi lake of 72.55 square kilometres, which
rank: 6
How large is the area of the Pihlajavesi lake?
The Pihlajavesi has an 712.59 km?.
name: Pihlajavesi area (km?): 712.59
largest Finland.
Table 27: FeTaQQA random winning samples. Red highlights facts that are hallucinations, Yellow highlights facts that are more faithful to the input.
31


Reviews:
[{"review_id": "n27GiXdrgo", "reviewer": "Reviewer_hu8D", "summary": "LLMs often struggle to generate fully grounded information to a provided context. There is a rich literature on the hallucination issues in generative models (https://arxiv.org/abs/2005.00661). This has become an even bigger priority as LLMs are being adapted widely. \n\nThis paper addresses the hallucination issues in LLMs and proposes a self-supervised method for synthetically generating a training set of negative responses. Using these negative examples, models are then trained to encourage the generation of grounded outputs over unfaithful ones using preference-based training. \n\nThe authors empirically demonstrate that the proposed method helps models generate more faithful texts, when evaluated automatically and by humans, on data-to-text generation and summarization.", "questions": "We are moving towards general purpose models, instead of a task-specific model. How does the proposed method generalize to other tasks, while achieving higher factuality/faithfulness for data-to-text generation and summarization? Will it be much harder to tune the value of alpha when tried with multiple tasks together?\n\nSFT on a specific task often  suffers from catastrophic forgetting. It would be interesting to see how post training baselines do on these tasks. The post training often aims to improve faithfulness along other criteria. \n\nCould you add a detailed analysis of synthetically generated data wrt faithfulness and fluency? Also how does the proposed method compare with other synthetic data generation methods? \n\nReference responses often have hallucinations. For example, XSum gold summaries often have hallucinated content. How does this effect the data generation? How do you ensure that an example is a positive example with respect to faithfulness?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is trying to address generative hallucination, a common problem in generative models. The paper is fairly well written and easy to follow. \n\nThe paper brings new insights on the behavior of preference-tuning with the synthetically generated negative examples.\n\nThe proposed approach appears to improve response faithfulness, at a minor cost of fluency.", "weaknesses": "The method for synthetically generating negative examples is not novel, as the authors claim. Several papers (see some pointers below) in the past have used similar methods for generating negative or diverse sets of samples to better calibrate their models. \n\nThe paper should present a detailed analysis of the synthetic data being generated. For example, it would be nice to see faithfulness score distributions in negative examples. Also a qualitative analysis showing types of hallucinations being generated would have been very interesting. \n\nFaithfulness is a pointwise metric, but the human evals mostly focus on SxS ratings. It would have been nice to see pointwise evaluations of faithfulness. \n\nIt would have been nice to see comparisons to other ways of generating negative examples, for example, prompting models to generate negative examples.\n\nOther interesting work in this space that might be interesting to discuss here: \n\nBrio or Slic: https://arxiv.org/abs/2203.16804, https://arxiv.org/abs/2210.00045\nCalibrating models for faithfuness: https://arxiv.org/pdf/2310.08764\nhttps://renzhaochun.github.io/assets/pdf/26596-Article%20Text-30659-1-2-20230626.pdf\nhttps://arxiv.org/pdf/1910.08684 from the point of sub-sequence sampling\nComparison with CoT or planning based methods for controlling hallucinations.\n\n- \nAfter the rebuttals, I have adjusted the score. Thanks for conducting additional experiments and evaluations. They will be valuable to the readers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mHvqr3eQdd", "reviewer": "Reviewer_fCdZ", "summary": "Scope is a self-supervised framework for improving the faithfulness of conditional text generation in LLMs. The approach uses a novel two-stage process: first fine-tuning on half the dataset, then using a mixture of that model and the base model to generate synthetic unfaithful samples for preference learning. The authors evaluate their method on data-to-text and summarization tasks.", "questions": "1. Have you investigated whether the method's effectiveness varies with model size? The limitation to 7B models leaves open questions about scalability.\n2. The choice of α in [0.4, 0.6] seems empirically driven. Could it vary for different tasks / models?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- They study an important problem in the subfield -- factuality and faithfulness issue in conditional text generation. The research is motivated.\n- Interesting two-stage training pipeline, and the approach to generate synthetic unfaithful samples.\n- Provides analysis of preference training dynamics as well as the ablation for the noise parameter $\\alpha$", "weaknesses": "- The preference learning stage uses β=0.1 for all models and datasets (A.2) without justification or ablation, despite this being a critical hyperparameter in preference learning.\n- No evidence the method works on larger models. For their main claimed contribution (\"improving faithfulness\" in finetuning), testing on 7B models seems insufficient. Is it possible to include a few larger models, e.g. 14B, if not 70B scales.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F28ThahYaL", "reviewer": "Reviewer_NUJj", "summary": "This paper introduces a new learning framework for conditional text generation for the purpose of increasing the faithfulness of generated text with respect to the context input. The learning framework consists of two main steps: finetuning and contrastive learning. For contrasting learning, dispreferred samples are created in an unsupervised manner by utilizing a decoding method for generating certain tokens from the untrained language model, thereby introducing tokens which are not contextually consistent with the task. In several experiments, SCOPE is shown to be more effective than other baselines, including various decoding methods. SCOPE shows to be more faithful than traditional finetuning and SCOPE output is more preferred by humans and by GPT-4 judges.", "questions": "What are the impacts of training set size on the SCOPE methodology? It seems that for a small dataset, utilizing half of the dataset for finetuning and utilizing the other half for preference optimization may not be as effective as traditional finetuning.\n\nIs there a way to rate the quality of the dispreferred samples in 3.2? For example, are there generated texts with more inconsistencies that are worse than other samples? Is there an impact on the potential difference in quality of the generated texts?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Relatively novel methodology that uses the process of collaborative decoding for the purpose of generating dispreferred text. The method of collaborative decoding has been used in the past to combine text from a general LLM and an expert LLM to improve performance. In this setting, however, collaborative decoding is utilized to generate inconsistencies in the text randomly based on a threshold alpha.\n\nExperiments in the tasks of summarization and data-to-text in several datasets which highlight the strength of SCOPE compared to traditional finetuning and other baseline decoding methods. Additionally, the large variety of metrics, human evaluation, and GPT-4 evaluation indicates that SCOPE is effective at increasing the faithfulness of the model for these two specific tasks.", "weaknesses": "Lack of human analysis of the unsupervised generated data: it is unclear whether the the dispreferred samples are worse due to the changes in the language distribution, or if the dispreferred samples contain information or text that is inconsistent with the context. If the unwanted generation mainly comes from the changes in the language distribution, then SCOPE is not learning to be more faithful to the context. This is a key distinction which may limit the interpretability of SCOPE for being a more faithful method of conditional text generation. One way to mitigate this concern would be to conduct a human analysis on a sample of generated data and determine if the generated data contains text inconsistent with the context or if the inserted text from the base LM is merely different in language distribution.\n\nLack of comparisons to instruction-tuned models. The methodology of SCOPE, at its essence, is very similar to the standard method of reinforcement learning with human feedback, where the model is first finetuned and then preference optimized with binary data consisting of a preferred sample and an unwanted sample. Due to the similarities, it would be valuable to compare SCOPE to RLHF methods and compare the degree of faithfulness of the model. There is evidence that instruction-tuned models are more \"helpful\" to users' inputs in the prompt, which correlates highly with faithfulness. Using the instruction-tuned model as a baseline, or running SCOPE on an instruction-tuned model is important to validate SCOPE as a novel, effective method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AOoQuWWUcF", "reviewer": "Reviewer_eSUQ", "summary": "The paper investigates the common issue of hallucinations in large language models (LLMs) during conditional text generation. To enhance faithfulness, the authors introduce SCOPE, a novel self-supervised fine-tuning approach that generates unfaithful samples and then trains the model to prefer context-grounded outputs over these fabricated examples. By using this approach, SCOPE achieves substantial improvements in generating grounded and faithful responses, as validated by automatic metrics and evaluations from both LLMs and human judges. The study's findings demonstrate SCOPE’s effectiveness across diverse tasks, including data-to-text and summarization, where it consistently outperforms existing self-supervised techniques.", "questions": "1. In the noisy generation step, is fluency in the output ensured? Since each token is selected based on the weighted sum of the original and fine-tuned LM distributions, could it happen that the token with the highest probability from the combined distribution is not among the top tokens in either individual distribution? For instance, if the two models’ distributions differ significantly, could an unsuitable token receive a relatively high score in each distribution and then emerge as the highest-scoring token after the weighted sum?\n2. Is the generated negative examples all unfaithful? it would be useful to provide statistics or evaluation results indicating the proportion of genuine negative samples. This could potentially be estimated using the evaluation metrics applied in the experiments.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The research problem addressed in this paper is significant, as faithfulness remains a major challenge for contextual generation in LLMs.\n2. The authors propose an innovative method for generating negative examples for DPO training, utilizing two models—the fine-tuned model and the pre-trained model—selecting the next token based on a weighted combination of their distributions. This approach is independent of external tools like NER.\n3. The experimental results demonstrate that their method outperforms baseline models across multiple tasks.", "weaknesses": "1. Some of the baselines appear to be training-free decoding methods, while others require training, similar to the method proposed in the paper. The authors should clarify which methods are training-free and which are not, as they require different resources to implement.\n2. Additional ablation studies are needed, such as applying the same training framework with other negative sampling methods like NER replacement or negation. Otherwise, the effectiveness of the negative sampling itself remains uncertain.\n3. Experiments should include the faithfulness score of the generated negative examples on the training set to directly demonstrate the impact of negative sampling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jqFXtsed43", "reviewer": "Reviewer_aJYu", "summary": "The paper presents SCOPE, a self-supervised framework that enhances the faithfulness of LLMs in conditional text generation tasks like summarization and data-to-text generation. The framework employs a two-stage approach: initial fine-tuning followed by preference tuning using contrastive learning. The key innovation lies in its method of automatically generating synthetic unfaithful samples by blending grounded and context-free generation, which are then used to train the model to prefer context-aligned outputs over hallucinated ones. Through comprehensive evaluation across multiple datasets, SCOPE demonstrates significant improvements in output faithfulness, achieving up to 14% gains according to automatic metrics, with additional improvements confirmed through both GPT-4 and human evaluations.", "questions": "1. What is the rationale behind splitting the dataset into 50-50 for the two phases? Would other split also work?\n2. Do you anticipate that SCOPE would perform similarly on domain-specific datasets, like those in healthcare or finance?\n3. Can SCOPE’s approach to unfaithful sample generation and preference tuning generalize to larger or different model architectures?\n4. The $\\alpha$ for CAD is set to very small values for tasks other than XSum. Why is it the case?\n5. The main evaluation metrics used are BLEU/NLI/ROUGE/AL. Did you consider any additional faithfulness metrics or error types that might capture different facets of faithfulness?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- An innovative self-supervised approach to improving LLM faithfulness through synthetic unfaithful sample generation\n- The method shows strong performance across multiple benchmarks, with notable improvement in data-to-text tasks. \n- The paper explains the methodology and experimental setup well, using helpful figures and clear explanations of core ideas, such as the noisy sample generation and preference tuning.", "weaknesses": "- While SCOPE performs well on general summarization and data-to-text tasks, evaluating it on domain-specific datasets (e.g., biomedical or financial) could better demonstrate its robustness in high-stakes settings where hallucinations are especially problematic.\n- SCOPE is tested on 7B models only; applying it to models of different sizes would clarify its scalability and help determine whether it generalizes effectively to larger different models. Perhaps the improvement using SCOPE on larger models will be marginal. \n- The evaluation on summarization task used ROUGE and AlignScore (an entailment metric). While SCOPE scores slightly lower ROUGE scores than other baselines, the authors did not use other faithfulness metrics in summarization research to further evaluate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)