TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
Authors: Zorik Gekhman
Jonathan Herzig
Roee Aharoni
Chen Elkind
Idan Szpektor
Venue: EMNLP 2023 Main (2023)
Keywords: factuality
attribution
consistency
hallucinations

Abstract: Factual consistency evaluation is often conducted using Natural Language Inference (NLI) models, yet these models exhibit limited success in evaluating summaries. Previous work improved such models with synthetic training data. However, the data is typically based on perturbed human-written summaries, which often differ in their characteristics from real model-generated summaries and have limited coverage of possible factual errors. Alternatively, large language models (LLMs) have recently shown promising results in directly evaluating generative tasks, but are too computationally expensive for practical use. Motivated by these limitations, we introduce TrueTeacher, a method for generating synthetic data by annotating diverse model-generated summaries using a LLM. Unlike prior work, TrueTeacher does not rely on human-written summaries, and is multilingual by nature. Experiments on the TRUE benchmark show that a student model trained using our data, substantially outperforms both the state-of-the-art model with similar capacity, and the LLM teacher. In a systematic study, we compare TrueTeacher to existing synthetic data generation methods and demonstrate its superiority and robustness to domain-shift. We also show that our method generalizes to multilingual scenarios. Lastly, we release our large scale synthetic dataset (1.4M examples), generated using TrueTeacher, and a checkpoint trained on this data.

TrueTeacher:
Learning Factual Consistency Evaluation with Large Language Models
Zorik Gekhman?.G. T.G.+ Jonathan Herzig Roee Aharoni
Chen Elkind Idan Szpektor
TTechnion - Israel Institute of Technology "Google Research
zorik@campus technion. il (zorik/jierzzi//reeeaaaroo chenell (strrrrrrrrnneeeeeesseeeeeerrrer Jegoekteee@gogle.ccmo
Abstract
(CNN) pilot and passenger died when civilian plane crashed at Eglin Air Force base Florida Thursday momning the said. The Club Beech aircraft belanged to local flying club, said Federal Aviation Administration spokeswoman Kathleen Bergen. She said...
Factual consistency evaluation is often con- ducted using Natural Language Inference (NLI) models, yet these models exhibit lim- ited success in evaluating summaries. Pre- vious work improved such models with syn- thetic training data. However, the data is typi- cally based perturbed aumann-rrilen sum- maries, which often differ in their character- istics from real model-generated summaries and have limited coverage of possible factual errors. Alternatively, large language models (LLMs) have recently shown promising re- sults in directly evaluating generative tasks, but are too computationally expensive for prac- tical use. Motivated by these limitations. we introduce True a method for gen- erating synthetic data by annotating diverse model-generated summaries using a I.I.M Un- like prior work, True'Teacher does not rely on numan-written human- written summaries, and multilingual
T5-large
T5-base
TS-small
A pilot and passenger died when a small plane crashed at...
Two people have died
group of US pilots has been ki led in plane crash in
plane crash at Florida
LLM
1
O
Figure 1: A real example from our data generation pro- cess. We fine-tune summarization models with differ- capacities, and use them to produce diverse set of model-generated summaries of CNN/DM articles, which we label for consistency using 540B LLM.
by nature. Experiments the TRUE bench- mark show that a student model trained us- ing our data, substantially outperforms both the state-of-the-art model with similar capac- ity, and the LLM teacher. In a systematic study, we compare Teachea to existing synthetic data generation methods and demonstrate its superiority and robustness to domain-shift. We also show that our method generalizes multi- lingual scenarios. Lastly, we release our large- scale synthetic dataset (1.4M examples), gen- erated using True Teacher, and a checkpoint trained on this data.!
Since factual consistency evaluation could be cast as Natural Language Inference (NLI) task, NLI models are often used to evaluate consistency (Falke et al., 2019;; Maynez et al., 2020; Laban et al., 2022). However, II.I models exhibit lim- ited success in evaluating factual consistency in summarization (Falke et al., 2019b; Kryscinski et al., 2020), since NLI datasets lack the entail- ment phenomena that naturally arise in abstrac- tive summarization (Khot et al., 2018). For ex- ample, single-sentence prmise-hypothert pairs are shorter than document-summary pairs (Mishra et al., 2021; Schuster al., 2022).
1 Introduction
Generative summarization models are prone to generate summaries that are factually inconsistent with respect to the corresponding input documents (Goodrich et al., 2019; Kryscinski et al., 2019), limiting their applicability in real-world scenarios.
To address this domain mismatch, previous work proposed various approaches for generating syn- thetic training data (Kryscinski et al., 2020; Yin et al., 2021; Utama et al., 2022; Balachandran et al., 2022). The data is typically generated by perturb-
Work done during an internship at Google Research. 'Our dataset and model are available at: https:// thub. com/ com/google google- researchh// google-research/treee

---

ing human-writtn summaries to introduce factual inconsistencies. While these perturbations are ef- fective, they are limited to factual error categories that can be covered by the perturbation logic. In addition, since simulating factual errors is chal- lenging, such perturbations may fail to introduce factual errors, leading to incorrect labels. Finally, since the synthetic summaries are based on human- written summaries, they may differ in style from real model- generated summaries, which can reduce the effectiveness of the synthetic data.
synthetic data generation. In contrast, Teachea demonstrates robustness by successfully generaliz- ing to documents from new domains.
Finally, we apply Truchea to generate multilingual synthetic data. While existing data generation methods are often limited to English (Utama et al., 2022; Balachandran et al., 2022), TrueTTeacher can use multilingual LLM. Results on the mFACE dataset (Aharoni al., 2022), show improvements on 35 out of 45 languages when us- ing our method. This demonstrates the usefulness of multilingual synthetic data and the effectiveness of TrueTeacher in generating such data.
An alternative approach to augmenting NLI mod- els with synthetic data, is to directly prompt large language models (LLMs) to evaluate factual consis- tency. Recently, there has been a growing evidence for the effectiveness of LLMs in evaluating gener- ative tasks (Koemi and Federmann, 2023; Wang et al., 2023; Liu et al., 2023), including factual consistency in summarization (Chen et al., 2023). However, LLMs are still too computationally ex- pensive to be heavily used in practice.
To summarize, this work includes the following contributions:
We introduce TrueTeacher, synthetic data generation approach based on annotating model model-generated summaries with LLMs, and demonstrate its effectiveness and robustness.
We evaluate FLAN-PaLM 540B on the task of factual consistency evaluation and show that its knowledge be distilled into signifi- cantly smaller model using our method.
To make the best of both worlds we propose TrueTTacher, a simple and effective synthetic data generation method that leverages model- -eeer--eeerated summaries and the reasoning abilities of LLMs (Huang and Chang. 2022). In TrueTeacher, we first train diverse collection of summarization models with different capacities. Next, els to summarize each document in a given corpus (Figure 1). The resulting documret--uummary pairs are then annotated by prompting LLM to predict the corresponding factual consistency label.
We conduct systematic study, re-evaluating existing synthetic data generation methods for the task in an apples-to-apples comparison and identify their limitations.
these mod-
We perform the first experiment in generating multilingual synthetic data for factual consis- tency, and demonstrate its usefulness.
We apply TracTeachhr using FLAN-PaLM 540B (Chung et al., 2022) to generate large-scale syn- thetic dataset, which is used to train student model. Experiments on the summarization sub- set of the TRUE benchmark (Honovich et al., 2022) show that augmenting existing NLI data with Trachea data improves a state-of-thh--ar model's ROC-AUC from 82.7 to 87.8, while main- taining similar model capacity. The resulting model even outperforms its LLM teacher, despite the latter having x50 larger capacity.
We release a large-scale dataset comprised of 1.4 million Trcchea examples, and verify its quality with human evaluation. We addi- tionally release a state-of-the-art consistency evaluation model trained on this data.'
2 TrueTTecherr
In this section we describe Trachea our approach for generating synthetic examples for the task of factual consistency evaluation summarization. Our main motivation is to use factual inconsistencies that occur in real gener--geerrttdd summaries, instead of relying on perturbed wwitten summaries. To this end, generate diverse set of summaries using gener- ative summarization models of different capacities, and leverage a LLM to label them for factual con- sistency. Some of the generated summaries are ex- pected to contain factual errors, and we hypothesize
We also compare Teachea to existing syn- thetic data generation methods. To this end, we design a systematic study to re-evaluate existing methods with "fair comparison" in challeng- ing setting. Our results indicate that existing ap- proaches fail to generalize documents derived from a distribution different from the one used for
AAs we also demonstrate in $4.3.

---

Summa tzation Datasets
Pre-rainnd Language Models
Summar zation Modes
does not require gold summaries, which allows it to be used with any collection of documents D, and makes more scalable than previous methods (Yin et 2021; Utama et al., 2022; Balachandran et al., 2022).
sd
[m
59111
sd2
Imz
Finally, LLM is prompted to label all the summaries in O for consistency w.r.t their source documents, resulting with labels (......,,11.......d
sd,
Imm
s
Summantzal Models
Model generated Summaziee
Syntheic Consistencyy
Figure illustrates real example of this process for a single document dj E D. Each document, summary, and label (d,, SiJ, (4j) are then used as a synthetic example for training factual consistency classifier. Since leverage LLMs for labeling, our approach is likely to benefit from the ongoing progress in LLMs quality. Furthermore, previous approaches often rely on language-specific components (e.g.. Information Extraction), which limits their appli- cability in multiple languages. Since recent LLMs are pretrained on multilingual data, our method can be easily applied to non- non-English languages, as we show in $5.
Documents
S/11
81,1
4.1
3/112
$12
LLM
4,
d
S/11
Lib
Figure 2: Our data generation process. We train a col- lection of generative summarization models, use them to summarize documents and label the resulting sum- maries for factual consistency using a LLM.
that groong-performing LLM can generalize the task and label them with sufficient quality to be use- ful for training. The usage of -enee--geeeeted sum- maries not only yields more realistic texts, but also allows to potentially include rare errors, which can be harder to incorporate with perturbation logic.
Experimental Sctup
We use Trucheacher to generate a synthetic dataset for factual consistency evaluation in summariza- tion ($3.1), and experiment with it to evaluate the effectiveness and usefulness of our method ($4).
Our data generation process is illustrated in Figure 2. First, we train variety of summa- rization models (upper diagram). We use a col- lection of one or more summarization training and different pre-
3.1 True Instantiation
sets = {ad1, sdl,..., trained LMs = [lm1, lm....., } to fine- tune collection of summarization models SM {sm1, sm,,, ..., smz}, where k different pretrained LMs allows to diversify the expected consistency errors, e.g., errors made by large or small models. The choice of summariza- tion training sets allows to control for the nature of the resulting summaries, e.g., focusing on abstra- tive training sets to increase output abstractiveness.
To apply TrueTre we instantiate the summa- rization datasets T, the pre-trained LMs and the documents corpus D. We use XSum (Narayan et al., 2018) T. T5 pre-trained models (Raf- fel al., 2020) LMs (T5-small, T5-base, T5-large. T5-3B, T5-11B}, and documents from CNN/DailyMail (Hermann al., 2015) as D). As our teacher model, we employ FLAN-PaLM 540B (Chung et al., 2022). This model was instruc- tion fine- tuned, including training on the closely- related NLI task. Therefore, we expect it to gen- eralize well to factual consistency evaluation. We use zero-shot prompting for simplicity, and since applying few-shot or chain-of- of-th-ughtt prompting did not improve performance in early experiments."
m. Using
Next, we obtain model- -enerrate summarics and annotate them (lower diagram). We choose a docu- ments corpus D {d,, d2, ...... dr} and use all the summarization models in SM summarize all the documents in D, resulting in a collection of model- generated output summaries O {81,1)... 8r,)) where $tj is the summary of document d, gener- ated by summarization model sm,.
"See $3.1 and $A.1 for our prompting implementation. 'https:/ //gs: thub. htt.//gogll.ressaac//LLLLLLL. e9e4ec6e2701 a 7677ff 10071 bock_split._pp py
'We note that the pretrained LM here refer to the mod- els that we are fine tuning for summarization, and they are different from the L.M that we use as the teacher.
"We validate this expectation in $4,1 and $4.4. "In $A.1 I we discuss potential reasons to this.

---

Summaries Source
Consistent
# Inconsistent
et al., 2022), WeCheck (Wu al., 2023), and the Ensemble from Honovich al. (2022).
T5-11B
233,815
39,423
T5-3B
229,097
45,662
We also compare TrueTeacher data generation mechanism to existing methods for synthetic data
TS-large
195,681
81,986
T5-base
161,177
118,480
generation. We consider the following approaches:
T5-small
88,129
190,012
(Yin et al., 2021). Reformatted NLI, question answering and summarization datasets, in- cluding the CNN/DM corpus. The summarization- based positive examples are based on concatenated gold summaries. The negative examples are then generated using word/entity replacements.
DocNLI
Total
907,899
475,563
Table Our generated dataset statistics.
Extensive implementation details about our FLAN-
PaLM usage provided in $A.1 and $A.2.
Applying TrueTeacher this setup resulted 1..4M synthetic training examples (Table 1), which we use to train a student model for factual consistency evaluation. 8 In $4, we provide evi- dence for the dataset's quality through human eval- uation ($4.4), its usefulness for improving NLI models in challenging setting ($4.1), and its supe- riority over other existing synthetic datasets ($4.2).
FactCC (Kryscinski et al., 2020). The docu- ments are from CNN/DM. The consistent sum- maries are randomly sampled sentences from the document, which are optionally injected with noise or paraphrased. The inconsistent summaries are ob- tained by rule-based transformations, such tence negation and entity/prononnnumbbr swaps.
FactEdit (Balachandran et al., 2022). The posi- tive examples are based on gold summaries from CNN/DM. For the negative examples, infilling model is trained using sentences from the docu- ments, employing the OpenTE framework (Banko et al., 2007) to mask predicates and arguments. Each predicate and argument phrase in the sum- mary is then iterativelly masked and infilled with the model's lower order beam candidates.
In early experiments, we also explored data fil- tering based on prompting FLAN-PPLMM for self- verification (details $A.5). This resulted in an increase in the labeling accuracy. Yet, surprisingly, training the student model on the filtered data did not improve performance in comparison to train- ing on the full dataset." Thus, for simplicity, conduct experiments using the full dataset.
3.2 Evaluation
Falsesum (Utama et al., 2022). The positive examples are based on gold summaries from CNN/DM. For the negative examples, predicates and arguments are detected in the document and the summary using the OpenlE (Banko et al., 2007) framework. Randomly selected predicates and ar- guments from the summary are then masked and infilled using predicates and arguments from the document, or by "hallucinating" new content. For this purpose dedicated infilling model is trained.
To compare between consistency evaluation mod- els, we use the TRUE benchmark (Honovich et al., 2022), focusing on its summarization subset: MNBM (Maynez et al., 2020), FRANK (Pagnoni et al., 2021), Svl (Fabbri et al., 2020), QAGS-X and QAGS-C (Wang et al., 2020). For additional details about these datasets, we refer the reader to Honovich et al. (2022). Following Honovich et al., use ROC-AUC binary clas- sification setting as our evaluation metric.
4 Experiments and Analysis
3.3 Baselines
Our main experiments are in $4.1 and $4.2, followed by various analyses and ablations in $4.3, $4.4, $4.5 and $4.6. We design our experiments to address the following research questions (RQs):
We compare the performance of factual consistency evaluation models trained on TrueTeacher data, against the top performing models on the TRUE benchmark: QuestEval (Scialom et al., 2021), Q² (Honovich et al., 2021), SUMMA Czs (Laban et al., 2022), T5-11B fine tuned on ANLI (Honovich
RQI: What is the performance of FLAN-PaLM
540B in factual consistency evaluation in sum- marization? Is it good choice for teacher?
'Implementation details our trained models are in $A.3. "This could be attributed to the high-quality of the initial labels and the student model's 's robustmess to noise.
lWee discuss WeCheck in $6, and refer the reader to Hon- ovich et al. (2022) for a detailed description of other busclines.

---

MNBM
QAGS-X
FRANK
Summ
QAGS-C
Average
EveltEval (Scialom et al., 2021)
65.3
56.3
84.0
70.1
64.2
68.0
Q' (Honovich al., 2021)
68.7
70.9
87.8
78.8
83.5
77.9
SUMMAC25 (Laban et al., 2022)
71.3
78.1
89.1
81.7
80.9
80.2
TB-11B w. ANLI (Honovich et al., 2022)
77.9
83.8
82.1
80.5
89.4
82.7
WeCheck (Wu al., 2023)
83.0
81.4
88.1
79.8
82.6
83.0
Ensemble (Honovich ct al., 2022)
76.6
85.8
91.2
82.9
87.7
84.8
FLAN-PaLM 540B (Chung al., 2022)
76.0
88.1
91.4
83.7
85.2
84.9
T5-11B w. ANLI+ TrucTTeacher full
78.1
89.4
93.6
88.5
89.4
87.8
Table 2: ROC-AUC results on the summarization subset of the TRUE benehmark (Honovich et al., 2022).
RQ2: Can TruheTeacher facilitate training of competitive model w.r.t. state-of-the-art models?
attributed large-scale knowledge distillation on specific task, while the LLM trained to per- form many tasks. Additionally, the smaller model is trained on summaries) which can further im- prove performance (Gururangan et al., 2020).
RQ3: What the quality of the data gener- ated using TrucTeaccher compared to existing syn- thetic data generation methods?
data (documents and
We address RQI and RQ2 $4.1. To address RQ1, we evaluate FLAN-PaLM 540B against com- petitive models for factual consistency evaluation. To address RQ2, we use our full dataset from $3.1 to train our best-performing model, and evaluate it in the exact same setting. Finally, RQ3 is ad- dressed in $4.2, where conduct systematic study, comparing existing methods Teache,, while controlling for factors such as the synthetic data size and the documents used for data synthesis.
4.2 Re-evaluating Synthetic Data Generation Methods - A Study
Previous studies synthetic data generation have used different experimental setups, making it dif- ficult to compare their results. In this section, we design a systematic study to re-evaluate existing methods in a standardized setup. We first discuss our study design choices followed by the results. Previous work has demonstrated that synthetic data can improve NLI-based models. However, they typically used relatively small-capacity mod- els, whereas Honovich al. (2022) recently demon- strated significant performance gains by scaling up to T5-11B fine-tuned on ANLI. We therefore adopt this competitive baseline, to which we add syn- thetic data from each method. For ablation, we include variants trained solely on synthetic data (without ANLI), and also repeat our study using the smaller-capacity T5-base model.
4.1 Main Results on the TRUE Benchmark
We address RQI by evaluating FLAN-PaLM 540B on the task and present the results in Table 2. FLAN-PaLM 540B achieves an impressive perfor- mance, with an average ROC-AUC of 84.9 com- pared to 83.0 of the best single-model baseline, and performs on-par with the Ensemble. This demon- strates the chosen LLM's capability for the task, teacher for smaller models. To address RQ2, we fine-tune T5-11B (Raffel al., 2020) over our full dataset ($3.1) mixed with ANLI (Nie et al., 2020). Table shows that including Trucheacher data the training set, substantially improves the strong-performing T5-11B w. ANLI baseline from an average ROC- AUC of 82.7 to 87.8 (+5.1), while maintaining ex- actly the same model capacity. This strong result demonstrates the high effectiveness of True Teachee in challenging setup. Notably, our model sets the new state-of-the-ard result on the benchmark, outperforming the x50 times larger LLM that we used as the teacher (84.9 - 87.8). This can be
and its potential
To preform a fair comparison, we restrict the number of examples from each evaluated method to 100k, randomly sampled with balanced labels.
To evaluate domain-shift robustness, we fur- ther restrict the synthetic training examples to ones that were generated only based on CNN/DM docu- ments, !I and then consider the XSum-based evalu- ation sets as out-of-domain."
Some methods are based exclusively on CNN/DM while others use additional datasets, more details in $3.3.
"SummEyal and QAGS-C are based on documents from CNN/DM. MNBM and QAGS-X use documents from XSum, and FRANK has documents from both CNN/DM and XSum. We split ANANK to FRANK-C C and FR FRANK-X which contain its CNN/DN based and XSum based subsets respectively.

---

CNNIDM--sse
XSUM-hased
Average scores
Training data
FRANK-C
FRANK
FRANK-X
Out-of-domain
ANLI
83.4
74.2
35.6
90.7
93.2
$8.0
73.9
81.1
85.0
82.0
83.7
(-0.4)
(-188)
(-7.8)
FactEidit ANLI
88.9
78.9
31.1
88.0
86.1
76.2
59.8
(3.0
740(-1100)
(-.4)-)))
DocNLI
89.1
72.9
83.0
89.2
92.4
83.8
67.0
81.7 (+0.6)
(-3.9)
80.4 (-1.5)
DucNLI+ ANLI
87.8
72.0
81.9
88.2
93.7
84.2
68.0
(-.66)
62.0
50.0 (-2.0)
FacCCC
83.1
79.0
$1.6
84.1
67.5
72.7
55.0
81.2 (+0.1)
63.1 (-19.9)
74.8 (-7.2)
FacIOC ANLI
84.7
84.7
89.5
89.6
84.2
81.3
82A (404)
Falsesum
90.3
85.4
85.8
89.8
84.5
70.8
53.9
87.2 (+6.1)
69.7 (-15.)
78.0 (-4.0)
Falaesum ANLI
TrueTeacher
84.9
$5.0
$5.8
93.6
94.4
$6.5
76.1
86.2 (+5.1)
85.7 (+0.7)
85.2 (+3.2)
Teacher
(16.4)
ANLI
74,9
63.7
73.1
81.3
80.6
77.2
77.0
70.6
78.3
74.8
FactEdit
73.6
51.9
48.0
58.4
60.1 (-10.5)
52.8 (-25.5)
60.2 (-14.6)
FactEdit ANLI
68.7
60.0
62.2
78.5
73.6
72.2
75.5
(37.0)
73.8 (-4.5)
71.00
DocNLI
71.4
66.5
66.7
77.9
81.0
75.2
71.6
68.2 (-2.4)
(-294)
72.5 (:2.3)
DocNLI ANLI
75.2
66.7
74.4
84.9
83.3
78.7
74,8
72.1
(+0.6)
(6.1))
FacICC
74.0
72.7
78.7
83.2
71.9
71.0
62.7
75.3 (+4.7)
68.5 (-9.8)
72.7 (-2.1
T3
FacICC ANLI
(-12))
Felsosum
80.9
74.2
82.0
86.4
71.6
65.0
53.1
(-8.4)
63.2 (-15.1)
7129
Falsesum
TraeTracher
77.3
73.6
79.1
88.0
82.6
79.9
78.3
76.7 (46.1)
80.3 (42.0)
79.4
TraeTsccher- ANIJ
80.4 (49.8)
(44.0)
81.9
Table 3: ROC-AUC results TRUE comparing different synthetic data generation methods. For each model size, average scores are compared to the corresponding ANLI-only baseline (difference is listed in parentheses).
Table 3 presents the results of our study. We cal- culate three average scores: for in-domain test sets based on CNN/DM documents, for out-of-domain test sets based on XSum documents, and for the original datasets from TRUE.
pared the in-domain setup, it is still the only method with higher out-of-domain score compared to the corresponding ANLI-only baseline. This demonstrates the robustness of TrueTeacher to do- main shift, which may be due to the use of model- generated summaries that increase the variability of the resulting synthetic data.
In-Domain Results Most methods outperform the corresponding ANLI-only baseline, demonstrat- ing the usefulness of synthetic data. Predictably, all methods improve with larger models and a comple- mentary effect is often observed when mixing syn- thetic data with ANLI. The best results are obtained by mixing ANLI with Falsesum or Teacher data and using T5- with substantial improve- ment over the corresponding ANLI-only baseline (in-domain score increase from 81.1 to 87.9).
Overall Results on TRUE Due to the poor out- of-domain performance of the existing methods, TrueTTeacher is the only method that consistently outperforms the ANLI-only baseline on the TRUE benchmark. Notably, TrueTeacher ANLI with T5- base (81.9) performs par with the ANLI-only baseline using T5-11B (82.0). Additionally, the TrueTeacche--aaedd variant using T5-1 1B (85.2) al- ready performs on-par with the 540B LLM teacher (84.9, Table 2), even though we used only 100k syn- thetic examples in this experiment, and did not use ANLI data. When comparing TrucTeacher + ANLI with T5-11B and 100k examples (Table 3) the equivalent variant using the full dataset (Table 2). observe a performance increase (86.1 - 87.8), which demonstrates Tracheasher's scalability. We conclude that TrueTeacher yields high quality data and generalizes well for new domains, which we at- tribute to the usage of model-generated summaries.
Out-of-domain Results While most methods perform well in-domain, their performance drops significantly on the out-of-domain test sets. Most of the evaluated methods underperform the corre- sponding ANLI-only baseline with similar model capacity. For some methods, performance dete- riorates dramatically; e.g. Falsesum despite its impressive in nn-oomain performance, its out-of- domain score falls significantly below the ANLI- only bascline. This suggests that some methods overfit to documents from the distribution used to generate the synthetic data. Based on this find- ing, we encourage future research prioritize out- of-domain evaluation. Interestingly, even though True'Teacher relative improvement is smaller com-
4.3 Qualitative Analysis
Figure presents a case study with a randomly sam- pled document, and the corresponding inconsistent summaries generated with each of the evaluated

---

Class
#Ex.
Precision
Recall
F1
Article: LOS ANGELES, California (CNN) - Los Angeles frrfighters and own: lnaterlogged sinkhols Two firafighters crawled out the truck's windows after it sank Tuesday momiig. No one was injure. The incident happened after four firefighters took tuck to the San Femanno Valley neighborhood of Valley Vilage, flooding had reported.....
city orews worked several hours Tueaday to rescue one
Consistent
41
80.0
97.6
87.9
22-ton firatruck that nearly swallwed
Inconsistent
59
98.0
83.1
89.9
Table 4: Human evaluation results.
Gold Summaries:
1. Los Angeles firetruck nearly swallowed by sinkhole Tuesday morning.
introduces a nuanced factual error by replacing "Los Angeles firefighters" with A firefighter and also by hallucinating new content (the text in bold red font). This case study further illustrates the challenges of perturbing texts to introduce factual inconsistencies and re-iterates the importance in using model--eneraaee summaries.
Firefighters in truck were responding to fooding call when incident
happened
Two frofighters escaped truck through windows; injuries reported.
Firefighters in truck were responding rescue when incident happened..
FactEdit
Angeles firetruck nearly destroyed by sinkhole Tuesday night. Firefighters truck responding emergency call when ( happened Two firefighter escaped truck through windows no injuries reported
DocNLI
LOS LOS ANGELES, California ([CNN) - Las Angeles firefighters and worked Two on Tuesday resque of their
FactCG
4.4 Human Evaluation
22-ton fire engine nearly
swallewed by a Binkhole filled with waterwater.
To further assess the quality of the synthetic data produced by Teaheeachhr, we perform human eval- uation carried out by domain experts. 13 We evalu- ate 100 examples from our dataset," using binary judgements based on the attribution definition from Rashkin et al. (2021). The labeling accuracy of the sampled examples from our data stands at 89%, which demonstrates its high quality. Table further presents the precision, recall and Fl scores for the consistent and inconsistent classes. More details on the human evaluation are available in $A.8.
Falsesum
Los Angeles firstruck nearly swallwwad by water. firefighter has rescued truck that sank in Los Angeles, causing extensive flooding.
True Teacher
Figure 3: A case study comparing factually inconsis- tent summaries of the same document generated us- ing different methods. Content replacements are high- lighted using the same color for the original and the replaced text. Added content is in bold red font.
methods. FactEdit used the second gold-summary and replaced "to flooding call" with "rescue", in- troducing grammatical error rather than a factual error, demonstrating the potential problems with using lower-beam completions proxy for factual errors. DocNLI uses all the gold summaries con- catenated. While replacing "morning" with "night" introduces a factual error, three other edits fail to introduce factual errors, demonstrating the limi- tations of using simple word/entity replacements. FactCC used the first sentence from the article and successfully introduced factual error by an entity swap from "firetruck" to "fire engine". The para- phrase highlighted in green increases the abstrac- tiveness, but the paraphrase in orange introduces a grammatical error that is less likely to be made by a strong summarization model. The noise in- jection used by FactCC (duplicating or removing random tokens) is colored in red, but its useful- ness is questionable. Falsesun uses the first gold summary, and its perturbation model predicts the removal of "Tuesday morning" and the replacement of the "sinkhole" argument with "water", failing introduce factual error, since the sinkhole is referred to as "water-logged sinkhole" in the ar- ticle. Finally, TrueTeacher uses an abstractive summary generated by a real summarization model.
4.5 Ablating Summary Distribution and Label Correctness
There are two key differences between TrueTeachie and perturbation-based synthetic data generation methods: (1) the distribution of the s 15 and (2) the correctness of the generated labels. Each of these differences may lead to the better quality of True'eeacher w.r.t the baselines. To mea- sure the impact of each difference, we isolate them in a controlled ablation study. We create 2 ab- lated variants, using Falsesum as a recent baseline method for synthetic data generation. The results are presented in Table 5,
LabelAblation is an ablation created by label- ing the document-summe summmnr pairs from Falsesum's data using FLAN-PaLM 540B. 17 Comparing
10 NLP researchers, each with at least one year of experi- ence in factual consistency evaluation.
L'W randomly sampled 50 positively and 50 negatively labeled examples from our synthetic dataset.
human written perturbed. "Both methods may yield wrong labels. Perturbations might not introduce inconsistencies. as seen $4.3, while TrueTeacher can have errors due to LLM mislabeling.
15 Model generatge
We used the same 100k examples as Falsesum + ANLI hascline, and the same LLM prompt as in TrueTeacher

---

Variant
Summary Distribution
Labeling Quality
T5-11B
T5-Base
Falsesum ANLI
Humtn-written perturbed
Falsesum
80.8
73.5
TrueTeacher + ANLI
Model-generated
FLAN-PaLM 540B
86.4 (+6.9%
81.9 (+11.4%)
LabelAblation
Human-written perturbed
FLAN-PaLM 540B
85.3 (+5.6 (%)
78.9 (+7.3%)
SummaryAblation
Model-gencrated
Falsesum (proxy)
85.5 (+5.8%)
79.1 (+7.6%)
Table 5: Average ROC-AUC on TRUE for the ablated variants. Falsesum ANLI and TraceTeacher ANLI are copied from Table for reference.
LabelAblation to Falsesum + ANLI allows us to examine the effect of using FLAN-PaLM labels instead of the original Falsesum labels, while controlling for the summaries distribution. LabelAblation outperforms Falsesum + ANLI by 5.6% which shows that performance gains can be obtained using summaries generated with exist- ing synthetic data generation methods combined with second-stage stage improved labeling quality. How- ever, TrueTeacher is substantially simpler and also results in better performance.
Coverage Density
Combined +
FactEdit
0.86
2.92
2.67
DocNLI
0.85
15.66
15,20
FacICC
0.93
8.16
7.93
Falsesum
0.88
2.98
2.76
TracTeachhr
0.86
2.41
2.15
Table 6: Average abstractiveness scores (lower is bet- ter), measured on a random sample of 5k examples.
measures to obtain a combined score. 18 Table 6 presents the abstractiveness scores, and density plot is available in the Appendix (Figure 5). We ob- serve higher abstractiveness for model-based meth- ods (FactEddit, Falsesum and TrueTeacher), suggest- ing that rule-based methods might be less useful with the recent shift towards abstractive summariza- tion. TrueTeecher produces the most abstractive summaries with lowest combined score.
SummaryAblat ion is an ablation created by flip- ping labels on random portion of True'Teacher data, such that the expected labeling accuracy is similar Falsesum (More details in $A.9). Com- paring SummaryAblat
Falsesum + ANLI al- lows us to examine the effect changing the sum- mary distribution from human-writted perturbed to mooel--generaiee beling quality. SummaryAblation outperforms Falsesum + ANLI by 5.8%. % similar improve- ment observed for LabelAblation (5.6%). This demonstrates that label correctness and summary distribution have similar effect on the perfor- mance, but they also have a complimentary effect as the best performance 86.4 ROC-AUC ob- tained only when they are combined together.
while controlling for the la-
5 Multi-Lingual Data Generation for Factual Consistency Evaluation
Utilizing a multilingual LLM enables straightfor- ward application of TrucTeacher to multiple lan- guages. This contrasts with recent approaches that rely on NLP components only available for high- resource languages, e.g., information extraction (Utama et al., 2022: Balachandran et al., 2022). In this section, we examine TrucTeacher's 's usefulness for multilingual factual consistency evaluation.
4.6 A Abstractiveness Analysis
Advances in large scale pretraining (Devlin et al.. 2019; Lewis et al., 2020) and the availability of rel- evant datasets (Narayan et al.. 2018), enabled rapid progress in abstractive summarization, which bet- ter imitates the way humans summarize (Koh et al., 2023) and is also preferred by humans (Goyal et al., 2022). This motivates us to focus on generating abstractive synthetic summaries.
We first generate multilingual synthetic data us- ing TrucTeacher. This time we train a single sum- marization model by fine tuning mT5-XXL, (Xue et al., 2021) on XLSum (Hasan et al., 2021) and use it to summarize documents from WikiLingua (Ladhak al., 2020), which then label for con- sistency with our LLM. For the purposes of this experiment we focus on subset of WikiLingua documents 4 languages: English (en), French
We compare the abstractiveness degree of differ- ent methods using the extractive fragment coverage and density measures from Grusky et al. (2018). Following Utama et al. (2022) we multiply these
We provide additional technical details in $A.6.

---

Improved languages
Avg. ROC-AUC Per lang.
vised consistency evaluation model that leverages probabilistic labels derived from aggregated scores of other consistency evaluation models. Our work proposes a simpler solution, that is also inherently 75.3
Training data
Per ex.
ANI.I+NNII
73.3
71.6
+ Terchee en + +rruTTaccher en.fe,es.ge
32/ / 45
75.7
73.8
35 45
77.2
multilingual.
Table 7: Multilingual results on the mFACE test set.
Another line of work for adapting NLI-based models for summarization, focuses on better pro- cessing of long texts, splitting the documents into sentences to create shorter premise-hypootheis pairs (Laban et al., 2022; Schuster et al., 2022).
(fe), Spanish (es) and German (de). 19, After gener- ating the dataset for these 4 languages, we sample 100k examples, by randomly sampling 25k in each language with balanced labels (as illustrated Ta- ble 9 in the Appendix). For ablation, we also cre- ate an English-only variant, by randomly sampling 100k English examples with balanced labels. 20
Recent work attempts to assess LLMs' capability for evaluating generative tasks (Koemi and Feder- mann, 2023; Wang et al., 2023; Liu et al., 2023). Luo et al. (2023) evaluated ChatGPT (OpenAI, 2022) speciffically on the task of factual consis- tency evaluation in summarization. Yet, Aiyappa et al. (2023) argued that ChatGPT's "closed" nature risks data leakage (training-test contamination).? Chen et al. (2023) performed study of LLMs as factual consistency evaluators, using a variety of prompting methods.
We use the resulted data to train multilingual con- sistency evaluation models and evaluate them on the mFac test set (Aharoni et al., 2022), containing 3150 examples in 45 languages. As a strong base- line we follow Aharoni et al. and fine-tune mT5- XXL (Xue et al., 2021) on the ANLI (Nie et al., 2020) and XNLI (Conneau et al., 2018) datasets. We then assess whether adding our synthetic data the training set can improve this model.
Previous work also attempted to distill knowl- edge from LLMs (West et al., 2022; Hsich et al., 2023), as well as to leverage LLMs for data anno- tation (Wang et al., 2021; Ding et al., 2022), and synthetic data generation (Agrawal al., 2022: Liu al., 2022: Bitton al., 2023). As far as we aware, our work is the first to leverage LLMs for data generation for factual consistency evaluation.
Table 7 presents the results overview, full re- sults in all 45 languages are available in Table 10 (Appendix). Adding English-only summarization- based synthetic data, already improves results on 32 out of 45 languages and increases the avg. ROC- AUC from 71.6 to 73.8. Yet, using the same amount of multi- lingual examples improved the performance even more, with avg. ROC AUC of 75.3. This demonstrates the added value in generating multi-lingual synthetic examples using TrueTeacher, laying the ground for future work.
7 Conclusion
We introduced TrucTeacher, simple and highly effective method for generating synthetic data for factual consistency evaluation. Instead of per- turbation of human-written summaries like done in previous work, TrueTeacher leverages realistic model generatee summaries, which are annotated by prompting a large language model.
6 Related Work
Previous work proposed methods for generating synthetic training data for factual consistency eval- uation, by perturbing gold summaries (Yin et al., 2021; Kryscinski et al., 2020; Balachandran et al., 2022: Utama et al., 2022; Soleimani et al., 2023), 21 A key advantage of True Teacher, is the ability to leverage real model-generated summaries, leading to superior performance and robustness. The utility of moder-ggnerated outputs was also highlighted by Wu et al. (2023), who proposed a weakly super-
Using our method, we generate large-scale synthetic dataset, which are making publicly available. Our experimental results show that this dataset substantially enhances the performance of state-of-the-ar model. In our systematic study, we compare TrucTeacher to existing approaches and further demonstrate its effectiveness and robust- ness. Our study highlights the importance of out-of- domain evaluation, which we hope will be adopted in future work. Lastly, we show that Teacheacher generalizes well to multilingual scenarios, present- ing additional advantage over existing methods.
"hhey the most prevalent languages in PaLM's pre- training data (Chowdhery al., 2022)
20 Also based on WikiLingua, generated with the same pro- cess like the 25k English subset our multilingual dataset. 21 We provide extensive review of these methods in $3.3.
Whil FLAN's instruction fine-tuning data is public.

---

8 Limitations
Roee Aharoni, Shashi Narayan, Joshua Maynez, Jonathan Herzig, Elizabeth Clark, and Mirella La- pata. 2022. mface: Multilingual summariza- tion with factual consistency evaluation. CoRR, abs/2212. 10622.
Noisy synthetic data Teacher relies on a LLM for labeling model generated summaries. This process may result in some frequency of noisy synthetic examples for which the label is incor- rect. This can affect the overall quality of the stu- dent model that trains on this data. In our experi- ments we validated the quality of our synthetic data with human evaluation, however this should be re- examined when generating data for new domains. In addition, we experimented with different filter- ing approaches. but found that training on filtered data with higher labeling accuracy, did not improve the performance of the student model. We encour- age future work to further examine such automatic filtering.
Rachith Aiyappa, Jisun An, Haewoon Kwak, and Yong- Ycol Ahn. 2023. Can trust the evaluation on chat- gpt? CORR, abs/2303.12767.
Vidhisha Balachandran, Hannaneh Hajishirzi. William W. Cohen, and Yulia Tsvetkov. 2022. Correcting diverse factual enrors in abstractive summarization via post-editing and language model infilling. In Proceedings of the 2022 Conference Empirical Methods in Natural Language Pro- cexsing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 9818-9830. Association for Computational Linguisties.
Michele Banko, Michael J. Cafarella, Stephen Soder- land, Matthew Broadhead, and Oren Etzioni. 2007, Open information extraction from the web. In IJCAI 2007, Proceedingy of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, In- dia, January 6-12, 2007, pages 2670-2676.
Reliance on LLMs In this work we use a 540B LLM to label 1.4M model generated summaries. This requires non-negligible resources that may not be available to the whole community. To mitigate this, we release our collected synthetic data and the corresponding model checkpoint. In addition, the decreasing inference cost of proprictary LLMs, and the availability of open-source LLMs (Touvron et al., 2023) can further assist.
Yonatan Bitton, Shlomi Cohen- Ganor, Ido Hakimi. Yoad Lewenberg. Roce Aharoni, and Enav Weinreb. 2023. q2d: Turning questions into dialogs to teach models how to search.
Effect of low-resuurce languages Our multilin- gual experiments ($5) focus subset of WikiLin- gua documents in only 4 languages: English (en), French (fe), Spanish (es) and German (de), that are the most prevalent in our LLM's pre-training data. As can be seen in our full results (Table 9 in the Appendix), our multilingual data successfully improves low-resource languages as well. We did not fully explore the effect of adding additional languages to our synthetic data, especially low- resource ones. We believe that there is trade- off between language coverage and labeling qual- ity. i.e while generating the synthetic data in low- resource languages will increase language cover- age, it can lead to poor labeling quality by our LLM. We did not fully explore the exact sweet-spot for how many languages to include in our synthetically labeled training data, leaving this for future work.
Shiqi Chen, Siyang Gao, and Junxian He. 2023. Eval- uating factual consistency of summaries with large language models. CoRR, abs/2305.14069.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emiil Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin. Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghe- mawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra. Kevin Robinson, Liam Fe- dus, Denny Zhou, Daphne Ippolito, David Luan, Hycontack Lim, Barret Zoph. Alexander Spiridonov, Ryan Sepassi. David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankara- narayana Pillai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Bren- nan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy
Douglas Eck. Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language modeling with pathways. CoRR, abs/2204.02311.
References
Priyanka Agrawal, Chris Alberti, Fantine Huot, Joshua Maynez, Ji Ma, Sehastian Ruder, Kuzman Ganchev, Dipanjan Das, and Mirella Lapata. 2022. Qameleon: Multilingual QA with only examples. CoRR, abs/2211 1.08264.
Hyung Won Chung. Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang.

---

Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suz- gun, Xinyun Chen, Aakanksha Chowdhery, Sha- ram Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Fdd H. Chi, Jeff Dean, Jacob Devlin, Adam Roherts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling instruction-fine finetuned language models. CoRR, abs/2210. 1416.
Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HILT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Vol- ume 1 (Long Papers), pages 708-719. Association for Computational Linguistics.
Alexis Conneau, Ruty Rinou,, Guillaume Lample, Ad- ina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: evaluating cross-lingual sentence representations. In Proceed- ings of the 2018 Conference Empirical Methods Natural Language Processing, Brussels, Belgium, October 31- November 4, 2018, pages 2475-2485. Association for Computational Linguistics.
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342-8360, Online, Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Pa- pers), pages 4171-4186. Association for Computa- tional Linguistics.
Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Is- lam, Kazi Samin Muhasshir, Yuan-Fang Li, Yong- Bin Kang. M. Sohel Rahman, and Rifat Shahri- yar. 2021. XI-sum: Large-scale multilingual ab- stractive summarization for 44 languages. In Find- ings of the Association for Computational Linguis- tics: ACIAJCCLP 2021, Online Event, August 1- 6, 2021, volume ACI/UCCILP 2021 of Findings of ACL, pages tional Linguistics.
Association for Computa-
Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing. Shafiq R. Joty, and Boyang Li. 2022. Is GPT-3 a good data annotator? CoRR,
Karl Moritz Hermann, Tomás Kociský, Edward Grefen- stette, Lasse Espeholt. Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Infor- mation Processing Systems 28: Annual Conference on Newral Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 1693-1701.
Alexander R Fabbri, Wojeiech Kryšciński, Bryan McCann, Caiming Xiong, Richard Socher. and Dragomir Radev. 2020. Summeval: Re- arXiv
evaluating summarization evaluation. preprint arXiv:2007.12622
Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajic Utama, Ido Dagan, and Tryna Gurevych. 2019a. Ranking generated summaries by correctness: An in- teresting but challenging application for natural lan- guage inference. In Proceedings of the 57th Confer- ence of the Association for Computational Linguis- tics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume Long Papers, pages 2210-2220. As- sociation for Computational Linguistics.
Or Honovich, Roce Aharoni, Jonathan Herzig, Ha- gai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE: re-evaluating fac- tual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, NAACI 2022, Seattle, WA, United States, July 10-15, 2022, pages 3905- 3920. Association for Computational Linguisties.
Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajic Utama, Ido Dagan, and Iryna Gurevych. 2019b. Ranking generated summaries by correctness: An in- teresting but challenging application for natural lan- guage inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, pages 22 2214-2220, Florence, Italy, Associa- tion for Computational Linguistics.
Or Honovich, Leshem Choshen, Roce Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend. 2021. 2S: Evaluating factual consistency in knowledge- grounded dialogues via question generation and question answering. In Proceedings of the 2021 Conference Empirical Methods in Natural Lan- guage Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 7856-7870 Association for Computa- tional Linguisties.
Ben Goodrich, Vinay Rao, Mohammad Saleh, and Pe- ter J. Liu. 2019. Assessing the factual accuracy of generated text. CoRR, abs/ Abs//190..133 13322.
Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022. News summarization and evaluation the era of GPT-3. CoRR, abs/2209. 12356.
Cheng-Yu Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ran- jay Krishna, Chen-Yu Yu Lee, and Tomas Pfister. 2023.

---

Distilling step-by-step! outperforming larger lan- guage models with less training data and smaller model sizes. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8003- 8017, Toronto, Canada. Association for Computa- tional Linguisties.
2020. BART: denoising sequence-to-son to-seuunce pre- training for natural language generation, translation. and comprehension. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7871 Association for Computational Linguisties.
Jie Huang and Kevin Chen- Chen-Chuan Chang. 2022. To- wards reasoning in large language models: survey. CoRR, abs/2212. 10403.
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2022. WANLI: worker and AI collabora- tion for natural language inference dataset creation. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages Association for Computational Linguistics.
Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. Scitail: A textual entailment dataset from science question answering. In Proceedingy of the Thirty- Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Arti- Sicial Intelligence (1AAI-18), and the 8th AAAI Sym- posium on Educational Advances Artificial Intel- ligence (EAAI-18) New Orleans, Louisiana, USA, February 2-7, 2018, pages 5189-51111 5197. AAAI Press.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang. Ruochen Xu, and Chenguang Zhu. 2023. G-eval: NLG evaluation using GPT-4 with better human alignment. CoRR, 0as/2303.166344
Tom Koemi and Christian Federmann. 2023. Large lan- guage models are mfte-of-theearr evaluators of trans- lation quality. CoRR, ftss/2322.142200
Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. 2023. Chatgpt factual inconsistency eval- uator for abstractive text summarization. CoRR, abs/2303.15621.
Huan Yee Koh, Jiaxin Ju, Ming Liu, and Shirui Pan. 2023. An empirical survey on long document sum- marization: Datasets, models, and metries. ACM Comput. Surk, 55(8)
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe. Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang. Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. CoRR, abs/2303. 17651.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Twasawa. 2022. Large lan- guage models are pero-shot reasoners. In NeurIPS.
Wojeiech Kryscinski, Nitish Shirish Keskar, Bryan Mc- Cann, Caiming Xiong. and Richard Socher. 2019. Neural text summarization: A critical evaluation. In Proceedings of the 2019 Conference on Empiri- cal Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 540-551. Association for Computational I Linguistics.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan T. McDonald. 2020. On faithfulness and fac- tuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5-10, 2020. pages 1906-1919 Association for Com- putational Linguistics.
Anshuman Mishra, Dhruvesh Patel, Aparna Vijayaku- mar, Xiang Lorraine Li, Pavan Kapanipathi, and Kar- tik Talamadupula. 2021. Looking beyond sentence- level natural language inference for question answer- ing and text summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HIL 2021, Online, June 6-11, 2021, pages 1322-1336, Associ- ation for Computational Linguistics.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong. and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing, EMNIP 2020, Online, November 16-20, 2020, pages 9332- 9346. Association for Computational Linguistics.
Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. Summac: Re-visiting nli- based models for inconsistency detection in summa- rization. Trans. Assoc. Comput. Linguistics, 10:163- 177.
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't 't give me the details, just the summary! topic-aware convolutional neural networks for treme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, Brussels, Belgium, October 31 November 4, 2018, pages 1797-1807. Association for Computational Linguistics.
Faisal Ladhak, Esin Durmus, Claire Cardie, and Kath- leen R. McKeown. 2020. Wikilingua: A new bench- mark dataset for cross-lingual abstractive summa- rization. CoRR, abs/2010.03093.
Mike Lewis, Yinhan Liu, Naman Goyal, Mar- jan Ghazvininejad, AbdeIrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer.
Yixin Nie, Adina Williams, Emily Dinan, Mohit
Bansal, Jason Weston, and Douwe Kiela. 2020. Ad- versarial NLI: A new benchmark for natural lan-

---

guage understanding. Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5-10, 2020, pages 4885-4901. Association for Computational Linguistics.
Prasetya Utama, Joshua Bambrick, Nafise Sadat Moosavi, and Iryna Gurevych. 2022. Falsesum: Generating document-level NLI examples for recog- nizing factual inconsistency in summarization. In Proceedings of the 2022 Conference of the North American Chupter of the Association for Computa- rional Linguisties: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10- 15, 2022, pages tational Linguistics.
OpenAI. 2022.
Chatgpt,
2ages2763333 2763-27767 Association for Compu-
Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. 2021. Understanding factuality abstrac- tive summarization with FRANK: A benchmark for factuality metrics. In Proceedings of the 2021 Con- ference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies, NAACI.HII. 2021, Online, June 6-11, 2021. pages 4812-4829, Association for Computational Linguistics.
Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions evaluate the fac- tual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, ACI. 2020, Online, July 5-10, 2020, pages 5008-5020. Association for Computa- tional Linguistics.
Jiaan Wang. Yunlong Liang, Fandong Meng. Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou, 2023. Is chatgpt good NLG evaluator? A prelimi- nary study. CoRR, abs/2303.04044.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yangi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn Res., 21:140: 1111140:144000 1-140:67.
Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021. Want to reduce label- ing cost? GPT-3 can help. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2021, Virtual Event / Punla Cana, Dominican Republic, 16-20 November, 2021, pages 4195-4205. Associa- tion for Computational Linguistics.
Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Michael Collins, Dipanjan Das. Slav Petrov, Gau- rav Singh Tomar, Iulia Turc, and David Reitter. 2021. Mcasuring attribution in natural language generation models. CoRR, abs/2112.12870
Tal Schuster, Sihao Chen. Senaka Buthpitiya, Alex Fabrikant, and Donald Metzler. 2022. Stretching sentence-pair NLI models to reason over long doc- uments and clusters. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates. December 7-11, 2022, pages 394 412. Association for Computational Lin- guistics.
Yixuan Weng. Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Kang Liu, and Jun Zhao. 2023. Large language mod- els are better reasoners with self veriffctiii....... CORR, abs/2212.00561.
Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang. Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language mod- els to commonsense models. In Proceedings of the 2022 Conference of the North American Chap- ler of the Association for Computational Linguis- tics: Human Language Technologies. pages 4602- 4625, Seattle. United States. Association for Com- putational Linguistics.
Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, and Patrick Gallinari. 2021. Questeval: Summariza- tion asks for fact-based evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Vir- tual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 6594-6604. Association for Computational Linguistics.
Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Sujian Li, and Yajuan Lv. 2023. Wecheck: Strong factual consistency checker via weakly supervised learning. Proceedings of the 61th Annual Meeting of the Asso- ciation for Computational Linguistics, ACL 2023.
Amir Soleimani, Christof Monz, and Marcel Worring. 2023. NonFactS: NonFactual summary generation for factuality evaluation in document summarization. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4495-6419, Toronto, Canada. Association for Computational Linguistics.
Linting Xue, Noah Constant, Adam Roberts, Mi- hir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mt5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 483 498. Association for Computational Linguis- tics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Manie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.132 13971.

---

Wenpeng Yin, Dragomir R. Radev, and Caiming Xiong. 2021. Docnli: A large-scale dataset for document-level natural language inference. In Find- ings of the Association for Computational Linguis- lics: ACL/UCNLP 2021, Online Event, August 1- 6, 2021, volume ACL/IICNLP 2021 of Findlings of ACL, pages 4913-4922, Association for Computa- tional Linguistics.

---

A Appendix
(CNN) Desperate migrants from Africa and the Middle Last keep heading to Europe, with 978 res- cued Friday in the Mediervanean Sea, the Italian Coast Guard said Saturday via Twitter. The migrants were picked up 30 miles off the coast of Lihya, said European Parliament member Matteo Salvini, the leader of Ital"" 's far-right Northern Leugue. In the first three months of 2015, htly registered more than 10,000 migrants arriv- ing. the International Organization, for Migrution said, and about 2,000 were rescued at sea during the first weekend of April in the Channel of Sicily. Most mi- grants recorded this year come from countries in West Africa as well as Somalia and Syria, the IMO said. They use Libya as country of transit. At least 480 migrants have died while crossing the Mediterranean since the beginning of the yeur, often becouse of bod weather and overcrowded vessels used by snugglers, the IMO said. Sometimes the captains and crews abandon the ships, leaving passengers fend for themselves. At this time last year, there were fewer than 50 deaths reported, the IMO said, Most of the migrants are asylum seekers, vic- tims of trafficking or violence, wnaccompanied children and pregnant women.
Premise:
A.1 FLAN-PaLM Prompt Design
To apply FLAN-PaLM for factual consistency eval- uation, we experimented with zero-shot, few-shot and chain-of-thought prompting strategies, and var- ious formats for each strategy. We chose the best performing strategy and format, based on the accu- racy on a development set. 23 Table presents the accuracy of each prompt type on the development set. We observed only minor performance differ- ences, and thus we opted for the simplest solution that is the zero-shot prompt. While we cannot know the exact reasons for why few-shot and chain-of- thought did not improve performance, we can offer potential explanations.
(1) Since the model was fine-tuned on NLI datasets, it is able effectively generalize to factual consistency evaluation, mak- ing further demonstrations via few-shot prompting unnecessary in this case. (2) The performance with the zero-shot prompt is already notably high (89%, $4.4) and thus our particular LLM is less likely to benefit from chain-of thain-of-thought prompting. (3) It could be the case that only a few reasoning steps are needed to evaluate consistency in our particular setup and thus chain-of-thought not necessarily better in this case.
Hypothesis: the migrants were picked up 30 miles off the coast of libya.
Can the kypothesis be inferred from the premise? An- swer using "Yes" or "No" only. Answer: Yes
Premise: (CNN) A nucleur submarine being repaired at Russian shipyard has caught on fire, according a law enforcement source speaking to Russia's state-rum news agency ITAR-Tass. "The submarine is in dry dock," Tass reports, citing the source, and there is communition on board. "The rubber insulation between the submarine's 's light and pressure hull is on fire"" " Tass reported. Russia's RIA Novosti news agency says insu- lation caught on fire welding work was being done on the submarine. Tass reported that the fire began sub in the Zvyozdochka shipyard in northwestern Russia. Zvyozdochka spokesman Yevgeny Gladyshev told the news agency that the sub had been undergoing repairs since November 2013. "Nuclear fuel from the sub's 's actor has been unloaded, " reportedly said, "There are no armaments chemically active, dangerous sub- stances, fissionable materials on it, Gladyshev said to Tass. "The enterprise's personnel left the premises when the submurine caught fire, no one has been injured. The fire presents no threat to people and the shipyard." " Hypothesis: "the rubber insudation between the subma- rine's light and pressure hull is on fire," russia 's ria novosti news agency says.
Below, we describe our top- nop-performing zero- shot, few-shot and chain-of-thought prompts.
Zero-shot Prompt Since FLAN-PaLM was in- struction fine-tuned on NLI, designed our prompt resemble an NLI prompt (e.g. using "premise" and "hypothesis" instead of "document" and "summary"). Our final prompt is as follows:
Premise: [document] Hypothesis: (summary) Can the hypothesis be inferred from the premise? Answer us- ing "Yes" "Ye""or or "No" only.
Can the hypothesis be inferred from the premise? An- swer using "Yes" or "No" only. Answer: No
Few-shot Prompt We use two sho- examples, one "consistent" and one "inconsistent". We randomly sample these examples from the development set examples shorter than 200 words, 23 We limit ourselves to two short examples since summarization examples can include long documents, and thus few-shot may lead to too long context length. Our final prompt is as follows:
Premise: [document]
Hypothesis: (summary)
Can the hypothesis be inferred from the premise? An- swer using "Yes" or "No" only.
Answer:
Chain-of-thought Prompt Following Kojima et al. (2022) we append "Let's think step by step" to the prompt to facilitate a step-by-step rea- soning before predicting the label. Our final prompt is as follows:
For development set we use the FactCC dataset (Kryscin- ski et al., 2020) with 1,431 examples containing summaries of documents from CNN/DailyMail,, manually annotated for factual correctness. Following (Utama et al., 2022), we merge the dev and test sets.

---

Premise: [document]
Prompt type
Dev accuracy
Hypothesis: (summaryy
Q: Can the hypothesis be inferred from the premise? Answer using "Yes" "ss" or "No" only.
zero-shot
93.6
few-shot
93.2
A: Let's think step by step
thain-of-thought
93.8
This prompt successfully unlocked a step by step reasoning. Below is FLAN-PaLM''s response for- mat, where (answer) is either "yes" or "no".
Table FLAN--PL.M accuracy on the development
set using different prompting strategies.
freasoning steps]. So, the answer is [answer].
Language
ISO 639-1
consistent
inconsistent
English
12,500
12,500
Example input:
Spanish
cs
12,500
12,500
French
fr
12,500
12,500
(CNN) Georgia Southern University was in mourning Thursday after five nursing students were killed the day before in multiveiicle wreck near Sa- vannah. Caitlyn Baggett, Morgan Bass, Emity Clark, Abbie Deloach and Catherine (McKay) Pitmaan all juniors were killed in the Wednesday morning crash they traveling hospital in Savannah, cording the school website. Fellow mursing students Britiney McDaniel and Megan Richards
Premise:
German
de
12,500
12,500
total
50,000
50,000
Table 9: Our multilingual dataset statistics.
injured
another person, who was identified by the their
Premise: [document] Hypothesis: [summary]
Georgia State Patrol. The young way finish their first set of clinical rotations Hypothesis: georgia southern university was ing after five nursing students died.
The model trained to predict "1" if the sum- mary is factually consistent and "ø" otherwise. We use a learning rate of 10-4 and a batch size of 32. During training, we use maximum input length of 512 tokens and truncate the premise if needed." During inference
Can the hypothesis Answer using "Yes" "No" only.
inferred from the premise?
A: Let's think step by step
maximum input length of 2048 tokens. We train for a maximum of 20 epochs, evaluate checkpoint every Ik steps and choose the checkpoint with the best ROC-AUC on a development set. In our study we make sure to use the same training regime for all baselines.
The output for this example is:
Georgia Southern University mourning Thursday after five nursing students were killed day before a multivehicle wreck near Savannah. So, the answer yes.
The ANLI-only results in Table are from experiments, while in Table use the results reported in previous work.
A.2 Inference with FLAN-PaLM
We used the zero-shot prompt (see $A.1). The vast majority of FLAN-PaLM's responses were either "Yes" or "No", and a tiny fraction of the responses were "It's impossible to say".
For the summarization models we fine tune the corresponding T5 models on the XSum training set (Narayan et al., 2018) in similar fashion and use the ROUGE score on the XSum development set as a stopping criteria.
During the labeling phase, we let FLAN-PaLM generate the output (predict mode), and label as "consistent" if the generated output is "Yes" and "inconsistent" in case the output is "No". We discard the "It's impossible to say" ex- amples. order to measure ROC-AUC in a binary classification setting, we compute the model's prob- ability of generating "Yes" (score mode) and use it as the example-level factual consistency score.
A.4 Additional Details About Our Dataset
As mentioned in $3.1, we create the dataset based on documents from CNN/DailyMail (Hermann et al., 2015). We do not use the gold summaries, and we only use examples from the training set.
In our experiments with the full dataset ($4.1), balance the labels by randomly sampling 475,563 positive examples (see Table 1).
A.3 Fine tuning T5
We fine tune our T5 models for factual consistency evaluation using the following input format:
In carly experiments we saw that training with longer maximum input length resulted with comparable performance.

---

FactEdit
Self-verification prompt
DocNLI
FactCC
Falsesum
Density
Ours
Yes
0.
Zero-shot prompt
No
011
0.0
5.0
10.0
17.5
No Yes
Extractiveness
Figure 5: Visualization of the density of the combined abstractivness score. The plot is actually measuring the extractiveness degree, so lower x-values mean higher abstractiveness.
Figure 4: Self- verifiiatitn prompting. If the LLM clas- sified the summary as consistent, we prompt it again and ask it for its certainty. If the answer is "Yes" (con- sistent with the original reasoning), we keep the exam- ple, otherwise we filter out.
When we qualitatively analyzed the filtered exam- ples, it seems that the majority of the filtered exam- ples indeed had wrong label, and that applying this filtering mechanism increases the labeling ac- curacy by approximately 5%.
A.5 Data Filtering with Self-verification
As mentioned in $3 also explored data filter- ing based on prompting FLAN-PaLM for self- verification. Our proccess is based on steps. (1)
While this filtering mechanism results in higher labeling accuracy, we did not observe a perfor- mance gain when filtering the training data in this way. For TruceTeacher + ANLI with T5-1 1B (on sample of 100k examples) of 86 ROC-AUC TRUE using the filtered data, slightly below the 86.4 using the unfiltered data (Table 3). As mentioned in Footnote we attribute this to the fact that the labeling accuracy is high to begin with (89,, section 4.4) and that the model is likely robust to some amount of labeling noise. Following this, for simplicity, our official method does not use filtering.
Detect potential examples in our dataset that are likely to be labeled incorrectly by the LLM. (2) Prompt the LLM to self-verify its earlier prediction and filter out examples that the model is uncertain of. This leads to a smaller dataset with improved labeling accuracy. (3) Train the factual consistency evaluation model on the filtered dataset. This ap- proach is based on 2 observations:
got average
1. In early experiments, we saw that our LLM has extremely high precision for the inconsistent class. This can also seen in our human eval- uation (Table 4). This means that almost all the errors occur when the LLM predicts that the summary is consistent. Following this, we only consider filtering examples classified as consistent by the LLM.
A.6 Abstractiveness Analysis: Additional Details
As our backbone metrics we use the Extractive Fragment Coverage and Densify measures defined by Grusky et al. (2018). Coverage measures the percentage of words in the summary that are part of an extractive fragment with the article, quanti- fying the extent to which a summary is derivative of a text. Density measures the average length of the extractive fragment to which each word in the summary belongs, quantifying how well the word sequence of summary can be described as a series of extractions. Our Combined score is obtained by multiplyng the Coverage and the Density scores, similar Utama et al. (2022). To further illustrated the differences in the abstractiveness of different methods, we include visualization of the density of the combined abstractivness score in Figure 5.
2. Inspired by the work of Weng et al. (2023) and Madaan et al. (2023), we use a self verification prompt. If the L.L classified the summary as consistent, we prompt it again and ask it for its certainty. If the answer is "Yes" (i.e. is consis- tent with the original reasoning path), keep the example, otherwise we filter it out. This proccess illustrated Figure
The self-veriffcation prompt as follows:
Premise: [document] Hypothesis: |summaryy] Are you sure that the summary can be inferred from the docu- ment? Answer using "Yes" " "No" only.
This approach filtered-out 15% of the dataset.

---

ANLI+XLLI 100K
+00K en/esideffe
article?". In our work we use the average attribu-
amharic
63.1
67.2
68.6
tion score (between 0 to and treat summaries
arabic
87.8
89.0
87.7
factually consistent if the score is larger than 0.5.
azcrbaijani
59.6
68.6
65.5
benguli
90.4
94.3
98.5
We focus on the test split of XLSum containing
burmese
59.0
64.5
57.9
3150 examples in 45 languages (i.e., 70 examples
chinesesimp.
87.6
chinese trad.
82.5
82.6
83.2
in each language). In $5 we refer to Table 7 with the results overview, and we provide the full results for all languages in Table 10.
english
80.2
74.7
80.0
french
91.9
94.1
97.1
gujarati
50.8
52.0
51.5
hausa
69.5
67.7
73.7
hindi
72.2
79.9
86.5
A.8
Human Evaluation
igho
62.2
62.8
75.7
indonesian
77.6
84.1
We instructed the participants review the docu- ment and its corresponding summary, and evalu- the summary based on the attribution definition provided by Rashkin et al. (2021), using binary judgements. To avoid common confusion be- tween factual inconsistency and contradiction, we also provided the following instruction:
jupanese
97.7
kirundi
83.5
koreun
87.3
82.3
89.9
kyrgyz
70.1
77.4
79.0
marathi
75.2
78.7
73.6
nopali
55.2
59.1
57.2
orama
81.2
83.7
83.3
pashto
56.4
68.2
67.7
persian
43.5
pidgin
70.0
81.4
portuguese
79.6
79.5
79.0
this task you evaluate factual consistency system-generated summary. The system 's goal is to summarize the original source document, while remain- ing truthful to Your goal is to evaluate whether the system-genee generateedded summary is consistent w.r.. the source document. Summary will be considered consistent if all the information in the summary can be verified from the source document (i.e.. for the summary to inconsistent, the document does not necessarily need to contradict it, it can also fail to support some facts).
punjabi
77.7
81.5
78.2
russian
88.8
85.1
81.2
scottish guelic
59.0
58.8
63.1
serbian cyrillic
84.2
79.3
85.5
serbian latin
39.7
42.2
sinhala
72.9
74.9
76.1
somali
spanish
80.7
85.9
89.1
swahili
88.1
89.2
92.2
tamil
63.9
69.8
66.0
telugu
55.9
62.3
60.4
thai
78.8
83.8
86.8
an early experiment, we found that using crowd workers without domain expertise and sub- stantial time investments resulted in extremely low- quality ratings. Following this, all our raters were NLP researchers, each with at least one year of spe- cific experience in the task of factual consistency evaluation, with significant time allocation and no more than 10 examples per rater. These steps ensured high quality ratings.
tigrinya
79.9
82.9
86.1
lurkish
87.0
86.6
86.6
55.5
uzdu
69.0
63.8
uzbck
54.6
59.3
vietnamese
89.8
84.4
88.1
welsh
83.0
83.4
83.9
yoruba
69.0
69.0
77.2
wins
15
25
# ANLI+XNLI
32
35
Per lang, avg.
73.3
75.7
77.2
Per example avg.
71.6
73.8
75.3
A.9 Adding noise TeacTeacher
Table 10: ROC-AUC results on the mFace test set.
In $4.5 we create SummaryAblation by flipping labels to a random portion of True Tracher'her's 's data, such that the expected labeling accuracy is sim- ilar to Falsesum. Falsesum's labeling method is coupled with the data generation, thus we need approximation for its labeling quality. We estimate Falesum's labeling accuracy as 83.5%, according to Utama al. (2022)'s human evaluation (we aver- age the Intrinsic and Extrinsic results), while ours is 89% ($4.4). So to mimic Falsesum's quality we flipped TrueTeeccher labels in order to add addi- tional 5.5% errors.
A.7 Using the mFace dataset
$5 we report results on the mFFace dataset (Aha- roni et al., 2022). Aharoni et al. performed large scale human evaluation of summaries of documents from the XLSum corpus (Hasan et al., 2021), pro- duced by different summarization models. Each summary was rated for quality, attribution and in- formativeness. We use the attribution scores in our work. The attribution evaluation is based on the attribution definition provided in Rashkin et al. (2021), with the participants asked "Is all the in- formation in the summary fully attributable to the
We found that it is sufficient to use one rater example (unlike in our experiments with the crowd workers).


Reviews:
[{"review_id": "idO3WKraBM", "reviewer": "Reviewer_AJ8x", "summary": "", "questions": "Q1. What kinds of filtering mechanisms on LLMs labeled synthetic data have you tried that mentioned in line 232? What kind of samples have been filtered in which settings? Could you elaborate more on what your observations are?\n\nQ2. In line 214, the authors mentioned they used closely-related NLI tasks for teacher model instruction fine-tuning. Is there a reason why you didn’t use publicly opened NLI datasets in this setting?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback. We are glad that you found our work strongly sound, well written and organized, and that you found our ablation studies insightful. \n\nWe will now provide a detailed response to each of your concerns/questions, and will incorporate your feedback into the camera-ready version. \n\n------------------------------------------------------------------\n\n# TrueTeacher can be considered a distillation recipe, thus the technical contribution is limited\n\nWe acknowledge that TrueTeacher is closely related to knowledge distillation, a point we've noted in the paper (line 123 and line 337), and we recognize the technical simplicity of our method. However, we'd like to emphasize several aspects of our contributions which we find exciting and also distinguish our work from typical “distillation work”. We will revise our paper to emphasize these points.\n-  The focus of our work is clearly not on distillation in NLP, but rather on the task of factual consistency evaluation. Accordingly, the design choices we made are motivated by limitations of recent synthetic data generation approaches for factual consistency evaluation that we improve upon using a fundamentally different approach. Thus, we see our work as relevant for the summarization community. \n- Distillation requires a dataset for labeling, and we put significant efforts in generating one tailored to our use case from scratch. To achieve this, we (1) trained a diverse set of models to generate summaries, hypothesizing that the best proxy for real factual errors can be obtained from the outputs of actual summarization models, and (2) trained these summarization models on an abstractive dataset to produce summaries that more closely resemble those generated by modern summarization models. \n- Other than distillation, we make additional substantial contributions: Our study reveals previously unknown limitations of the existing synthetic data generation methods, and encourages future researchers to perform domain-aware evaluation. Our work provides useful insights about the applicability of LLMs for factual consistency evaluation, and about the feasibility of multilingual data generation (which our work is the first to explore in this context). \n- As mentioned in the paper and the responses, we experimented with several advanced prompting techniques, such self-verification and few-shot / chain-of-thought prompting (line 217 and line 231), yet the simplest solution worked best empirically. We believe that the technical simplicity of TrueTeacher is actually an advantage, as this makes our approach easy to adopt. To ensure that TrueTeacher can be applied easily, we further release our dataset and a strong model trained on it (T5-11B w. ANLI + TrueTeacher full, Table 2). We hope that it will provide a good starting point for future work in this important field. \n\n\n------------------------------------------------------------------\n\n# Q1: What kind of filtering did you try?\nWe will now describe our filtering attempts and will add the relevant details and results to the revised version.\n\nThe main idea in our filtering experiment is:\n1. Detect potential examples in our dataset that are likely to be labeled incorrectly by the LLM.\n2. Prompt the LLM to self-verify its earlier prediction and filter out examples that the model is uncertain of. This leads to a smaller dataset with improved labeling accuracy.\n3. Train the factual consistency evaluation model on the filtered dataset.\n\nOur filtering approach is based on 2 observations:\n1. In early experiments, we noticed that our LLM has extremely high precision for the negative (inconsistent) class. This can be also seen in our human evaluation results in Table 4. This means that almost all the errors occur when the model predicts that the summary is consistent (failing to identify inconsistency). Following this, we only consider filtering examples that the LLM classified as consistent.\n2. Inspired by the work of [Weng et al. (2023)](https://arxiv.org/pdf/2212.09561.pdf) and [Madaan et al. (2023)](https://arxiv.org/pdf/2303.17651.pdf), we use a self verification prompt. If the LLM classified the summary as consistent, we prompt it again and ask it for its certainty. If the answer is “Yes” (i.e. it is *consistent* with the original reasoning path), we keep the example, otherwise we filter it out.\n\nOur self verification prompt (the main prompt is presented in the paper in section A.1, line 973):\n> ***Premise:*** *{document}* ***Hypothesis:*** *{summary}* ***Are you sure that the hypothesis can be inferred from the premise? Answer using \"Yes\" or \"No\" only.***\n\nThis approach filtered-out 15% of the dataset. When we qualitatively analyzed the filtered examples, it seems that the majority of the filtered examples indeed had a wrong label, and that applying this filtering mechanism increases the labeling accuracy by approximately 5%. \n\nWhile this filtering mechanism results in higher labeling accuracy, we did not observe a performance gain when filtering the training data in this way. For `TrueTeacher+ANLI` with `T5-11B` (on a sample of 100k examples) we got an average of 86 ROC-AUC on TRUE using the filtered data, slightly below the 86.4 using the unfiltered data (Table 3). As mentioned in footnote 7,we attribute this to the fact that the labeling accuracy is high to begin with (89%, line 471) and that the model is likely robust to some amount of labeling noise. Following this, for simplicity, we decided to remove this result from the paper. We will include it in the camera-ready version.\n\n------------------------------------------------------------------\n\n# Q2: What kind of NLI tasks was the LLM trained on?\nTo ensure anonymity, we defer the specification of the exact LLM we used to the camera-ready version. However, we would like to clarify that the LLM’s instruction fine-tuning dataset is fully disclosed and based only on **publicly available datasets** (including NLI datasets). The camera-ready version will provide the exact details for these datasets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well-written and well-organized. The ablation studies cover various research questions. The authors also provide human evaluation of model-generated summaries that are classified by LLMs that indeed performs well on a factual inconsistency task.\n", "reasons_to_reject": "The technological contribution of this paper is limited since the proposed method has been explored actively in the distillation setup.", "questions_to_authors": "Q1. What kinds of filtering mechanisms on LLMs labeled synthetic data have you tried that mentioned in line 232? What kind of samples have been filtered in which settings? Could you elaborate more on what your observations are?\n\nQ2. In line 214, the authors mentioned they used closely-related NLI tasks for teacher model instruction fine-tuning. Is there a reason why you didn’t use publicly opened NLI datasets in this setting?\n", "paper_topic_and_main_contributions": "Factual consistency evaluation is often conducted using NLI datasets which shows limitations in evaluating summaries. Previous works have improved such models using synthetic data which is different from real-world data. LLMs could be an alternative solution to evaluating factual information while it is computationally expensive. To overcome this, the author proposed a method called TrueTeacher which generates synthetic data using LLMs. The author first trains summarization models and feeds documents to these models. Then the author makes LLMs to predict factual consistency labels and used further for factual consistency in summarization task which achieves the competitive performance compared to strong baselines. \n\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "Foot note 2 should be in the page 3 for better readability."}, "extra_text": {}}, {"review_id": "ZGNc5NSYsW", "reviewer": "Reviewer_LjUF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback. We are happy that you found our work strongly sound and our results insightful.\n\nWe will now provide a detailed response to each of your concerns/questions and will do our best to incorporate your feedback into the camera-ready version.\n\n------------------------------------------------------------------\n\n# The prompt mechanism is relatively simple\nWe experimented with several more complex prompting techniques such as **few-shot** and **chain-of-thought** prompting, as well as selecting examples with high confidence using a **self-verification** prompt (i.e. data filtering). For more details we refer you to:\n- Our response to **reviewer TpWY**, where we give extensive details about our **few-shot** and **chain-of-thought** efforts, since they specifically requested these details.\n- Our response to **reviewer AJ8x**, where we give extensive details about our data filtering effort (using a **self-verification** prompt), since they specifically requested these details.\n\nEventually, the technically-simplest solution worked best, and thus we included it in our main results. While we believe that the simplicity of TrueTeacher will make it easy to adopt, we will also include the more complex approaches in the camera-ready version.\n\n------------------------------------------------------------------\n\n# NLI-style prompt can cause inheritance of limitations of NLI models\n\nThe NLI-style prompt performed well since the LLM was exposed to NLI tasks during instruction fine-tuning (lines 213-214). We believe that prompting an LLM with an NLI-style prompt is not equivalent to using an NLI model, as LLMs are pre-trained on a plethora of text and multiple NLP tasks and thus generalize well (e.g. [Wei et al. (2022)](https://arxiv.org/pdf/2109.01652.pdf)). Therefore, an LLM with an NLI-style prompt is more likely to generalize to the summarization domain (e.g. to long premises) compared to a model that was fine-tuned only on an NLI dataset. We also see that our LLM outperforms the SOTA NLI-specific model (84.9 vs. 82.7 ROC-AUC, Table 2). Furthermore,  our final model also benefits from training on a large amount of documents and model-generated summaries, which leads to superior performance, even compared to our LLM (87.8 vs. 84.9 ROC-AUC, Table 2). This is in line with previous work that demonstrated the utility of target-domain unlabeled data (e.g. [Gururangan et al. (2020)](https://aclanthology.org/2020.acl-main.740.pdf)). We will address this in the camera-ready version to provide more clarity.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The framework incorporates the current progress of pretrained language models/large language models in factual consistency of summarizations. \n- The experimental results are rich to demonstrate the performance of TrueTeacher under different scenarios, the finding of generalization ability of TrueTeacher could especially benefits the future research.", "reasons_to_reject": "- The design of the labeling prompt mechanism is relatively simple, and still in a NLI style, would is likely to inherit the previous limitations of using NLI in factual consistency evaluation. The work does not fully address this problem or provide useful breakdown analysis of how it could be resolved. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This work is about a new framework TrueTeacher for summarization consistency by using pretrained language models to generate candidate summaries from documents without using the golden summaries and using another pretrained language model to label the consistency. TrueTeacher demonstrate competitive performance compared to previous single summarization models and the labeling mechanism is better combined with the summarization generation component in TrueTeacher compared to other summarization generation alternatively. Lastly, TrueTeacher has better generalizability in the multilingual settings.", "excitement": 3, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": " "}, "extra_text": {}}, {"review_id": "T6y6SbzTFr", "reviewer": "Reviewer_TpWY", "summary": "", "questions": "Line 335: I find the claim on TrueTeacher+ANLI outperforming 540B LLM less convincing. \n* 540B LLM is fully zero-shot and to my understanding does not involve any optimization at all (e.g., post-hoc calibration, prompt selection). On the other hand, TrueTeacher+ANLI has access to a development set (Line 1004). It would be fairer if the 540B LLM approach also utilize the development set.\n* It is mentioned in Line 216 that few-shot / chain-of-thought prompting is not effective in early phase of the study, it would be useful to describe these efforts in more details.\n\nSec 4.3: From the qualitative analysis I think there are two main difference between TrueTeacher and the other synthetic data generation methods: (1) the distribution of the summary (machine-generated vs. human-written + perturbation); (2) the correctness of the generated labels (e.g., falsesum failing in introducing factual error). Which aspect is more important? Is there a way to separate these two aspects in a controlled way? Would love to hear the authors thoughts on this.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback. We are happy that you found our work strongly sound and our results insightful, especially as we believe that our ablation study is a key contribution of this work in addition to the proposed method. We also believe that our dataset will be a useful resource for future research. \n\nWe will now provide a detailed response to each of your concerns and will do our best to carefully revise our work following your feedback. Regarding your second question about the importance of the different aspects (data distribution vs. synthetic label correctness), we performed additional detailed ablation experiments with insights we find interesting.\n\n------------------------------------------------------------------\n\n# TrueTeacher can be considered a distillation recipe, thus the technical contribution is limited\n\nWe acknowledge that TrueTeacher is closely related to knowledge distillation, a point we've noted in the paper (line 123 and line 337), and we recognize the technical simplicity of our method. However, we'd like to emphasize several aspects of our contributions which we find exciting and also distinguish our work from typical “distillation work”. We will revise our paper to emphasize these points.\n-  The focus of our work is clearly not on distillation in NLP, but rather on the task of factual consistency evaluation. Accordingly, the design choices we made are motivated by limitations of recent synthetic data generation approaches for factual consistency evaluation that we improve upon using a fundamentally different approach. Thus, we see our work as relevant for the summarization community. \n- Distillation requires a dataset for labeling, and we put significant efforts in generating one tailored to our use case from scratch. To achieve this, we (1) trained a diverse set of models to generate summaries, hypothesizing that the best proxy for real factual errors can be obtained from the outputs of actual summarization models, and (2) trained these summarization models on an abstractive dataset to produce summaries that more closely resemble those generated by modern summarization models. \n- Other than distillation, we make additional substantial contributions: Our study reveals previously unknown limitations of the existing synthetic data generation methods, and encourages future researchers to perform domain-aware evaluation. Our work provides useful insights about the applicability of LLMs for factual consistency evaluation, and about the feasibility of multilingual data generation (which our work is the first to explore in this context). \n- As mentioned in the paper and the responses, we experimented with several advanced prompting techniques, such self-verification and few-shot / chain-of-thought prompting (line 217 and line 231), yet the simplest solution worked best empirically. We believe that the technical simplicity of TrueTeacher is actually an advantage, as this makes our approach easy to adopt. To ensure that TrueTeacher can be applied easily, we further release our dataset and a strong model trained on it (T5-11B w. ANLI + TrueTeacher full, Table 2). We hope that it will provide a good starting point for future work in this important field. \n\n------------------------------------------------------------------\n\n# The claim that TrueTeacher+ANLI outperforms 540B LLM is less convincing\n\nYou raised 2 concerns in this context:\n1. The 540B LLM is fully zero-shot.\n2. It would be fairer if the 540B LLM approach also utilized the development set.\n\nThank you for pointing these out. Regarding (1), we would like to clarify that our LLM was instruction fine-tuned, including on NLI datasets, and we are using a prompt that resembles the NLI task in those datasets. Thus, even though our inference is in a zero-shot format, our setting is not fully zero-shot. This can also explain why few-shot prompting does not improve performance (see results below), since our LLM is already familiar with the task. For (2) we would like to clarify that the LLM-based approach utilizes the development set, since we used it to choose the best performing prompt. We explored several zero-shot, few-shot and chain-of-thought prompts, and for each prompt type we experimented with different formats. For the few-shot approach, we also used few-shot examples from the development set. These details are indeed missing in the current manuscript, we will add them to the camera-ready version.\n\nIn terms of how can `TrueTeacher+ANLI` outperform the `540B LLM`, we hypothesize that while the LLM is trained on many tasks, the smaller model should only master one specific task (factual consistency evaluation), which is easier (as we noted in line 336). Additionally, the smaller model is trained on target-domain data which can further improve its performance [(Gururangan et al. (2020))](https://aclanthology.org/2020.acl-main.740.pdf). We will try to better clarify this in the camera-ready version.\n\n------------------------------------------------------------------\n\n# Details about our few-shot / chain-of-thought prompting efforts\n\nFollowing your request we provide more details about these efforts, and we will revise the paper to include these details (and results). As mentioned above, we experimented with different formats and chose the best performing prompt on the development set. The details of the zero-shot prompt are provided in the paper (line 970). Below, we describe our top-performing few-shot and chain-of-thought prompts. In addition, we add a table with the accuracy of each prompt type on the development set. The table shows that the differences are minor, so we opted for the simplest solution that is the zero-shot prompt. We attribute this to the fact that our particular LLM was already exposed to NLI datasets, and is able to effectively generalize to factual consistency evaluation. \n\n### Accuracy of Different Prompt Types on the Development Set\n\n| Prompt type       | Dev accuracy |\n|-------------------|--------------|\n| zero-shot         | 93.6         |\n| few-shot          | 93.2         |\n| chain-of-thought  | 93.8         |\n\n### Final few-shot prompt\nWe use 2 few-shot examples, 1 positive (consistent) and 1 negative (inconsistent). We randomly sample these examples from all the examples from the development set which are shorter than 200 words. We limit ourselves to 2 examples and only to short examples since summarization examples can include long documents, and thus few-shot may lead to too long context length. Our final prompt is as follows:\n\n> ***Premise:*** *(CNN) Desperate migrants from Africa and the Middle East keep heading to Europe, with 978 rescued Friday in the Mediterranean Sea, the Italian Coast Guard said Saturday via Twitter. The migrants were picked up 30 miles off the coast of Libya, said European Parliament member Matteo Salvini, the leader of Italy's far-right Northern League. In the first three months of 2015, Italy registered more than 10,000 migrants arriving, the International Organization for Migration said, and about 2,000 were rescued at sea during the first weekend of April in the Channel of Sicily. Most migrants recorded this year come from countries in West Africa as well as Somalia and Syria, the IMO said. They use Libya as a country of transit. At least 480 migrants have died while crossing the Mediterranean since the beginning of the year, often because of bad weather and overcrowded vessels used by smugglers, the IMO said. Sometimes the captains and crews abandon the ships, leaving passengers to fend for themselves. At this time last year, there were fewer than 50 deaths reported, the IMO said. Most of the migrants are asylum seekers, victims of trafficking or violence, unaccompanied children and pregnant women.*\n>\n> ***Hypothesis:*** *the migrants were picked up 30 miles off the coast of libya.*\n>\n> ***Can the hypothesis be inferred from the premise? Answer using \"Yes\" or \"No\" only.***\n> \n> ***Answer:*** *Yes*\n> \n> &nbsp;\n> \n> ***Premise:*** *(CNN) A nuclear submarine being repaired at a Russian shipyard has caught on fire, according to a law enforcement source speaking to Russia's state-run news agency ITAR-Tass. \"The submarine is in a dry dock,\" Tass reports, citing the source, and there is no ammunition on board. \"The rubber insulation between the submarine's light and pressure hull is on fire,\" Tass reported. Russia's RIA Novosti news agency says insulation caught on fire as welding work was being done on the submarine. Tass reported that the fire began on a sub in the Zvyozdochka shipyard in northwestern Russia. Zvyozdochka spokesman Yevgeny Gladyshev told the news agency that the sub had been undergoing repairs since November 2013. \"Nuclear fuel from the sub's reactor has been unloaded,\" he reportedly said. \"There are no armaments or chemically active, dangerous substances, fissionable materials on it,\" Gladyshev said to Tass. \"The enterprise's personnel left the premises when the submarine caught fire, no one has been injured. The fire presents no threat to people and the shipyard.\"*\n>\n> ***Hypothesis:*** *\"the rubber insulation between the submarine's light and pressure hull is on fire,\" russia's ria novosti news agency says.*\n>\n> ***Can the hypothesis be inferred from the premise? Answer using \"Yes\" or \"No\" only.***\n>\n> ***Answer:*** *No*\n> \n> &nbsp;\n> \n> ***Premise:*** *{document}*\n>\n> ***Hypothesis:*** *{summary}* \n> \n> ***Can the hypothesis be inferred from the premise? Answer using \"Yes\" or \"No\" only.***\n> \n> ***Answer:***\n\n### Final chain-of-thought prompt\n\nFollowing [Kojima et., al (2023)](https://arxiv.org/pdf/2205.11916.pdf) we append`Let's think step by step` to the prompt to facilitate a step-by-step reasoning before predicting the label. Our final prompt is:\n\n> ***Premise:*** *{document}* \n> \n> ***Hypothesis:*** *{summary}*\n> \n> ***Q: Can the hypothesis be inferred from the premise? Answer using \"Yes\" or \"No\" only.***\n> \n> ***A: Let's think step by step***\n\n\nThis prompt successfully unlocked step by step reasoning and the response format is of the form `{reasoning steps}. So, the answer is {answer}.`, where {answer} is either yes or no.\n\nExample input:\n\n> ***Premise:*** *(CNN) Georgia Southern University was in mourning Thursday after five nursing students were killed the day before in a multivehicle wreck near Savannah. Caitlyn Baggett, Morgan Bass, Emily Clark, Abbie Deloach and Catherine (McKay) Pittman -- all juniors -- were killed in the  Wednesday morning crash as they were traveling to a hospital in Savannah, according to the school website. Fellow nursing students Brittney McDaniel and Megan Richards were injured as was another person, who was not identified by the Georgia State Patrol. The young women were on their way to finish their first set of clinical rotations. \"Today should have been a day of celebration for this bright group of students,\" at St. Joseph's/Candler hospital said in a Facebook posting. \"It was their last day of clinical rotations ... in their first year of nursing school.\" Clinicals include hands-on instruction at a health care facility. A post commander for the Georgia State Patrol said a tractor-trailer smashed into an eastbound line of cars that had slowed for a prior accident on Interstate 16. \"He came along from behind them and he just did not stop for those cars,\" Sgt. Chris Nease said. There were four passenger vehicles and three tractor-trailers involved in the 5:45 a.m. accident. The women who were killed were in two cars, a Toyota Corolla and a Ford Escape. One of their vehicles caught on fire, Nease said, but it will take an investigation to determine whether the women died on impact. CNN Savannah affiliate WTOC reported one witness tried to help. \"Right about the time I got here, the car was just about catching on fire,\" Cayne Monroe told the station. \"The car just burned up really quickly. And I run up there, but there was nothing anyone could do. I've never witnessed something like that in my life. It was pretty tragic.\" The state patrol said the truck driver is from Louisiana. The 55-year-old man had not been charged as of Thursday evening, Nease told CNN. \"Every one of our students contributes in no small measure to the Eagle Nation,\" university President Brooks A. Keel said in a statement. \"The loss of any student, especially in a tragic way, is particularly painful. Losing five students is almost incomprehensible.\" Georgia Southern flew flags at half-staff and counseling was offered to students. A campuswide vigil was held Thursday night. On the university's Twitter page, a tear was added to the profile logo of the eagle mascot. The school has a student body of about 20,000 and is in Statesboro, about 60 miles from Savannah. \"You could tell that they really loved what they did,\" Sherry Danello, vice president of patient care services and chief nursing officer at St. Joseph's/Candler, said on the hospital's Facebook posting. \"They didn't just go through the task, they really connected to the patients.\" Luke Bryan, a country music star and school alumnus, tweeted his condolences: \"Praying for everyone at Georgia Southern and the families who lost loved ones.\" CNN's Matthew Stucker contributed to this report.*\n>\n> ***Hypothesis:*** *georgia southern university was in mourning after five nursing students died.*\n> \n> ***Q: Can the hypothesis be inferred from the premise? Answer using \"Yes\" or \"No\" only.***\n> \n> ***A: Let's think step by step***\n\n\nThe output for this example is:\n\n> *Georgia Southern University was in mourning Thursday after five nursing students were killed the day before in a multivehicle wreck near Savannah. So, the answer is yes.*\n\n\n------------------------------------------------------------------\n\n#  The contribution of the data distribution vs. the correctness of the labels\n\nThank you for this question. We believe the analysis below, which we conducted following your question, provides additional insights that will greatly improve our work. We performed an ablation study where we control the two aspects, i.e. the distribution of the summaries vs the quality of the generated labels. A TL;DR is that we show that they are both equally important, and have a complimentary effect when combined together. We describe our ablation study in detail below, which we will add to the paper. We will also revise the introduction to include both factors as motivation for TrueTeacher, stressing that perturbations of gold-summaries not only lead to unnatural summaries distribution, but they can also fail to introduce factuality errors. \n\n### Our additional ablation study\n\nTo control for the summaries distribution and the labeling quality, we created 2 ablated variants, using Falsesum as a recent relevant baseline method for synthetic data generation. Results on TRUE are presented in the table below. The first row (**variant #1**) is the `Falsesum + ANLI` variant from the paper (see Table 3), where the summaries distribution is **human-written + perturbation** and the labeling quality is of the **Falsesum** method. The last row (**variant #4**) is the `TrueTeacher + ANLI` variant from the paper (see Table 3), where the summaries distribution is **model-generated** and the labeling quality is of our **LLM**. Our ablated variants are in the second and the third rows (**variants #2 and #3**). \n\n| Variant | Summary Distribution                                 | Labeling Quality                      | T5-11B (ROC AUC) | T5-Base (ROC AUC)|\n|-----------|---------------------------------------------------------|-----------------------------------------|-------------------------|---------------------------|\n| 1          | Human-written + perturbation (Falsesum)  | Falsesum                                | 80.8                      | 73.5                         | \n| 2          | Human-written + perturbation (Falsesum)  | LLM                                         | 85.3 (**+5.6%**)  | 78.9  (**+7.3%**)     |\n| 3          | Model-generated (TrueTeacher)                 | Falsesum (proxy)                    | 85.5  (**+5.8%**) | 79.1  (**+7.6%**)    |\n| 4          | Model-generated (TrueTeacher)                 | LLM                                         | 86.4   (**+6.9%**) | 81.9   (**+11.4%**) | \n\n**Variant #2** was created by taking the document-summary pairs from Falsesum’s data and labeling them with our 540B LLM teacher. This variant has the same summary distribution as **variant #1** (i.e. **human-written + perturbation**) and the same labeling quality as **variant #4** (both are labeled using the LLM). Comparing **variant #2** to **variant #1** allows us to examine the effect of using our LLM for labeling (instead of using the original Falsesum labels), while controlling the summaries distribution. **Variant #2** improves **variant #1** by **5.6%**, while **variant #4** improves by **6.9%** \\[1\\]. This demonstrates that performance gains can be obtained using summaries generated with existing synthetic data generation methods combined with second-stage improved labeling quality. However, our data generation method is substantially simpler and also results in better performance (since **variant #4** outperforms **variant #3**).\n\nFinally, we would like to estimate the effect of changing the summaries distribution from **human-written + perturbation** to **model-generated**, while controlling the labeling quality. **Variant #3** is created by flipping labels on a random portion of TrueTeacher’s data, such that the *expected* labeling accuracy is the same as Falsesum’s \\[2\\]. Comparing **variant #3** to **variant #1** allows us to examine the effect of using **model-generated** summaries (instead of **human-written + perturbation**), while controlling the labeling quality. **Variant #3** improves by 5.8%, a similar improvement compared to **variant #2** (5.6%). This demonstrates that the data distribution and the correctness of the labels have a roughly similar effect on the performance, but they also have a complimentary effect and the best performance of 86.4 can be obtained only when they are combined together.   \n\n\\[1\\]: We focus the analysis on T5-11B, yet we observe similar trends with T5-Base.\n\n\\[2\\]: Falsesum’s labeling method is coupled with the data generation, thus we need an approximation for its labeling quality. Falesum’s estimated labeling accuracy is 83.5% (according to [Utama et al. (2022)](https://aclanthology.org/2022.naacl-main.199.pdf)’s human evaluation), while ours is 89% (line 471). So to mimic Falsesum’s quality we flipped TrueTeacher’s labels in order to add additional 5.5% errors.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* TrueTeacher is a simple, effective and strong method for improving factual consistency evaluation in summarization.\n* Detailed comparison and ablation study demonstrating that TrueTeacher outperforms other generation methods, improves out-of-domain generalization, and remains effective for multilingual settings.\n* The 1.4M generated data (to be released with the paper) may be useful to the research community.", "reasons_to_reject": "* Limited technical contribution: The proposed method can be considered as a distillation recipe and similar approaches have been studied for other NLP problems (e.g., commonsense knowledge and reasoning). To this end the technical contribution of this paper is limited.", "questions_to_authors": "Line 335: I find the claim on TrueTeacher+ANLI outperforming 540B LLM less convincing. \n* 540B LLM is fully zero-shot and to my understanding does not involve any optimization at all (e.g., post-hoc calibration, prompt selection). On the other hand, TrueTeacher+ANLI has access to a development set (Line 1004). It would be fairer if the 540B LLM approach also utilize the development set.\n* It is mentioned in Line 216 that few-shot / chain-of-thought prompting is not effective in early phase of the study, it would be useful to describe these efforts in more details.\n\nSec 4.3: From the qualitative analysis I think there are two main difference between TrueTeacher and the other synthetic data generation methods: (1) the distribution of the summary (machine-generated vs. human-written + perturbation); (2) the correctness of the generated labels (e.g., falsesum failing in introducing factual error). Which aspect is more important? Is there a way to separate these two aspects in a controlled way? Would love to hear the authors thoughts on this.", "paper_topic_and_main_contributions": "The paper introduces TrueTeacher, a method to generate large-scale training data for improving factual consistency evaluation in summarization. \n\n__Method:__ TrueTeacher first use different summarization models to produce large-scale machine-generated summaries, then use a 540B large language model to annotate the factual consistency of the machine-generated summaries. TrueTeacher can be considered as a distillation method (540B LLM -> 11B T5 model).\n\n__Empirical Findings:__\n* Models trained with data generated by TrueTeacher (combined with ANLI data) achieves state-of-the-art results on the TRUE benchmark.\n* With a controlled analysis in 4.3 on different synthetic data generation method, the authors demonstrates the effectiveness of TrueTeacher compared to other methods such as FactEdit, DocNLI, FactCC, FalseSum; moreover, TrueTeacher outperforms these methods on out-of-domain evaluation. \n* An additional set of experiment is done in a multilingual setting, showing that TrueTeacher is also applicable and brings improvements in such settings.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "* It would be helpful to provide more context in the abstract; currently it covers many aspects and is a bit confusing to a first-time reader.\n* Line 569: specifically"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces TrueTeacher, a method to enhance factual consistency evaluation in summarization by generating extensive training data. TrueTeacher utilizes various summarization models to create machine-generated summaries on a large scale and employs a 540B large language model to assess the factual consistency of these summaries. This approach can be seen as a distillation method, with the 540B LLM as the teacher model for a smaller student model (T5). Models trained using TrueTeacher and ANLI data achieve state-of-the-art performance on the TRUE benchmark. The authors demonstrate its effectiveness through a controlled analysis comparing TrueTeacher to other methods such as FactEdit, DocNLI, FactCC, and FalseSum, especially in out-of-domain evaluations.
Additionally, TrueTeacher is proven to be applicable and beneficial in multilingual settings. The paper also promises to release 1.4M generated data for the research community's benefit. Even though the paper is well-written and includes comprehensive ablation studies and human evaluations of model-generated summaries, its technical contributions are limited.
Decision: Accept-Main