Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Authors: Mateusz Lango
Ondrej Dusek
Venue: EMNLP 2023 Main (2023)
Keywords: data-to-text generation
hallucinations
decoding approaches
natural language genereation
TL;DR: We significantly reduce hallucinations in data-to-text generation by combining the generator language model (LM) with a “text critic” classifier, which guides the decoding by assessing the match between the input data and the text generated so far.

Abstract: Hallucination of text ungrounded in the input is a well-known problem in neural data-to-text generation. Many methods have been proposed to mitigate it, but they typically require altering model architecture or collecting additional data, and thus cannot be easily applied to an existing model. In this paper, we explore a new way to mitigate hallucinations by combining the probabilistic output of a generator language model (LM) with the output of a special “text critic” classifier, which guides the generation by assessing the match between the input data and the text generated so far. Our method does not need any changes to the underlying LM's architecture or training procedure and can thus be combined with any model and decoding operating on word probabilities. The critic does not need any additional training data, using the base LM's training data and synthetic negative examples. Our experimental results show that our method improves over the baseline on  the WebNLG and OpenDialKG benchmarks.

Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Mateusz Lango and Ondřej Dušek
Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics
Prague, Czech Republic
{ (lango, odusek .ff cuni. cz
Abstract
ence problems between the data and the generated text. However, use of text classifiers in generation typically involves producing many outputs with a base model and reranking them afterwards (Hark- ous et al., 2020).
Hallucination of text ungrounded in the input is a well-known problem in neural textxt generation. Many methods have been proposed to mitigate it, but they typically require alter- ing model architecture collecting additional data, and thus cannot be easily applied to existing model. In this paper, we explore a new way mitigate hallucinations by combining the probabilistic output of generator language model (LM) with the output of special "text critic" classifier, which guides the generation by assessing the match between the input data and the text generated so far. Our method does not need any changes to the underlying LM's architecture or training procedure and can thus combined with any model and decoding op- erating word probabilities. The critic does not need any additional training data, using the base LM's training data and synthetic negative examples. Our experimental results show that our method improves over the baseline on the WebNL.G and OienDi benchmarks.
In this paper, we propose a novel critic-driven decoding approach that combines the probabilistic output of a conditional language model (LM) with the output of a specialized text critic classifier that guides the generation process by evaluating the co- herence of the textual prefix generated so far with the input data. This allows us to influence the gen- eration on-the-fly, without the need to overgenerate many outputs. Furthermore, our approach does not require modifications to the underlying L.M or additional fine-tuning. This ensures compatibility with wide range of existing models and decoding algorithms that operate on word probabilities. Fi- nally, our method does not rely on the collection of additional data, as training data for the critic can be synthesized from the data-to-text dataset used to train the underlying conditional LM.
1 Introduction
We verify the effectiveness of our critic-driven decoding in experiments on the WebNLG (Gardent et al., 2017) and OpenDialKG (Moon al., 2019) benchmarks, with both automatic and manual eval- uation of text hallucinations in the model outputs.
Hallucination, i.e., generated text lacking ground- ing in the input data, is major challenge in neural data-to-text generation (Raunak et al., 2021; Re- buffel et al., 2022; Corbelle et al., 2022; Ji et al.. 2023). Hallucinations misleading information, significantly undermining the quality and reliability of the generated output. While many approaches have been proposed to address this problem, they often involve modify- ing the underlying model architecture (Rebuffel et al., 2022) or acquiring additional data (Wang, 2019; Thomson et al., 2020), making them im- practical for existing models. At the same time, popular metrics for evaluating hallucinations based on text classification models, e.g. NLI-based metrics (Honovich al., 2021; Dušek and Kasner, 2020). This indicates that text classifiers have the potential to accurately identify and assess coher-
The results show that our method is able to limit hallucinations and produce a more faithful text, yet close to the base LM's output. Our implementation of the proposed method is publicly available.'
lead to inaccurate or
Critic-driven decoding
Recall that rugo-segressive conditional LMs for data-to-text generation rely on the following proba- bility factorization:
P(y x) = II P(uluse-,2))
(1)
h/ttps://gitaee thub. com/langus0/ critic- ceatic-wware-oee -ediic--aaeede

---

where x is the data representation and y is the gen- crated text. We use Øşj denote all the tokens y1, y2: 1/, "........... Vj-
3 Training text generation critic
The critic model P(clys æ) is a binary classifier that checks the correspondence between the lin- earized data representation æ and the so far gen- erated prefix of the output text Vsi. We assume an encoder pretrained LM as the backbone of the critic. The model input contains and yei split by separator token.
In our approach, we introduce to this probability an additional text generation critic which evaluates the match between the generated text and the data representation. The output of the critic can be seen as binary variable, equal to if the text matches the input data and 0 otherwise. This leads the following probability factorization:
Positive instances for the critic's train- ing are constructed from examples (x, in the underlying LM's dataset as prefixes: (æ, vi), (I. Vg2), (I, Vsa). (I, Vsn). Negative examples must be synthesized and are crucial for training the critic, as they teach it how to detect that the generated text starts deviating from the input data (i.e. hallucinations). Therefore, we explore five ways of generating negative examples (see Appendix G for examples):
P(ylz, c) II 2(,
(2)
i.c. generation of text y given the data representa- tion and the output of the critic c. By apply- ing simple probability transformations (see Ap- pendix A), we obtain the following factorization:
P(w,Mgi-1, e) x P(cluga, P(ulls---)))
(3)
base - for each positive example, we replace the last token with a random one. To make the training set more challenging, the tokens are sampled from amother text reference for the same data (if available) or another random sentence from the dataset.
This formulation combines two probabilities: the probability of standard conditional LM P(uilvs 1, and the probability of the match between the text and the data as evaluated by the critic model P(elyss, x).
The critic is modelled as a binary classifier, con- ditioned on the data, past tokens and the token currently being decoded. It is thus run at cach de- coding step, and it is evaluating the viability of the current prefix of the generation output (assuming future generation will be successful). The proposed formulation can be used with any auto-regressive conditional LM without modification, as the opera- tion is identical to Eq. 1. The critic can be trained separately from the LM since our formulation im- plies the conditional independence of those two models.
base with full sentences a randomly selected sentence of the reference text y is replaced with random sentence from the dataset. Neg- ative examples are then generated in the same way positive examples, but starting from the first token that deviates from the reference. In addition. instances where random token the reference is replaced by wrong one are also generated in the same way.
3. vanilla LM- for each positive example we probe an unconditional LM to get a list of the five most likely next tokens. We randomly select a token from this list and construct a negative example.
The above factorization leads us to a practi- cal proposal of critic-driven decoding algorithm. First, additional critic model is trained, which is able to approximate P(clgg 2) (details are dis- cussed in Sec. 3). We then perform standard greedy decoding with the LM, but using the updated for- mula for calculating probabilities of the next tokens (Eq. 3). In practice, our implementation operates on logarithms rather than raw probabilities and uses additional weight À that adjusts the influence of the critic the final result:
4. fine-tuned LM similar to the previous, but using the LM conditioned on the data.
5. fine-tuned LM with full sentences - the LM conditioned on the data is used to generate a textual description of the data. The nega- tive examples constructed for each token starting from the one where the model starts deviating from the reference.
In P(ullUi-1,,
& A P(cluss.)) x) + In P(N1U5)-1,
All critic model variants are trained by optimiz- ing binary cross-entropy loss.
(4)

---

4 Experimental evaluation
critic model
accuracy
F1
1. base
0.969
0.970
We compare all critic variants defined in Sec. 3 with the baseline LM on both automatic and manual metrics, focusing the number of hallucinations.
2. base w/full sent.
0.984
0.975
3. vamilla. LM
0,931
0.798
4. finc-tuned LM
0.920
0.718
5. fine-tuned LM wffull sent.
0.929
0.714
4.1 Experimental setup
Table I: The classification performance of different
We performed most experiments on the WebNLG benchmark (Gardent et al., 2017) containing data expressed as RDF triples and corresponding text references, which is prominent in many previous works tackling hallucination. We also evaluate our approach on the OpenDialKG dataset (Moon et al.. 2019), which contains dialogues annotated with RDF triples representing the information expressed in each utterance. We use it in similar way as WebNLG, treating the utterances as textualisations of the data, ie. without taking dialogue history into account. The BART-base encoder-decoder model (Lewis et al., 2020), finetuned on WebNLG, is used as the base NI.G system (see Appendix C for training details).
critic models measured on the validation test.
2002), METEOR (Banerjee and Lavie, 2005) and BERTScore (Zhang et al., 2020) as well as measures particularly targeting hallucinations: BLEURT (Sellam al., 2020) and the NLI-based metric proposed by Dušck and Kasner (2020).
Overall results on WebNLG are presented in Table 2. Except for the critic trained on full LM- generated sentences (var. 5), all the other vari- ants of critic-driven decoding slightly improve performance according to BLEU, METEOR, and BERTScore. Higher gains, up to 2.5% absolute on the whole test set, can be observed on measures tar- geting hallucinations, i.e. NLI and BLEURT. Note that our approach achieves this without modify- ing the original LM. The base critic achieves the highest scores across most evaluation metrics.
Five different critic models were trained as dis- cussed in Sec. 3 with classification heads on top of XLM-RoBERTa-bas model (Conneau et al., 2019), Appendix D for details. The vanilla LM critic uses BART-base without any fine-tuning, the fine-tuned LM variants (4 & 5) use the base LM to generate training data. The critics' classifica- tion performance given in Table 1. This shows that the critics are able to learn the synthetic data well, which is, however, not necessarily reflected in performance when used during generation.
Interestingly, both critics trained on data gener- ated with the fine-tuned LM (i.e. the same system as used for decoding) failed to improve the NLI measure and only one improved BLEURT. This shows that effective critic can be trained sepa- rately from the NLG system.
We use greedy decoding by default. To speed up computation of critic-driven decoding, we first eval- uate the second term of Eq. 3, i.e. the conditional LM, and we run the critic model only for l = most probable tokens, modifying its probabilities accordingly. The critic weight À = 0.25 (see Eq. 4) was used for all the models for WebNLG and A - 1 for Op. We found that the output of the critic can be noisy when evaluating the match be- tween the data and only few initial tokens of the text. Therefore, we add a simple linear warmup for X for the first five tokens: while decoding the i-th token, A = min( ( 1) (cf. Appendix B for choice of k and warmup).
Analysis of introduced changes We also mea- sured to what extent the critic-based approaches change the outputs compared to the baseline. i.c. the percentage of modified outputs as well the number of added and removed words.? Results in Tab. 4 show that critic-based approaches preserve many outputs (30-70%) and generally only change a few words, while keeping the output lengths similar. This suggests that our critics make small changes and only where necessary.
Out of domain generalization The test data of the WebNLG dataset contains about 46% of in- stances from categories not present in the training data. Therefore, we also provide the fine- fine-grained results for both in-domain and out-domain part of the test set in Table 2. The in-domain results are naturally better, but we can observe consistent im-
4.2 Analysis of decoding performance with automatic measures
The system outputs were evaluated using stan- dard automatic metrics BLEU (Papineni et al.,
"Replacing word counts as one addition and one deletion.

---

MET
BERT Score
NLI
BLEURT
decoding approach
BLEU
EOR
all
ood
ind
ood
ind
baseline
45.09
0.373
0.911
0.841
0.783
0.889
0.128
-0.026
0.257
1. critic (base)
45.48
0.377
0.913
0.855
0,801
0.901
0.155
0.010
0.277*
2. critic (base with full sentences)
44.90
0.371
0.913
0.868
0.820*
0,909
0.153
0,007
0.274
3. critic (vanilla LM)
45.44
0.377
0.913
0.859
0.811
0,900
0.139
-0.002
0.258
4. critic (finc-tuned LM)
45.41
0.373
0.911
0.834
0.772
0.886
0.128
-0.021
0.254
5. critic (fine-tuned LM full sentences)
45.59
0.374
0.912
0.839
0.779
0.889
0.136
-0.013
0.261
Table 2: Results of automatic evaluation the WebNLG test set. NLI and BLEURT are reported for the whole test
set (all) as well as separately for its out-of-domain (ood) and in-domain (ind) parts. "*" marks statistical significance at a = 0.05 level (NLI: exact test for proportions, BLEURT: unpaired t-test).
BLEU
METEOR
BERTScore
NLI
BLEURT
baseline
11.74
0.149
0.775
0.748
-0.933
critic (hase)
9.67
0.137
0.771
0.796
-0,905
critic (base with full sentences)
11.88
0.151
0.776
0.754
-0.920
critic (yanilla LM)
10.37
0.139
0.763
0.713
-0,980
critic (fine-tuned LM)
10.76
0.143
0.768
0.739
-0.964
critic (fine-tuned LM with full sentences)
11.41
0.149
0.771
0.712
-0.956
Table Results of automatic evaluation on the OpenDialKG test set.
provements of our critic-aware approach on both in-domain and out-of-domain data.
critic model
mod [%]
add.
rem.
base
66.3
4.54
4.58
base w/full sent. vanilla LM
72.8
5.42
4.72
Statistical analysis We performed statistical hy- pothesis testing to compare the results of the base- line with our approach with critic (base with full sentences). As expected, the differences on text quality measures (BLEU, METEOR, BERTScore) are not statistically significant, in contrast to the differences on measures targeting hallucinations, which are statistically significant the 0.05 significance level (cf. Table 2).
72.8
5.03
5.39
fine-tuned LM
48.5
2.52
2.71
fine-tuned LM w/full sent.
31.9
1.63
1.76
Table 4: Percentage of modified outputs and average number of words added/removed by different critics compared standard decoding on WebNLG.
4.3 Manual analysis of decoding performance
To verify the automatic metric results, we per- formed a small-scale in-house human evaluation. We sampled 100 instances from the test set of the WebNLG dataset and annotated for them the out- put of all the systems under study (600 system outputs in total). The annotation was performed by five NLP expert annotators, who assessed the presence of minor hallucinations (mostly typos in named entity names), major hallucinations (output containing fact(s) not supported by the data), omis- sions (missing information), disfluencies (grammar errors or hard-to-read text) and repetitions (infor- mation mentioned twice). Finally, the annotators ranked the system outputs for each example from best to worst, with ties allowed. The annotation was blind, with system order randomly shuffled for each example. Results are summarised in Table 5 (see Appendix for inter-annotator agreement).
Beam search experiment To verify the con- sistency of our critic's improvements, we rum additional experiments with stronger baseline. i.e. beam search decoding. The results, confirming greedy decoding results, are in Appendix F.
Results on OpenDialKG are presented in Ta- ble 3 and are mostly in line with those obtained for WebNLG, The base critic approach (var. 1) ob- tained gain 5 percentage points on NLI and of points on BLEURT over the baseline. The values of basic word-overlap metrics are lower, but our qual- itative assessment did not confirm any quality drop. The second critic variant (base with full sentences), which offered high performance of WebNLG, also performed well on OpenDialKG. scored best on standard text quality metrics while offering improvements over the baseline on hallucination- focused metrics (NLI, BLEURT).
All critic-driven approaches achieved better av-

---

decoding approach
min. hal.
maj. hal.
omi.
disfl.
rep.
avg. rank
baseline
0.22
0.40
0.25
0.20
0.08
3.61
1. critic (base)
0.21
0.30
0.20
0.17
0.04
3.38
2. critic (base with full sentences)
0.21
0.29
0.27
0.11
0.08
3.43
3. critic (vanilla LM)
0.18
0,29
0.23
0.19
0.05
3.54
4. critic (fine-tuned LM)
0,22
0.37
0.26
0.21
0.07
3.53
5. critic (fine-tuned LM with full sentences)
0.20
0.37
0.26
0.18
0.07
3.54
Table 5: Results of manual evaluation on a sample of 100 examples from the WebNLG (percentage of examples with minor and major hallucinations, omissions, disfluencies, repetitions; average relative ranking).
erage ranks than the baseline, with the base critic (var. having the best rank. The rank difference compared to the baseline not large (0.23), but increases for more complex instances: in instances with three or more triples, the difference is 0.33, for instances with file or more triples, it is 0.53. More importantly, the base critic reduced the rate of major hallucination by 10% absolute. Again, the improvements are bigger for more complex in- stances (15.3% for 2 3 triples, 20% for 2 5). also performed better on all other criteria, produc- ing a more fluent output, with fewer omissions and repetitions, as well as a slightly reduced number of minor hallucinations.
so-called contrast candidates but does not use it during decoding.
Our method is closely inspired by works on class- conditional LMs, which use the Bayes rule to intro- duce additional conditioning (Cohn-Gordoy et al.. 2018; 8: Dathathri et al., 2020). In particular, for- mulation similar ours is used by FUDGE (Yang and Klein, 2021) impose certain requirement, such as a level of formality, on the text produced by LM. However, these works do not address the issue of hallucinations.
The use of randomly generated words as negative samples to improve natural language generation has also been explored by Welleck et al. (2020). In contrast to this work, their unlikelihood training technique is mainly aimed at limiting repetitive text generation and requires training a new model, as it modifies the training objective.
Other critic variants were also effective in re- ducing hallucinations; in particular, the vanilla LM critic (var. 3) was the most effective in reducing both major and minor hallucinations. The fine- tuned LM approaches (vars. 4 & 5) only provided very limited benefits.
Summary
Our paper introduces a novel critic-driven decoding approach to mitigate hallucinations data-to-text generation. By using the output of a specialised text critic classifier, we guide the generation process to produce more grounded output without requiring any modifications to the underlying LM. The exper- imental results the WebNLG and OpenDialKG benchmarks show that the proposed method has the potential limit hallucinations without hindering other text quality metries.
5 Related works
Hallucination in NLG is a widely studied problem, with many different mitigation methods proposed, including data cleaning or various model architec- ture modifications (see Ji et al. (2023) for detailed review). Mitigation methods most similar to ours include the controlled generation approach by Fil- ippova (2020), which uses special control codes control hallucinations. This was followed by Rashkin et al. (2021), who combine control codes with resampling of several texts and selecting the best one according to the metrics. However, both approaches require training new LM with control codes and, in the latter case, additional resampling of whole texts. Cao et al. (2020) proposed two- step generate & refine procedure, which model- independent but requires training of an additional correcting LM and decoding the sequence twice. Similarly to our approach, Chen et al. (2021) use a text classifier to select the best output among the
A Acknowledgmentss
This work was supported by the European Re- search Council (Grant agreement No. 101039303, NG-NLG) and used resources of the LINDAT/ CLARIAH-CZ Research Infrastructure (Czech Ministry of Education, Youth, and Sports project No. LM2018101). The authors would like thank Ondfej Plátek, Patrícia Schmidtová and Sourabrata Mukherjee, who kindly provided manual annota- tions for this work.

---

Limitations
Natural Language Generation: Use Case On Me- teorology. In Proceedings of the 15th International Natural Language Generation Conference.
While our approach strives remove many hal- lucinations possible from the NLG output, certain proportion still remains for all our setups. The performance of the approach is limited by the base LM and its proposed most likely next tokens (as a limited number of next tokens is considered at each step, cf. Sec. 4). Furthermore, the use of the critic slows down the decoding. For application to other datasets, the critic may become less effective if the underlying training data is too noisy.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung. Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representa- lions.
Ondfej Dušek and Zdenčk Kasner. 2020. Evaluating semantic accuracy of data-to-text generation with nat- ural language inference. In Proceedings of the 13th International Conference on Natural Language Gen- eration, pages 131-137, Dublin, Ireland. Association for Computational Linguistics.
References
Katja Filippova. 2020. Controlled hallucinations: Learning to generate faithfully from noisy data. Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 864 870, Online. As- sociation for Computational Linguistics.
Satanjeev Banerjee and Alon Lavie. 2005. METEOR:
An automatic metric for MT evaluation with im- proved correlation with human judgments. In Pro- ceedings of the ACL Workshop on Intrinsic and Ex- trinsic Evaluation Measures for Machine Transla- tion and/or Summarization, pages 65-72, Ann Arbor, Michigan. Association for Computational Linguis- tics.
Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltracee 2017. Creating training corpora for NI.G micro-planners. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 179-188, Vancouver, Canada. Association for Computational Linguistics.
Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020. Factual error correction for abstrac- tive summarization models. In Proceedings of the 2020 Conference on Empirieal Methods in Natural Language Processing (EMNLP), pages 1251-6258, Online. Association for Computational Linguistics.
Hamza Harkous, Isabel Groves, and Amir Saffari. 2020. Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity. In COLING, Online. ArXiv: 2004.06577.
Sihao Chen, Fan Zhang, Kazoo Sone, and Dan Roth.
2021. Improving faithfulness in abstractive sum-
marization with contrast candidate generation and selection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, pages 5935-5941, Online. Association for Computational Linguistics.
Or Honovich, Leshem Choshen, Roce Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend. 2021. q²: Evaluating factual consistency in knowledge- grounded dialogues via question generation and ques- tion answering. In Proceedings of the 2021 Confer- ence Empiricul Methods in Natural Language Pro- cessing, pages 7856-7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Reuben Cohn-Gordon Noah Goodman, and Christo- pher Potts. 2018. Pragmatically informative image captioning with sharacter-level inference. In Pro- ceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- une 2 (Shori Pupers), pages 439-443, New Orleans, Louisiana, Association for Computational Linguis- tics.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38
Günter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter. 2017. nelf-normalizig neural networks. In Proceedings of the 31st International Conference on Neural Information Processing Sys- tems, NIPS' 17, page 972-981, Red Hook, NY, USA. Curran Associates Inc.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2019. Unsupervised cross-ling linss-l representation learning at scale. CoRR, abs/1911.0211.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising equeence-to-sqquene pre-training
Javier González Corbelle, Alberto Bugarín-Dizz Jose Maria Monso-Moral, and Juan Taboada. 2022. Dealing With Hallucination And Omission In Neural

---

for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computa- tional Linguistics.
Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511-3535, Online. Association for Computational Linguisties.
Seungwhan Moon, Pararth Shah, Anuj Kumar, and Ra- jen Subba. 2019. OpenDialKG: Explainable conver- sational reasoning with attention- walks over knowledge graphs. In Proceedings of the 57th An- nual Meeting of the Association for Computational Linguistics, pages 845 854, Florence, Italy. Associa- tion for Computational Linguisties.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinherger, and Yoav Arizi. 2020. BERTScore: Evaluating generation with BERT. In Interna- tional Conference on Learning Representations.
A Derivation of the proposed probability factorization that incorporates a critic model
Kishore Papineni, Salim Roukos, Todd Ward, and Wei jing Zhu. 2002. Bleu: method for automatic evalu- ation machine translation, pages 31 1-318.
Hannah Rashkin, David Reitter, Gaurav Singh Tomar, and Dipanjan Das. 2021. Increasing faithfulness in snowledge-grounderr dialogue with controllable features. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I Ith International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 704-718, Online. Association for Computa- tional Linguistics.
By applying the conditional probability formula and the product rule to Eq. 2. we obtain the follow- ing:
P(ui (5
P(Ugi-1,3,4 C.
P(c|) Vsi-1, 1 2))
Vikas Raunak, Arul Menezes, and Marcin Junczys- Dowmunt. 2021. The Curious Case of Hallucinations in Neural Machine Translation. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1172-1183, Online. Association for Computational Linguisties.
P(Usi-1, X,
P(clus, Plysi-
P(ygi-1: I.
P(d x ((yluss,
Clement Rebuffel, Marco Roberti, Laure Soulier, Geof- frey Scoutheeten, Rossella Cancelliere, and Patrick Gallinari. 2022. Controlling hallucinations at word level in data-to-text generation. Data Mining and Knowledge Discovery, 36(1):318-35
where the last line comes from the fact that when computing the probability of the next token yi, the previous tokens V-i-1 are fixed, the critic's score for the previous tokens P(clygi-1, x) is a constant and does not affect the result.
Thibault Sellam, Dipanjan Das, and Ankur P. Parikh. 2020. Bleurt: Learning robust metrics for text gener- ation. In ACL.
B Sensitivity analysis of the hyperparameters of critic-aware decoding
Craig Thomson, Ehud Reiter, and Somayajulu Sripada. Basketball
2020, Spokkset::asketballl A robust and maintain- able dataset for Natural Language Generation. In IntelLanG- Intelligent Information Processing and Natural Language Generation, page 9, Online.
B.1
The number of most probable considered tokens
To speed up computations of critic-driven decoding. we run the critic model only for k most probable tokens according to the LM and modify its prob- abilities with Eq. 3. The results in the paper are reported for & = 5, but we performed additional experiments with A = 15 to investigate how it will affect the performance. The results are given in Table 6. In general, we observe minor differences in comparison to = Some metries has been slightly improved, but it probably does not counter- balance the additional computational cost.
Hongmin Wang. 2019. Revisiting challenges in data-to- text generation with fact grounding. In Proceedings of the 12th International Conference on Natural Lan- guage Generation, pages 311-322 Tokyo, Japan. Association for Computational Linguistics.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Di- nan, Kyunghyun Cho, and Jason Weston. 2020. Neu- ral text generation with unlikelihood training. In 8th International Conference Learning Represen- tations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. ReenRevie..net.

---

decoding approach
BLEU
METEOR
BERTScore
NLI
BLEURT
baseline
45.09
0.373
0.911
0.841
0.128
1. critic (base)
45.57
0.378
0.914
0.857
0.157
2. critic (base with full sentences)
44.96
0.371
0.913
0.867
0.155
3. critic (unconditional LM)
45,53
0.377
0.913
0.865
0.141
4. critic (conditional LM)
45,41
0.373
0.911
0.834
0.129
5. critic (conditional LM with teacher forcing)
45.59
0.374
0.912
0.839
0.136
Table Results of automatic evaluation on WebNLG dataset for 1:
0.06
0.00
0.86
b5sc nhul sent
base auu sent (warmsp!
vonila LN
vanili (Mbaampp))
38
Figure I: The accuracy of the critic (base) as a function prefix length on validation set
36
somt.eamm-u up!
10'
Figure 2: BLEU as a function of À parameter for sys- outputs generated with different critic variants and with/without warm-up of X.
B.2 The importance of linear warm-up
A practical motivation for using linear warm-up of the À parameter can be found in Figure 1, which shows the accuracy as function of text prefix length for one of the critic models (base, var. 1). can be observed that at the beginning the gener- ation process (i.e. for very short prefixes) the accu- racy of the critic is low, but grows rapidly with the length of the prefix, reaching a high level around the length of tokens.
0:0
The importance of linear warm-up is investigated by comparing the decoding performance with constant and with linear warm-up (i.e. X; min( 1) A). The results of this experiment for BLEU and BLEURT measures are depicted in Fig- ure 2 and 3, respectively. It can be observed that the linear warm-up provides better performance for almost every model.
0.34
012
0.10
boselworm-upp
base hhot sent wwammup
0.00
vanila
vonils (Noarrmpp
8.64
FELM
willl
10
107
C Hyperparameters of BART fine-tuning
Figure 3: BLEURT function of À parameter for system outputs generated with different critic variants and with/without warm-up of X.
As conditional language model, we used BART- base model (Lewis al., 2020) fine-tuned with default architecture provided by HuggingFace li- brary. A with learning rate 77 = 2- 10 and
parameters 3 (0.9, 0.997), c = 10 was used

---

optimizer. Additionally, we applied polynomial scheduler of 1 with warmup equal to 10% of op- timization steps. The training was scheduled for 20 epochs with early stopping on validation loss (patience 10 epochs). We used batch size equal to 8 and label smoothing with 0.1 smoothing factor.
G Examples of negative instances generated by different approaches for critic training set construction
Let consider the following data representation:
(A-Ree A--Rosa Luna length 125800.0 (millimetres)); (A A--Rosa Luna I power type MTU Friedrichshafen) and the reference:
D Details on critic model training
The A-Rosa Luna powered by a MTU Friedrichshafen engine and 125.8 metres in
The architecture of the critic model consisted of pretrained LLM-RoBERTa-base model (Conneau et al., 2019) and a classification head on top of the representation of the first token. The classification head contained fully connected layer with SELU activation function (Klambauer et al., 2017) and one additional classification layer with sigmoid ac- tivation. The number of neurons in the first layer was set the dimensionality of the output embed- ding.
length.
The positive examples for the critic consist on all the possible prefixes generated from the reference, i.e. "The", "The A-Rosa", "The A-Rosa Luna", etc. The negative examples generated by different approaches are as follows:
base - the negative examples are generated with random words
The critic models were trained discussed in Sec. 3 by optimizing the cross-entropy loss. A was used as an optimizer with a learn- ing rate of of == 10 5 The training was continued until the convergence, i.e. lack of the improvement on validation loss.
"The Cruises", "The A-Rosa operated", "The A-Rosa Luna located"....
2. base with full sentences sentence or a token from the reference is replaced with random sentence/token and all possible negative ex- amples generated
All the experiments with the critics (both critic one
training and decoding) were performed GPU: Vidia Quadro P5000 16 GB. During decod- ing the BART-based language model was loaded with bitsandbytes library (8-bit mode).
"The Cruises", "The Cruises Luna", "The Cruises Luna is", "The A-Rosa operated", "The A-Rosa operated is",
E Inter-annotator agreement
3. vanilla LM - the incorrect next words sam- pled from the five most probable tokens cording (unconditioned) LM
To estimate the inter-annotator agreement, one of the annotators re-annotated 10 x 6 model outputs) instances originally annotated by different anno- tator. 86% of annotations were identical. In terms of Cohen's kappa, 0.19 agreement minor hallucinations, 0.68 for major, 0.88 for omis- sions, 0.48 for repetitions and 0.07 for disfluencies.
"The United", "The A-Rosa is", "The A-Rosa Luma powers"....
obtained for
4. fine-tuned LM with full sentences for a given data the NLG system generated the fol- lowing output: "The A-Rosa Luna is 125.8m long and powered by MTU Friedrichs- burger", which is used to generate negative examples by comparing it against the refer- ence
F Comparison with a stronger baseline
One simple method which generates multiple out- puts and generally tends to offer texts of higher quality beam search. We run additional exper- iments with beam size equal to 5 and present the result in the Table 7. The improvements for this stronger bascline are consistent with these reported in the main paper for greedy decoding.
"The A-Rosa Luna is 125.8m". "The A-Rosa Luna 125.8m long", "The A-Rosa Luna is 125.8m and", "The A-Rosa Luna is 125,8m and is....

---

BLEU
METEOR
BERTScore
BLEURT
baseline
47.57
0.380
0.916
0.852
0.176
1. critic (base)
47.75
0.387
0,918
0.886
0.202
2. critic (base with full sentences)
46.06
0.376
0.917
0.898
0.212
3. critic (vanilla LM)
46,56
0.379
0.913
0.881
0.161
4. critic (fine-tuned LM)
49.04
0.385
0,919
0.866
0.196
5. critic (fine-tuned LM with full sentences)
43.74
0.372
0.909
0.861
0.123
Table 7: Results of automatic evaluation on the WebNLG set while using beam search (beam size equal 5).
5. fine-tuned LM the incorrect next words are sampled from the five most probable tokens according to data-conditioned LM
"The A-Rosa Luna is 125.8m", "The A-Rosa Luna is supplied", "The A-Rosa Luna is powered with"....


Reviews:
[{"review_id": "osuSimRAod", "reviewer": "Reviewer_QRL1", "summary": "", "questions": "- What is the agreement score from 5 annotators?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and for taking the time to read this paper.\n\n>It only showed on one dataset and evaluated “in-domain” results. (...)\n\n(1) Our approach is evaluated on both in-domain and out-of-domain data. The test data of the WebNLG dataset contains about 46% of instances from categories not present in the training data. We add the fine-grained results below. The in-domain results are naturally higher, but one can observe consistent improvements on both in-domain and out-of-domain data. Additionally, we also offer results on one more dataset in (4).\n\t \t \t \t \t \t\n|                                     |   NLI   |               |           |  BLEURT |               |           |\n|-------------------------------------|:-------:|:-------------:|:---------:|:-------:|:-------------:|:---------:|\n|                                     | overall | out-of-domain | in-domain | overall | out-of-domain | in-domain |\n| baseline                            | 0,841   | 0,783         | 0,889     | 0,128   | -0,026        | 0,257     |\n| critic (base)                       | 0,855   | 0,801         | 0,901     | **0,155**   | **0,010**         | **0,277**     |\n| critic (base with full sentences)   | **0,868**   | **0,820**         | **0,909**     | 0,153   | 0,007         | 0,274     |\n| critic (vanilla LM)                 | 0,859   | 0,811         | 0,900     | 0,139   | -0,002        | 0,258     |\n| critic (fine-tuned LM)              | 0,834   | 0,772         | 0,886     | 0,128   | -0,021        | 0,254     |\n| critic (fine-tuned LM w/full sent.) | 0,839   | 0,779         | 0,889     | 0,136   | -0,013        | 0,261     |\n\n\n>The proposed critic sampling strategy is not novel. This paper reminds me of PPLM (Plug and Play Language Models: A Simple Approach to Controlled Text Generation) and its literature of controllable generation. To me this is a subset of the controllability and focusing on reducing hallucination.\n\n(2) We agree that our approach is related to PPLM (hence Dathathri et al. are cited in our Related Work section). However, we believe that there are important differences: \n\n- PPLM performs “ex post facto” optimization of the LM activation values and requires gradient computations during decoding. In contrast, our critic-based approach only modifies the probabilities of the next token. Even though PPLM does not require LM retraining, it does require access to its implementation to cache historical activation values of neurons and to perform specific backpropagation during decoding. Our approach can potentially work even with non-differentiable models (since only output probabilities are modified) and does not require any changes in the LM implementation. Note that in practice it may be difficult to access some (e.g. proprietary) model implementations.\n\n- Second, the PPLM attribute model is conditioned on specific historical activation values of a given LM (see Eq. 3 of Dathathri et al.). In contrast, some of our critics (e.g. the well-performing critic-base) have been trained completely independently of the LM itself and, once trained, can be straightforwardly used in decoding any LM.\n\n- Although controllability and hallucination mitigation are related, we argue that controlling sentiment or term usage (Dathathri et al.) is easier than ensuring the fidelity of generated text (our paper).\n\n>More hallucination tasks are expected to be evaluated, especially summarization tasks and open-domain QA tasks.\n\n(3) We agree that more tasks could be evaluated, but the scope of the current short paper is limited to the challenging data-to-text task.\n\n>More hallucination reduction methods are expected to be compared. For examples, Improving Factual Consistency in Summarization with Compression-Based Post-Editing, or check section 5 in this paper to get some ideas (Survey of Hallucination in Natural Language Generation).\n\n(4) Thank you for pointing out interesting works. Unfortunately, the paper you mentioned and also other methods from Section 5.2.3 (Post-Processing methods, which are the most related to our work) are designed for the summarization task and are not directly applicable to our data-to-text task. This shows that there is a certain research gap that our paper fits in.\nThe only work on a related task is Neural Path Hunter (NPH, by Dziri et al.), which post-edits utterances from a dialog agent using small knowledge graphs. We were not able to replicate their experiments in this short time due to problems with unresolved dependencies etc., but  we will contact Dziri et al. to compare both methods directly and provide the results in the final version. Currently, we were able to run our approach with “critic (base)” on their OpenDialKG dataset, using the data in the same way as WebNLG (i.e. without taking dialog history into account). On measures targeting hallucinations, we obtained +5% on NLI and +3% on BLEURT. Basic word-overlap metrics are lower, but our qualitative assessment did not confirm any quality drop. For the final version of the paper, we'll prepare the results for all the critics, including a variant conditioned on dialog history.\n\n|                | BLEU  | METOR | BERTScore | NLI   | BLEURT |\n|----------------|-------|-------|-----------|-------|--------|\n| baseline       | **11,73** | **0,149** | **0,775**     | 0,748 | -0,933 |\n| critic (base)  | 9,66  | 0,137 | 0,771     | **0,796** | **-0,905** |\n\n>What is the agreement score from 5 annotators?\n\n(5) In our study, we didn’t have overlapping annotations so the inter-annotation agreement was not originally measured. We asked one annotator to re-annotate 10 (x 6 model outputs) instances originally annotated by a different annotator. 86% of annotations were identical. In terms of Cohen’s kappa, we obtained 0.19 agreement for minor hallucinations, 0.68 for major, 0.88 for omissions, 0.48 for repetitions and 0.07 for disfluencies.\n\n>One reason that why this can show improvement is that the critic classifier is trained to learn those unlikelihood samples (e.g., NEURAL TEXT DEGENERATION WITH UNLIKELIHOOD TRAINING). Even though this line of work focuses more on the training stage not inference stage, but I'd recommend the authors to check and compare.\n\n(6) Thank you for pointing out this interesting work. We will include it in the revised version of Related Work section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- It showed several negative sampling strategies and its performance on guiding a trained generator, and showed that by human evaluation it can reduce hallucination.", "reasons_to_reject": "- It only showed on one dataset and evaluated “in-domain” results. For hallucination, it is more important to show its generalization ability, especially nowadays we have generators that are pretrained on billions of tokens already. \n\n- The proposed critic sampling strategy is not novel. This paper reminds me of PPLM (Plug and Play Language Models: A Simple Approach to Controlled Text Generation) and its literature of controllable generation. To me this is a subset of the controllability and focusing on reducing hallucination. \n\n- More hallucination tasks are expected to be evaluated, especially summarization tasks and open-domain QA tasks. \n\n- More hallucination reduction methods are expected to be compared. For examples, Improving Factual Consistency in Summarization with Compression-Based Post-Editing, or check section 5 in this paper to get some ideas (Survey of Hallucination in Natural Language Generation).", "questions_to_authors": "- What is the agreement score from 5 annotators?", "paper_topic_and_main_contributions": "* This paper is about a decoding strategy that takes a classifier output into account for resampling.\n* The main contribution is the exploration of showing different results on negative sample generation, and showed that the in-domain generation results can reduce hallucinations.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "- One reason that why this can show improvement is that the critic classifier is trained to learn those unlikelihood samples (e.g., NEURAL TEXT DEGENERATION WITH UNLIKELIHOOD TRAINING). Even though this line of work focuses more on the training stage not inference stage, but I'd recommend the authors to check and compare."}}, {"review_id": "xVqB0VvwoP", "reviewer": "Reviewer_GpK3", "summary": "", "questions": "A: Your results suggest that your method is able to reduce the overall hallucination rate. However, this reduction does not necessarily translate to higher output quality. Have you analysed what may be the reason for this? Is it that the number of hallucinations is small to impact corpus-level aggregated metrics? Or is it, perhaps of more significance, that some good quality outputs have seen their quality go down after employing the critic?\n\nB: You mention that your critic preserves many outputs (L240) with percentages of around 30-70%. This leads you to mention that the critics \"make small changes and only where necessary\". However, preserving the outputs does not necessarily mean that the critic only provides changes *only where necessary*. In fact, without further evidence as to how the critic behaves for originally good quality outputs, it is hard to back up this statement. Have you done further experiments that suggest it is indeed the case that the critic only makes changes where necessary?\n\nC: You mention in L274-279 that some methods are more effective than others for reducing hallucinations. It would be very interesting to analyse why this is the case. Have you done some qualitative experiments that provide some hints as to why this may be the case?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and for taking the time to read this paper.\n\n>the authors do not compare their proposed methods with any other baselines for mitigation of hallucinations; although they have mentioned these alternatives (e.g., changes to the underlying architecture or generation of multiple hypotheses), they have not shown how these previous methods fare against theirs; this is particularly critical for their story: while this new method does not require changing the underlying architecture or generation of multiple hypotheses, it does require training a classifier that, I would assume, needs to be trained on in-domain data. Moreover, using the critic at test time also incurs inference costs. Thus, this method also comes with its own set of overheads/computation burdens. I believe it is critical to compare with one such method (e.g., generation of multiple outputs w/o critic) both at the level of the output quality, output hallucination and computational cost.\n\n(1) Thank you for this comment. One simple method which generates multiple outputs and generally tends to offer texts of higher quality is beam search. We run additional experiments with beam size equal to 5 and present the result in the table below. The improvements for this stronger baseline are consistent with these reported in the paper for greedy decoding.\t \t\n|                                            | BLEU       | METEOR    | BERTScore | NLI       | BLEURT    |\n|--------------------------------------------|------------|-----------|-----------|-----------|-----------|\n| baseline                                   | 47,573     | 0,380     | 0,916     | 0,852     | 0,176     |\n| critic (base)                              | 47,752     | **0,387** | 0,918     | 0,886     | 0,202     |\n| critic (base with full sentences)          | 46,056     | 0,376     | 0,917     | **0,898** | **0,212** |\n| critic (vanilla LM)                        | 46,561     | 0,379     | 0,913     | 0,881     | 0,161     |\n| critic (fine-tuned LM)                     | **49,043** | 0,385     | **0,919** | 0,866     | 0,196     |\n| critic (fine-tuned LM with full sentences) | 43,740     | 0,372     | 0,909     | 0,861     | 0,123     |\n\n>the analysis of the mitigation method could benefit from finer-grained studies apart from corpus-level/aggregated metrics of quality or measures of edit percentage (e.g., studying how the method impacts the outputs depending on the quality of the original output)\n\n(2)  Thank you for this comment. We’ve already provided some additional insights, such as the number of added/removed words and modified outputs reported in the paper. We will report some additional fine-grained results on the additional page provided upon acceptance.\n\n>the authors only tested on one dataset, which may limit the generalisability of their conclusions\n\n(3) We have run additional experiments on the OpenDialKG dataset with our “critic (base)” approach -- please see our response to reviewer QRL1, item (4)\n\n>A: Your results suggest that your method is able to reduce the overall hallucination rate. However, this reduction does not necessarily translate to higher output quality. Have you analysed what may be the reason for this? Is it that the number of hallucinations is small to impact corpus-level aggregated metrics? Or is it, perhaps of more significance, that some good quality outputs have seen their quality go down after employing the critic?\n\n(4) Thank you for this comment - we were also interested in this matter. \n\nFirst of all, we would like to point out that our method does improve the text quality metrics in a vast majority of configurations, albeit slightly.\n\nWe attribute the lack of a larger metric gain to two factors. (1) Our method generally modifies a small number of words in the generated texts, which can be hard to capture by n-gram based metrics. (2) The automatic measures are generally imperfect and have some shortcomings. The manual evaluation of text quality (expressed by ranking the model outputs) demonstrated that all critic approaches obtained better average rank than the baseline.\n\n>B: You mention that your critic preserves many outputs (L240) with percentages of around 30-70%. This leads you to mention that the critics \"make small changes and only where necessary\". However, preserving the outputs does not necessarily mean that the critic only provides changes only where necessary. In fact, without further evidence as to how the critic behaves for originally good quality outputs, it is hard to back up this statement. Have you done further experiments that suggest it is indeed the case that the critic only makes changes where necessary?\n\n(5) It is true that the critic can sometimes suggest changes which are not correct, but this is relatively rare. For instance, critic (base) introduces errors to ~5% of originally correctly generated sentences while correcting >32% of incorrect outputs (in terms of hallucinations).\n\n>C: You mention in L274-279 that some methods are more effective than others for reducing hallucinations. It would be very interesting to analyse why this is the case. Have you done some qualitative experiments that provide some hints as to why this may be the case?\n\n(6) We suspect that lower improvements of critics trained on data generated with fine-tuned LMs are related to the fact that LMs often generate correct data verbalizations that differ from the reference. In such a situation, the text suggested by LM is correct but the comparison to the reference constructs a negative example for the critic's training data. This can lead the critic both to over-focus on generating the exact reference and to learn to negatively assess other possible correct data verbalizations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- the idea of employing a critic to guide generation is, to the best of my knowledge, novel and has some interesting properties (e.g., not requiring changes to the original LM architecture)\n- different ways of training the critic were put forward and analysed extensively, including with human evaluation; the results show that the method holds some promise", "reasons_to_reject": "- the authors do not compare their proposed methods with any other baselines for mitigation of hallucinations; although they have mentioned these alternatives (e.g., changes to the underlying architecture or generation of multiple hypotheses), they have not shown how these previous methods fare against theirs; this is particularly critical for their story: while this new method does not require changing the underlying architecture or generation of multiple hypotheses, it does require training a classifier that, I would assume, needs to be trained on in-domain data. Moreover, using the critic at test time also incurs inference costs. Thus, this method also comes with its own set of overheads/computation burdens. I believe it is critical to compare with one such method (e.g., generation of multiple outputs w/o critic) both at the level of the output quality, output hallucination and computational cost. \n- the analysis of the mitigation method could benefit from finer-grained studies apart from corpus-level/aggregated metrics of quality or measures of edit percentage (e.g., studying how the method impacts the outputs depending on the quality of the original output)\n- the authors only tested on one dataset, which may limit the generalisability of their conclusions", "questions_to_authors": "A: Your results suggest that your method is able to reduce the overall hallucination rate. However, this reduction does not necessarily translate to higher output quality. Have you analysed what may be the reason for this? Is it that the number of hallucinations is small to impact corpus-level aggregated metrics? Or is it, perhaps of more significance, that some good quality outputs have seen their quality go down after employing the critic?\n\nB: You mention that your critic preserves many outputs (L240) with percentages of around 30-70%. This leads you to mention that the critics \"make small changes and only where necessary\". However, preserving the outputs does not necessarily mean that the critic only provides changes *only where necessary*. In fact, without further evidence as to how the critic behaves for originally good quality outputs, it is hard to back up this statement. Have you done further experiments that suggest it is indeed the case that the critic only makes changes where necessary?\n\nC: You mention in L274-279 that some methods are more effective than others for reducing hallucinations. It would be very interesting to analyse why this is the case. Have you done some qualitative experiments that provide some hints as to why this may be the case?", "paper_topic_and_main_contributions": "The paper addresses the problem of on-the-fly mitigation hallucinations in data-to-text generation. Instead of modifying the architecture or generating several hypotheses, the authors propose using a text critic classifier to guide the generation process. This critic is constructed to encourage the generated text to match the input data. The results suggest that the critic approach does not have significant impacts on the quality of the generated text but can mitigate the impact of hallucinations.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3cvNpi6PWF", "reviewer": "Reviewer_eK7W", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and for taking the time to read this paper.\n\n>I know other baselines require additional efforts and might not be directly comparable to your method, but the lack of baselines makes it hard to establish your contribution, as most improvements seem to be marginal (are they statistically significant?)\n\n(1) We performed additional statistical tests to compare the results of our baseline and our approach with critic (base with full sentences). As expected, the differences on text quality measures (BLEU, METEOR, BERTScore) are not statistically significant, but the differences on measures targeting hallucinations are statistically significant at the standard alpha=0.05 significance level. For the NLI metric, p-value was equal to 0.0225 (exact test for proportions) and for BLEURT p-value is 0.0360 (unpaired t-test).\n\n>I know it's a very easy critique to make, but I think it's justified in this case. Another dataset can really help in making your findings more robust.\n\n(2) We have run additional experiments on the OpenDialKG dataset with our “critic (base)” approach -- please see our response to reviewer QRL1, item (4)\n\n\n>Short papers are very challenging to write, but I think this is one of the first papers I read without a \"conclusion\" section, which harms the paper's self-containment.\n\n(3) Thank you for this comment. We will include a formal conclusion section in the revised version of the paper, using the additional page. Currently, the introduction section contains the main highlights of the paper and summarizes the most important findings.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The paper is well-written and easy to follow. \n2) The idea is simple (in a good way, IMO) and well-motivated. \n3) Unlike previous methods, the additional effort for mitigating hallucinations seems to be minimized here, which will likely lead to a broader use for real-world applications. \n4) The authors will share their code, which is much appreciated.", "reasons_to_reject": "1) I know other baselines require additional efforts and might not be directly comparable to your method, but the lack of baselines makes it hard to establish your contribution, as most improvements seem to be marginal (are they statistically significant?)\n\n2) I know it's a very easy critique to make, but I think it's justified in this case. Another dataset can really help in making your findings more robust. \n\n3) Short papers are very challenging to write, but I think this is one of the first papers I read without a \"conclusion\" section, which harms the paper's self-containment.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors present a new method to mitigate hallucinations in data-to-text generation. \nThe method can be combined into an existing system with minimal effort (relative to prior methods) without the need to annotate additional data.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Eq 3 needs to be better aligned."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method for reducing hallucinations by additionally incorporating a classifier's probability to assess the correctness between the generated text so far during decoding. Reviewers agree that this method is useful, simple (in a positive way), and easy to apply. The idea is also well-motivated, and the authors have shown extensive results.

A common drawback of this work, as raised by reviewers, is that the paper evaluates its method only on a single dataset, thereby raising questions about its generality. Additionally, there is a lack of comparison with other hallucination mitigation baselines.
Decision: Accept-Main