Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation (2023)

Abstract: Hallucination of text ungrounded in the input is a well-known problem in neural data-to-text generation. Many methods have been proposed to mitigate it, but they typically require altering model architecture or collecting additional data, and thus cannot be easily applied to an existing model. In this paper, we explore a new way to mitigate hallucinations by combining the probabilistic output of a generator language model (LM) with the output of a special “text critic” classifier, which guides the generation by assessing the match between the input data and the text generated so far. Our method does not need any changes to the underlying LM’s architecture or training procedure and can thus be combined with any model and decoding operating on word probabilities. The critic does not need any additional training data, using the base LM’s training data and synthetic negative examples. Our experimental results show that our method improves over the baseline on the WebNLG and OpenDialKG benchmarks.

## introduction hallucination, i.e., generated text lacking grounding in the input data, is a major challenge in neural data-to-text generation @xcite @xcite . hallucinations can lead to inaccurate or misleading information, significantly undermining the quality and reliability of the generated output. while many approaches have been proposed to address this problem, they often involve modifying the underlying model architecture @xcite or acquiring additional data @xcite , making them impractical for existing models. at the same time, popular metrics for evaluating hallucinations are based on text classification models, e.g. nli-based metrics @xcite . this indicates that text classifiers have the potential to accurately identify and assess coher-ence problems between the data and the generated text. however, use of text classifiers in generation typically involves producing many outputs with a base model and reranking them afterwards @xcite . in this paper, we propose a novel critic-driven decoding approach that combines the probabilistic output of a conditional language model (lm) with the output of a specialized text critic classifier that guides the generation process by evaluating the coherence of the textual prefix generated so far with the input data. this allows us to influence the generation on-the-fly, without the need to overgenerate many outputs. furthermore, our approach does not require modifications to the underlying lm or additional fine-tuning. this ensures compatibility with a wide range of existing models and decoding algorithms that operate on word probabilities. finally, our method does not rely on the collection of additional data, as training data for the critic can be synthesized from the data-to-text dataset used to train the underlying conditional lm. we verify the effectiveness of our critic-driven decoding in experiments on the webnlg @xcite and @xcite benchmarks, with both automatic and manual evaluation of text hallucinations in the model outputs. the results show that our method is able to limit hallucinations and produce a more faithful text, yet close to the base lm's output. our implementation of the proposed method is publicly available. foot_0 ## critic-driven decoding recall that auto-regressive conditional lms for data-to-text generation rely on the following probability factorization: in our approach, we introduce to this probability an additional text generation critic which evaluates the match between the generated text and the data representation. the output of the critic c can be seen as a binary variable, equal to 1 if the text matches the input data and 0 otherwise. this leads to the following probability factorization: this formulation combines two probabilities: the probability of a standard conditional lm p (y i |@xmath0, x) and the probability of the match between the text and the data as evaluated by the critic model p (c|@xmath1, x). the critic is modelled as a binary classifier, conditioned on the data, past tokens and the token currently being decoded. it is thus run at each decoding step, and it is evaluating the viability of the current prefix of the generation output (assuming future generation will be successful). the proposed formulation can be used with any auto-regressive conditional lm without modification, as the operation is identical to eq. 1. the critic can be trained separately from the lm since our formulation implies the conditional independence of those two models. the above factorization leads us to a practical proposal of a critic-driven decoding algorithm. first, an additional critic model is trained, which is able to approximate p (c|@xmath2, x) (details are discussed in sec. 3). we then perform standard greedy decoding with the lm, but using the updated formula for calculating probabilities of the next tokens (eq. 3). in practice, our implementation operates on logarithms rather than raw probabilities and uses an additional weight λ that adjusts the influence of the critic on the final result: the critic model p (c|@xmath3, x) is a binary classifier that checks the correspondence between the linearized data representation x and the so far generated prefix of the output text @xmath4. we assume an encoder pretrained lm as the backbone of the critic. the model input contains x and @xmath5. positive instances for the critic's training are constructed from examples (x, y) in the underlying lm's dataset as prefixes: (x, y 1 ), (x, @xmath6, (x, @xmath7, ..., (x, @xmath8. negative examples must be synthesized and are crucial for training the critic, as they teach it how to detect that the generated text starts deviating from the input data (i.e. hallucinations). therefore, we explore five ways of generating negative examples (see appendix g for examples): all critic model variants are trained by optimizing binary cross-entropy loss. ## experimental evaluation we compare all critic variants defined in sec. 3 with the baseline lm on both automatic and manual metrics, focusing on the number of hallucinations. ## related works hallucination in nlg is a widely studied problem, with many different mitigation methods proposed, including data cleaning or various model architecture modifications (see @xcite for a detailed review). mitigation methods most similar to ours include the controlled generation approach by filippova (2020), which uses special control codes to control hallucinations. this was followed by @xcite , who combine control codes with resampling of several texts and selecting the best one according to the metrics. however, both approaches require training a new lm with control codes and, in the latter case, additional resampling of whole texts. @xcite proposed a twostep generate & refine procedure, which is modelindependent but requires training of an additional correcting lm and decoding the sequence twice. similarly to our approach, @xcite use a text classifier to select the best output among the so-called contrast candidates but does not use it during decoding. our method is closely inspired by works on classconditional lms, which use the bayes rule to introduce additional conditioning @xcite . in particular, a formulation similar to ours is used by @xcite to impose a certain requirement, such as a level of formality, on the text produced by a lm. however, these works do not address the issue of hallucinations. the use of randomly generated words as negative samples to improve natural language generation has also been explored by @xcite . in contrast to this work, their unlikelihood training technique is mainly aimed at limiting repetitive text generation and requires training a new model, as it modifies the training objective. ## summary our paper introduces a novel critic-driven decoding approach to mitigate hallucinations in data-to-text generation. by using the output of a specialised text critic classifier, we guide the generation process to produce more grounded output without requiring any modifications to the underlying lm. the experimental results on the webnlg and opendialkg benchmarks show that the proposed method has the potential to limit hallucinations without hindering other text quality metrics. for natural language generation, translation, and comprehension. in proceedings of the 58th annual meeting of the association for computational linguistics, pages 7871-7880, online. association for computational linguistics. seungwhan moon, pararth shah, anuj kumar, and rajen subba. 2019. opendialkg: explainable conversational reasoning with attention-based walks over knowledge graphs. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 845-854, florence, italy. association for computational linguistics. kishore papineni, salim roukos, todd ward, and wei jing zhu. 2002. bleu: a method for automatic evaluation of machine translation. pages 311-318. hannah rashkin, david reitter, gaurav singh tomar, and dipanjan das. 2021. increasing faithfulness in knowledge-grounded dialogue with controllable features. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 704-718, online. association for computational linguistics. vikas raunak, arul menezes, and marcin junczys-dowmunt. 2021. the curious case of hallucinations in neural machine translation. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 1172-1183, online. association for computational linguistics. clement rebuffel, marco roberti, laure soulier, geoffrey scoutheeten, rossella cancelliere, and patrick gallinari. 2022. controlling hallucinations at word level in data-to-text generation. data mining and knowledge discovery, 36(1):318-354. thibault sellam, dipanjan das, and ankur p. parikh. 2020. bleurt: learning robust metrics for text generation. in acl. craig thomson, ehud reiter, and somayajulu sripada. 2020. sportsett:basketball -a robust and maintainable dataset for natural language generation. in intellang -intelligent information processing and natural language generation, page 9, online. hongmin wang. 2019. revisiting challenges in data-totext generation with fact grounding. in proceedings of the 12th international conference on natural language generation, pages 311-322, tokyo, japan. association for computational linguistics. sean welleck, ilia kulikov, stephen roller, emily dinan, kyunghyun cho, and jason weston. 2020. neural text generation with unlikelihood training. in 8th international conference on learning representations, iclr 2020, addis ababa, ethiopia, april 26-30, 2020. openreview.net. kevin yang and dan klein. 2021. fudge: controlled text generation with future discriminators. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 3511-3535, online. association for computational linguistics. tianyi zhang, varsha kishore, felix wu, kilian q. weinberger, and yoav artzi. 2020. bertscore: evaluating text generation with bert. in international conference on learning representations.