KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection
Authors: Sehyun Choi
Tianqing Fang
Zhaowei Wang
Yangqiu Song
Venue: EMNLP 2023 Main (2023)
Keywords: Hallucination
Knowledge Grounding
Natural Language Generation
Constrained Decoding
Controllable Text Generation
TL;DR: A constrained decoding method using MCTS for faithful natural language generation grounded on reference knowledge.

Abstract: Large Language Models (LLMs) have demonstrated remarkable human-level natural language generation capabilities. However, their potential to generate misinformation, often called the *hallucination* problem, poses a significant risk to their deployment. A common approach to address this issue is to retrieve relevant knowledge and fine-tune the LLM with the knowledge in its input. Unfortunately, this method incurs high training costs and may cause catastrophic forgetting for multi-tasking models. To overcome these limitations, we propose a knowledge-constrained decoding method called KCTS (Knowledge-Constrained Tree Search), which guides a frozen LM to generate text aligned with the reference knowledge at each decoding step using a knowledge classifier score and MCTS (Monte-Carlo Tree Search). To adapt the sequence-level knowledge classifier to token-level guidance, we also propose a novel token-level hallucination detection method called RIPA (Reward Inflection Point Approximation). Our empirical results on knowledge-grounded dialogue and abstractive summarization demonstrate the strength of KCTS as a plug-and-play, model-agnostic decoding method that can effectively reduce hallucinations in natural language generation.

KCTS: Knowledge-Constraine Tree Search Decoding with Token-Level Hallucination Detection
Schyun Choi, Tianqing Fang, Zhaowei Wang and Yangqiu Song Department of Computer Science and Engineering, HKUST, Hong Kong SAR (schoiaj, tfangaa, zwanggy, yasong)@cse.use ust.
Abstract
to hallucination, poses significant risk the reliability of the texts generated by these models. Previous research has attempted to mitigate this issue by augmenting the input of the lan- guage model with relevant knowledge (Asai et al.. 2023), involving knowledge retrieval (Robertson and Zaragoza, 2009; Karpukhin et al., 2020; He et al., 2022; Shi et al., 2023) to identify relevant information and a reader language model that takes both the context and the retrieved knowledge in- put to generate response (Lewis et al., 2020; Izac- ard and Grave, 2021; Shuster et al., 2021; Borgeaud et al., 2022; Peng et al., 2023). Some works also proposed joint-train methods of the retriever and reader modules for better performance (Zhong et al., 2022b; Rubin and Berant, 2023). Whilc these approaches have demonstrated potential, it involves pre-training fine-tuning the reader lan- guage model, which poses significant challenges. First, the ever-increasing size of language models makes training them computationally expensive, which is becoming increasingly prohibitive, not to mention that some API-based LLMs (c.g., Ope- nAI APIs²) are not trainable by end users. Second, many state-of-the-ar language models are designed to be multi-task zero-shot models through instruc- tion tuning, aiming to perform well across various tasks. However, fine-tuning a language model ex- tensively on a specific task can lead to catastrophic forgetting (French, 1999; Kirkpatrick et al., 2017: He et al., 2021), causing the model to lose its gener- alizability across different tasks and compromising its overall performance. To address these chal- lenges, there is pressing need for methods that do not require updating weights of language models, enabling efficient knowledge-gen grrdgdegroonded generation without sacrificing the generalization capabilities of the model.
Large Language Models (LLMs) have demon- strated remarkable human-level natural lan- guage generation capabilities. However, their potential to generate misinformation, often called the hallucination problem, poses sig- nificant risk to their deployment. A com- mon approach to address this issue to re- trieve relevant knowledge and fine-tune the LLM with the knowledge in its input. Unfor- tunately, this method incurs high training costs and may cause catastrophic forgetting for multi- tasking models. To overcome these limitations, we propose knowledge-constraing decoding method called KCTS Swwwledde-Constrannee Tree Search), which guides a frozen LM to generate text aligned with the reference knowl- edge at each decoding step using knowledge classifier score and MCTS (Monte-Care Tree Search). To adapt the requence-level knowl- edge classifier to token-level guidance, we also propose novel token-level hallucination de- tection method called RIPA (Reward Inflection Point Approximation). Our empirical results knowledge-groundeh dialogue and abstrac- tive summarization demonstrate the strength ICTTS' plug-and-play. model-agnostic decoding method that can effectively reduce hallucinations in natural language generation.
1 Introduction
Recent progress in instruction-tunn language mod- els (LMs) has brought forth strong general-pune purpose language that can perform well on various tasks in a zero-shot setting (Tay, 2023; Ouyang et al., 2022; Chung et al., 2022; Sanh et al., 2022). How- ever, there have been numerous studies indicating current language models may generate non-factual information that is not supported by evidence with high level of confidence (Ji al., 2023a; Liu al., 2023b). This phenomenon, often referred
Although designing a decoding method for LLMs is a natural way to mitigate hallucinations
https:/ggiiha..oo////KU/tAnnnncn Constrained- Constrained-Decodinng
http::///plafoomooppaaa.coo wttp:p/a....mmmmoooooppprrennnn.

---

without fine-tuning, current works in plug-and- play guided decoding (Dathathri al., 2020; Yang and Klein, 2021; Liu et al., 2021; Chaffin et al., 2022a) are still inapt to directly be adapted to the knowledge-groundee scenarios due to their inabil- ity to identify the necessary knowledge required for generation, which leads to hallucination. There- fore, in this work, we propose novel approach knowledge-consttained decoding (KCD), which applies an auxiliary knowledge classifier on top of frozen LM to detect hallucinations, and uses its hhowledg--rruuddddess score to guide the de- coding process. By incorporating these classifiers during decoding, we aim to constrain the gener- ated text, ensuring its faithfulness to the reference knowledge. In addition, we propose a novel token- level hallucination detection method, RIPA (Re- ward Inflection Point Approximation), which is trained to predict the starting point of the halluci- nating token and enables effective adaptation of the knowledge classifier defined on the sequence level to the token level.
in model-generraeed text, several detection meth- ods and benchmarks have been designed (Thorne et al., 2018; Pagnoni et al., 2021; Wang et al., 2020; Min et al., 2023; Manakul et al., 2023; Chen et al., 2023). In this work, instead of directly measuring hallucination, we aim to mitigate hallucination in gewledge-groundes systems, which naturally re- quires the response to be faithful to the knowledge.
Knowledge Grounded Generation Knowledge Grounded Generation is mainly driven by retriev- ing relevant knowledge (Karpukhin et al., 2020; Su et al., 2022) and training the generator to pro- duce responses augmented on the retrieved knowl- edge (Lewis et al., 2020; Izacard and Grave, 2021; Rashkin et al., 2021; Mialon et al.. 2023). An- other line of work (Févry et al.. 2020; Verga et al.. 2021; Zhong et al., 2022b) learns and stores entity or fact representations and provides them as input to the generator. While these methods all address the problem of snowledge-grounded they all require the full fine-tuning of the genera- tor, which may degenerate the zero-shot ability of the base model due to catastrophic forgetting, and incur significant computational cost. A recent work (Peng et al., 2023) improves the grounded- ness of ChatGPT responses by incorporating the knowledge context in the prompt and providing textual feedback. While this work introduces a non-traine non-training method to knowledge- grodndeerr gen- eration, strongly dependent on the base LM's ability to understand the textual feedback and gen- erate reference knowledge to begin with. In con- trast, we propose to mitigate this problem with an approach that does not involve the fine-tuning of the generator weights and is model-
generation,
To sum up, our contributions are two-fold: First, we introduce KCTS Search), a discriminator guided decoding method that constrains the generation to be grounded on the reference knowledge, together with a novel token-level approximation method of the future reward (groundedness) through the Reward Inflec- tion Point Approximation (RIPA). Second, our ex- tensive experiments in knowledge-groundde logue and abstractive summarization tasks show the strength of KCTS, even outperforming Chat- GPT and GPT 3.5 in some dimensions.
Constrained Tree
dia-
2 Related Work
Mcasuring Hallucinations Hallucination of lan- guage models or the generation of contents that are either non-factual or not supported by evidence have been studied and reported in various fields (Ji et al., 2023b; Bang et al., 2023), such as machine translation (Raunak et al., 2021), abstractive sum- marization (Maynez et al., 2020; Lee et al., 2022), Open Domain Dialogue (Ji et al., 2023e; Xu et al., 2023), Question Answering (Lin et al., 2022), or image captioning (Rohrbach et al., 2018). Recently developed LLMs such as Bing Chat, or perplex- ity.ai even serve as generative search engines, al- though their seemingly fluent and informative re- sponses are not always verifiable (Liu et al., 2023b). To automatically detect and quantify hallucination
Guided Decoding Guided decoding includes su- pervised controllable text generation (Keskar al., 2019; Arora et al., 2022), discriminator-guidee de- coding (Dathathri al., 2020; Yang and Klein, 2021; Krause et al., 2021: Meng et al., 2022: Wang et al., 2022b) and constrained decoding (Qin et al., 2020, 2022; Lu et al., 2021, 2022: Kumar et al., 2022; Liu et al., 2023c; Geng et al., 2023), in which the user can control the sentiment or style of the generated text or constrain the generation to lcxical constraints. Plug-and- LM (PPLM) (Dathathri et al., 2020) introduces key concept of Bayesian decomposition P(y]ær, c) xx P(ylær) P(cly), where is the control attribute. PPLM trains a small dis- criminator on a frozen LM and performs gradient

---

ascent from the discriminator to maximize P(dly). FUDGE (Yang and Klein, 2021) instead performs weighted decoding (WD) by directly re-weighing the token probabilities (yt (et, with an auxil- iary classifier probability P(CUse)) To perform re- ranking every step. ((sl)))((clv) is decomposed into token-level and token-level attribute classi- fier P(c))<)) is used. NADO (Meng et al., 2022) proposes to sample from similar token-level dis- tribution that is also weighted by P(cly<t), which is defined as an approximation of the sequence- level oracle p(cly) GeDi (Krause et al., 2021) and DExperts (Liu et al., 2021) also take the weighted decoding approach but avoid enumerating the vo- cabulary for computing P(c)))c),)) by training gen- erative classifiers.
From an optimization perspective, obtaining a generation that best grounded in the knowledge while being faithful to the task instruction can be written as the equation below:
y" = arg max PLM (
(3)
Then, given the auto-regressive nature of lan- guage models, Eq. (3) can be decomposed into token-level as found in FUDGE (Yang and Klein, 2021):
97 - -arg mag 2 k). (4) Vi
Token-Level Groundedness Knowledge ground- ednss f (or hallucination in the opposite perspec- tive) is can be modeled as an entailment fact verification problem. However, to guide the generation at each step, we need to define [W k) for partially gen- erated yer- Following NADO (Meng et al., 2022), we define fyyc, k) as the approximation of future groundedness, as denoted in Eq. (5):
Constrained decoding methods focus con- straint satisfaction, such lexical constraints or hight-hand-sie coherence (Qin al., 2020, 2022). As constraint satisfaction can measured after a sequence is fully generated, search-based meth- ods (Lu et al., 2022; Chaffin et al., 2022a; Lamprier et al., 2022) that take the estimate of the future score (reward) in each decoding step has been pro- posed. Unlike weighted decoding, these methods commit to token not only based on the current token's score but also on the estimate of future rewards. This may guide the generation process towards a better reward in the end.
at the sequence level, which
y ~ P(yluct, f(yce, k) f(y,k). (5)
The KCTS Method
In this section, we introduce the framework of ICTT, which consists of a chowledge-oonstriined tree search decoding module ($4.1) and a token- level hallucination detection module, RIPA (54.2). We will also introduce a sister method Knowledge Weighted Decoding (KWD) ($4.3), which the Weighted Decoding variant using RIPA.
Our method builds on guided decoding method- ology but does not control a fixed set of attributes or lexical constraints but groundedness (faithfulness) to reference knowledge.
3 Problem Statement
4.1 Monte-Carlo Tree Search Decoding
We propose improve instruction-tunel LMs' fac- tual generation ability under a constrained decod- ing setting. The problem can be formulated as
While weighted decoding (WD) re-weighs the to- ken distribution with the score every step, it selects the most grounded to- ken in a greedy manner, which may lead subop- timal solution. This is especially problematic given that the groundedness is only well-defined after the sequence is fully generated and the guidance signal from the classifier at each step is merely an approx- imation. To this end, we propose to use Monte- Carlo Tree Search Algorithm (MCTS) (Coulom, 2007; Chaffin et al., 2022a), which can provide better estimate of the future knowledge grounded- ness through multiple simulations, have been proven effective in other scenarios such as senti- ment polarity control (Chaffin et al., 2022a,b), con- ditional generation (Chevelu et al., 2009; Scialom
PLM (yla,, k, ax),
(1)
where y is generated text, is input text with the task description, k is the piece of knowledge that y must be constrained to, and og is the attribute denoting the groundedness of y to
Let [y,,k] = P(ax = 1 k) be a function that defines the groundedness of the generation to k. Following the Bayesian decomposition in the literature (Dathathri et al., 2020; Yang and Klein, 2021). apply the Bayes rule Eq. (1) and
obtain the Eq. (2) below:
PLM (u/a, ir, k av) x (o (2)

---

al., 2021; Leblond al., 2021; Lamprier al., 2022), and alignment (Feng et al., 2023; Liu et al., 2023a).
we just evaluated back to the root. Follow- ing (Chaffin al., 2022a), we used mean ag- gregation of all simulations played after this node. This leads to
We will first define some notations used for the MCTS tree. Let the root node be the currently generated sequence Vet, each node represent a token, and p(v) be parent node of v. Let Vet be the token sequence obtained by traversing down the tree from the root the node and appending V/ce all tokens in the path except v.
V(((s:)) - N₂ VVV(((8:)) + f(Vce k)
n4.
(7)
for all 8; the path from the leaf node the root. These values will be used in Eq. (6) select the nodes in Step in the next simu- lation.
Next, the main steps of MCTS are described below in the foloowing order: Selection, Expan- sion, Rollout and Backpropagation.
These steps are repeated for predefined num- ber of simulations, then a child node of the root that has the highest visit counts gets selected as the next token and the next root of the tree.
1. Selection:
Starting from the root, we traverse the tree down until we reach a leaf node, se- lecting the children using the PUCT algo- rithm (Rosin, 2011: Silver al., 2017):
4.2 Teken-Level Hallucination Detection
We first model f as fact verification prob- lem (Thorne et al., 2018) and train binary clas- sifier [ly k)) Py(ax = 1/y, k) on the sequence- level. To adapt to token-level groundedness (y=t), previous methods trained classifier with random input sequence truncation (Yang and Klein, 2021; Chaffin et al., 2022b) or token-level label- ing (Meng et al., 2022). The random truncation approach can be sample inefficient as only a part of the input is used during training and it may add noise to the training since the input sequence may no longer contain hallucinated content after trunca- tion while still receiving negative label. Although the token-level labeling approach can be more sam- ple efficient, may correlate benign tokens before hallucinated text with hallucination label.
pfet(() V(s.) + Cpucr P(ye, (a, yea.) 1+ na
N
(6)
where (s;;) is the estimated groundedness value of node 8;, n, is the visit count of the node 8j (i.e., number of simulations after the node), and N; is the number of visit count of the parent of 8t. Cpuct is a hyperparameter that controls the trade-off between exploration and exploitation, with higher Cpuct encouraging exploration. The child node s; with higher picl(() value will be selected.
2. Expansion: If the selected leaf node is not EOS (an end-of--entence token, or a terminal state), the node is expanded in depth with & children by decoding for one step using the
LM and selecting top-k tokens as the children.
To alleviate these shortcomings, we pro- pose novel approach called Reward Inflection- Point Approximation (RIPA) to approximate fu- ture ʃ for un-finished token sequence by explic- itly providing token-level label for grounded- ness. A schematic diagram of the comparison of RIPA and previous approaches can be found in Fig- ure 1. Inspired by the "Hallucination Snowballing" effect (Zhang et al., 2023), where the language model's initial hallucination leads to further unsup- ported claims down the line, we hypothesize that identifying such an inflection point for grounded- ness is more effective upproximation of the future score. Hence, RIPA trains the classifier to iden- tify the starting point, or the inflection point, of the reward (groundedness) with token-level labels that start with 1 (positive) and become 0 (nega- tive) after the first hallucinated token. This aligns
RIPA
3. Rollout (Evaluation): from the selected leaf node s, generate until EOS using the language model, then evaluate the groundedness of the generated sequence, fy, k). Let this be the value of s, Vss) [w, k). However, such full rollout can be costly and result in high variance (Lamprier et al., 2022). Using the definition of in Eq. (5), the groundedness value of (y: k) can be approximated from the geenrrtde sequence marked by the current node. Hence, instead of performing full rollout, we propose to directly evaluate with the approximated token-level grounded- ness score: (s)) f(yc,, k).
4. Backpropagation: Then, this score is back- propagated recursively from the node that

---

Sourrcce Labsk
Given that the datasets contain diverse topics. k is highly likely to be irrelevant the con- text œ. Then, although the relevance between y and æ remains unchanged, the groundedness of on becomes negative, as is no longer based on k. All tokens in are labeled 0.
Grounabal
Tallectrotion
Tokert Labela
Token Labeling
Randoes Truscation
Partial Hallucination: Similar above, given a training example (y, x, k) ~ D, first randomly swap k with another knowledge k'. Then, randomly sample position < T. where 7 is the length of y. and truncate the response y to i'th token and obtain Vi- An LM is then asked to complete the sequence y: by sampling from PL.M (y/æ, yr, k) in zero-shot manner, by including the knowledge text & in- side the instruction. Notice that the goal here is to utilize the hallucination of LMs: hence, we sampled the completion with a tempera- ture greater than 1. This introduces more ran- domness to the generation (Zhang et al., 2021; Chang et al., 2023) and allows rare tokens to be selected more easily, which in turn ditions the LM toward hallucination (Zhang al., 2023; Aksitov et al., 2023). In this ap- proach, only the completion tokens (y>+)) are labeled as
Kaowtedge:
Discrimiaater
Tupelo, Missisipps
Elvie
bern
Califoriia
Figure I: Our proposed token-level hallucination detec- tion method RIPA. Previous token-level attribute clas- sifier training randomly truncates the input sequence for classification (FUDGE) or labels all tokens in the sequence with the same sequence-level label (NADO). RIPA explicitly labels each token with groundedness, letting the discriminator associate only the ballucination sequences with the negative label.
with our definition in Eq. (5) that the label at each position to be the expected future groundedness of the preceding sequence yise all the sequences after the first hallucination tokens include at least one hallucination token and therefore are labeled as hallucinations.
As a result, RIPA does not associate benign to- kens preceding the hallucination starting point with hallucination labels, which can lead to more sta- ble training. Additionally, it is trained to predict 0 for all tokens after hallucination is detected, which will further discount future exploration under that node in MCTS, discouraging the selection of that token. This also suggests that RIPA opens up more opportunities to explore better grounded sequences within the fixed budget of MCTS. Together, RIPA and MCTS (i.e., KCTS) provide a better estimate of Eqs. (4) and (5), and they lead to a more efficient allocation of MCTS simulation budget and better knowledge-constrainee generation.
We used balanced mixture of the two to obtain the training set. For tasks in which æ and & are indistinguishable (e.g., summarization), the prob- lem becomes P(ylk). Therefore, only the partial hallucination approach was employed. Detailed hy- perparameters we used in each task are presented in Appendix C, and the quality analysis of the partial hallucination data is in Appendix D.
4.3 Knowledge Weighted Decoding (KWD)
In addition to ICTS,, also propose sister method named Knowledge Weighted Decoding (KWD), which applies RIPA module as the guid- ance classifier to the previous decoding method, FUDGE (Yang and Klein, 2021). As discussed in Related Work, FUDGE is a weighted decoding algorithm that re-ranks the top-k tokens proposed by the language model using a classifier. They originally proposed to use classifier trained with random truncation ($4.2, also see Fig. 1), which might be inoptimal for knowledge grounded gen- eraiton. On the other hand, KWD uses RIPA as the guidance classifier module for improved guidance signal at the token-level. This method serves as a
RIPA Training Training RIPA requires fine- grained token-level annotation of hallucination, which is difficult to acquire through human annota- tion. Alternatively, take 2 simple approaches to generating synthetic data listed below.
1. Knowledge Shuffle: Given training exam- ple (y, x. k) ~ D from dataset D, randomly swap with another knowledge from the knowledge source to form a negative example.

---

bridge between previous methods and KCTS and will be used in the ablation studies.
first used Knowledge-Fi (KF1; Lian al., 2019).
which measures the unigram overlap between the generated and knowledge tokens, and K-Copy, as defined in Eq. (8),
5 Experiments Setup
this section, we will first describe the tasks se- lected for snowledge-constranee decoding ($5.1), then the evaluation metrics used ($5.2). Finally, the baselines and implementation details are provided in $5.3 and $5.4.
LD(y, k) max M1) [kl)
(8)
where LD stands for Levenshtein Distance be- tween generated response and reference knowledge string. This metric captures the amount of verbatim copies of the knowledge in a generation. The pur- pose of this metric is to monitor if the model simply copies the knowledge as a response, which defeats the purpose of using a generative LM. Hence, an ex- cessively high copy rate (e.g.. 27%% may indicate reduced utility of the response.
5.1 Datasets
To show the strength of the guided decoding method in the knowledge-grounded generation, we have selected two well-studied tasks from the litera- ture: knowledge- knowleedge-roomded dialogue and abstractive summarization. In both tasks, the language model is given piece of reference knowledge in the in- put and asked to generate a response using that knowledge.
Finally, we also utilize UniEval (Zhong et al., 2022a), multifaccted, model-based evaluator trained using Boolean QA format. For the dia- log task, we utilize Naturalness, Coherence (with dialogue context), and Groundedness (to the knowl- edge), and for summarization, we take Coherence (within the summary), Consistency (with the arti- cle), Fluency, Relevance (to the gold answer) as fine-grained evaluation dimensions. For summa- rization, we also employ MFMA (Lee et al., 2022) preitrained metric, which showed SOTA-level cor- relation with human labels CNN/DM (See al., 2017) data split of FRANK (Pagnoni al., 2021) and QAGS (Wang et al., 2020) benchmark.
Knowledge Grounded Dialogue Knowledge- grounded dialogue (KGD) can be formulated as modeling PLM (w|ær, k),, where y is the generated response, æ is dialog history, and & is the relevant knowledge. We experimented with gold knowl- edge, as the focus of this study was to show the potential of constrained decoding in knowledge grounding. We used the Wizard of Wikipedia (WoW) dataset's (Dinan al., 2019) unseen topic portion of the test set as the benchmark dataset for this task.
5.3 Baselines
Summarization Abstractive summarization can whooledge-groundee
We use popular APT-based LLMs and publicly available instruction tunddd language models of var- ious sizes as the initial baseline. We experimented with the LLMs provided through OpenAl API; namely, ChatGPT (gpt-3.55 10t-333-tturbo000000 GPT 3.5 (text-davinci-00) -003). For instruction- tuned models, studied two different sizes (XL & XXL) of the Flan-T5 (FT5) model family (Chung et al., 2022), and T0++ (Sanh et al., 2022). Note that they are not directly compared with our method due to the significant differences in terms of the model size and training cost.
be naturally considered generation task, as the generated summary should only contain the information from the reference document. fact, improving factual consis- tency in abstractive summarization is a challenging task (Pagnoni et al., 2021; Wang et al., 2020). We used CNN/DM (See et al., 2017) dataset as the benchmark dataset of our study.
and
5.2 Evaluation Metrics
We use various evaluation metrics applied for knowledge grounding and natural language gen- eration. We categorize the metrics into 3 cate- gories: token-based knowledge-basee and multi- faceted. For token-based automatic metrics, used BLEU-4 (Papineni al., 2002), Rouge- I. (Lin, 2004), ChrF (Popović, 2015), and ME- TEOR (Banerjee and Lavie, 2005), following Peng et al. (2023). For knowledge-base based metrics, we
While FT5 and TO++ models have been fine- tuned on some dialogue data, knowledge-grounded dialogue is still unscen task for these mod- els. Hence, we first gathered zero-shot results from various instruction-tuner models and experi- mented with guided decoding. On the other hand, CNN/DM summarization task is included in the TO dataset mixture and the Natural Instructions

---

K-Overlap K-Copy
Token Overlap
UniEval C
Type
Model
KFI
F1
BLEU
Rougel.
Chri
METEOR
N
G
f
ChaGGP
49.41
39.71
30.32
6.91
26.24
34,95
31,67
57.62
96.41
96.15
95.82
LLM
GPT-3.5
25.91
28.22
22.32
3.01
18.70
27.86
23.06
42.77
98.07
92.42
92.63
SET
FT5-X XL.
39.85
37.79
28.08
9.41
25.11
31.17
25.40
76.44
92.36
95.16
97.90
FT5-XL
34,50
37.07
21.18
6.81
19.64
24.88
18.53
71.69
82.21
75.70
88.75
Zero- Shot
FT5-XXL
28.20
32.33
19.11
17.55
24.15
72.37
84.24
75.51
85,89
I0++
26.94
28.80
17.57
4.13
16.14
19.84
13.37
52.79
85.26
70.14
88.61
FUDGE
55.30
54.04
29.43
11.72
27.35
31.50
26.00
73.68
88.20
83.53
94.54
Decoding Baselines
NADO
50.20
50.10
27.86
10.57
26.01
29.84
24.51
74.14
88.35
81.10
92.76
MCTS
55.54
54.21
29.56
11:69
27.48
31,60
26.08
74.54
88.16
83.90
95.07
KWD
58.19
56.58
30.71
12.74
28.27
33.40
28:10
70.27
90.51
87.86
97.54
Ours
KCTS
56.06
$1.90
30.54
11.42
27.43
35.22
28.92
62.32
92,78
91.78
98.30
Table 1: Results on WoW Test set (unseen topics). SFT stands for supervised fine-tuning, and FT' is shorthand for Flan-T5. Under the UniEvaal metrics, each letter stands for the following: N Naturalness, C Coherence, G Groundedness. For all metries, larger number preferred, except for K-Copy. Note that the performance of LLM in the upper half is for reference only. For each column, boldface denotes the best score out of the KCD methods under the FT5-XL backbone, and underline indicates the second best.
Main Evaluation
K-Overlup K-Copy
Token Overlap RougeL
UniEval
T
KF1
BLEU
C
G
f
In this section, we provide the evaluation results on the above-mentioned two tasks and conduct more analysis to demonstrate the reasons behind the suc- cess of our method.
5
48.78
48.22
10.17
25.39
90.58
85.87
90.58
10
48.24
48.05
9.98
25.87
90.22
86.41
85,43
51.49
11.07
56.06
51.90
11.42
27.43
92.78
91.78
98.30
Table 2: Ablation study on the number of initial tokens to be constrained in the knowledge with KCTS.
6.1 KGD Results
Results Analysis We report the performance of zero-shot LLMs and various ceerruttoonfinetunen models in the upper half of Table 1 and the per- formance of directly comparable decoding- -aaeed baselines and our methods in the lower half. We also studied the performance of a Supervised- FineTTuned (SFT) version of FT5-XL. for the KGD task. Note that the performance on the upper half (LLM and SFT) is only used to provide an overviewed understanding of how powerful each language models when tested WoW and are not directly compared our methods. The instruc- tions used for each model are listed in Appendix A. From the results in the upper half of Table 1, ChatGPT shows a strong ability to generate re- sponses that show high overlap with the knowl- edge. On the other hand, FT5-XL, while being the smallest in size (3B) out of all the models stud- ied, showed the best performance in generating responses that are the most grounded to the ref- erence knowledge, as indicated by the KF1 and Groundeness column of the Table Therefore, we selected - as our base model in our further studies for guided decoding..
dataset (Wang et al., 2022a), which was part of the Flan finetuning (Chung et al., 2022). There- fore, performing Knowledge-Constrainee Constrained Decod- ing (KCD) on CNN/DM test can be considered as guiding an already-finetume model to improve the factuality dimension further.
Then, apply weighted decoding (WD) & con- strained decoding baselines, namely FUDGE (Yang
and Klein, 2021), NADO (Meng al., 2022), and MCTS (Chaffin et al., 2022a), on the KCD set- ting directly, which serve as the strong baselines directly comparable to our method.
5.4 Implementation Details
To train the classifiers, we applied lightweight adapters through LoRA (Hu et al., 2021) only to the decoder layers of the language model and added a single linear layer top of the last hidden states. This only adds 0.21 % of additional training weights that can be disabled or enabled at test time, which does not hurt the rich multi-task ability of the base cntructiio-followinn model. See Appendix for more details about model training.
*ee also conducted preliminary experiment of KCD application to GPT-3.5 in Appendix B.

---

K-Overlap K-Copy
Token Overlap
Unitival
MPMA
Type
Model
KF1
F1
BLEU
RougeL
ChrF
METEOR
Coh.
Cons.
fluency
Relv.
score
CharGPT
29.43
17.92
40.45
11,75
27.85
42.96
37.66
93.85
91.67
87.15
87.11
80.62
L.MM
GPT-3.5
27.54
16.94
38.96
10.78
26.63
41.17
35.38
92.56
90.33
85.73
85.78
78.74
FTS-XL
17.04
10.18
32.21
8.74
24.02
30.27
24.47
84.82
86.02
89.90
81.28
64.55
SFT
FTS-XXL
17.45
10.42
31.55
8.43
23.38
29.95
23,91
87,
88,58
90.00
82.28
68.37
T0++
22.79
13.65
38.82
13.64
28.06
38.53
33.68
86.57
87.47
89.03
81.09
69.38
FUDGE
18.68
10.70
33.51
9.32
24.83
31.06
24.93
90.52
90.61
83.37
82.00
71.35
Decoding
NADO
20.35
11.72
35.10
10.93
26.22
33.50
27.34
92.26
93.72
88.41
84.49
72.01
Baselines
MCTS
17.86
34.59
9.00
25.85
30.90
25.12
94.30
94.28
86.51
85.90
71.28
KWD
20.39
11.63
36.24
12.30
27.20
34.25
28.46
96.24
96.64
91.60
88.48
85.11
Ours
KCTS
22.97
13.29
38.27
14.21
28.10
37.18
31.37
95.85
96.03
90.24
87.16
85.36
Table 3: Results CNN/DM Test set. The guided decoding conducted with FT5-XL model as the base model. Coh., Cons., and Relv. stand for coherence, consistency, and relevance, respectively. As the performance LLMs is for reference, we highlight the best scores on the last two groups with boldface and second- best with underline.
The comparison of baseline decoding methods and our proposed KCTS can be found in the lower half of Table 1. The penultimate group contains the results of baseline decoding methods, guided by their own proposed approximation of token-level classifiers. FUDGE and MCTS both use random truncation approximation, and NADO uses a token- level labeling approach. All decoding methods showed improvement over the nucleus sampling baseline regarding groundedness, indicated by a higher KFI score and Groundedness column of UniEval. The results also clearly show that the RIPA provides a better token-level guidance sig- nal for KCD, both KWD and KCTS outperform baselines in all dimensions except the naturalness. KCTS also resulted in the highest/ f activation, con- firming the hypothesis that MCTS, which also es- timates future rewards for token selection through simulations, can produce a higher reward.
Model
Relv.
Gr.
K-Copy
ChatGPT
3.00
2.73
2.62
0.04
FT5-XI.
2.64
2.30
1.95
0.12
Overall
FUDGE
2.82
2.35
2.19
0.21
KCTS
2.92
2.55
2.37
0.17
ChatGPT
3.00
2.75
2.60
Non -Copy
FT5-XL
2.61
2.31
1.81
FUDGE
2.78
2.40
1.97
IKC
2,91
2.61
2.24
Table 4: Human Evaluation in 3-point Likert scale WoW Test Set. F1., Relv., and Gr. stands for Fluency, Relevance, and Groundedness, respectively. The inter- rater agreement by Krippendorff alpha (Krippendorff, 2011) was 0.57, 0.46, 0.77, 0.31. Non-Copy means average scores of examples that annotators agreed the generation does not copy the knowledge.
Human Evaluation We also conducted human evaluation of the generated responses assess their quality. We randomly sampled 100 examples and asked 3 different evaluators to measure their flu- ency, relevance to the dialogue context, grounded- ness to reference knowledge, and if the response is an unnatural copy of the knowledge. The hu- man evaluation results in Table further confirm the strength of KCTS, which received better scores in all dimensions than the baselines. Furthermore, KCTS resulted in higher relevance and grounded- ness while copying less knowledge than FUDGE, which suggests that KCTS responses have higher perceived utility. The results the Non-Copy group also show that KCTS outperforms baselines even excluding the snowledge-copier responses.
Does KCTS really estimate future grounded- ness? To show that KCTS guides the future gen- eration trajectory towards a more grounded re- sponse in the end, we experimented with constrain- ing the token generation for initial T' tokens, then letting the original language model complete the sentence with nucleus sampling. The results in Table 2 indicate that the initially grounded tokens provide good context to the LM that leads to a more grounded response generation, following the intuition of using MCTS with RIPA to fulfill the definition of future groundedness f(ye,, k) R fly P(vlg<r), k). Moreover, this provides an additional performance/speed trade-off parameter.

---

Model
Fluency
Grounded
Complete
including groundedness and completeness.
ChatGPT
3.00
2.93
2.88
Conclusion
FTS-XL
2.81
2.60
2.13
FUDGE
2.89
2.90
2.31
In this work, we proposed ICTS, new guided decoding approach mhowledge-groundng natural language generation. Our token-level hallucination detection module RIPA used with MCTS decoding has shown effectiveness alogue and abstractive summarization tasks regard- ing both automatic metrics and human evaluation, surpassing previous guided decoding baselines and even ChatGPT in some metrics. Applying ICTS only the first few tokens also proved helpful in the knowledge-grounee other dimension of performance/speed trade-ofT.
KCTS
2.95
2.97
2.40
Table 5: Human Evaluation on CNN/DM. This follows 3-point Likert scale, with agreement alpha of 0.35, 0.44, and 0.19.
srowledge-groundee di-
6.2 Summarization Results
Results Analysis We used the same models as in $6.1. From the results found in Table 3, it can observed that ChatGPT again outperforms other models in most dimensions, except for BLEU and Rouge metrics. On the other hand, the instruction- tuned models show different trend than with the KGD setting; TO++ model outperforms FT5 mod- els. presumably because Flan finetuning introduces vast amount of additional tasks than TO finetun- ing dataset mixture, leading to a deteriorated per- formance in some of the tasks. This finding aligns with our motivation for not finetuning the base LM directly for knowledge grounding.
generation, adding an-
Limitations
One limitation of the approach is additional compu- tational cost and inference time. For NADO, which computes f(y-c,, k) for all V once, requires two forward passes (1 for LM, for discriminator) per token. FUDGE needs to enumerate the top-k tokens to feed into the discriminator, taking & dis- criminator forward passes per token. For KCTS (MCTS), it takes N LM and discriminator forward passes each to perform N simulations per token. However, assert that the generation speed and groundedness are in a tradeoff relationship, the or N can be reduced for a faster generation. future work, the time complexity can be further reduced by training a smaller auxiliary weight for discriminator or employing carly-stopping heuris- tics to reduce the number of simulations of MCTS decoding (Baier and Winands, 2012). As this study focused on showing the constrained-decodiin methods' strengths in the -edge-groundedd generation, we did not con- sider knowledge retrieval and experimented with gold knowledge. Constraining on retrieved knowl- edge can be considered to test in more realistic deployment scenario.
For efficiency, have continued to use the FT5-XL model throughout guided decoding ex- periments with the summarization task. KCTS again showed superior performance over the base- line methods, with significant improvements in to- ken overlap and MFMA. RIPA-guided decoding also outperformed all bascline methods in UniEval metrics, with K KWD showing slightly better per- formance than KCTS. This might be partially at- tributed to KCTS having higher knowledge overlap with the original article (KF1= 22.97) than KWD (KFI 20.39), which may suggest the summaries were "less abstractive" and situated in the out-of- distribution of the training data of the UniEval model. Overall, knowledge-bar based, reference-based, and learned metrics consistently support the effec- tiveness of KCTS.
Human Evaluation We have randomly selected 50 samples for human evaluation, with the same 3 human evaluators from the KGD task. The eval- uators asked to evaluate the summaries in dimensions:
ChatGPT was generally preferred over smaller instruction tuned models during human evaluation, even with KCD applied. However, since the details about training data, model architecture, mecha- nisms behind ChatGPT are not public, we cannot ascertain that this is fair comparison. Moreover. as our method is model-agnostic, it could also be applied to large language models by those with access to further improve them.
fluency, groundedness, and complete- ness. Groundedness is analogous the precision of knowledge, while completeness similar to recall, It can be observed from Table 5 that the evaluators preferred KCD over the baselines in all dimensions,

---

Ethics Statement
We would also like to thank the Turing Al Com- puting Cloud (TACC) (Xu et al., 2021) and HKUST iSING Lab for providing us computation resources on their platform.
Our experiments are conducted on two datasets, namely the Wizard of Wikipedia (WoW) dataset (Dinan et al., 2019) for Knowledge Grounded Dialogue and CNN/DM (See et al., 2017) dataset for abstractive summarization. The knowledge part in WoW is retrieved from Wikipedia, which is open-access, and the project* is under MIT license without any copyright issues. CNN/DM a well-studied summarization dataset that crawled from CNN and Daily Mail news, where the data is under an apache-2.0 license that is safe to use for research studies. Both datasets do not involve privacy problems about any specific entities (e.g.. person or company) and are widely used in NLP research papers.
References
Renat Aksitov, Chung-Ching Chang, David Reitter, Sia- mak Shakeri, and Yunhsuan Sung. 2023, Character- izing attribution and fluency tradeoffs for retrieval- augmented large language models.
Kushal Arora, Kurt Shuster, Sainbayar Sukhbaatar, and Jason Weston. 2022. Director: Senerator-classifie for supervised language modeling. In Proceedings of the 2nd Conference of the Asia-Pucific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference Natu- ral Language Processing (Volume I: Long Papers), pages 512-526, Online only. Association for Compu- tational Linguistics.
Human evaluation was performed by 3 post- graduate NLP researchers with least I year of experience in the field to ensure quality. Two of the three annotators employed for the human eval- uation were authors of the paper, and the third an- notator was another postgraduate student in NLP within the same institution for broader and more objective perspective. The authors' involvement in the anmotation process was part of their academic responsibilities, and no additional compensation was provided. The third annotator was compen- sated for their time and effort at the hourly rate equivalent to 7.65 USD/hr, which was in line with the university guidelines and higher than the local law of minimum wage.
Akari Asai, Sewon Min, Zexuan Zhong, and Danqi Chen. 2023. Retrieval-based language models and applications. In Proceedings of the 61 Annual Meet- ing of the Association for Computational Linguistics (Volume 6: Tatorial Abstracts), pages 41-46, Toronto, Canada. Association for Computational Linguisties.
Hendrik Baier and Mark H. M. Winands. 2012. Time management for monte-carlo tree search in go. In Advances in Computer Games, pages 39-51, Berlin, Heidelherg. Springer Berlin Heidelberg.
Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with im- proved correlation with human judgments. In Pro- ceedings of the ACL Workshop on Intrinsic and Ex- trinsic Evaluation Measures for Machine Transla- tion and/or Summarization, pages 65-72, Ann Arbor, Michigan, Association for Computational Linguis- tics.
Our method focuses on factual natural language generation in terms of groundedness to the refer- ence knowledge. It does not verify the factual correctness of the provided reference, so if the knowledge source contains non-factual informa- tion, KCD is likely to also generate misinformation. We urge the users to verify the knowledge source carefully before usage.
Yejin Bang. Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do. Yan Xu, and Pascale Fung. 2023. A multitask, multilin- gual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.
Acknowledgements
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katic Milli- can, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoe, Aidan Clark, Diego De Las Casas, Aurelia Guy. Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang. Loren Mag- giore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings
The authors of this paper were supported by the NSFC Fund (U20B2053) from the NSFC of China, the RIF (R6020-19 and R6021-20), and the GRF (16211520 and 16205322) from RGC of Hong Kong. We also thank the support the UGC Research Matching Grants (RMGS20EG01-D. RMGS20CRI1, RMGS20CR12, RMGS20EG19, RMGS20EG21, RMGS23CR05, RMGS23EG08).
Shtpss//huggingiiae...odaaasssss dailymail

---

of Machine Learning Research, pages 2206-2240. PMLR.
Tim Dettmers, Mike Lewis, Sam Shleifer. and Luke Zettlemoyer. 2022b. 8-bit optimizers via boic-wwise quantization. 9th International Conference on Learn ing Representations, ICLR.
Antoine Chaffin, Vincent Claveau, and Ewa Kijak. 2022a. PPL-MCTS: Constrained textual generation through discriminator-guien Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 2953-2967, Seattle, United States, Association for Computational Linguistics.
MCTS decoding. In
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of Wikipedia: Knowledge-powers agents. In Proceedings of the International Confer- ence on Learning Representations (ICLR).
conversational
Xidong Feng, Ziyu Wan, Muning Wen, Ying Wen, Weinan Zhang, and Jun Wang. 2023. Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309. 17179.
Antoine Chaffin, Thomas Scialom, Sylvain Lamprier, Jacopo Staiano, Benjamin Piwowarski, Ewa Kijak, and Vincent Claveau. 2022b. Which discriminator for cooperative text generation? In SIGIR '22: The 45th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, Madrid, Spain, July 11 15, 2022, pages 2360-2360 2365. ACM.
Thibault Févry, Livio Baldini Soares, Nicholas FitzGer- ald, Eunsol Choi, and Tom Kwiatkowski. 2020. En- tities as experts: Sparse memory access with entity supervision. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 4937-4951, Online. Association for Computational Linguistics.
Thung-Ching Chang, David Reitter, Renat Aksitov, and Yun-Hsuan Sung. 2023. Kl-divergence guided tem- perature sampling.
Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian Hc. 2023. Felm: Benchmarking factuality evaluation of large language models.
Robert M. French. 1999. Catastrophic forgetting in con- nectionist networks. Trends in Cognitive Sciences, 3(4):128-135.
Saibo Geng, Martin Josifosky, Maxime Peyrard, and Robert West. 2023. Flexible grammar-baa based con- strained decoding for language models.
Jonathan Chevelu, Thomas Lavergne, Yves Lepage, and Thierry Moudenc. 2009. Introduction of a new para- phrase generation tool based on Monte-Carke sam- pling. Proceedings of the ACL-IJCNLP 2009 Con- ference Short Papers, pages 249-252, Suntec, Singa- pore. Association for Computational I. Linguistics.
Hangfeng He, Hongming Zhang, and Dan Roth. 2022. Rethinking with retrieval: Faithful large language model inference.
Hyung Won Chung, Le Hou, Shayne Longpre. Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu, Zhuyun Dai. Mirac Suzgun, Xinyun Chen, Aakanksha Chowdh- ery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu. Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Fed H. Chi, Jeff Dean, Ja- cob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling instruction-finetune finetuned language models.
Tianxing Hc, Jun Liu, Kyunghyun Cho, Myle Ott. Bing Liu, James Glass, and Fuchun Peng. 2021. Analyzing the forgetting problem in pretrain-linetuning of open- domain dialogue response models. In Proceedings of the 16th Conference of the European Chapter of the Association for Compulational Linguisties: Main Volume, pages 1121-1133, Online, Association for Computational Linguistics.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhi Yuanzhi Li. Shean Wang. Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.
Rémi Coulom. 2007. Efficient selectivity and backup operators in monte-carlo tree search. In Comput- ers and Games, pages 72-83, Berlin, Heidelberg. Springer Berlin Heidelberg.
Gautier Izacard and Edouard Grave 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation Computational Linguistics: Main Volume, pages 874-880, Online. Association for Computa- tional Linguistics.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach controlled text generation. In International Conference on Learning Representa- tions.
Ziwei Ji. Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023a. Survey of halluci- nation in natural language generation. ACM Comput. Surv, 55(12).
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022a. Llm.int80:: 8-bit matrix mul- tiplication for transformers at scale, arXiv preprint arXiv:2208.07339.

---

Ziwei Ji. Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023b. Survey of halluci- nation in natural language generation. ACM Comput. Surv. 55(12).
Rémi Leblond, Jean- rean-Baptiste Alayrac, Laurent Sifre, Miruna Pistar. Lespiau lean-Baptiste loannis Antonoglou, Karen Simonyan, and Oriol Vinyals. 2021. Machine translation decoding beyond beam search. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 84 10-8444, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, and Pascale Fung. 2023c. RHO: Reducing hallucination in open-domain dialogues with knowledge grounding. In Findings of the As- sociation for Computational Linguistics; ACL 2023, pages 4504-4522, Toronto, Canada, Association for Computational Linguistics.
Hwanhee Lee, Kang Min Yoo, Joonsuk Park, Hwaran Lee, and Kyomin Jung. 2022. Masked summariza- tion to generate factually inconsistent summaries for improved factual consistency checking. In Find- ings of the Association for Computational Linguis- tics: NAACL 2022, pages 1019-1030, Scattle, United States. Association for Computational Linguistics.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Fdunnv, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 6769-6781, Online. Association for Computational Linguistics.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2020. serrieval-uugmentedd generation for knowledge- intensive nlp tasks. In Advances in Neural Infor- mation Processing Systems, volume 33, pages 9459- 9474. Curran Associates, Inc.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for control- lable generation.
Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, and Hua Wu. 2019. Learning select knowledge for response generation dialog systems. In Proceed- ings of the Twenty-Eighth International Joint Con- ference Artificial Intelligence, IJCAI-19, pages 5081 5081-5087. International Joint Conferences on Arti- ficial Intelligence Organization.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Baaaska-Barwiiska, Demis Hassabis, Clau- dia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(1333521-3522
Chin-Yew Yew Lin. 2004, ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, 1, Barcelona, Spain. Association for Computational Linguisties.
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher. and Nazneen Fatema Rajani. 2021. GeDi: Gener- ative discriminator guided sequence generation. In Findings of the Association for Computational Lin- guisticx: EMNLP 2021, pages 4959-4952, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Mcasuring how models mimic human falsehoods. In Proceedings of the 60th Anmual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214 14-3252, Dublin, Ireland. Association for Computational Linguistics.
Klaus Krippendorff. 2011. Computing krippendorff's alpha-reliability.
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta. Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021. DExperts: Decoding- time con- trolled text generation with experts and anti-experts. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume I: Long Papers), pages 66791,,06, Online. Association for Computational Linguistics.
Sachin Kumar, Biswajit Paria, and Yulia Tsvetkov. 2022. Gradient- based constrained sampling from language models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages
Abu Dhabi, United Arab Emirates. Association for Computational Linguisties.
Sylvain Lamprier, Thomas Scialom, Antoine Chaffin, Vincent Claveau, Ewa Kijak, Jacopo Staiano, and Benjamin Piwowarski. 2022. Generative coopera- tive networks for natural language generation. In Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pages 11891-11905. PMLR.
Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, and Asli Celikyil- maz. 2023a. Making ppo even better: Value Value-guided monte-carlo tree search decoding.
Nelson F. Liu, Tianyi Zhang, and Percy Liang. 2023b. Evaluating verifiability in generative search engines.

---

Xin Liu, Muhammad Khalifa, and Lu Wang. 2023c. BOLT: Fast energy-based controlled text generation with tunable biases. In Proceedings of the 6 An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Pupers), pages 186-200, Toronto, Canada. Association for Computational Lin- guistics.
Long Ouyang, Jeff Wu, Xu Jiang. Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback.
Ximing Lu, Sean Welleck, Peter West, Liwei Jiang. Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lian- hui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith, and Yejin Choi. 2022. NeuroLogic eseuqu decoding: Constrained text generation with lookahead heuris- tics. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies. pages 780-799, Seattle, United States. Associa- tion for Computational Linguistics.
Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. 2021. Understanding factuality in abstrac- tive summarization with FRANK: A benchmark for factuality metrics. In Proceedings of the 2021 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 48 1812-4829, Online. As- sociation for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311-318, Philadelphia. Pennsylvania, USA. Association for Computational Linguistics.
Ximing Lu, Peter West, Rowan Zellers. Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Neuro- Logic decoding: (un superviedd neural text genera- tion with predicate logic constraints. In Proceedings of the 2021 Conference of the North American Chap- ler of the Association for Compulational Linguisties: Human Language Technologies. pages 4288 4288-4299. Online. Association for Computational Linguistics.
Baolin Peng. Michel Galley, Pengcheng Hc, Hao Cheng, Yujia Xie. Yu Hu, Qiuyuan Huang, Lars Liden. Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feed- back.
Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Zero-resource black-box hal- lucination detection for generative large language models.
Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 992-395, Lisbon, Portugal. Association for Computational Linguistics.
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, and Sayak Paul. 2022. Pefi:: State- of-the-art art parameter-effice efficcient finettnnig methods. https:/ //githu ctt.coo/////// www.hhgggggggggggggggggggggeeeet
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan Mc 2020. On faithfulness and factu- ality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906-1919, On- line. Association for Computational Linguistics.
Lianhui Qin. Vered Shwartz, Peter West, Chandra Bha- gavatula, Jena D. Hwang. Ronan Le Bras, Antoine Bosselut, and Yejin Choi. 2020. Back to the future: Unsupervised backprop- based decoding for counter- factual and abductive commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 794 805, Online. Association for Computa- tional Linguistics.
Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang. 2022. Controllable text generation with neurally- decomposed oracle. In Advances in Neural Informa- tion Processing Systems, volume 35, pages 28125- 28139. Curran Associates, Inc.
Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. 2022. COLD decoding: Energy-basee based con- strained text generation with langevin dynamics. In Advances in Neural Information Processing Systems.
Grégoire Mialon, Roberto Dessi, Maria Lomeli, Christo- foros Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi- Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. 2023, Augmented language mod- els: a survey.
Alec Radford, Jeff Wu, Rewon Child, David Luan. Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Hannah Rashkin. David Reitter, Gaurav Singh Tomar, and Dipanjan Das. 2021. Increasing faithfulness in knowledge knowledge-grounded dialogue with controllable features. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- ral Language Processing ( (Volume I: Long Papers),
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit Tyyer, Luke Zettlemoyer, and Hannanch Hajishirzi. 2023. Factscore: Fime-grained atomic evaluation of factual precision in long form text generation.

---

pages 704-718, Online. Association for Computa- tional Linguistics.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784-3803, Punta Cana, Do- minican Republic. Association for Computational Linguisties.
Vikas Raunak, Arul Menezes, and Marcin Junczys- Dowmunt. 2021. The curious case of hallucinations in neural machine translation. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1172-1188 Online, Association for Computational Linguisties.
David Silver, Julian Schrittweserr, Karen Simonyan. Toannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap. Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the game of go without human knowledge. Nature. 550(7676):354-359
Stephen Robertson and Hugo Zaragoza. 2009, The probabilistic relevance framework: Bm25 and be- yond. Foundations and Trends in Information Re- trieval, 3:333-389.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018, Object hallu- cination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035-4045, Brussels, Belgium. Association for Computational Linguistics.
Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang. Yushi Hu, Mari Ostendorf, Wen tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2022. One embedder, any task: Instruction finetuned text em- beddings.
Christopher D. Rosin. 201 I. Multi-armed bandits with episode context. Annals of Mathematies and Artifi- cial Intelligence, 61(3):203-22) -300.
Yi Tay. 2023. A new open source flan 20b with ul2.
James Thorne, Andreas Vlachos, Christos Christocdoulopoulls,, and Arpit Mittal. 2018. FEVER: large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguisties.
Ohad Rubin and Jonathan Berant, 2023. Long-range language modeling with self-vetrieval.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach. Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani. Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang. Mattco Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang. Tr- ishala Neeraj, Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Techan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In International Conference on Learning Representations.
Pat Verga, Haitian Sun, Livio Baldini Soares, and William Cohen. 2021. Adaptable and interpretable neural MemoryOver symbolic knowledge. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies. pages 3678-3691, Online. Association for Computational Linguistics.
Alex Wang, Kyungbyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the fac- tual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages ciation for Computational Linguistics.
Thomas Scialom, Paul-Alexis Dray. Jacopo Staiano, Sylvain Lamprier, and Benjamin Piwowarski. 2021. To beam not to beam: That is question of co- operation for language gans. Advances in Neural Information Processing Systems, volume 34, pages 26585-26597. Curran Associates, Inc.
Online. Asso-
Yizhong Wang, Swaroop Mishra, Pegah Alipoormo- labashi, Yeganch Kordi, Amirreza Mirzaci, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasckaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Puro- hit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel. Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Necraj Varshney. Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A. Sumanta Patro, Tanay Dixit, and Xudong Shen. 2022a. SuperrNaturallnsttuution General- ization via declarative instructions on 1600+ NLP
Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer- generator networks. In Proceedings of the 55th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 1073- 1083, Vancouver, Canada. Association for Computa- tional Linguistics.
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. 2023. Replug: eetrieval-augmeent black-box language models.

---

tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5085-5109, Abu Dhabi, United Arab Emirates. Association for Computational Linguisties.
Zexuan Zhong, Tao Lei, and Danqi Chen. 2022b. Train- ing language models with memory augmentation. In Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing, pages 5657-5673, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Zhaowei Wang, Hongming Zhang, Tianqing Fang. Yangqiu Song, Ginny Wong, and Simon See. 2022b. Subeventwriter: Iterative sub-event sequence genera- tion with coherence controller. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1590-1604
A Instruction Templates
The instruction used for different models and tasks are listed in Table For ChatGPT with KGD task, we used the chat completion API, where each dialogue turn is separated and formatted as user/assistant message, and the instruction was given as the system message at the end.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer. Patrick von Platen, Clara Ma. Yacine Jernite. Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 45, Online. Association for Computational Linguistics.
B Application to LLMs
As a preliminary study, we have experimented with applying knwwledge--onsstrineed decoding on LLMs, GPT-3.5 text-davinci-000) in our study, One limitation of the OpenAI API is that it does not return the token probability distribution over the whole vocabulary; most top-5 log probabilities are returned. This significantly limits the search space for all WD methods, which may reduce the ability to guide the generation toward the objec- tive. Hence, we propose a new method called Pre- KWD where we use a proxy model to propose top-k tokens first, re-rank the tokens with RIPA, then include it in the API request in the logit bias field, which is added to the logit of the LLM before sampling. This can be denoted as:
Kaiqiang Xu, Xinchen Wan, Hao Wang, Zhenghang Ren, Xudong Liao, Decang Sun, Chaoliang Zeng, and Kai Chen. 2021. Tace: A full-stack cloud computing infrastructure for machine learning tasks. arXiv preprint
10.00156..
Yan Xu, Deqian Kong, Dehong Xu, Ziwei Ji, Bo Pang. Pascale Fung, and Ying Nian Wu. 2023. Diverse and faithful knowledge-groundee dialogue generation via sequential posterior inference. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org.
Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 351 3-335,, Online. Association for Computational Linguistics.
Z - Z. + + log f( log
k)]
(9)
Where Zi is the logit of token i, Z is the logit of the base LLM, and Ž is the logit of the smaller proxy model. a is another hyper- parameter that controls the strength of logit bias. Since GPT2 (Radford et al., 2019) shares its vo- cabulary with GPT3 family, Z was computed with GPT2-XL.
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. 2021. Trading off diversity and quality in natural language generation. Proceed- ings of the Workshop on Human Evaluation of NLP Systems (HumEval), pages 25-33, Online. Associa- tion for Computational Linguistics.
We have randomly sampled 100 examples from the WoW test set for this experiment. The results in Table 7 shows that applying post- -uidduuidnnee does not improve much, as the search space is limited: without sufficient width, the generation is bound to what the base model believes. On the other hand, although the overlap between tokens pro- posed by the proxy model and actual token distri- bution is unknown, the empirical results suggest
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023. How language model hallucinations can snowball.
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022a. Towards unified multi- dimensional evaluator for text generation. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 2023- 2038, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
'I turn, we call the original KWD as Post-KWD here.

---

Model
Task
Instruction
##* Document:
ARTICLE
Summarization
FT5, TO, GPT3.5
Given the article, generate faithful summary.
History:
DIALOG
Knowledge:
KGD
KNOWLEDGE
Given the dialog history and relevant knowledge. generate knowiedgeable, useful, and helpful response.
Summarize the following text:
Summarization
ChatGPT
ARTICLE
(content:
role: user)
(content: turn 2, role: assistant)
KGD
Use following knowledge, but directly copy, generate concise response: "KNOWLEDGE", role: system)
(content:
Table Instruction templates for different models for different tasks.
K-Overlap
Token Overlap Rougel.
UniEval
Decoding
K-Copy
BLEU
Chrl
METEOR
GPT-3.5
25.75
28.40
23.71
3.91
28.53
24.46
40.42
98.70
94.19
Post-KWD
26.94
29.53
23.80
3.41
19.78
28.58
24.32
45.62
97.80
94.12
Pre-KWD
27.44
29.15
23.86
3.91
19.93
28.02
23.64
39.24
98.90
94.43
Pre+Post-KWD
27.92
30.51
24.00
4.00
19.96
28.83
23.97
41.43
98.72
95.18
Table GPT3.5 KWD on 100 random examples from WoW test set (unseen topics).
that this method can successfully add bias toward tokens that are grounded reference knowledge. Finally, using both pre-guidance and regular post- reweighting together can result in the most faithful generation.
applied repetition penalty (Keskar et al., 2019) of 1.2 for MCTS following the original implementa- tion.
The synthetic data generated has the following statistics: for WoW, 8,832 partial hallucination ex- amples generated using FT5-XL in zero- shot manner, with temperature T 1.4 encour- age hallucination. 10,000 knowledge shuffle exam- ples were also sampled, along with 20,000 original examples, leading to balanced mixture of 20k pos- itive examples and 18.8k negative examples. For CNN/DM, 12,811 partial hallucination negative ex- amples were generated using the same procedure. The final dataset included 13,180 positive exam- ples as well. We then applied random 9:1 split obtain the training and test sets.
Limitations Applying KWD to LLM incurs ((T) times more cost, where 7' is the number of generated tokens. This is because we need to query the API 1 token at time, leading to redundant computation of the prompt tokens. This be easily mitigated with attention key-value caching by the API provider, which we hope to be enabled the future.
Implementation Detail
The response length was set to 64 tokens for sum- marization and 32 for KGD. For nucleus sampling, top-p was 0.95 with temperature= 1, which also ap- plies OpenAl models. For all decoding methods studied, we applied top-k filtering with k 50. In addition, in NADO, the constraining factor a was set to 0.25, and in MCTS, the constant Cpuct = and the number of simulations N = 50. We also
All experiments were conducted with NVIDIA GPUs with CUDA 11.7 with CuDNN7.52 enabled. We used either RTX A6000 48GB or RTX 3090 24GB GPUs. All classifier training was performed with an effective batch size of 64 for KGD and 32 for summarization for 2000 steps. For efficiency, we loaded the models in 8-bit quantization with

---

bitsandbytes (Dettmers et al., 2022a,b), both during training and inference.
Ascuracy
All implementations were based on the hug- gingface transformers (Wolf et al., 2020) and peft (Mangrulkar et al., 2022) library. We also utilized evaluate' library for metric implemen- tations. All the pretrained model weights were downloaded from huggingface hub
Rercan ("UOE)
(46.0)
Assurssy
0.9
0.85
0.8
D Analysis on Partial Hallucination Data
0.76
CNNDM
WiW
Dstaset
In this section, evaluated the utility of the gen- erated Partial Hallucination data in two aspects: hallucination success and relevance to practical de- coding scenarios.
Figure 2: Classitier performance test split of pseudo- data generated for CNN/DM and WoW.
To evaluate the hallucination success rate, we compared the groundedness the partial hallucination data through UniEval. The score drop was from 0.95 0.63 (KGD grounded- ness), and from 0.88 to 0.36 (Summarization Con- sistency), which suggests that our silver-standard partial hallucination label is, to a large extent, valid.
From the results in Figure 2, we can first observe that f achieves very high accuracy, suggesting that its good validator of groundedness defined by the dataset. RIPA closely follows f, outperform- ing other token-level classifiers. The random trun- cation approach shows suboptimal performance. which aligns with our hypothesis that this approach may add noise during training. Token-labeling ap- proach from NADO shows the lowest accuracy, which can be attributed to their classifier design not being standard binary classifier; their classi- fier outputs V independent output scores Z; such that Z, = f(y<r., k), where is vocab size, and Y/4 the i-th token in the vocabulary. This can be interpreted as having V different independent classifiers. While this allows for faster inference, some of the classifier may receive far less number of training examples depending on the token dis- tribution in the training dataset. Moreover, since this approach labels all tokens with the same la- bel, it also suffers from the problem of associating benign tokens before hallucination with negative label. RIPA, on the other hand, alleviates this prob- lem and consequently achieves the best accuracy out of the token-level classifiers.
of the original to
Another concern is that the tighttemperature sampling used to generate partial hallucination data may lead to sequences that the model does not nor- mally generate. Then, the discriminators trained on this dataset may not be exposed the usual tokens generated by the LM and diminish their utility guided decoding. To ascertain the rele- vance of the hallucinated sequences, we examined if the sampled tokens with high temperature fall into the top-50 most probable tokens of our base LM (flan-t5-x1), this the search width for each step of KWD and KCTS. It was found that 68% of the sampled tokens for summarization (78% for Knowledge Grounded Dialoguc) are included in the top-50, which suggests that the synthetic negative data generated using high temperature are relevant to the real use-case.
E Classifier Performance
F Generated Examples
We report the performance of the knowledge- groundedness classifiers used for decoding on the test split of the pseudo-dataset generated in Fig- ure 2. Notice that both dialogue and summarization datasets are balanced (Appendix C), so comparing accuracy would suffice. We also included the accu- racy of the sequence-level groundedness classifier f for reference.
We included some example outputs from each method for the case study for KGD in Table and summarization in Table 9. Example 1&2 of Table 8 show the strength of our approach. In Example 1, while the FT5-XL fails to address the knowledge entirely and FUDGE outputs "sweater" which is not mentioned, ICTS uses the knowledge in a nat- ural way to generate the response. ChatGPT also generates well-grounded response. In example 2,
'haps://hugginghaeeccodecss coldesseevalutte//nnee Shanss//hugginggfecccoommdddd

---

Context
hey, do you like cardigans?
Knowledge
more modern version the garment has no huttons and hangs open by design.
ChatGPT
Yes, I do like cardigans. particularly like the more modern version that has no buttons and hangs open by design.
Example
FT5-XI.
love cardigans.
FUDGE
A cardigan sweater with buttons.
KCTS
like cardigans, but don't It like buttoned version.
Context
love dogs! my favorite breed the goldendoodle! they're crossbreed between and golden retriever and poodle.
have seen one before and they are so cute.
they are so fluffy and cute right?! the original purpose of cross was actually to develop guide dogs suitable those with allergies dogs! has been successful for that purpose?
Knowledge ChatGPT
their coats do not shed, which reduces dander.
Example
Yes, the goldendoodle's coat doesn't 't shed much, making them great for people who suffer from allergies. The breed's 's hypoallergenic coat means...
FTS-XI.
That's true!
FUDGE
Yes, they are very popular with people with allergies.
KCTS
Yes, they are great breed for people with allergies because their coats do not shed which reduces the amount of dander that can bc...
Context
i drink protein powder with nothing but water. really? have you ever tried fish protein powder? sounds gross, but its made for humans to drink.
does it smells and taste good. maybe could give trial. am trying to gain muscles here.
Knowledge
fish protein powders have various sanitary processing, purity and functional characteristics which establish them human food ingredients.
Example 3
ChatGPT
Fish protein powder actually popular supplement among bodybuilders due to its high protein content and amino acid profile. made through sanitary processing method ensure...
FTS-XL
yes it smells very good and tastes very good.
FUDGE
yes, it is good source of protein.
KCTS
yes, it is good source of protein and has good taste.
Table 8: Example generations selected from WoW test set.
FT5-XL. is not very informative, FUDGE makes unsupported claim about goldendoodle's 's popular- ity, while KCTS generates the most knowledgeable response. On the other hand, Example 3 shows the failure case for all models. While the provided knowledge is about "fish protein powders" being sanitary, ChatGPT mentions high protein content and other models mention taste.
Table 9 includes two success cases for KCD. ChatGPT response tends to be more detailed, which may contribute to having higher completeness in human evaluation, but can be not sufficiently con- cise to serve as a good summary. FT5-XL may gen- erate seemingly unrelated comments, and FUDGE can degenerate in fluency during token re-ranking. KCTS generates the most concise and faithful sum- mary that captures the core of the article.

---

Article
(CNN/Their relationship led time ber, Mary Kay Letourneau Faalsuu wants world know that she and her husband, Vili Fualsau, are still happily togetheer, She was mamried 34-ycar-old teacher mother of
Scattle in 1996 student. I Leinumem guve hirth to her young lnver's child and went on to serve more than seven years in prison charges related to their sexual relationship. interview with the couple, who will celebrate their 10th wedding anniversary on May 20. The pair wed soon after she was released from prison in 2005 and are now the parents of two teen girls. Acconding Barbura Walters, "Mary Kay Waltets
she bogan allair with Huslamn,
13-ycar-old
Friday, ABC's "20/2"" will air exclusive
ABC, during the interview work in spite of their huge difference in their age. "She also discusses her surprising plans to tench her
makes their marriage
registered offender lifted," posting the network's is site says. "Vili Fualan,, meanwhile, discusses his bouts with alcoholism, depression and why be belseves the system failed him while hushand 31.
again, well her intentions
Exumple
still minor." Letournewu Fualanu now 53, and ber
ChaiGPT
Mary Kay Lekoureeu Fualauu and her husband, Vili Fualaau, are still huppily together and are celebrating their wedking anniverstry May 20. Letoumenu Fualarn served seven years jail for having an affair with Fualaan, her 13 (3-year old student,
FTS-XL
It's been more then 10 years, and no. that's a big difference in age.
FUDGE
teacher mather who was teen them teen ager.
A
in began affair with student, Fualaon,
KCTS
Mary Kay Letournema Fualaon, who gave birth young. lover's child and than seven years in prison, will tell Barbara Walters how she and her husband, Vili Hualam, are still happily together.
Article
Tim Sherwood's 's penchant for giket put the Villa sense and isn't hard to imagine what he thought about Carles N' choice of attire on Thesday. The Villa wide turned for training at the club's Bodymoer Ileuhh complex wearing outrageoudly flamboyant shirt and tronser combo. Shay Given cortainly wase't a of N' Zogbia's flowery ensemble, made worse by his decision wear sunglasses in the dressing mom. Charles N Zoghia poses in his flowery
the spotlight
Aston Villa training on Tuesday N "'Zgbiia has leatured regularly under Sherwood since his appointment Februsry. N'Zogbia is no stranger to garish outfit as seen here in a camoutlage jump
Sherwood became Famous wearing a gilet during time Tortehha manager last season. The Asion Villa goalkeeper took to Twitter to ridicule the Frenchman's choice of altire describing
the worst ever'. The Irvhnd insernational posted picture of Zogbia the social networking ing site writing 'OMG get on Charles Voohhii's gear. fhoo #Worstever N N'Zoghia has featured regularly under Sherwood since he was appointed in February, but after catching glimpse former Newcastle
Esample 2
his outlit may have bad few choice words clear of the Premier Leagne relegation zone after Saturday's 's 1-0 win against Tottenham at White Hart Lane and face Liverpool the FA Cup semi final Wembley
Villa moved points
weekend.
CharGPT
Aston Villa footballer Carles N Zoggia caused stir when nimed up to training wearing on outragoous floral shirt and trousers combo, topped off with sunglasses the dressing room, Even poalkceper Shay Given ridiculed
Twitter, calling "the worst ever'.
FTS-XL
Villa keeper Shay Given trolled Carles N Zogobii's flowery outlit.
FUDGE
garish autfit maining Tuesday and Tim Sherwood dammed "hee worst ever' gealkeeper Shay Given.
Aston Villa's Carles a'ggg
KCTS
Aston Villa's ' Carles N Zoggga nimed up for training on Thesday wearing flowery shirt and trouser comba, Shay Given took Twitter ridicule Frenchman's chaice attire describing the "worst ever'.
Table 9: Example generations selected from CNN/DM test set.


Reviews:
[{"review_id": "iLCVBjMDeM", "reviewer": "Reviewer_rGKp", "summary": "", "questions": "Eq (5): going from string to probability in the function f would require some more explanation or more specific pointers to the NADO paper sections.\n\nIn the Rollout paragraph of section 4.1, what is the criteria to stop when generating partial sequences, if EOS is not used since it is too costly?\n\nI am not sure about the strong statement at line 344 that RIPA does not associate benign tokens with hallucination labels, since facts that appear after an hallucinated token may still be grounded (e.g., Barack Obama is Norwegian and his wife is Michelle Obama).\n\nWhen training RIPA, how can you guarantee that the examples obtained with the Partial Hallucination method have the last part that is actually based on the provided knowledge? What if the model does not use the knowledge and outputs something from its own knowledge? I imagine that using high temperature to create more random sequences is yielding examples which are very different from what would be sampled during decoding with the normal temperature.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed feedback. The following paragraphs respond to the reasons to reject together with the question.\n\n**Additional explanations to RIPA**\n\n*1. Guarantee of hallucination in partial-hallucination data*\n\nThe intuitive guarantees lie in two aspects. First, as the knowledge is randomly sampled, and both of our datasets contain diverse topics, it’s highly likely the random knowledge is irrelevant to the context. In this sense, if the LM uses the knowledge provided, it’s expected to generate ungrounded outputs, i.e., hallucination. Second, high temperature sampling in decoding will bring larger randomness, leading to more irrelevant content. This phenomenon has been reported in the NLG literature (Zhang et al., 2020, Chang et al., 2023) where larger temperature brings randomness and diversity while sacrificing quality. While the LM may contain parametric knowledge about the context, high temperature allows more rare tokens to be easily selected, which in turn conditions the LM toward hallucination, as pointed out in Zhang et al. (2023) and Aksitov et al. (2023).\n\nIt is a valid concern that high temperature sampling may lead to sequences that the model does not normally generate. We consider this to be a trade-off between hallucination success rate and quality. Moreover, we checked if the sampled tokens with high temperature fall into the top-50 most probable tokens of our base LM (flan-t5-xl), as this was the search width for each step of KWD and KCTS. It was found that 68% of the sampled tokens for summarization (78% for Knowledge Grounded Dialogue) are included in the top-50, which suggests that the synthetic data generated using high temperature are relevant to the real use-case.\n\nIn addition, when we observed the groundedness score difference from the original to the partial-hallucination data through UniEval, the score drop was 0.95 -> 0.63 (KGD groundedness), and 0.88 -> 0.36 (Summarization Consistency), which suggests that our silver-standard partial hallucination label is, to a large extent, valid.\n\n\n*2. RIPA and benign tokens*\n\nThe intention behind the statement was that RIPA does not associate benign tokens “before” the hallucination with the hallucination label, which NADO or FUDGE may do. Given the autoregressive nature of LM and the definition of $f$ having to approximate the groundedness of the whole sequence that would follow the current sequence, we believe the labels should become 0 for all tokens after hallucination. In other words, if the partial-sequence only consists of grounded tokens, the full sequence following it has a chance to be grounded, but if it contains at least one hallucinated content, all generations that can follow from it will be considered hallucinated sequences.\n\nPractically, this assumption helps MCTS decoding to reject hallucination token better. As RIPA is trained to predict 0 for all tokens following hallucinated tokens, all descendants of a hallucination token in the MCTS tree will be evaluated to have lower scores, which will in turn decrease the value of the hallucination token through backpropagation. This will prevent the token from being selected.\n\n**KWD showing better results in human eval**\n\nWe respectfully ask for your clarification on “human eval”, since KWD was not included in the human evaluation. In the following paragraphs, we will assume that you are referring to the unieval results on the summarization task (CNN/DM dataset, Table 3 in our paper), in which KWD achieved a higher score than KCTS. \n\nWe would first like to stress that UniEval is a model-based learned metric, which is strongly influenced by its training data. We hypothesize that higher knowledge overlap (KF1 of KCTS=22.97 vs KWD=20.39) with the original article might indicate the generated summary was slightly “less abstractive”, which might situate in the out of distribution of the summarizations included in the training of UniEval metrics. Also, we would like to point out that other dimensions of evaluation in summarization tasks, including Knowledge Overlap, Token Overlap, and learned (MFMA), prefer KCTS consistently. Overall, along with the results on the Knowledge-Grounded Dialogue (KGD) task, most knowledge based, reference based, and learned metrics show the effectiveness KCTS.\n\nIn addition, the $f$ column of Table 1 shows that MCTS decoding leads to generations that the groundedness classifier finds more grounded (looking at rows FUDGE/NADO vs MCTS and KWD vs KCTS). This finding supports our hypothesis that MCTS decoding guides the generation towards higher reward of $f$ at the end of sequence. We believe this is another evidence that KCTS is the better-suited RIPA-guided decoding method.\n\nWe will include more details about KWD and the discussion above in the final version of our paper.\n\n**Probability in Eq. (5)**\n\nThank you for pointing out the misnotation of Eq. (5); the input of function $f$ should not have been probability but the decoded token sequence. We will make the update in the final version as follows:\n\n$$\ny \\sim P(y|y_{<t}, x), f(y_{<t}, k) \\approx f(y, k).\n$$\n\n**Stopping criteria for the Rollout**\n\nTo avoid expensive rollout, we evaluated the node directly; that is, we did not generate any tokens following the node but applied $f$ right on top of the node. Generative Cooperative Network (Lamprier et al., 2022) is a previous work that also employed MCTS with NLG, and they had the same approach of not performing rollout to avoid high variance (which we have missed the citation at line 288; we will also update this in the final version). We further formulated the discriminator score of partial sequence ($y_{< t}$) as the approximation of full sequence following it ($y$) as in Eq. (5), which would make the prediction more stable.\n\n\n**References**\n\nHugh Zhang, Daniel Duckworth, Daphne Ippolito, & Arvind Neelakantan. (2020). Trading Off Diversity and Quality in Natural Language Generation.\n\nChung-Ching Chang, David Reitter, Renat Aksitov, & Yun-Hsuan Sung. (2023). KL-Divergence Guided Temperature Sampling.\n\nMuru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. 2023. How language model hallucinations can snowball.\n\nRenat Aksitov, Chung-Ching Chang, David Reitter, Siamak Shakeri, & Yunhsuan Sung. (2023). Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models.\n\nSylvain Lamprier, Thomas Scialom, Antoine Chaffin, Vincent Claveau, Ewa Kijak, Jacopo Staiano, and Benjamin Piwowarski. 2022. Generative cooperative networks for natural language generation. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 11891–11905. PMLR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Reducing hallucination and improving groundedness is an hot and important topic in the LLM research community and any effort in this direction is extremely valuable and of interest.\n\nThe method is slightly better (larger for summarization) than the baselines on the automatic metrics but is significantly better according to the human evaluations.", "reasons_to_reject": "The synthetic data generation methods used to train RIPA may have some issues since it seems that there are no guarantees that generated tokens are hallucinated or not factual. \n\nAccording to human evals, KWD is better for summarization than the proposed method. KWD still uses RIPA but it is only described with a quick one liner. Human evals, which are more important, may tell the more nuanced story that KCTS may not be always the best RIPA guided method, and require a bit of restructuring of the paper.", "questions_to_authors": "Eq (5): going from string to probability in the function f would require some more explanation or more specific pointers to the NADO paper sections.\n\nIn the Rollout paragraph of section 4.1, what is the criteria to stop when generating partial sequences, if EOS is not used since it is too costly?\n\nI am not sure about the strong statement at line 344 that RIPA does not associate benign tokens with hallucination labels, since facts that appear after an hallucinated token may still be grounded (e.g., Barack Obama is Norwegian and his wife is Michelle Obama).\n\nWhen training RIPA, how can you guarantee that the examples obtained with the Partial Hallucination method have the last part that is actually based on the provided knowledge? What if the model does not use the knowledge and outputs something from its own knowledge? I imagine that using high temperature to create more random sequences is yielding examples which are very different from what would be sampled during decoding with the normal temperature.", "paper_topic_and_main_contributions": "This paper proposes a Knowledge-Constrained Tree Search decoding method that is more faithful to the reference knowledge provided to the model. Together with this method, there is a token-level approximation method to predict if the generated content is grounded in knowledge (Reward Inflection Point Approximation, RIPA). In addition, there are experiments in knowledge-grounded dialogue and abstractive summarization tasks where the paper method outperforms GPT models in some dimensions.\n\nOne problem that the authors tackle is to understand at token-level when the output starts to be not grounded in the knowledge. This is well-defined at sentence level, i.e. after the full generation, when a classifier can be used to get a desired signal. At token level, they use a Monte-Carlo Tree Search Algorithm to explore multiple generation paths, and instead of using a groundedness signal from a classifier at the end of a generation only, they incorporate the same signal for partial paths.\n\nThe authors describe RIPA, a model that detects the first hallucinated token in the sequence generation, trained on synthetic data obtained from knowledge swapping/shuffling to create grounded and ungrounded (random) instances.\n\nAutomatic metrics and human evals show that the models using RIPA are better than the baselines.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "a04mKVP3oe", "reviewer": "Reviewer_nrcv", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort invested by the reviewer in thoroughly evaluating our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposes a novel token-level hallucination identification method and a knowledge-constrained decoding method to reduce hallucinations in instruction-tuned LLMs.\n2. This paper conducts extensive experiments and evaluations to validate the effectiveness of the proposed decoding method.", "reasons_to_reject": "N/A", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a decoding method to reduce hallucinations in text generation. \nThey apply an auxiliary knowledge classifier on top of a frozen LM to detect hallucinations, and use its knowledge-groundedness score to guide the decoding process.\nIn addition, they propose a token-level hallucination detection method to predict the starting point of the hallucinating token and enables effective adaptation of the knowledge classifier to the token level.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BIDpdyDPku", "reviewer": "Reviewer_jKoz", "summary": "", "questions": "Why Knowledge-Weighted Decoded (KWD), the RIPA guided variant, is not clearly explained? How does it perform an ablation study?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive feedback. The following responds to the questions, which simultaneously addresses the reasons to reject.\n\n**Details of KWD**\n\nThe reason we did not explain KWD in more detail was because KWD is a method that combines RIPA and weighted decoding (WD), which was explained thoroughly in the related work (line 174). KWD can be considered as a RIPA classifier-guided FUDGE.\n\n**How does it relate to ablation studies**\n\nKWD stands in the middle of Weighted Decoding (WD) baselines (FUDGE, NADO*) and KCTS, because KWD adds RIPA classifier on WD, and KCTS further uses MCTS Decoding with RIPA. Therefore, FUDGE/NADO vs KWD shows the effectiveness of RIPA by keeping the decoding algorithm the same, and KWD vs KCTS shows the effectiveness of RIPA + MCTS.\n\n**Although FUDGE and NADO are both categorized as WD, they have a different implementation of the classifiers: FUDGE employs a binary classifier and enumerates over top-k tokens to perform WD, while NADO’s classifier outputs a $V$-dimensional vector ($V$ is the vocabulary size) that can be added to the logits directly. This is also explained in lines 630-632 of the paper, and further detail can be found in NADO paper (Meng et al., 2022). KWD uses the approach in FUDGE.*\n\n**References**\n\nTao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang. 2022. Controllable Text Generation with Neurally-Decomposed Oracle. In Advances in Neural Information Processing Systems, volume 35, pages 28125–28139. Curran Associates, Inc.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper describes a knowledge-constrained tree search decoding mechanism applied on top of a large language model to reduce hallucinations. The paper is well written, clear, detailing the related work. Authors propose to guide decoding through a Monte-Carlo tree search algorithm which estimates the future groundedness. The Reward Inflection Point Approximation (RIPA), a binary classifier, is then used to predict the inflection point where a token is not longer grounded to the knowledge source. For training RIPA sequences were automatically generated by knowledge shuffle or by response truncation + generation with high temperature. This mechanism was evaluated on two datasets the wizard of Wikipedia and CNN/DM for grounded dialogue and summarisation respectively. Distinct metrics were used such as token-based (BLEU, ROUGE, METEOR), knowledge-based (Knowledge-F1, K-Copy) and UniEval. For summarisation the metric MFMA was also used to find correlations with human labels. The results show the effectiveness of the proposed methods KCTS.", "reasons_to_reject": "The Knowledge-Weighted Decoded (KWD), the RIPA guided variant which is not clearly explained in the paper.", "questions_to_authors": "Why Knowledge-Weighted Decoded (KWD), the RIPA guided variant, is not clearly explained? How does it perform an ablation study?", "paper_topic_and_main_contributions": "The paper describes a knowledge-constrained tree search decoding mechanism applied on top of a large language model to reduce hallucinations. The paper is well written, clear and the related work is well described. Authors propose to guide decoding through a Monte-Carlo tree search algorithm which estimates the future groundedness. The Reward Inflection Point Approximation (RIPA), a binary classifier, is then used to predict the inflection point where a token is not longer grounded to the knowledge source. For training RIPA sequences were automatically generated by knowledge shuffle or by response truncation plus generation with high temperature. This mechanism was evaluated on two datasets the wizard of Wikipedia and CNN/DM for grounded dialogue and summarisation respectively. Distinct metrics were used such as token-based (BLEU, ROUGE, METEOR), knowledge-based (Knowledge-F1, K-Copy) and UniEval. For summarisation the metric MFMA was also used to find correlations with human labels. The results show the effectiveness of the proposed method KCTS. My unique concern regards the technique of Knowledge-Weighted Decoded (KWD), or the RIPA guided variant, which is not explained in the paper.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "There is no mention about how evaluators were rewarded by their labour."}, "extra_text": {}}]

Author rebuttal:

Meta-review: KCTS, a knowledge-constrained decoding method, guides a frozen LM to generate text aligned with the reference knowledge at each decoding step using a knowledge classifier score and MCTS. KCTS is a plug-and-play, model-agnostic decoding method that can effectively reduce hallucinations in natural language generation without incurring high training costs or causing catastrophic forgetting.
The proposed approach is shown to be effective in both automatic and human evaluations.
Decision: Accept-Main