Title: STICKING TO THE FACTS: CONFIDENT DECODING FOR FAITHFUL DATA-TO-TEXT GENERATION

Abstract: Neural conditional text generation systems have achieved significant progress in recent years, showing the ability to produce highly fluent text. However, the inherent lack of controllability in these systems allows them to hallucinate factually incorrect phrases that are unfaithful to the source, making them often unsuitable for many real world systems that require high degrees of precision. In this work, we propose a novel confidence oriented decoder that assigns a confidence score to each target position. This score is learned in training using a variational Bayes objective, and can be leveraged at inference time using a calibration technique to promote more faithful generation. Experiments on a structured data-to-text dataset -WikiBio (Lebret et al., 2016) -show that our approach is more faithful to the source than existing state-of-the-art approaches, according to both automatic metrics and human evaluation.

1. INTRODUCTION:

Conditional text generation is the task of generating some target text y conditioned on source content x. It is the essence of many natural language processing problems, such as text summarization (Mani, 1999) , where x is a long document and y is a more concise version, machine translation (Koehn, 2009) , where x and y represent equivalent text in different languages, and data-to-text generation (Kukich, 1983; McKeown, 1992) , where x is a structured table and y is a textual description.

While traditionally done with template-based approaches (Becker, 2002; Foster & White, 2004; Gatt & Reiter, 2009; Reiter et al., 2005) , recently neural encoder-decoder approaches (Sutskever et al., 2014; Cho et al., 2014; Bahdanau et al., 2014) have become a popular approach. In this formulation, the source content is encoded with a neural architecture, and the decoder autoregressively produces a token at each output position based on its internal state and the source representation. By leveraging continuous representations with rich non-linearities, encoder-decoder approaches can generate highly fluent text (Rush et al., 2015; Radford et al., 2019) without the need for cumbersome handcrafted rules and templates.

However, encoder-decoder architectures are inherently difficult to control, and have been shown to be prone to hallucination, i.e., generating text that is fluent but unfaithful to the source (Vinyals & Le, 2015; Koehn & Knowles, 2017; Wiseman et al., 2017; Lee et al., 2018) . This severe shortcoming can often limit the use of neural approaches in many real world systems, where it is not acceptable to produce output that is even occasionally unfaithful.

In this work, we focus on data-to-text generation since the structured form of source content x makes it relatively easy to evaluate faithfulness using both human evaluation and domain-specific automatic metrics (Dhingra et al., 2019) . In particular, we focus on the WikiBio (Lebret et al., 2016) dataset, where the task is to generate a sentence summarizing a tabular biography of a person. Figure 1 shows an example.

First note that the reference contains information such as bonanno crime family and informant that are true, but cannot be inferred from the source. This source-reference divergence exists in many large-scale generation datasets (Wiseman et al., 2017; Dhingra et al., 2019) . Secondly, most generation systems are agnostic to this divergence and trained to maximize the log-likelihood of reference. This can often encourage the models to output phrases that are unsupported by the source. For example, Figure 1 shows the output of a state-of-the-art generation baseline, the Pointer-Generator

Frank Lino:



FBI surveillance photo:



Birth date:

October 30, 1938

Birth place Gravesend, Brooklyn, New York,

United States:

Source (Wikipedia infobox):

Reference: Frank ``Curly '' Lino (born October 30, 1938 Brooklyn) is a Sicilian-American Caporegime in the Bonanno crime family who later became an informant.

Baseline: Frank Lino (born October 30, 1938 in Brooklyn, New York, United States) is an American criminal defense attorney.

Our model: Frank Lino (born October 30, 1938 in Brooklyn, New York, United States) is an American.

Target::

Figure 1 : Example in the WikiBio dataset (Lebret et al., 2016) showing the biography of Frank Lino. The baseline Pointer-Generator (See et al., 2017) exhibits hallucination.

network (See et al., 2017) , which contains the phrase criminal defense attorney that is false (but loosely related to FBI in the table). Thus, hallucination can often result from the coupling of model shortcomings (e.g. lack of formal reasoning, learning false correlations), and noise/artifacts in the training data.

In this work, we propose a confidence oriented approach which assigns a learned confidence score to each decoder position, and then uses the score in two ways to reduce hallucination: (1) In test, it uses confidence to adjust the output probabilities by a calibration technique (Braverman et al., 2019) .

(2) In training, we employ a variational Bayes objective to jointly learn the confidence score while allowing the model to skip tokens with a low confidence score to avoid training on reference phrases that are difficult to infer from the source. In Figure 1 , our approach leads to a faithful generation that omits the occupation.

Empirically, when evaluated on the WikiBio dataset (Lebret et al., 2016) , we show that our approach is considerably more faithful to the source than existing state-of-the-art solutions, according to both PARENT precision (Dhingra et al., 2019) and human evaluation.

2. RELATED WORK:

Improving the fidelity and accuracy of text generation systems is an important research topic that has spawned a variety of different approaches. Some focus on blending extractive and abstractive methods, e.g., allowing the model to copy tokens directly from the source (Gu et al., 2016; See et al., 2017) , separating content selection from generation (Zhou et al., 2017; Gehrmann et al., 2018) and utilizing topic information from the source to make informed generation (Narayan et al., 2018) .

Other approaches have proposed generating more accurate text using semiparametric approaches (Guu et al., 2018; Pandey et al., 2018; Peng et al., 2019) , reinforcement learning-based rewards (Paulus et al., 2018; Pasunuru & Bansal, 2018) , semi-Markov models to learn neural templates (Wiseman et al., 2018) , data augmentation (Ma et al., 2019; Kedzie & McKeown, 2019) , content planning (Puduppully et al., 2019a) , and constrained vocabulary decoding (Wu et al., 2017) . While many leverage the structure of the source (Liu et al., 2018; Marcheggiani & Perez-Beltrachini, 2018) or task-based insights (Puduppully et al., 2019b) , our approach is complementary in that it uses general machine learning techniques to build a confidence oriented decoder, that is more faithful to the source and robust to divergence/noise in the training data. Furthermore, many previous works rely on automatic metrics such as BLEU, which can be poorly correlated with human judgment of faithfulness (Wiseman et al., 2017; Dhingra et al., 2019) . In contrast, we evaluate on PARENT precision (Dhingra et al., 2019) , a metric specifically designed to capture faithfulness in data-to-text generation, and conduct a rigorous human evaluation to assess hallucination in our models.

3. PRELIMINARIES:

We first review the existing encoder-decoder framework (Sutskever et al., 2014; Bahdanau et al., 2014) which this work is based on. Let x = x 1 x 2 . . . x S , be the source input of length S and y = y 1 y 2 . . . y T be the target sequence of length T . Each token x i , y i takes one value from a vocabulary V . Our goal is to model the conditional distribution P (y|x) = T t=1 P (y t |y <t , x),

where y <t = y 1 . . . y t-1 is the prefix of y up to the (t -1) th token. The source can be encoded by any neural network function enc, such as a convolutional neural network (CNN, LeCun et al., 1990) , long-short-term memory (LSTM, Hochreiter & Schmidhuber, 1997) , or Transformer (Vaswani et al., 2017) . Let s 1 , ..., s S = enc(x 1 , ..., x S ).

Define e x ∈ R d as the d dimensional embedding of token x. Then, the probability of each target token is computed as:

EQUATION

where the context vector v t is given by:

EQUATION

In this work, we use a Luong-style attention (Luong et al., 2015) for the first term on the right hand side of Eq. 2, as defined in Eq. 3; and the second term is given by an RNN1 hidden state at position t, as defined in Eq. 4 (let [•] denote concatenation):

α s,t = exp(s s W h t ) s exp(s s W h t )

(3)

EQUATION

In case the encoder-decoder is equipped with a copy mechanism, the generation probability is mixed with a probability of copying from the source (Gu et al., 2016; See et al., 2017) :

EQUATION

where p gen t is the probability of doing generation instead of copying at step t, and β s,t is an attention weight that the copy mechanism is paying to position s in the source. The sum is taken over all positions s where the word x s is the same as y t .

4. MODEL:

Our approach is based on a confidence score at each decoding position, and the idea is originated from an observation on the hallucinations shown in Figure 1 : hallucination often occurs when the table lacks some field that usually exists (e.g. Occupation), and the baseline model is trying to make it up (more examples are found in Appendix). In such cases, the generation probabilities for the hallucinated tokens can still be high (e.g. > 0.5), so it is difficult to tell hallucination from faithful generation. However, if some usual field is missing in the table, one might tell the difference by assessing the attention in the encoder-decoder. Our preliminary investigation supported this idea: we defined an attention score A t using the length of the attention vector relative to the hidden state, as the following (let • denote the Euclidean norm):

A t := a t a t + h t (6)

and compared the scores between hallucinated and faithful generations. We found that A t drops in the hallucinated cases. Then, we refined our attention score with the copy mechanism:

EQUATION

As illustrated in Figure 2 , Ãt is usually high (∼ 0.9) when the model is copying from the source (e.g. Campbell), medium (∼ 0.4) when it is generating based on some source information (e.g. <unk>), and low (∼ 0.2) when generating function words, template elements, or when the corresponding field is missing (more examples are found in Appendix).

Christian Campbell:

Birth name Christian Campbell

Birth date 1979:



Birth place Bahamas:



Occupation Poet:

Years active 2010 --present Thus, the attention score can recognize faithful content words, but it cannot distinguish hallucination from function words and template elements. Therefore, we further introduce an unconditioned language model called base-LM. The base-LM does not have access to the source data, so if it can predict a token as precisely as the encoder-decoder, that token is probably an element of a general linguistic construction that does not convey source information. Then, we define confidence score as an interpolation between the encoder-decoder P (y t |y <t , x) and the base-LM P (y t |y <t ):

EQUATION

For function words and template elements, we expect both P (y t |y <t , x) and P (y t |y <t ) to be high, so the confidence score C(y t |y <t , x) will be high no matter what the attention score is. On the other hand, we expect P (y t |y <t , x) to be higher than P (y t |y <t ) for content words, and the confidence score will largely depend on the attention score in this case.

From a high level, our confidence score keeps a balance between two factors:

• How much the model should rely on the source for predicting this position, is reflected by combining the encoder-decoder with the base-LM.

• How much the model actually relies on the source for this position, is measured by the attention score.

Intuitively, it is reasonable for a system to depend mostly on language modeling to predict function words that make the generation fluent, but it should consult more of the source data to predict content words. For example, given a partial generation "Christian Campbell is ", one could predict that the next token is mostly likely "a", "an" or "the", based on language modeling. However, if a model predicts "American" as the next token to "Christian Campbell is an ", it should be based on a field such as "Nationality: U.S." in the source, rather than the language tendency that "American" is likely to appear after the phrase "is an". A typical neural network can make predictions based on both reasons with little controllability; this, we contend, is a general cause of hallucination. By introducing the confidence score and enforcing a "confident prior" on our model, we can gain some control on these two.

In order to further strengthen control, we tweak the input-feeding of our RNN models as below:

EQUATION

EQUATION

) Here, g t is the hidden-state of the base-LM, and the input embedding e yt-1 in Eq. 9 is weighted by a component of the previous confidence score. We found this weighting scheme to decrease dependence of the base-LM on content words, seemingly resulting in a model of soft templates, as shown in Figure 2 (see Appendix for more examples). We also added stop-gradients (SG) to prevent information at the current step from being propagated to previous steps, and used Eq. 10 instead of Eq. 4. They seem to result in more reasonable attention scores.

The confidence score is leveraged in two ways, as we describe in the following:

• At test time, we augment the generation probability with the confidence score, using a calibration technique (Section 4.1). It allows us to weigh more on the confidence of generation, without sacrificing perplexity (Braverman et al., 2019) . 

Q(z | y, x) P(z | x)

Random Sub-sequence 

4.1. CALIBRATION:

In order to use the confidence score to promote faithful generation, we apply a calibration technique (Braverman et al., 2019) which augments the generation probability as follows:

P κ (y t |y <t , x) := SG(C(y t |y <t , x)) κ SG(P (y t |y <t , x))

w∈V SG(C(w|y <t , x)) κ SG(P (w|y <t , x))

.

Here, P κ (y t |y <t , x) is a one-parameter family of probability distributions called the calibration of P (y t |y <t , x) to C(y t |y <t , x). Note that SG stops gradients so that only the parameter κ is updated during training. In order to learn κ, we minimize the negative log-likelihood of P κ (y t |y <t , x) jointly with P (y t |y <t , x) and P (y t |y <t ):

L joint (y, x) := T t=1

-log P κ (y t |y <t , x) -log P (y t |y <t , x) -log P (y t |y <t ).

(

EQUATION

Since P (y t |y <t , x) is a special case of P κ (y t |y <t , x) (namely at κ = 0), the training perplexity of P κ (y t |y <t , x) is at most P (y t |y <t , x). In practice, κ is initialized as 0 and found converging to a positive value (Section. 5.4). Therefore, the calibration trick can improve confidence of generation without sacrificing perplexity.

4.2. TRAINING WITH A VARIATIONAL BAYES OBJECTIVE:

In practice, the training data of a conditional text generation task almost always contain noises and/or scarce examples. In order to reduce the impact of such outliers and train a confident generation model, we allow the model to skip some tokens in the training data when it feels unconfident. For this purpose, we use the confidence score to sample a sub-sequence of each training target, and minimize the negative log-likelihood on that confident sub-sequence. However, since the confidence score itself needs to be trained, we use a variational Bayes objective to formalize the problem.

Specifically, for each target y = y 1 y 2 . . . y T , we define z = z 1 z 2 . . . z R = y ι(1) y ι(2) . . . y ι(R) as a latent sub-sequence of y, which consists of confident tokens of length R. Here, ι : |R| → |T | is an inclusion of indices. We assume that z is generated by the calibrated probability P κ z r | z <r , x :

EQUATION

Then, we connect P (z|x) to the probability of training target P (y|x). From Bayes rule:

EQUATION

)

We assume that P (y|z, x) = 1 for every training example because the training data is uniquely given. Then, we regard z as a sequential "keep/skip" labeling over y, and define a probability distribution

Q t := Q t (keep) ∝ C(y t |z ι(s)<t , x) ρ Where keep means y t is in z Q t (skip) ∝ γ ρ

Where skip means y t is not in z (15)

to sample a sub-sequence according to the confidence score (Figure 3 ). Here, ρ and γ are hyperparameters. Now let

EQUATION

and the idea is to use Q(z|y, x) as an approximation to the unknown posterior P (z|y, x) in Eq. 14.

By taking -log(•) of both sides in Eq. 14 and trivially introducing log Q(z|y, x), we get

EQUATION

Then, we take the expectation E Q(z|y,x) [•] of both sides and note that E Q(z|y,x) [log Q(z|y,x) P (z|y,x) ] = KL[Q(z|y, x)||P (z|y, x)] ≥ 0, so

EQUATION

)

The variational Bayes objective is to minimize the upper bound on the right hand side of Eq. 18.

In practice, it is computationally expensive to explicitly calculate E Q(z|y,x) [•] by enumerating all sub-sequences of y, because the number of sub-sequences is exponential to the length T . Thus, we apply a Monte Carlo method which calculates E Q(z|y,x) [•] by sampling from Q(z|y, x). In order to back-propagate gradients through the expectation E Q(z|y,x) [•] as well, the loss function is given as follows (Paisley et al., 2012) :

L var (y, x) := 1 K K k=1 z k ∼Q(z|y,x) log Q z k | y, x -log P z k | x + λ SG log Q z k | y, x -log P z k | x log Q z k | y, x . (19)

Here, K is the number of samples taken, and λ is a hyper-parameter controlling how fast the gradients go through E Q(z|y,x) [•] . However, since we define P (z|x) in Eq. 13 by the calibrated probability, which only learns one parameter κ, we add joint learning terms into Eq. 19 to make the final objective:

L(y, x) := 1 K K k=1 z k ∼Q(z|y,x) log Q z k | y, x + L joint (z k , x) + λ SG log Q z k | y, x -log P z k | x log Q z k | y, x . (20)

In order to sample sub-sequences from Q(z|y, x), we apply the same Gumbel-max trick as in Kool et al. (2019) . Although the random sampling is purely based on a learned probability distribution, without any constraints to make it fluent, surprisingly the model still learns to generate fluent text.

5. EXPERIMENTS:

Although our approach could apply to many conditional text generation tasks, in this work we consider data-to-text generation, in which the source is some structured data and the target is natural language text describing the data. Usually, the data have concise semantics and simple structure, which makes it easy to check the facticity of the generation. For automatic metrics, we report BLEU (Papineni et al., 2002) , as well as PARENT (Dhingra et al., 2019) , a metric designed to mitigate the shortcomings of BLEU on structured data-to-text generation.

For human evaluation, we obtain crowd-source annotations on examples randomly chosen from predictions on the WikiBio test set, the same 1000 for each model. Examples from different models are mixed and randomly shuffled, with model names hidden from the raters. We instruct the raters to grade on each of 3 criteria: faithfulness, coverage, and fluency, as below:

• Faithfulness (precision) -We define a sentence to be faithful if all the information in the proposed sentence is supported by the table or the reference. A single hallucinated piece of information makes the sentence non-faithful. • Coverage (recall) -The number of table cells that contain information present in the sentence.

• Fluency -A sentence is defined to be fluent if it is clear, natural, and grammatically correct.

Raters choose among three options: Fluent, Mostly Fluent, Not Fluent.

An ideal system would always produce fluent and faithful text with high coverage. The complete annotation instruction is given in the Appendix. We assigned 5 raters who are well-trained on this task, and conducted 5-way annotation on a pilot batch of 100 examples. The unanimous agreement rate is 86% for faithfulness and 99% for fluency, with 74% of the examples judged as faithful and 98% as fluent.

5.2. EXPERIMENT SETTING:

We compare the following systems:

• BERT-to-BERT (Rothe et al., 2019) : A Transformer encoder-decoder model (Vaswani et al., 2017) where the encoder and decoder are both initialized with BERT (Devlin et al., 2018) . • Structure Aware Seq2Seq (Liu et al., 2018) : A state-of-the-art method on the WikiBio dataset in terms of BLEU. • Pointer-Generator (See et al., 2017) : A Seq2Seq with attention and copy mechanism (our implementation). • Confident BERT-to-RNN (This Work): A Transformer encoder initialized with BERT checkpoint, and a GRU (Cho et al., 2014) decoder with our confident decoding method. • Confident Pointer-Generator (This Work): Pointer-Generator model with confident decoding.

We built our systems using Tensorflow (Abadi et al., 2016) . Infoboxes are linearized into sequences, with field names and values separated by special tokens. For BERT-to-BERT and Confident BERTto-RNN, we pre-trained a BERT checkpoint on the Books corpus (Zhu et al., 2015) Table 2 : Ablative tests on three components of our confident decoding method, and a truncation test.

5.3. MAIN RESULTS:

Table 1 shows the results. According to human evaluation, our approach gives a clear improvement in faithfulness over the baselines, with some drop in coverage. To further measure the validity of our confidence score, we postprocessed the output to remove words with lower confidence than 0.125. This thresholding technique gives further gains to faithfulness, while sacrificing some fluency.

Among the automatic metrics, PARENT precision and recall seem correlated to faithfulness and coverage respectively, and our approach achieves the highest precision score. BLEU, perhaps because of its length penalty that rewards longer generations, seems more correlated to coverage rather than faithfulness. Regarding the baselines, we see that BERT-to-BERT is the most fluent while Pointer Generator is the most faithful, suggesting that pretraining might help fluency while the copy mechanism can be valuable for faithfulness.

5.4. ABLATIONS:

Our confident decoding method has three novel components: (1) The use of a base-LM to define confidence score;

(2) The calibration technique to adjust output probability; and (3) The variational Bayes objective to train a confident model. In this section, we assess the effects of each component by an ablative study. We start from the Confident Pointer-Generator, and in each test replace one component by a trivial alternative: (1) In order to assess the effects of using the base-LM in the confidence score, we instead use P (y t |y <t , x) directly as confidence, and train models with the same hyper-parameter search. The results on WikiBio are shown in Table 2 as "No base-LM". (2)

We use P (y t |y <t , x) instead of P κ (y t |y <t , x) at test time ('No calibration"), to assess the effects of calibration. The model is the same as Confident Pointer-Generator. The learned κ was 0.035.

(3) Instead of the variational Bayes objective, we use the joint training loss L joint in Eq. 12 without sampling sub-sequences from training targets (No variational).

As we can see from Table 2 , all three components improve PARENT precision. While the improvement by calibration is the smallest, the technique also improves PARENT recall and BLEU score at the same time, making it an easy choice. The other techniques trade recall for precision, making them useful for tasks that require a high degree of faithfulness. When all three components are disabled, the model is exactly the same as our implementation of the Pointer-Generator. Every component improves PARENT precision upon it as well. Especially, comparing Pointer-Generator with "No variational" shows again that joint training with calibration improves all metrics.

We also note that the average lengths of generations by our confident decoding models are shorter. While there exists heuristics such as length penalty (Wu et al., 2016) to encourage longer generation at inference time, shorter generation is not trivial. In the "Truncated" setting, we truncate predictions by the Pointer-Generator two words each to match the average length of Confident Pointer-Generator.

The PARENT precision by our confident decoding method is not trivially achieved by truncation.

6. CONCLUSION:

In this work, we have proposed a confidence oriented decoder that achieved more faithful generation on the WikiBio dataset than existing state-of-the-art approaches. Our method is general in principle, so it could potentially be combined with other approaches and/or adapted to other forms of conditional text generation such as document summarization and machine translation. For Pointer-Generator and Confident Pointer-Generator, we use GloVe (Pennington et al., 2014) as the input word embedding and truncate the vocabulary size to 5,000; the two models share the same hyper-parameter settings. For our confident decoding models, there are additional hyper-parameters ρ, γ as defined in Eq. 15, and K, λ as defined in Eq. 19. The hyper-parameters are shown in Table 3 .

7. APPENDIX:

The optimizer is Adam (Kingma & Ba, 2015) .

7.2. HUMAN EVALUATION INSTRUCTIONS:

Here are the detailed instructions for our human evaluation template:

A writer has the following background knowledge and is given a table: <Reference sentence shown as background knowledge> <Table>

The writer read the table and produced the following sentence: <Sentence generated by model> We wish to evaluate the quality of the sentence.

1. How fluent is the sentence? Mostly fluent: the repetition "designer designer" is not natural, but the sentence makes sense.

alfred angas scott ( 1875 -1923 ) was a motorcycle designer , was born in manningham bradford , lived in england u.k. , was british by nationality , and was buried at the undercliffe cemetery in bradford .

Mostly fluent: there are no mistakes, but the sentence contains too much information. This is unnatural.

2. Please compare carefully the content of the sentence to the content of the Coverage: 4 Faithful: All the information is supported because England is located in Europe.

We have also discussed with the lead annotators about many other examples. We have made sure that: (a) Valid inferences (e.g. inferring nationality from birth place) are considered faithful; (b) If a piece of information exists or can be inferred from the table, the corresponding cell should be highlighted, even if the information was also in the background knowledge; (c) Only one cell should be highlighted for one piece of information. Known for: English and Welsh dictionary thomas edwards ( 1779 -4 june 1858 ) was an english author , <unk>, and <unk>, who spent most of his life in the english and english literature of the english and english dictionary literature .

thomas edwards ( 1779 -4 june 1858 ) was a welsh author . 

7.3. HALLUCINATION EXAMPLES:

In Table 4 , we show some typical case where the Pointer Generator baseline hallucinates but the Confident Pointer Generator does not. The examples are taken from the WikiBio validation set.

In the first five examples (i.e. "Rohan Robertson", "Walter Smallwood", "Nellie Wong", "tight end" and "Constant Vanden Stock"), some information is missing in the table while the baseline made it up. Our Confident Pointer Generator model learns to omit the missing fields, although by doing this, two of the generated sentences are no longer fluent (i.e. "tight end" and "Constant Vanden Stock").

In the next example (i.e. "Richard Lloyd"), the baseline seems to have learned weird language modeling from some similar training points, and tries to generate more than the table contents; our Confident Pointer Generator is more conservative.

In the last two examples (i.e. "Robert I. Marshall" and "Thomas Edwards"), there are corresponding fields in the table but the baseline didn't learn to generate the text correctly, possibly because these

:

While it is possible our approach could extend to other types of decoders, our current formulation of the confidence score specifically uses RNN with attention.

Figures and Tables:

Figure 2: Example of learned attention score, base-LM probability, and confidence score. For content words the base-LM probability is lower, and the confidence score depends more on the attention score.

Figure 3: Example of sampling a sub-sequence according to the confidence score. Our variational Bayes objective combines the sampling probability Q(z|y, x) and the generation probability P (z|x).

Performance on WikiBio test set. Two Fluency measures differ in whether to include sentences graded as Mostly Fluent. Starred numbers are statistically significant against baselines (p < .001), by bootstrap test.5.1 DATASET AND EVALUATION METRICSThe WikiBio dataset(Lebret et al., 2016) contains 728,321 biographies paired with infoboxes, taken from the Sep-2015 dump of English Wikipedia, and splitted into train/valid/test sets in a 8 : 1 : 1 ratio.The biography text is the first sentence of the Wikipedia page (26.1 words on average). Infoboxes have 12.1 non-empty fields on average.

Table content: <table><tr><td>Automatic Metrics</td><td>Human evaluation</td></tr></table>

only, since the original BERT was trained on Wikipedia that overlaps with the test targets in WikiBio. Other hyper-parameter settings are given in Appendix.

Table content: <table><tr><td/><td>BLEU</td><td>PARENT (Prec. / Rec. / F1)</td><td>Avg Len.</td></tr><tr><td>Confident Pointer-Generator (This Work)</td><td>38.10</td><td>79.52 / 40.60 / 51.38</td><td>17.0</td></tr><tr><td>No base-LM</td><td>39.39</td><td>78.77 / 41.55 / 52.08</td><td>17.9</td></tr><tr><td>No calibration</td><td>37.89</td><td>79.47 / 40.47 / 51.26</td><td>16.9</td></tr><tr><td>No variational</td><td>41.29</td><td>78.25 / 42.40 / 52.52</td><td>18.9</td></tr><tr><td>Pointer-Generator</td><td>41.07</td><td>77.59 / 42.12 / 52.10</td><td>19.1</td></tr><tr><td>Truncated</td><td>35.50</td><td>77.68 / 38.16 / 48.66</td><td>17.1</td></tr></table>

Hyper-parameters. We did a hyper-parameter search for ρ, γ and λ, within the range [0.

Table content: <table><tr><td>25, 0.5, 0.75, 1],</td></tr></table>

Fluent: It is clear, natural, and the grammar is correct. It reads like if it was found in a book.• Mostly Fluent: It has a few errors or it does not sound natural, but you can understand it.• Not Fluent: It has many errors and/or you can hardly understand it.

Table content: <table><tr><td>Examples:</td><td/></tr><tr><td>Sentence</td><td>Decision</td></tr><tr><td>alfred angas scott ( 1875 -1923 ) was a motor-</td><td>Not fluent: the sentence stops abruptly. It is</td></tr><tr><td>cycle designer born in manningham bradford ,</td><td>unnatural, it does not make any sense.</td></tr><tr><td>who lived in</td><td/></tr><tr><td>alfred angas scott ( 1875 -1923 ) was a motor-</td><td/></tr><tr><td>cycle designer designer born in manningham</td><td/></tr><tr><td>bradford .</td><td/></tr></table>

table. How many cells from the table did the writer use to produce the sentence? (Click on the cells in the table above to update the counter) 3. A sentence is faithful if it contains only information supported by the table or the writer's background knowledge. It should not add any additional information, even if the information is true or interesting. Please compare once again the content of the sentence to the content of the table and background knowledge. How faithful is the sentence?• Faithful: every part of the sentence is supported by the table and/or background knowledge.• Mostly Faithful: every part of the sentence can be linked to some evidence in the table or the background knowledge, but it is not fully supported. This should only be used for rare edge cases. • Not Faithful: The sentence contains information that is not supported by the table or background knowledge.

Table content: <table><tr><td colspan="5">The examples are based on the following background knowledge and table:</td></tr><tr><td colspan="5">alfred angas scott ( 1875 -1923 ) was a british motorcycle designer .</td></tr><tr><td colspan="2">birth date birth place</td><td colspan="3">death date nationality residence</td><td>occupation</td></tr><tr><td>1875</td><td>manningham,</td><td>1923</td><td>british</td><td colspan="2">england, uk. motorcycle designer</td></tr><tr><td/><td>bradford, united</td><td/><td/><td/><td>and manufacturer</td></tr><tr><td/><td>kingdom</td><td/><td/><td/></tr><tr><td>Examples:</td><td/><td/><td/><td/></tr><tr><td>Sentence</td><td/><td/><td>Decision</td><td/></tr><tr><td colspan="3">alfred angas scott ( 1875 -1923 ) was a british</td><td colspan="2">Coverage: 4</td></tr><tr><td colspan="3">motorcycle designer and founder of the scott</td><td colspan="3">Not faithful: the table does not mention that</td></tr><tr><td colspan="2">motorcycle company .</td><td/><td colspan="3">A. A. Scott was the founder of the Scott Com-</td></tr><tr><td/><td/><td/><td>pany .</td><td/></tr><tr><td colspan="4">alfred angas scott ( 1875 -1923 ) was a euro-</td><td/></tr><tr><td colspan="2">pean motorcycle designer .</td><td/><td/><td/></tr></table>

Generation examples. Red tokens are hallucinated.

Table content: <table/>

Peer review (iclr2020):
This paper presents a method for conditional text generation that has higher factual precision, minimizing hallucination of facts. The method involves predicting confidence of generation at each time step and using this confidence measure to skip tokens during generation and calibrate output probabilities in test time. Their method achieves SoTA performance on automatically measured precision and human evaluated *faithfulness.* However their method does see a drop in recall (automatic metric and human evaluation). Comments and issues, - The intuitive explanation for the confidence score is a little confusing. In Section 4, page 3, you say that *If a token is likely a content word (i.e. when its generation probability by the encoder-decoder is much higher than the unconditioned language model), but the attention score is low, then the token might not be predicted based on the source, and could be hallucination.* However, this doesn*t seem like an airtight conclusion. Isn*t it possible that the base-LM and enc-dec model have similar probabilities for a content word with the enc-dec attention being low? This seems possible given your observation that low attention to the source is what may be causing content hallucination. This same thing is essentially restated in section 4.1 *we expect P(y_t |y_<t, x) to be higher than P(y_t | y_<t) for content words so the confidence score will largely depend on the attention score*, which seems more tangled up since P(y_t |y_<t, x) inherently depends on the attention score. This is all clarified when you explain the alteration made to the base-LM. I would recommend rewording/rearranging some of the earlier explanation for the efficacy of the confidence score since it seems that the alteration to the base-LM is an essential part of the explanation. - Need some explanation for Equation 6. I don*t really get the intuition behind it. - The presented results are pretty good! However, I would like to see some numbers on average score across a few runs. - It would also be good to see results on one more dataset like E2E. - Provide a little more detail on human evaluation, you don*t even mention if the evaluation was done with crowd-workers or another pool of people like grad students. How many annotators? What is the inter-annotator agreement? What was the prompt/structure? Human evaluation of models is notoriously difficult, more details would give some more weight to the results. I think this is a well written paper with thought out experiments. I recommend it be accepted to ICLR. I*d also be curious to see some future work that improves, or at least maintains recall, while keeping the higher precision. Minor requests/recommendations: - Include more examples of generations. Could be an appendix.