How Decoding Strategies Affect the Verifiability of Generated Text (2020)

Abstract: Recent progress in pre-trained language models led to systems that are able to generate text of an increasingly high quality. While several works have investigated the fluency and grammatical correctness of such models, it is still unclear to which extent the generated text is consistent with factual world knowledge. Here, we go beyond fluency and also investigate the verifiability of text generated by state-of-the-art pre-trained language models. A generated sentence is verifiable if it can be corroborated or disproved by Wikipedia, and we find that the verifiability of generated text strongly depends on the decoding strategy. In particular, we discover a tradeoff between factuality (i.e., the ability of generating Wikipedia corroborated text) and repetitiveness. While decoding strategies such as top-k and nucleus sampling lead to less repetitive generations, they also produce less verifiable text. Based on these finding, we introduce a simple and effective decoding strategy which, in comparison to previously used decoding strategies, produces less repetitive and more verifiable text.

## introduction recent years have led to a considerable surge of interest in and capabilities of pre-trained language models (lms). today, they play a critical role in many nlp tasks, such as text classification, machine comprehension and natural language inference @xcite @xcite , to name just a few. they serve as a pre-training objective for downstream applications and they have been used to showcase and measure the general progress in nlp @xcite . several works @xcite show the remarkable fluency and gram-matical correctness of text decoded from modern lms. additionally, recent works @xcite @xcite demonstrate that beyond general linguistic capabilities, language models can also pick up factual knowledge present in the training data. however, it is unclear if lms are able to convey such knowledge at decoding time when producing long sequences-do they generate fluent, grammatical but "babbler-level" text or can they produce utterances that reflect factual world knowledge? understanding this behaviour becomes crucially important as the downstream adoption of automatically generated text increases. already today lms face growing scrutiny from the media and the broader society, as well as from the researchers themselves. for example, @xcite initially argued against releasing their models in order to prevent automatic generation of fake news @xcite . several blogs and web resources demonstrate that differentiating between human and machine-generated text has become surprisingly difficult. 1 with that in mind, we set out to study stateof-the-art auto-regressive transformer-based language models through the lens of their verifiability. specifically, we use wikipedia to first create a set of natural language prompts to initiate generation. next, we use transformer models of various sizes and trained with different corpora to generate sentences off these prompts with varying decoding configurations. finally, following earlier work in fact checking @xcite , we use wikipedia again to verify each sentence as supported, refuted, or unverifiable using both an off-the-shelf automatic fact-checking system and human annotators. we define verifiability metrics on top of the automatic and human fact-checkers' evaluation outcomes (see figure 1 for a high-level overview). the truthfulness of generated text can be traded off with other properties. for example, a decoding algorithm can generate the same true fact over and over again to produce many verifiable utterances, but this would be a poor outcome in terms of repetitiveness. similarly, a model might generate ungrammatical text that cannot be verified as supported or refuted at all, and hence not as factually wrong either. our experiments show that the text generated from auto-regressive transformer-based lms, especially in their large versions (1.4b parameters), is almost always grammatical and fluent regardless of the configuration, but that repetitiveness can vary a lot. we hence focus on this dimension in our analysis and define metrics that combine repetitiveness with verifiability. one of our main findings is that while sampling methods, such as top-k and nucleus, produce more natural and less repetitive text, they also generate fewer supported and more refuted statements. beam search, on the other hand, shows much better performance along these dimensions at the cost of producing highly repetitive text. based on these observations, and inspired by findings in @xcite , who showed how the probability of human text under language models is varying from token to token, we introduce a simple strategy: delayed beam search (delayedbs). in delayedbs, we iterate between sampling and finding most likely utterances. by simply injecting stochasticity in the beginning of a sentence and then switching to beam search, we generate text that is less repetitive while at the same time scores well in terms of our verifiability metrics. our main findings hold across several experimental settings, with varying training set size and model size. to summarize, we make the following contributions: (i) we propose an experimental methodology to assess machine generated text with respect to repetitiveness and verifiability. (ii) we assess a wide range of decoding algorithms with respect to these dimensions, (iii) we introduce a novel decoding strategy that addresses some of the shortcomings of existing solutions, (iv) we carry out an annotation campaign to validate our findings and assess the quality of the automatic fact checking system. multiple efforts focus on improving text decoding with respect to different criteria. vijayakumar et al. (2016) and li et al. (2016) introduce alternative scoring strategies to diversify the hypothesis tree explored by beam search. @xcite propose top-k sampling, i.e., sampling from the top k tokens with the highest probability to generate stories. @xcite find that for the same neural language model, the choice of the decoding strategy can have a dramatic effect on the fluency and repetitiveness of the generation. they propose nucleus sampling as a way to increase diversity of the generated text while improving fluency. in our work, we find that while this strategy does create more fluent and less repetitive text, it does also result in a less factually true generation. @xcite choose to separate the generation and diversification steps altogether, and focus on leveraging content selection to map the input to diverse sequences. we describe various generation strategies in more detail in section 3. @xcite note that with nucleus sampling, per-token probabilities can be very low which they attribute to the likelihood training objective. they propose a novel unlikelihood training objective which lowers the probability of tokens in the context of the model. their approach is orthogonal to the decoding strategy and testing alternative training objectives is out of the scope of our paper. a recent approach by @xcite learns to distinguish human from machine generated text. @xcite investigate generating and detecting fake news using neural language models. @xcite propose a variation of the gpt-2 language model to explicitly generate malicious claims. instead of directly optimizing for generating fake or factual news, we are interested in investigating the relationship between the verifiability of the existing language models and different decoding strategies they are coupled with. fact checker a text generator a stephen hawking. stephen william hawking (8 january 1942 -14 march 2018) was an english theoretical physicist, cosmologist, and author who was director of research at the centre for theoretical cosmology at the university of cambridge at the time of his death. lm he was the author of several books, including a brief history of time. he was born in oxford, england, the son of frank and his wife jane hawking. he was the author of several books, such as a brief history of time. he was born with a rare form of dyslexia. his father was an electrical engineer and physicist. x x supported x x x verified x x x x unique supports per generation [spg] = 2/5 supports per verified [spv] = 2/3 unique supports per generation [uspg] = 1/5 wikipedia prefix lucy hawking was born in england to stephen hawking and jane hawking. generation strategy sentence processing the movie a concerning the life of stephen hawking, and his first wife, jane hawking. hawking 1 was born on 8 january 1942 in oxford to frank and isobel eileen hawking. wikipedia evidence ir sd refuted x unique supports per unique verified [uspv] = 1/2 figure 1: high level description of our experimental methodology that combines a language model (lm) with a fact checker, usually implemented combining an information retrieval (ir) and a stance detector (sd) component. @xcite . given that recent studies @xcite @xcite point to repetitiveness as one of the main problems affecting the generation of state-of-the-art models, we mainly consider this dimension in our analysis. ## background language models (lms) assign probabilities to sequences of tokens. given a context, that is, a sequence of tokens c @xmath0, w 2 , . . . , w t-1 ], autoregressive lms commonly estimate the probability distribution of the next target using neural models @xcite @xcite with: open-ended text generation as described in @xcite , the task of open-ended text generation involves producing a coherent completion of the provided context. we consider the common left-to-right generation, where a token at position t in the sequence is generated by considering the probability distribution over the vocabulary defined in equation 1. once a decision is made for w t according to a decoding strategy, it is incorporated into the context and the process is iterated -i.e., the token at position t + 1 is generated by considering p(w t+1 | c t+1 = [w 1 , . . . , w t ]). in this work, we consider different decoding strategies of selecting w t given p(w t | c t ). ## evaluating verifiability in this section we first describe the tools used to evaluate the verifiability of the generated text. we then formally introduce our repetitiveness and verifiability metrics. the high level overview of our evaluation setup is shown in figure 1 . for the purpose of this analysis, we consider both the text generator and the fact checker as black boxes which produce and assess text respectively. more specifically, the text generator gets in input a prefix p and produces a sequence of tokens that can be interpreted as a completion of p. we segment the generated completion into sentences and consider the first k sentences. the fact checker gets in input a sentence and outputs a positive (supported), negative (refuted) or unverifiable (not enough info) response as well as textual evidence used for the judgment. we consider a sentence as verified if the output label is either supported or refuted. our metrics assess the generation process given a set of prefixes p . the set p can be seen as the data source for our verifiability probe. let g @xmath1, ..., s p k ] be the sequence of sentences generated by the lm from prefix @xmath2. we indicate with v @xmath3, while with s @xmath4. to assess the verifiability of the generated text we introduce the following two metrics: supports per generation (spg): is the fraction of supported sentences among the generated ones: supports per verified (spv): is the fraction of supported sentences among the verified ones: spg can be interpreted as a sort of a recall metric while spv as a precision one. note that a generation could achieve a high score in terms of spg and spv by repeating the same supported sentence over and over again. to capture this behaviour, we define the unique variants of our metrics. we consider two sentences as equivalent if they have the same factuality label (i.e., sup-ported or refuted) and the decision is justified by the same evidence. for a set of equivalent sentences, we consider only the one which appeared first in the generation as unique. we denote the set of unique sentences as s p @xmath5, v p @xmath6. we introduce: unique supports per generation (uspg): the fraction of unique supported sentences among the generated ones: unique supports per unique verified (uspv): the fraction of unique supported sentences among unique verified sentences: in this section we describe in detail the implementational choices for all components in figure 1 . prefix dataset we retrieve title and description of the top-1000 most visited wikipedia pages of 2017 and 2018. for each page, we concatenate the title and the first sentence in the description to create a string prefix for the language model. we use 2018 data as validation set and run parameter sweeps over it. we tested the best configuration of every decoding strategy on 2017 data (test set). we ensure no overlap between 2017 and 2018 prefixes. language model we consider three sizes of language models (small, medium, large) based on the transformer architecture @xcite , with 124m, 354m and 1.4b parameters respectively. we train models on four corpora: (i) wikipedia, an english wikipedia dump consisting of roughly 2 billion words; (ii) books, the toronto books corpus @xcite , which consists of fiction books totaling about half a billion words; ## results we summarize the main results in sampling strategies (i.e., top-p and top-k) outperform the other strategies in terms of repetitiveness metrics, that is, they are able to generate text with a higher degree of diversity, consistently with previous works @xcite . however, diversity comes at a price, as the verifiability metrics are low (in particular, precision values -they generate more refuted sentences). intuitively, random choices might hamper verifiability when sampling a token in specific positions of the sentence, for instance, in a named entity, potentially making the overall sentence non factual. we notice that this problem gets even worse by increasing k or p. following a generation path that maximizes likelihood is a better approach for verifiability. in particular, bs achieves the highest performance in terms of spg and spv. nevertheless, generation diversity drops, consistently with previous works @xcite @xcite . solutions such as groupbs and siblingbs have been proposed to mitigate this problem, and their numbers actually look slightly better than bs in terms of repetitiveness metrics. when we assess diverse verifiability (that is, we consider distinct supported/refuted sentences), likelihood and sampling based strategies are similar in terms of recall (i.e., uspg), while likelihood-based solutions outperform both top-k and top-p in terms of precision (i.e., uspv) by a large margin -they generate less sentences refuted by the fact checker. delayedbs tries to combine the best of these two approaches, by defining a hybrid strategy that starts a sentence by sampling tokens and ends it by following a max-likelihood path. it achieves results comparable to likelihood-based solutions in terms of precision and recall for diverse verifiability while being much less repetitive (it almost doubles the number of distinct 4-grams). interestingly, it is sufficient to sample just the first token with high uncertainty (top-100) and finish the sentence with beam search to trigger this behaviour (figure 5 in the appendix section reports a detailed ablation study for the delay length). another way of mitigating repetitiveness is through n-gram blocking. we combine it with bs, sweeping over the values of n between 3 and 20. in line with our expectations, low n values score low in verifiability metrics, as the model is forced to explore less likely parts of the solution space in order to avoid generating previously observed n-grams. unsurprisingly, the diversity of the solution drops as n increases. in this sense, bs b and delayedbs attempt to strike a similar balance between diversity (introduced via n-gram blocking in bs b and via sampling in delayedbs) and verifiability (achieved by incorporating bs). figure 3 highlights this analogy further. overall, we achieve the best uspg performance by combining 20-gram blocking and bs -we believe it is due to the fact that n-gram blocking prevents bs from repeating the same phrases multiple times, while remaining 0 5 10 15 20 0 5 10 15 20 0 5 10 15 20 spg dbs: sampling length l bs b : n-gram size 60 65 70 75 80 85 0 5 10 15 20 0 5 10 15 20 spv dbs: sampling length l bs b : n-gram size 30 40 50 60 70 80 90 100 0 5 10 15 20 0 5 10 15 20 4-gram proportion dbs: sampling length l bs b : n-gram size bs b delayedbs figure 3: spg, spv and 4-gram proportion values for bs b and delayedbs, by varying the sampling length l for delayedbs (bottom axis) and the n-gram blocking size for bs b (top axis). ablation studies we experiment with different training corpora (figure 2a ) and different sizes of the transformer model (figure 2b ), using the validation set. we report uspv vs uspg values, taking inspiration from the popular precision-recall curve. the average perplexity of the small transformer model is the lowest for wikipedia (8.31) compared to books (53.08), openwebtext (11.14) and ccnews (12.23). even though all prefixes are likely to be in the corpus, wikipedia performance in terms of uspg is low regardless of the decoding strategy. this counter-intuitive behaviour seems to occur mainly due to the tendency of the small model trained on wikipedia to generate endless, unverifiable entity lists, mimicking wikipedia lists. ccnews leads to the best performance in terms of recall (uspg) for all decoding strategies, but also in terms of precision (uspv) for top-k and delayedbs. we did explore several other dimensions, including grammaticality (through a syntactic parser) and relevance (i.e., tf-idf score with the prefix wikipedia page) during our experiments (see we report examples of text generated by the large transformer model using different decoding strategies in the appendix section ( ## conclusion and discussion we presented a systematic analysis of the verifiability of text generated by a wide range of decoding strategies from large autoregressive language models. we assessed generated sentences with an offthe-shelf automatic fact-checker as well as through human annotations. we found that sampling decoding strategies produce text that is less verifiable, but also less repetitive when compared to strategies that consider most likely sequences according to the model distribution. we proposed a hybrid decoding strategy, combining the non-repetitive nature of sampling solutions with the verifiable generation of likelihood-based approaches. in our analysis, we considered the most viewed wikipedia pages in 2017 and 2018. our rationale was that such pages would represent topics that are likely to be highly covered in a random web crawl (e.g., openwebtext and ccnews). results (not reported in the paper) with a random set of wikipedia pages showed lower values in terms of spg and uspg (i.e., recall metrics). a potential line of future work could be to investigate relationships among training corpora and generation. we considered each sentence as a single claim to keep our experimental setting clean and avoid noise from an automatic claim extractor. however, some generations contain multiple claims that could be independently assessed. studying such phenomena is an interesting future direction.