GSD: Generalized Stochastic Decoding

1 INTRODUCTION . Benefiting from large-scale pre-trained language models ( Radford et al. , 2019 ; Lewis et al. , 2020 ; Tseng et al. , 2020 ; Sun & Yang , 2020 ) , considerable advances have been observed in many natural language generation ( NLG ) tasks . Nevertheless , given these pre-trained models , only limited decoding strategies are available to use , such as Greedy , Sampling , Beam Search , Top-k ( Fan et al. , 2018 ; Holtzman et al. , 2018 ) , and the recently proposed nucleus sampling ( i.e. , top-p ) ( Holtzman et al. , 2020 ) . These decoding algorithms are essential because virtually all text generation tasks need them to transfer inferred predictions to successive text ( Basu et al. , 2021 ) . However , few of these decoding algorithms can resist the risk of text degeneration . That is , text produced by these algorithms exhibits quite different characteristics from those used by humans , which contains many generic words , repeated loops , and irrelevant sentences . This problem has been confirmed by many previous works ( Holtzman et al. , 2020 ; Welleck et al. , 2020b ) . Some researchers deem that it is ineluctable because generators do empower high probabilities to the correct words , but locally , the highest probability can only be assigned to the text with generic , repetitive , or meaningless phrases . However , we suppose that this problem is triggered by the distinct manners between humans and the decoding algorithms about how text is produced . Concretely , it is very odd for humans to be told to utter more diversely or more fluently . Diverse or not , it usually depends on personal habits of language usage . Similarly , fluency or not , it is generally determined by people ’ s familiarity with that language ( patients with language disorders are excluded ) . For most decoding algorithms that are currently popular and widely used since virtually all of them must be configured with at least one hyper-parameter before being available to use . Then , these hyper-parameters become the only exceptions in the decoding system that are not controlled by the algorithms , which brings many uncertainties to the decoding system . Based on these observations , we propose a novel decoding algorithm with a hyper-parameter-free design . All behaviors of our decoding method are automatically controlled and dynamically adjusted by the algorithm itself , which is distinctive from all popular decoding algorithms currently developed . We name this novel method as Intrinsic Decoding . Detailed comparisons between Intrinsic Decoding and existing algorithms will be described in the following sections . More importantly , we also propose a generalized decoding framework ( GSD ) to connect these decoding algorithms in formal mathematical theorems , including both existing decoding methods and our proposed Intrinsic Decoding . The relationships between GSD and these algorithms can be vividly expressed in Figure 1 . 2 BACKGROUND . 2.1 PROBLEM STATEMENT . Given a vocabulary V and a pre-trained generator G , a general text generation task is to produce a sequence of words x = { xi } , xi ∈ V conditioned on some specific types of context xδ , where δ represents a span containing x . The context can be either text ( e.g. , in summarization ( Shen et al. , 2019b ; Matsumaru et al. , 2020 ) , translation ( Chen et al. , 2020 ; Wang et al. , 2020 ) , story-telling ( Ippolito et al. , 2020 ) tasks ) , images ( e.g. , image caption ( Alikhani et al. , 2020 ) ) , or structured input ( e.g. , AMR-to-text ( Liu et al. , 2021 ) , tabular-to-text ( Li & Rush , 2020 ) ) . For open-domain generation tasks ( Goldfarb-Tarrant et al. , 2019 ; Prabhumoye et al. , 2019 ) , both xδ and x are text tokens . Specifically , G can be pre-trained using any properly configured neural models , such as GRU ( Su et al. , 2020b ; Cho et al. , 2014 ; Shi et al. , 2020 ) , Transformer Vaswani et al . ( 2017 ) ; Thapliyal & Soricut ( 2020 ) , it accepts a span of context xδ , performs inference , and finally gives a batch of predictions indicating the possibility for each word x ∈ V that may appear after each individual x ∈ δ . Formally , we represent such predictions by a distribution P̂ as defined in Equation 1 . P̂ ( x|xδ ) = softmax ( Gxδ ) ( 1 ) In the following sections , we refer to P̂ as the inference distribution , which is fully controlled by G. Based on P̂ , the decoding task aims to answer the question of how to get x from P̂ ( x|xδ ) ? Intuitively , a naive solution is to argument-maximize P̂ iteratively until encountering an End-ofSequence ( EOS ) token , which is known as Greedy Decoding . Simple as it is , Greedy Decoding is fast to run and straightforward to interpret . While just as implied by its name , Greedy Decoding shares the same weakness as other greedy-based algorithms . That is , locally optimized searching does not make any promise to the global best answer . Indeed , both Greedy Decoding and Beam Search are approximations of the MAP decoding . To visualize the detailed decoding process of these deterministic algorithms , we plot the probabilities in each step to generate an example . As depicted in Figure 2.1 , probabilities assigned by Greedy Decoding and Beam Search are pretty different from probabilities of the samples obtained from data generated by humans . get x from P̂ ( x|xδ ) is to sample , instead of to search the solution space defined by V and G. This strategy inspires a batch of new decoding algorithms developed for various NLG tasks . These algorithms are generally referred to as stochastic approaches , which will be detailed in the next section . 2.2 RELATED WORK . In this section , we discuss the properties of some popular decoding algorithms that are widely used ( Kang & Hashimoto , 2020 ; Su et al. , 2020a ) in both industry and research NLG domains . For deterministic decoding , many recent works ( Murray & Chiang , 2018 ; Stahlberg & Byrne , 2019 ; Welleck et al. , 2020a ) have been proposed to investigate how to better control the quality of generated text . For instance , Murray & Chiang ( 2018 ) draws an important conclusion that for beam search , it is not always true that a wider beam will help translation , sometimes it hurts . Stahlberg & Byrne ( 2019 ) shows constraining search with a minimum translation length is at the root of the problem of empty translations . Recently , some works have been proposed to investigate how to explicitly control the property of the generated text by manipulating the decoding hyper-parameters under some specific principles . For example , Pang & He ( 2020 ) model generation as an offline reinforcement learning task with reference demonstrations . It aims to maximize quality given model-generated histories . Nadeem et al . ( 2020 ) propose to identify key properties that are shared among some decoding algorithms and investigate what will happen if meeting or violating the identified properties . ( Basu et al. , 2021 ) propose to design a feedback-based adaptive top-k decoding algorithm that generates text with a predetermined perplexity . However , none of them aim to design a generalized decoding framework that can be used to describe and connect existing popular decoding algorithms , which is just the focus of our paper . For clarity , we denote the decoding probability assigned by a decoding algorithm by P ( x|xδ ) , while keeping to use P̂ ( x|xδ ) as in Section 2.1 to represent the generation probability inferred by a pretrained NLG model G. Recall the sub-vocabulary observations about how human communicates introduced in Section 2.1 . Surprisingly , we find that these stochastic decoding algorithms are also related to some sub- vocabulary equivalents or subsets if formally described by mathematical language . To describe these different stochastic decoding methods uniformly , we use Equation 2 to abstract the relationship between P ( x|xδ ) and P̂ ( x|xδ ) . Different from V , ∆ defined in Equation 2 is exactly the sub-vocabulary equivalent in mathematics containing relevantly important words for the decoding steps . It is also the key difference to discriminate between these stochastic decoding methods . P ( x|xδ ) = { φ ( P̂ ( x|xδ ) ) , x ∈ ∆ 0 , x /∈ ∆ ( 2 ) We briefly introduce the theoretical backgrounds of Sampling Decoding , Top-k Decoding , and Nucleus Decoding ( i.e. , Top-p ) , especially how different ∆ is constructed by these stochastic methods . Among these methods , Sampling Decoding may be the simplest one because of its most flexible hypothesis of ∆ : ∆ is free to include any elements in V . In other words , ∆ is V itself . Therefore , there is no algorithm-specific sub-vocabulary . All words in the entire dictionary space are candidates to sample from . This behavior can be formally described by Equation 3. φ ( P̂ ) = P̂ ( x|xδ ) , ∆ = V ( 3 ) Sampling Decoding extends the search space to V while maintaining high decoding efficiency compared to the deterministic methods . This is quite different from Beam Search , whose speed is very sensible to the search width parameterw . However , as remained by Holtzman et al . ( 2020 ) , the problem with Sampling Decoding is that it has a long probability tail , which brings a lot of irrelevant samples in the decoding process . To alleviate this problem , a ∆-limited decoding method , Top-k sampling , is proposed to decode text from pre-trained generators ( Fan et al. , 2018 ; Holtzman et al. , 2018 ; Radford et al. , 2019 ) . Unlike Sampling Decoding , Top-k sampling has a limited subset of ∆ , which consists of the top-k most promising candidates for the algorithm to consider . Equation 4 formally describes the conditions of how a proper ∆ subset should be constructed . S = { Ω|min { P̂x∈Ω ( x|xδ ) } ≥ max { P̂x∈V−Ω ( x|xδ ) } , |Ω| = k , x ∈ V } ( 4 ) Given Equation 4 , Top-k sampling can be precisely defined by Equation 5 . Notably , S is defined as a super-set containing all valid ∆ sets that can be used by Top-k. A benefit of defining ∆ in such a way is that it takes into account all special and general cases of P̂ . This definition holds even if there exist identical elements in P̂ . φ ( P̂ ) = P̂ ( x|xδ ) / ∑ x∈∆ P̂ ( x|xδ ) , ∆ ∈ S ( 5 ) Top-k sampling has a similar subset ∆ to integrate the relevantly significant words for a given context . Promising as it is , the challenge for top-k is that the hyper-parameter of k is hard to specify ( Holtzman et al. , 2020 ) : large k is problematic for flat P̂ , while small k is precarious for peaked P̂ . Based on the observation , Holtzman et al . ( 2020 ) proposes a novel method to define ∆ , the size of which is dynamically changed for different shapes of P̂ . As described by Equation 6 , the limitation for ∆ is relaxed to the aggregated mass of the probabilities , instead of the size of ∆ containing these probabilities . This method is known as Nucleus Sampling or Top-P Decoding , the definition of which is formally described by Equation 7 . S = { Ω| ∑ x∈Ω P̂ ( x|xδ ) ≥ p , x ∈ V } ( 6 ) φ ( P̂ ) = P̂ ( x|xδ ) / ∑ x∈∆ P̂ ( x|xδ ) , ∆ ∈ S , |∆| = min { |Ω| , ∀Ω ∈ S } ( 7 ) Despite the relatively complex form of definition , Equation 7 can be efficiently executed by filtering the accumulated sum of P̂ ) . Given the lesson learned from Top-k , an akin question for Top-p sampling is how the hyperparameter p is determined ? General answer for this question is that small p values force the decoding algorithm to generate fluent expressions , while large p values encourage diverse generations . Then , what if p is small because we want fluent terms while P̂ is flat ? Or what will happen if P̂ is peaked , but p has been set to a large value to favor our diverse demand ? Indeed , similar questions could be endless as long as there is a hyper-parameter affecting the performance of the decoding algorithm . Except for Sampling Decoding and Greedy Search , all decoding algorithms discussed so far involve at least one hyper-parameter managing specific properties of that method . Unfortunately , the two exceptions both have known issues that stop them from being considered as potential alternatives . These are the key observations inspiring our work .

Summary: This paper proposes a fairly general approach to generating from language models: words are sampled from a renormalized distribution consisting of the smallest set of words in the vocabulary which together have probability at least $\rho$, where $\rho$ is a function of the entropy of the distribution. The authors show that several standard decoding algorithms arise from particular choices of $\rho$. The authors then propose to use a particular $\rho$, essentially $\tanh(\text{entropy}/\text{average batch entropy})$, which they argue is attractive because it has no hyperparameters. The authors evaluate the generations obtained from decoding with their approach by comparing them to baseline generations in terms of perplexity assigned to generations by GPT-2, self-BLEU, and other recent automatic metrics.