Follow the Wisdom of the Crowd: Effective Text Generation via Minimum B ayes Risk Decoding (2023)

Abstract: In open-ended natural-language generation, existing text decoding methods typically struggle to produce text which is both diverse and high-quality. Greedy and beam search are known to suffer from text degeneration and linguistic diversity issues, while temperature, top-k, and nucleus sampling yield diverse but often lower-quality outputs. In this work, we build upon Minimum Bayes Risk Decoding (MBRD), a family of decoding methods based on Bayesian risk minimization, to address this diversity-quality trade-off. Inspired by the principle of the wisdom of the crowd, MBRD seeks to select a candidate from a pool of candidates that has the least expected risk under a generative model according to a given utility function. The crowd of candidates serves as an approximation for the distribution over human-generated references. We show that MBRD generalizes numerous decoding methods, including majority voting, and can be used as a drop-in replacement for existing sampling methods. Across a wide range of tasks—such as summarization, data-to-text, translation, and textual style transfer—MBRD yields 3-7 ROUGE and BLEU point improvements, including state-of-the-art results on WebNLG and WMT’16.

## introduction the past five years have witnessed the emergence of a new paradigm in natural-language processing (nlp), with large statistical language models achieving remarkable performance on a wide range of tasks and benchmarks. these language models have been subject to, and highly benefited from, numerous advancements in model architectures, training procedures, optimization techniques, and dataset size, among other factors. additionally, a large and continually-increasing number of tasks have been unified under the framework of 0 1 2 3 4 5 6 conditional text generation with natural-language prompts @xcite ). yet standard decoding approaches for text generation such as greedy decoding, beam search, and temperature/probability-based sampling have remained relatively unchanged, despite suffering from well-known and well-studied issues such as degeneration @xcite . in this work, we present a family of decoding methods based on minimum bayes risk decoding (mbrd). this method is inspired by the principle of the wisdom of the crowd, which suggests that the collective opinion of a large group of diverse critical thinkers is often better than the opinions of individual experts. mbrd builds on a large body of classical work @xcite @xcite which incorporates mbrd methods from decision theory into natural-language processing, and especially recent work @xcite that utilizes mbrd to improve neural machine translation. this paper utilizes these ideas in the context of text generation to develop a drop-in replacement for existing sampling methods which can be used with any underlying probabilistic generative model of language. mbrd generalizes two existing decoding methods from different areas of nlp. from one perspective, it can be seen as a soft version of the majority voting mechanism, which has recently been applied to a variety of nlp problems, including but not limited to arithmetic and symbolic reasoning @xcite , quantitative reasoning @xcite , and text-to-code translation @xcite . from another perspective, it can be seen as a generalization of prompt-and-rerank @xcite , a recently-proposed method for textual style transfer (tst). at a high level, mbrd involves first generating multiple candidates using a stochastic sampling method such as temperature sampling, then comparing each candidate to the other candidates according to a utility function, and finally selecting the candidate with the highest expected utility (or by duality, lowest expected risk). for most tasks, our utility functions of choice measure the semantic or lexical alignment between each candidate and the full candidate set (which serves as an approximation for the distribution over human-generated references, as we will explain in §3.1). we conduct extensive experiments across a wide range of conditional text generation tasks: summarization, data-to-text, translation, tst, and image captioning. mbrd significantly improves performance across the board, delivering 3-7 rouge improvements on highly-competitive benchmarks. these improvements are consistent across models, datasets, and even in the multimodal setting of image captioning. as a byproduct, we set new stateof-the-art results on both webnlg @xcite and wmt16 @xcite . taken together, our results show that mbrd can be used as a drop-in replacement for existing text decoding methods to improve text generation across a wide range of nlp settings. 2 2 prelude: probabilistic text generation 2.1 notation and setup let @xmath0, with v the set of tokens in a vocabulary and the superscript * the kleene star. let θ indicate the model parameters. then p θ (y | x) corresponds to the probability of generating the text y given the text x under the probabilistic model p θ . 3 modern probabilistic text generators such as neural language models or machine translation systems take an autoregressive (causal) approach to esti-mate p θ (y | x) and define it as follows: given an input text x, the decoding objective at inference time is to solve the following: ## minimum risk bayes decoding in this section, we first present the minimum risk bayes decoding (mbrd) framework for text generation. we then describe how mbrd generalizes two existing methods in nlp: majority voting @xcite and prompt-and-rerank @xcite . ## ablation studies & additional findings 6.1 varying the candidate pool size here we examine the effect of varying the number of candidate generations (i.e., |c|). when |c| = 1, there is a single candidate-this is equivalent to the sample-once method in section 4.4. as we increase |c|, the candidate set c becomes better representative of the underlying distribution of possible human-written references, and one would hope that our method would select high-quality generations. in with bertscore delivers 5.9 r-1, 5.7 r-2, 6.7 r-l, and 5.1 bleu score improvements. 10 we observe similar phenomena in other nlg tasks and datasets. these findings uphold the advantage of average crowd prediction over individual predictions. ## conclusion in this work, we investigated the foundations and benefits of mbrd, a family of decoding methods based on an empirical risk minimization approach in bayesian statistics. mbrd can be utilized with any generative language model and any stochastic sampling method. to the best of our knowledge, our work is the first to demonstrate the efficacy of mbrd across a wide range of open-ended text generation tasks-including summarization, data-to-text generation, textual style transfer, and image captioning, in addition to machine translation, delivering 3-7 rouge and bleu score improvements over standard sampling methods. furthermore, it achieves new state-of-the-art results in webnlg and wmt16, outperforming the fewshot performance of palm 540b. as a whole, these results show that mbrd can be used in a drop-in manner to improve a wide range of nlg tasks.