Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding (2024)

Abstract: One of the most important challenges in text generation systems is to produce outputs that are not only correct but also diverse. Recently, Minimum Bayes-Risk (MBR) decoding has gained prominence for generating sentences of the highest quality among the decoding algorithms. However, existing algorithms proposed to generate diverse outputs are predominantly based on beam search or random sampling, thus their output quality is capped by these underlying decoding algorithms. In this paper, we investigate an alternative approach – we develop diversity-promoting decoding algorithms by enforcing diversity objectives to MBR decoding. We propose two variants of MBR; (i) Diverse MBR (DMBR) that adds a diversity penalty to the decoding objective and (ii) k-medoids MBR (KMBR) that reformulates the decoding task as a clustering problem. We evaluate DMBR and KMBR on a variety of directed text generation tasks using encoder-decoder models and a language model with prompting. The experimental results show that the proposed methods achieve a better trade-off than the diverse beam search and sampling algorithms overall. Our code is available at https://github. com/CyberAgentAILab/diverse-mbr/.

text: Findings of the Association for Computational Linguistics ACL 2024, pages 8494–8525 August 11-16, 2024 ©2024 Association for Computational Linguistics
One of the most important challenges in text generation systems is to produce outputs that are not only correct but also diverse. Recently, Minimum Bayes-Risk (MBR) decoding has gained prominence for generating sentences of the highest quality among the decoding algorithms. However, existing algorithms proposed to generate diverse outputs are predominantly based on beam search or random sampling, thus their output quality is capped by these underlying decoding algorithms. In this paper, we investigate an alternative approach – we develop diversity-promoting decoding algorithms by enforcing diversity objectives to MBR decoding. We propose two variants of MBR; (i) Diverse MBR (DMBR) that adds a diversity penalty to the decoding objective and (ii) k-medoids MBR (KMBR) that reformulates the decoding task as a clustering problem. We evaluate DMBR and KMBR on a variety of directed text generation tasks using encoder-decoder models and a language model with prompting. The experimental results show that the proposed methods achieve a better trade-off than the diverse beam search and sampling algorithms overall. Our code is available at https://github. com/CyberAgentAILab/diverse-mbr/.
1 Introduction
There are many reasons why natural language generation systems want to produce outputs that are not only correct but also diverse. For example, in systems involving reranking candidate outputs, the reranking algorithms are more effective when the candidates are diverse (Gimpel et al., 2013; Li and Jurafsky, 2016; Li et al., 2016; Choudhary et al., 2017). In image captioning, an image may contain many concepts with multiple levels of detail. To achieve human-level image captioning, it is important for models to be able to output a variety of captions covering such diverse information (Wang and Chan, 2019). As an application,
Krause et al. (2017) shows that a diverse set of image captions can be transformed into an entire descriptive paragraph explaining the image. For question generation tasks, a diversity-promoting question generation system can improve question answering systems (Sultan et al., 2020) and enhance the engagement in chatbots (Laban et al., 2020).
This importance of diversity in text generation has brought many studies aimed at producing diverse outputs instead of the most probable ones. The majority of the approaches are based on either random sampling (Fan et al., 2018; Ippolito et al., 2019; Holtzman et al., 2020; Hewitt et al., 2022) or beam search (Cho, 2016; Li and Jurafsky, 2016; Vijayakumar et al., 2016, 2018; Kulikov et al., 2019; Tam, 2020), thus the output quality is bounded by the quality of either random sampling or beam search.
In this paper, we develop diverse decoding methods by extending Minimum Bayes Risk (MBR) decoding (Goel and Byrne, 2000; Kumar and Byrne, 2002, 2004; Eikema and Aziz, 2022). MBR decoding is shown to generate higher quality sentences than random sampling and beam search in directed text generation tasks including machine translation, text summarization, and data-to-text (Freitag et al., 2022; Suzgun et al., 2023). The procedure of the MBR decoding consists of the following steps. First, it samples a set of candidate outputs from the probability model. Then, it computes the similarity of each sequence to the others according to a utility function. Finally, it selects the sequence that maximizes the expected utility over the sequences. A naive approach to generate k outputs using MBR is to select the top-k outputs with the highest expected utility. However, it tends to select a set of similar sentences with a large overlap (see Appendix C for examples).
We propose two approaches to promote diversity in MBR: Diverse MBR (DMBR) and k-Medoids
8494
MBR (KMBR). DMBR extends MBR by introducing a diversity penalty to the objective so that it maximizes the weighted sum of the expected utility and diversity. DMBR can tune the quality-diversity trade-off by the weight hyperparameter. KMBR selects a set of sentences by solving the k-medoids problem (Rdusseeun and Kaufman, 1987; Kaufman and Rousseeuw, 2009). k-medoids is a variant of k-means where the center points are restricted to be one of the data points instead of anywhere in the space. We pick a center point from each cluster to generate diverse and high quality outputs.
We evaluate DMBR and KMBR on machine translation, image captioning, question generation, generative common sense reasoning, and text summarization. The experimental results show that DMBR and KMBR achieve better trade-offs than diverse beam search and sampling algorithms. We also observe that DMBR and KMBR achieve higher Oracle quality scores in all the tasks.
2 Background
Sequence-to-sequence generation is the task of generating an output sequence y given an input sequence x. Probabilistic text generators define a probability distribution pθ(y|x) over an output space of hypotheses Y conditioned on an input x. The set of complete hypotheses Y is:
Y := {BOS ◦ v ◦ EOS|v ∈ V∗}, (1)
where ◦ is a string concatenation and V∗ is the Kleene closure of a set of vocabulary V . Typically, the goal of decoding is to find the hypothesis that best matches the human preference Phuman:
h∗ = argmax h∈Y Phuman(h|x). (2)
In this paper, we consider a variant of the set decoding problem (Meister et al., 2021) where the task is to generate a set of k sentences H that maximizes the sum of Phuman and the diversity according to the preference of human dhuman:
H∗ = argmax H⊆Y
∑ h∈H Phuman(h|x) + dhuman(H).
(3) In this paper, we use automated evaluation metrics to approximate dhuman. In particular, we consider P-BLEU, distinct-n, and P-SentBERT as measures of diversity (Shen et al., 2019; Li et al., 2016; Reimers and Gurevych, 2019).
2.1 Decoding Algorithms for Diversity
There have been two major approaches to diversityaware text decoding: random sampling and diversity-aware beam search.
Random sampling. Random sampling is commonly used to generate diverse outputs in both directed and open-ended text generation tasks. A simple solution is to use an ancestral sampling with a temperature parameter to control the stochasticity of the sampling. There have been a lot of studies on biasing the ancestral sampling to generate higher quality outputs while maintaining the diversity of the randomized algorithm. Prior work shows that top-k sampling that restricts the sampling to the ktop most likely tokens at each step is a better alternative to controlling the temperature in a story generation task (Fan et al., 2018; Ippolito et al., 2019). Nucleus sampling (Holtzman et al., 2020) is similar to top-k sampling and has shown to be more effective than top-k sampling in WebText dataset (Radford et al., 2019). Nucleus sampling truncates all tokens except those in the nucleus, the smallest possible set of tokens that covers a fraction p of the model probability. Epsilon sampling (Hewitt et al., 2022) is also a variant of ancestral sampling that truncates tokens whose probability is less than a threshold. They are shown to be more effective than nucleus sampling in the WebText dataset. However, these random samplings improve the diversity of outputs at the expense of the quality of outputs (Ippolito et al., 2019).
Diversity-aware beam search. Another line of work is to generate a diverse set of outputs by introducing diversity objectives to the beam search. Beam search is known to produce higher quality sequences than random sampling in a wide range of tasks (Graves, 2012; Sutskever et al., 2014). Diversity has been induced in the beam search procedure in various forms. Li and Jurafsky (2016) propose to add a diversity constraint to standard beam search so that the number of descendants from the same parent hypothesis is capped at some upper bound. Noisy Parallel Approximate Decoding induces noise to the hidden state of the decoder to generate randomized outputs (Cho, 2016). Diverse beam search (DBS) adds a diversity term to the reranking objective, penalizing sequences with a small Hamming distance to other groups of the sequences (Vijayakumar et al., 2016, 2018). Iterative beam search runs beam search multiple times
with a constraint that any partial hypothesis that has been generated in previous iterations is prohibited (Kulikov et al., 2019). Clustered beam search prunes similar sequences by clustering the candidates using a word embedding at each decoding step (Tam, 2020). Post-Decoding Clustering (PDC) encourages diversity by running the clustering after generating a large number of outputs, selecting good candidates from each cluster (Kriz et al., 2019; Ippolito et al., 2019). Best-k search maintains best-first queue in a course of beam search, resulting in higher quality and diversity outputs (Xu et al., 2023a).
2.2 Minimum Bayes Risk (MBR) Decoding
One of the most common decision rules to solve the decoding problem is maximum-a-posteriori (MAP) decoding such as beam search. MAP decoding finds the most probable translation under the model.
hMAP = argmax h∈Y P (h|x). (4)
In this paper, we denote P (h|x) as P (h) for simplicity. Although it seems intuitive to compute this MAP objective, previous work has pointed out two critical problems with this strategy. First, because the size of the hypothesis set |Y| is extremely large, it is intractable to solve optimally. Second, the MAP objective often leads to low quality output (Stahlberg and Byrne, 2019; Holtzman et al., 2020; Meister et al., 2020). Indeed, Stahlberg and Byrne (2019) show that hMAP is often found to be the empty sequence in their experimental setting.
Unlike MAP decoding, which searches for the most probable output, MBR decoding searches for the output that maximizes expected utility, which is equivalent to minimizing risk (Goel and Byrne, 2000; Kumar and Byrne, 2002, 2004). The procedure consists of two components: a text generation model and a utility metric. The model Pmodel(y) estimates the probability of an output y given an input sentence x. The utility metric u(h,y) estimates the quality of a candidate output h given a reference output y. Given a set of candidate hypotheses Hcand ⊆ Y , MBR decoding selects the best hypothesis according to its expected utility:
hhuman = argmax h∈Hcand
∑ y∈Y u(h,y) ·Phuman(y). (5)
Since Phuman is unknown, MBR instead uses the
model probability Pmodel to approximate Phuman.
hmodel = argmax h∈Hcand
∑ y∈Y u(h,y) · Pmodel(y). (6)
For the rest of the paper, we will denote Pmodel as P for simplicity, unless otherwise noted. Since integration over Y is computationally intractable, Eq. (6) is approximated by a Monte Carlo estimate (Eikema and Aziz, 2022; Farinhas et al., 2023) using a set of reference hypotheses Href ⊆ Y sampled from the model P :
hMBR = argmax h∈Hcand
1
N
∑
y∈Href u(h,y), (7)
where N = |Href |. Standard practice is to use the same set of hypotheses for the candidate pool (H) and the reference pool (Href ), therefore H = Href .
3 Minimum Bayes Risk Decoding with Diversity
We now introduce MBR decoding to the set decoding problem with diversity objective (Eq. 3). A naive way to generate k sentences by MBR decoding is to select the top-k hypotheses by Eq. (6):
HMBR = argmax H⊆Hcand |H|=k
∑
h∈H
1
N
∑
y∈Href u(h,y) (8)
However, it results in a set of similar sentences with a large overlap. In fact, it often finds almost duplicated sentences in machine translation tasks (Tables 2 and 3 in Appendix C).
3.1 Diverse MBR (DMBR) We propose Diversity MBR (DMBR) decoding, a variant of MBR decoding with a diversity penalty d : 2Y → R added to the decoding objective. The objective of the DMBR is the following:
HDMBR =
argmax H⊆Hcand |H|=k
∑
h∈H
  1 N ∑
y∈Href u(h,y)
 + d(H)
(9)
The objective for HDMBR consists of two terms: quality objective and diversity objective. The quality objective is the expected utility, the same as the original MBR HMBR. d(H) is the diversity objective that penalizes ac-
cording to a user-defined diversity objective. In
this paper, we aim to promote diversity by minimizing the pairwise similarity among the outputs. Since the utility function typically computes the similarity between each pair of texts, we minimize the following pairwise utility:
d(H) = − ∑
h∈H
∑
h′∈H\{h}
λ
|H|u(h,h ′). (10)
Assuming u > 0, Eq. (9) with the pairwise similarity results in a non-monotonic submodular function maximization problem (Buchbinder and Feldman, 2018) (proof in Appendix A). Because solving a non-monotonic submodular function maximization problem is NP-hard (Feige, 1998), we deploy a greedy heuristic algorithm. We greedily select a hypothesis that maximizes the objective until we have k hypotheses. This procedure is guaranteed to find a solution with an approximation factor of (1− 1e ), provided that λ is small enough to ensure the function is non-decreasing; otherwise, the approximation factor is slightly worse than (1 − 1e ) (Nemhauser et al., 1978). Note that DMBR still needs to compute u(h, r) for every pair of candidates and references. Thus, even with the approximation, DMBR is at best as slow as MBR.1
3.2 k-Medoids MBR (KMBR)
As an alternative approach to promote diversity, we propose k-Medoids MBR (KMBR) decoding. kMedoids is a clustering problem similar to k-means in that it chooses centers to minimize the total distance of data points to the closest center points. The difference is that k-Medoids needs to choose actual data points as centers (Rdusseeun and Kaufman, 1987; Kaufman and Rousseeuw, 2009). Intuitively, k-Medoids center points are supposed to be representative of different clusters of hypotheses. Thus, picking the center points for clusters is likely to result in a set of diverse and high-quality hypotheses. KMBR can be understood as a generalization of the vanilla MBR decoding, which is solving the 1-Medoid problem to find the single most centered hypothesis out of the sampled hypotheses (Jinnai and Ariu, 2024). We use the negative utility as a distance and consider the problem of picking a set of k center points. The total distance from the reference set (= data points) to the picked center
1In our experiments, DMBR with 128 samples was roughly 2-4 times slower than diverse beam search with k = 4 on g4dn.xlarge instances on AWS EC2 (4 vCPU cores, 16 GB memory, and an NVIDIA T4 GPU).
points is minimized as follows:
HKMBR = argmax H⊆Hcand |H|=k
∑
y∈Href min h∈H
−u(h,y). (11)
k-medoids can be used with arbitrary dissimilarity measures. Because the k-medoids problem is NP-hard to solve exactly, we deploy Partition Around Medoids (PAM) to compute HKMBR approximately (Park and Jun, 2009). Similarly to DMBR, KMBR needs to compute the utility function for every pair of hypotheses so it is still slower than MBR even with the approximation algorithm.
4 Experiments
We evaluate DMBR and KMBR in machine translation, image captioning, question generation, generative common sense reasoning, and text summarization. All experiments use BERTScore (Zhang* et al., 2020) as the utility function of the MBR. We compare the performance of the sampling algorithms, beam search, diverse beam search (DBS) (Vijayakumar et al., 2018), MBR, diverse MBR (DMBR), and k-Medoids MBR (KMBR).
We evaluate the quality and the diversity of the generated texts. As a quality metric, we report the mean, max, and min quality over each set of k sentences measured by BLEU, ROUGE-L, or METEOR (Papineni et al., 2002; Lin and Och, 2004; Banerjee and Lavie, 2005). We use distinct-n (Li et al., 2016) and pairwise-BLEU (P-BLEU) (Shen et al., 2019) as diversity metrics.
Due to limitations in computational resources, we run experiments using the first 1000 entries of the dataset for all the experiments in this paper. We use Huggingface’s Transformers library for running all the experiments (Wolf et al., 2020). We initialize the clusters for PAM used in KMBR by k-medoids++ with maximum iterations set to 300. For reproducibility, all the experiments are conducted using publicly available pretrained models and datasets. We use sacreBLEU system (Post, 2018) to compute BLEU scores.
Examples of generations are shown in Appendix C. See Appendix E for additional figures and the summary of the experimental results in tables.
4.1 Machine Translation We use the WMT’19 dataset (Barrault et al., 2019) to evaluate the performance on machine translation tasks. WMT’19 dataset examines translation
between English and other languages in the news domain. We run experiments on two language pairs: German → English (De→En) and Russian → English (Ru→En) using the pretrained models of each language pair provided by fairseq (Ng et al., 2019).
We set the number of outputs to be k ∈ {4, 8, 12}. We compare the performance of sampling algorithms, diverse beam search, and the proposed methods. For sampling algorithms, we evaluate ancestral sampling, nucleus sampling with p ∈ {0.8, 0.9, 0.95}, top-k sampling with ktop ∈ {5, 10, 50}, epsilon sampling with ϵ ∈ {0.01, 0.02, 0.04}, and temperature sampling with T ∈ {0.8, 0.9} (Holtzman et al., 2020; Fan et al., 2018; Hewitt et al., 2022). For DBS (Vijayakumar et al., 2018), we set the number of groups to be equal to the beam width (k). The strength of diverse penalty λ for DBS is set to {0.0 (beam search), 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0}. For MBR-based decoding methods, we set the sample size N = 128 per source sentence (see Eq. 7) and use epsilon sampling with ϵ = 0.02 (Hewitt et al., 2022; Freitag et al., 2023). The diversity penalty λ is set {0.0 (vanilla MBR), 0.1, 0.3, 0.5, 1.0, 2.0}.
Due to space limitations, we discuss the essential results here and show the detailed results in
Tables 9, 10, and 11 in Appendix E.
DMBR achieves higher diversity than baselines. Figure 1 shows the P-BLEU and distinct-2 as a function of mean BLEU score. The results show that DMBR achieves higher diversity (lower PBLEU and higher distinct-2) than DBS and sampling algorithms with the same mean BLEU score.
DMBR achieves more flexibility than DBS on the quality-diversity trade-off. We observe that DBS does not increase the diversity by increasing the diversity penalty larger than 10.0 (Table 9 in Appendix). On the other hand, the diversity of DMBR continue to increase with larger λ, achieving higher maximum diversity than DBS. DMBR also achieves higher diversity than the sampling algorithms.
DMBR achieves higher oracle score than vanilla MBR. The Oracle score indicates the score of the highest-scoring output in a set of k outputs (Vijayakumar et al., 2018). It is intended to evaluate how well the obtained diversity benefits in producing output close to a particular correct answer. Figure 2 shows the P-BLEU and distinct2 as a function of max BLEU score (i.e., Oracle score). We observe that DMBR achieves a slightly
lower max BLEU score than DBS, yet with higher diversity. Interestingly, we observe that DMBR and KMBR achieve higher max scores than vanilla MBR (Table 9). This shows the potential of DMBR to further improve the quality of the output of MBR. This is analogous to DBS achieving a higher Oracle score than beam search (Vijayakumar et al., 2016). KMBR achieves the highest max BLEU score over all algorithms compared.
DMBR outperforms DBS with varying output sizes. Figure 3 shows the quality and diversity trade-off with varying output size k ∈ {4, 8, 12}. DBS shows no degradation of P-BLEU but shows a lower distinct-2 score with a larger output size. Overall, we observe that DMBR outperforms DBS in both diversity metrics with all the output sizes we evaluated.
DMBR improves with a larger number of samples. Figure 4 shows the performance of DMBR and KMBR with varying numbers of samples N ∈ {32, 64, 128}. We observe that the qualitydiversity trade-off is improved with a larger sample
size, yet it does not significantly improve by doubling the sample size. The results indicate that one can improve both quality and diversity by increasing the sample size, but it comes at the cost of inference time. Note that the computational complexity of DMBR is quadratic to the number of samples.
Choice of the underlying sampling algorithm matters. Figure 5 shows the comparison of DMBR with varying sampling algorithms. We evaluate ancestral sampling, nucleus sampling with p ∈ {0.8, 0.9, 0.95}, top-k sampling with ktop ∈ {5, 10, 50}, and epsilon sampling with ϵ ∈ {0.01, 0.02, 0.04} (Holtzman et al., 2020; Fan et al., 2018; Hewitt et al., 2022). The temperature is set to 1 for all the runs. We observe that the choice of the sampling algorithm changes the Pareto front of the DMBR. Using less biased sampling algorithms such as ancestral sampling, DMBR improves the diversity, and using more focused sampling algorithms such as epsilon sam-
pling, DMBR improves the mean quality of the outputs.
4.2 Image Captioning using BLIP-2
We evaluate the performance of the proposed methods on image captioning using MS COCO dataset (Lin et al., 2014). We use BLIP-2 (Li et al., 2023) with Flan T5-xl (Chung et al., 2022) finetuned for MS COCO. We load the model in 8- bit to reduce the VRAM consumption. The number of outputs k is set to {4, 8, 12}. We evaluate the performance of epsilon sampling with ϵ ∈ {0.01, 0.02, 0.04}. The diversity penalty for DBS is set to {0.0, 0.2, 0.5, 1.0, 2.0, 5.0}. For DMBR, we generate N = 64 samples using epsilon sampling with ϵ = 0.02. The results on k = 4 are shown in Figure 6 (a, d, g). DMBR achieves lower P-BLEU and higher distinct-2 than DBS and epsilon sampling.
Evaluation of Semantic Diversity. While the machine translation task requires generating a text semantically the same to the input text, the image captioning task allows more diversity in the contents of the output (Wang and Chan, 2019). To evaluate the semantic diversity beyond the surface diversity, we employ sentence BERT (Reimers and Gurevych, 2019). We compute the embedding of each output using the sentence BERT and compute the cosine similarity of each pair of outputs. The cosine similarity over a pair of sentences is shown to have a high correlation to human preference (Tevet and Berant, 2021). We evaluate the pairwise sentence BERT (P-SentBERT), the average cosine similarity of the sentence embeddings over a set of pairs of outputs. We use ALL-MPNET-BASE-V2 model. The model is based on MPNet (Song et al., 2020) and has shown to be effective for a variety of sentence embedding tasks. The result in Figure 6g shows that DMBR achieves better (lower) P-SentBERT than DBS and epsilon sampling.
4.3 Question Generation using Language Model
The goal of question generation is to generate a question on a topic in natural language from a paragraph of text (Mulla and Gharpure, 2023). We use a Stanford Question Answering Dataset (SQuADv2) to evaluate the decoding algorithms for question generation (Rajpurkar et al., 2016, 2018). SQuADv2 is a reading comprehension dataset consisting of questions and answers on Wikipedia articles. We use a language model Zephyr-7B β (Tunstall et al., 2023) with prompting as a text generation model. We use the following prompt:
Given a paragraph provided by the user, generate a very short question one can answer by a word to test the understanding of the paragraph. Make sure that the question is very short. Do NOT include the answer. We generate k ∈ {4, 8, 12} outputs. For DBS, we set the diversity penalty to {0.0, 0.5, 1.0, 2.0, 5.0}. For MBR we generate 128 samples with epsilon sampling with ϵ = 0.01. The diversity penalty λ for DMBR is set to {0.0, 0.1, 0.3, 0.5, 1.0, 2.0}. The mean METEOR and the diversity metrics are shown in Figure 6 (b, e, h). Compared with the same METEOR score, DMBR has better distinct2 and P-SentBERT than DBS. The P-BLEU and P-sentBERT of DMBR are slightly worse than DBS and epsilon sampling compared with the same mean METEOR score.
We speculate that DMBR underperforms DBS for SQuADv2 and CommonGen (Section 4.4) because DBS is more likely to generate a set of sequences of varying lengths. Table 1 shows the average standard deviation of the sequences generated by DMBR and DBS with varying diversity penalties. DBS tends to generate a set of diverse lengths of sentences than DMBR in these domains. Because P-BLEU is sensitive to the difference in length whereas distinct-n and P-SentBERT are less sensitive, P-BLEU is a favorable metric for DBS. As such, DMBR underperforms DBS for SQuADv2 and CommonGen in P-BLEU but not in distinct-n and P-SentBERT. It implies that if a practitioner benefits from varying sequence lengths, DBS may be preferred over DMBR, and if not, DMBR may be preferred over DBS.
4.4 Generative Common Sense Reasoning using Language Model
CommonGen is a constrained text generation task to evaluate the ability of common sense reasoning of the system (Lin et al., 2020). Given a set of common concepts, the task is to generate a coherent sentence describing an everyday scenario using these concepts. We use Zephyr-7B β with prompting as a text generation model. We use the following prompt:
Generate a short and interesting sentence using all the words provided from the user as is. Make sure that it is short and all the words are included without rewording. The other experimental setting is the same as in Section 4.3. The mean METEOR and the diversity metrics are shown in Figure 6 (c, f, i). DMBR has better distinct-2 than DBS and epsilon sampling but slightly worse P-BLEU. We observe that the coverage of the input concepts is low (< 18%) in all settings in our experiments. See Appendix B for the analysis of the coverage.
4.5 Text Summarization
We use the XSum dataset as a benchmark for an abstractive single-document summarization (Narayan et al., 2018). We use a BART model pretrained on XSum dataset (Lewis et al., 2020). We evalu-
ate the quality of the outputs by ROUGE-L using HuggingFace’s evaluate library (Lin, 2004). We generate k = 4 outputs per each input document. For DBS, we set the diversity penalty to {0.0, 0.5, 1.0, 2.0, 5.0}. For MBR we generate 64 samples with epsilon sampling with ϵ = 0.02. The diversity penalty λ for DMBR is set to {0.0, 0.1, 0.3, 0.5, 1.0, 2.0}. The results are shown in Figure 7. DMBR achieves better diversity than DBS measured by P-BLEU and distinct-n.
5 Conclusions
We study the problem of generating a set of texts with high quality and diversity. Our approach is to promote diversity to MBR which is shown to generate high quality texts so that it can generate high quality and diverse outputs. We extend MBR and propose DMBR and KMBR that seek to optimize both the diversity and the quality when selecting a set of outputs. Because both algorithms are too expensive to compute exactly, we devise approximate algorithms to make it feasible. We evaluate DMBR and KMBR on machine translation, image captioning, question generation, generative common sense reasoning, and text summarization tasks and show that overall they achieve better qualitydiversity trade-off than DBS. We also observe that both methods, especially KMBR, achieve a higher max BLEU score than MBR, analogous to DBS achieving a higher BLEU score than beam search.
6 Limitations
Our experiments are focused on directed text generation tasks. Open-ended and directed text generation tasks are different tasks for text generation algorithms. This distinction separates what kind of text generation algorithms are suitable for each task. While beam search variants tend to perform better in directed text generation tasks, stochastic decoding algorithms tend to do better in open-ended text generation tasks (Holtzman et al., 2020; Basu et al., 2021; Hewitt et al., 2022; Xu et al., 2023b). The evaluation of the methods in open-ended text generation tasks is future work.
We rely on automatic evaluations to evaluate the quality and the diversity of the generated texts. Human evaluation is desirable, especially for evaluating diversity. Although the automatic evaluation metrics for diversity used in this paper (e.g., P-BLEU, distinct-n, P-SentBERT) are shown to correlate with human evaluation, there is still a
clear gap between automatic metrics and humans (Tevet and Berant, 2021).
DMBR and KMBR are much slower than DBS as they require the computation of the MBR objective which needs a computation of the utility function for N2 times. Although recent work has shown that the computation of the MBR objective can be significantly reduced (Cheng and Vlachos, 2023; Jinnai and Ariu, 2024; Deguchi et al., 2024; Vamvas and Sennrich, 2024), it is not directly applicable to DMBR and KMBR. Reducing the inference time of these algorithms will be future work.
We use a simple greedy algorithm to compute the DMBR objective. More sophisticated approximation algorithms may improve the performance of DMBR (Feldman et al., 2017; Sakaue, 2020).
We consider the Monte Carlo estimate (Eq. 7) as the target quality objective for simplicity. Exploring other quality objective functions such as model-based estimate (Jinnai et al., 2024) is future work.
Acknowledgements
We thank all the reviewers for their constructive comments throughout the manuscript review. We thank Naoto Ohsaka for providing insights on approximation algorithms for a non-monotonic submodular function maximization problem.
References Satanjeev Banerjee and Alon Lavie. 2005. METEOR:
An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019. Findings of the 2019 conference on machine translation (WMT19). In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 1–61, Florence, Italy. Association for Computational Linguistics.
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney. 2021. Mirostat: A neural text decoding algorithm that directly controls perplexity. In International Conference on Learning Representations.
Niv Buchbinder and Moran Feldman. 2018. Submodular functions maximization problems. In Teofilo F. Gonzalez, editor, Handbook of Approximation Algorithms and Metaheuristics, Second Edition, Volume 1: Methologies and Traditional Applications, pages 753–788. Chapman and Hall/CRC.
Julius Cheng and Andreas Vlachos. 2023. Faster minimum Bayes risk decoding with confidence-based pruning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12473–12480, Singapore. Association for Computational Linguistics.
Kyunghyun Cho. 2016. Noisy parallel approximate decoding for conditional recurrent language model. arXiv preprint arXiv:1605.03835.
Sajal Choudhary, Prerna Srivastava, Lyle Ungar, and João Sedoc. 2017. Domain aware neural dialog system. arXiv preprint arXiv:1708.00897.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
Hiroyuki Deguchi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe, Hideki Tanaka, and Masao Utiyama. 2024. Centroid-based efficient minimum bayes risk decoding. arXiv preprint arXiv:2402.11197.
Bryan Eikema and Wilker Aziz. 2022. Sampling-based approximations to minimum Bayes risk decoding for neural machine translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10978–10993, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia. Association for Computational Linguistics.
António Farinhas, José de Souza, and Andre Martins. 2023. An empirical study of translation hypothesis ensembling with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11956–11970, Singapore. Association for Computational Linguistics.
Uriel Feige. 1998. A threshold of ln n for approximating set cover. J. ACM, 45(4):634–652.
Moran Feldman, Christopher Harshaw, and Amin Karbasi. 2017. Greed is good: Near-optimal submodular maximization via greedy optimization. In Proceedings of the 2017 Conference on Learning Theory, volume 65 of Proceedings of Machine Learning Research, pages 758–784. PMLR.
Markus Freitag, Behrooz Ghorbani, and Patrick Fernandes. 2023. Epsilon sampling rocks: Investigating sampling strategies for minimum Bayes risk decoding for machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9198–9209, Singapore. Association for Computational Linguistics.
Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. 2022. High quality rather than high model probability: Minimum Bayes risk decoding with neural metrics. Transactions of the Association for Computational Linguistics, 10:811–825.
Kevin Gimpel, Dhruv Batra, Chris Dyer, and Gregory Shakhnarovich. 2013. A systematic exploration of diversity in machine translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1100–1111, Seattle, Washington, USA. Association for Computational Linguistics.
Vaibhava Goel and William J Byrne. 2000. Minimum bayes-risk automatic speech recognition. Computer Speech & Language, 14(2):115–135.
Alex Graves. 2012. Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711.
John Hewitt, Christopher Manning, and Percy Liang. 2022. Truncation sampling as language model desmoothing. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3414– 3427, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations.
Daphne Ippolito, Reno Kriz, João Sedoc, Maria Kustikova, and Chris Callison-Burch. 2019. Comparison of diverse decoding methods from conditional language models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3752–3762, Florence, Italy. Association for Computational Linguistics.
Yuu Jinnai and Kaito Ariu. 2024. Hyperparameter-free approach for faster minimum bayes risk decoding. arXiv preprint arXiv:2401.02749.
Yuu Jinnai, Tetsuro Morimura, Ukyo Honda, Kaito Ariu, and Kenshi Abe. 2024. Model-based minimum bayes risk decoding. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR.
Leonard Kaufman and Peter J Rousseeuw. 2009. Finding groups in data: an introduction to cluster analysis. John Wiley & Sons.
Jonathan Krause, Justin Johnson, Ranjay Krishna, and Li Fei-Fei. 2017. A hierarchical approach for generating descriptive image paragraphs. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 3337–3345. IEEE Computer Society.
Reno Kriz, João Sedoc, Marianna Apidianaki, Carolina Zheng, Gaurav Kumar, Eleni Miltsakaki, and Chris Callison-Burch. 2019. Complexity-weighted loss and diverse reranking for sentence simplification. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3137–3147, Minneapolis, Minnesota. Association for Computational Linguistics.
Ilia Kulikov, Alexander Miller, Kyunghyun Cho, and Jason Weston. 2019. Importance of search and evaluation strategies in neural dialogue modeling. In Proceedings of the 12th International Conference on Natural Language Generation, pages 76–87, Tokyo, Japan. Association for Computational Linguistics.
Shankar Kumar and William Byrne. 2002. Minimum Bayes-risk word alignments of bilingual texts. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pages 140–147. Association for Computational Linguistics.
Shankar Kumar and William Byrne. 2004. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 169–176, Boston, Massachusetts, USA. Association for Computational Linguistics.
Philippe Laban, John Canny, and Marti A. Hearst. 2020. What’s the latest? a question-driven news chatbot. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 380–387, Online. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110–119, San Diego, California. Association for Computational Linguistics.
Jiwei Li and Dan Jurafsky. 2016. Mutual information and diverse decoding improve neural machine translation. arXiv preprint arXiv:1601.00372.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730–19742. PMLR.
Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. CommonGen: A constrained text generation challenge for generative commonsense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1823–1840, Online. Association for Computational Linguistics.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
Chin-Yew Lin and Franz Josef Och. 2004. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL04), pages 605–612, Barcelona, Spain.
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer.
Clara Meister, Ryan Cotterell, and Tim Vieira. 2020. If beam search is the answer, what was the question? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2173–2185, Online. Association for Computational Linguistics.
Clara Meister, Martina Forster, and Ryan Cotterell. 2021. Determinantal beam search. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6551–6562, Online. Association for Computational Linguistics.
Nikahat Mulla and Prachi Gharpure. 2023. Automatic question generation: A review of methodologies, datasets, evaluation metrics, and applications. Prog. in Artif. Intell., 12(1):1–32.
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–1807, Brussels, Belgium. Association for Computational Linguistics.
George L Nemhauser, Laurence A Wolsey, and Marshall L Fisher. 1978. An analysis of approximations for maximizing submodular set functions—i. Mathematical programming, 14:265–294.
Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook FAIR’s WMT19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 314–319, Florence, Italy. Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Hae-Sang Park and Chi-Hyuck Jun. 2009. A simple and fast algorithm for k-medoids clustering. Expert systems with applications, 36(2):3336–3341.
Martin F Porter. 1980. An algorithm for suffix stripping. Program, 14(3):130–137.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789, Melbourne, Australia. Association for Computational Linguistics.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
LKPJ Rdusseeun and P Kaufman. 1987. Clustering by means of medoids. In Proceedings of the statistical data analysis based on the L1 norm conference, neuchatel, switzerland, volume 31.
Nils Reimers and Iryna Gurevych. 2019. SentenceBERT: Sentence embeddings using Siamese BERTnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
Shinsaku Sakaue. 2020. Guarantees of stochastic greedy algorithms for non-monotone submodular maximization with cardinality constraint. In International Conference on Artificial Intelligence and Statistics, pages 11–21. PMLR.
Tianxiao Shen, Myle Ott, Michael Auli, and Marc’Aurelio Ranzato. 2019. Mixture models for diverse machine translation: Tricks of the trade. In International conference on machine learning, pages 5719–5728. PMLR.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and TieYan Liu. 2020. Mpnet: Masked and permuted pretraining for language understanding. Advances in Neural Information Processing Systems, 33:16857– 16867.
Felix Stahlberg and Bill Byrne. 2019. On NMT search errors and model errors: Cat got your tongue? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3356– 3362, Hong Kong, China. Association for Computational Linguistics.
Md Arafat Sultan, Shubham Chandel, Ramón Fernandez Astudillo, and Vittorio Castelli. 2020. On the importance of diversity in question generation for QA. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5651–5656, Online. Association for Computational Linguistics.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, page 3104–3112, Cambridge, MA, USA. MIT Press.
Mirac Suzgun, Luke Melas-Kyriazi, and Dan Jurafsky. 2023. Follow the wisdom of the crowd: Effective text generation via minimum Bayes risk decoding. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4265–4293, Toronto, Canada. Association for Computational Linguistics.
Yik-Cheung Tam. 2020. Cluster-based beam search for pointer-generator chatbot grounded by knowledge. Computer Speech & Language, 64:101094.
Guy Tevet and Jonathan Berant. 2021. Evaluating the evaluation of diversity in natural language generation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 326–346, Online. Association for Computational Linguistics.
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023. Zephyr: Direct distillation of lm alignment. arXiv preprint arXiv:2310.16944.
Jannis Vamvas and Rico Sennrich. 2024. Linear-time minimum bayes risk decoding with reference aggregation. arXiv preprint arXiv:2402.04251.
Ashwin Vijayakumar, Michael Cogswell, Ramprasaath Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2018. Diverse beam search for improved description of complex scenes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R. Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2016. Diverse beam search: Decoding diverse solutions from neural sequence models. arXiv preprint arXiv:1610.02424.
Qingzhong Wang and Antoni B. Chan. 2019. Describing like humans: On diversity in image captioning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 4195–4203. Computer Vision Foundation / IEEE.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Jiacheng Xu, Caiming Xiong, Silvio Savarese, and Yingbo Zhou. 2023a. Best-k search algorithm for neural text generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12385– 12401, Toronto, Canada. Association for Computational Linguistics.
Nan Xu, Chunting Zhou, Asli Celikyilmaz, and Xuezhe Ma. 2023b. Look-back decoding for open-ended text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1039–1050, Singapore. Association for Computational Linguistics.
Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations.
A Proof of Submodularity
We show that Eq. (9) with the pairwise similarity objective (Eq. 10) is a submodular function maxi-
mization problem. Let f be the objective function:
f(H) = ∑
h∈H
  1 N ∑
y∈Href u(h,y)
 
− ∑
h∈H
∑
h′∈H\{h}
λ
|H|u(h,h ′).
Then,
f(H) + f(H ′)
= ∑
h∈H
  1 N ∑
y∈Href u(h,y)
 
+ ∑
h∈H′
  1 N ∑
y∈Href u(h,y)
 
− ∑
h∈H
∑
h′∈H\{h}
λ
|H|u(h,h ′)
− ∑
h∈H′
∑
h′∈H′\{h}
λ
|H ′|u(h,h ′)
≥ ∑
h∈H∪H′
  1 N ∑
y∈Href u(h,y)
 
+ ∑
h∈H∩H′
  1 N ∑
y∈Href u(h,y)
 
− ∑
h∈H∪H′
∑
h′∈H∪H′\{h}
λ
|H ∪H ′|u(h,h ′)
− ∑
h∈H∩H′
∑
h′∈H∩H′\{h}
λ
|H ∩H ′|u(h,h ′)
= f(H ∪H ′) + f(H ∩H ′).
Thus, f is a submodular function.
B Evaluation of the Coverage for CommonGen
The task encourages the generated sentence to contain as many input concepts as possible, so we also consider the coverage of the input concepts. The definition of coverage is the number of captured concepts divided by the number of input concepts. To compute the captured concepts, we use a Porter stemmer (Porter, 1980) to extract the stems of the input concepts and the generated sequences and compute the number of overlaps. Overall, the coverage is low (< 18%) for all the decoding algorithms. For k = 4, the average coverage is 15.56, 15.42, 16.03, 15.12, and 15.93 for beam
search, DBS-1.0, MBR, DMBR-1.0, and KMBR, respectively. We speculate this is because the text generation model (Zephyr-7b β) is not trained to follow the instructions on the lexical constraints without few-shot prompting.
C Examples of Generations
We show outputs generated by the decoding algorithms. Tables 2, 3, 4, 5, 6, and 7 are the examples of the generations with various sampling algorithms for each domain evaluated in Section 4. The examples show the generations of the first input source of each dataset.
D Evaluation of Oversampling Strategy
We additionally evaluate the performance of an oversampling strategy as a baseline. Oversampling strategy generates N(> k) samples and then selects k samples out of the N , maximizing the objective in Eq. (3):
H∗ = argmax H⊆Y
∑ h∈H Phuman(h|x) + dhuman(H).
(12) Because Phuman and dhuman are inaccessible, we approximate them using a model and Eq. (10):
H∗ = argmax H⊆Y
∑ h∈H P (h|x)− ∑
h∈H
∑
h′∈H\{h}
λ
|H|u(h,h ′). (13)
The results with N = 128 and k = 4 are shown in Table 8. Overall, we observe that it performs slightly worse than DMBR, and the hyperparameter λ is dependent on the generation probability of the sentences which in turn depends on sequence length (Table 9). The result indicates that the improvement of DMBR comes from the use of the utility function to select high-quality samples in addition to the oversampling.
E Additional Figures and Tables
E.1 Additional Figures The evaluation of the diversity as a function of min BLEU over the outputs on WMT’19 datasets is present in Figure 8. DMBR achieves a better tradeoff than DBS and sampling algorithms with the same min BLEU score.
The Oracle (max) and the min quality scores on MS COCO, SQuADv2, CommonGen, and XSum with an output size of 4 are present in Figures 9 and 10. We observe similar trends as in machine translation tasks.
Figures 11 and 12 show the P-SentBERT as a function of the max and min BLEU and METEOR scores on MS COCO, SQuADv2, and CommonGen. The result indicates that DMBR and DBS are successfully generating diverse outputs, not only lexically but also semantically.
E.2 Summary of the Results Tables 9, 10, 11, 12, 13, 14, and 15 show the summary of the experimental results described in Section 4.
F Pretrained Models and Codes used in the Experiments
We list the pretrained models and codes we used in the experiments in Table 16.
Text Generation Models
WMT’19 (Section 4.1) Ng et al. (2019) https://github.com/facebookresearch/fairseq/blob/main/examples/wmt19/README.md MS COCO (Section 4.2) Li et al. (2023) https://huggingface.co/Salesforce/blip2-flan-t5-xl-coco SQuADv2 (Section 4.3) Tunstall et al. (2023) https://huggingface.co/HuggingFaceH4/zephyr-7b-beta
CommonGen (Section 4.4) Tunstall et al. (2023) https://huggingface.co/HuggingFaceH4/zephyr-7b-beta XSum (Section 4.5) Lewis et al. (2020) https://huggingface.co/facebook/bart-large-xsum
Models for Evaluation
WMT’19 (Section 4.1) sacreBLEU: Post (2018) https://github.com/mjpost/sacrebleu MS COCO (Section 4.2) Sentence BERT: Song et al. (2020) https://huggingface.co/sentence-transformers/all-mpnet-base-v2 SQuADv2 (Section 4.3) Sentence BERT: Song et al. (2020) https://huggingface.co/sentence-transformers/all-mpnet-base-v2 CommonGen (Section 4.4) Sentence BERT: Song et al. (2020) https://huggingface.co/sentence-transformers/all-mpnet-base-v2 CommonGen (Section 4.4) Porter stemmer: Porter (1980) https://www.nltk.org/_modules/nltk/stem/porter.html
Table 16: List of pretrained models we used in the experiments.

heading: Future Work
text: The evaluation of the methods in open-ended text generation tasks is future work. We rely on automatic evaluations to evaluate the quality and the diversity of the generated texts. Human evaluation is desirable, especially for evaluating diversity. Although the automatic evaluation metrics for diversity used in this paper (e. g. , P-BLEU, distinct-n, P-SentBERT) are shown to correlate with human evaluation, there is still a
clear gap between automatic metrics and humans (Tevet and Berant, 2021). DMBR and KMBR are much slower than DBS as they require the computation of the MBR objective which needs a computation of the utility function for N2 times. Although recent work has shown that the computation of the MBR objective can be significantly reduced (Cheng and Vlachos, 2023; Jinnai and Ariu, 2024; Deguchi et al. , 2024; Vamvas and Sennrich, 2024), it is not directly applicable to DMBR and KMBR. Reducing the inference time of these algorithms will be future work. We use a simple greedy algorithm to compute the DMBR objective. More sophisticated approximation algorithms may improve the performance of DMBR (Feldman et al. , 2017; Sakaue, 2020). We consider the Monte Carlo estimate (Eq. 7) as the target quality objective for simplicity. Exploring other quality objective functions such as model-based estimate (Jinnai et al. , 2024) is future work