Asking a Language Model for Diverse Responses (2025)

Abstract: Large language models increasingly rely on explicit reasoning chains and can produce multiple plausible responses for a given context. We study the candidate sampler that produces the set of plausible responses contrasting the ancestral (parallel) sampling against two alternatives: enumeration, which asks the model to produce n candidates in one pass, and iterative sampling, which proposes candidates sequentially while conditioning on the currently generated response set. Under matched budgets, we compare these samplers on quality, lexical and computation flow diversity, and efficiency. Our empirical results demonstrate that enumeration and iterative strategies result in higher diversity at comparable quality. Our findings highlight the potential of simple non-independent sampling strategies to improve response diversity without sacrificing generation quality.

## introduction large language models (llms) have shown strong performance across a wide range of applications @xcite . in particular, the ability to generate explicit reasoning chains that guide planning and decision-making has become a cornerstone of recent progress @xcite @xcite . many of these applications benefit from access to multiple plausible responses for a given context, including test-time control @xcite @xcite , majority voting or best-of-n @xcite , conformal generative modeling @xcite , reasoning with diverse decoding paths @xcite and ambiguity resolution @xcite @xcite . a necessary component of these pipelines is a candidate sampler that returns a set of n re-* these authors contributed equally to this work sponses in context. the candidates are commonly obtained by ancestral sampling from the model distribution, or from variations such as temperature, top-p, top-k @xcite @xcite @xcite . beyond being in some sense the natural approach, ancestral sampling also benefits from being simple to implement and readily parallelizable across devices, as each response is sampled independently of the others. nevertheless, ancestral sampling suffers from repetitions of high-probability sequences, which motivated researchers to propose non-independent algorithms, including arithmetic sampling @xcite , diverse, stochastic, and determinantal beam search modifications @xcite @xcite . these approaches, wellstudied in the literature, are based on search-style algorithms on top of a language model's output probability, which still scores each sample separately, possibly with the help of a separate dissimilarity function. in this work, we take a substantially different approach and ask whether we can use the standard llm generation pipelines to enable efficient non-independent sampling, by processing multiple candidates at the same time. in particular, we are interested in a candidate sampler that: we compare the commonly used parallel sampling strategy (ancestral sampling) with two alternative sampling strategies, which we define as enumera-tion and iterative approaches, and study them from the perspective of quality, diversity, and efficiency. our main finding is that the enumeration and iterative strategies are simple and promising alternatives to the standard parallel approach. we find that our non-independent iterative and enumeration strategies result in higher lexical and computational flow diversity. such approaches can be seen in a way as upper-bound oracles to diverse generation, in the sense that they fully model the joint distribution over samples and are only limited by the instruction-following performance of the llm. our implementation is released as open-source. foot_0 ## methodology we consider tasks for which there are multiple valid responses. in the context of this work, we consider a valid response to contain both a derivation and a final answer, so different derivations leading to the same answer are valid responses. given a model p θ and a prompt c, our goal is to produce a set @xmath0, . . . , y (n) } of n responses. we keep all decoding hyperparameters fixed across methods and vary only the sampling protocol. ## experimental setup we evaluate on gsm8k @xcite , a grade school math problem-solving benchmark. each problem has a single gold answer, but multiple valid solutions may lead to it. therefore, a candidate is y (i) = (r (i) , a (i) ), with r (i) the solution (reasoning) and a (i) the final extracted answer. ## conclusion we study the problem of generating a diverse set of responses. we propose two non-independent approaches for sampling responses from a language model, namely enumeration and iteration strategies, and compare them against parallel algorithms based on ancestral sampling. on gsm8k, we find that our non-independent approaches can provide higher diversity of the samples, while maintaining simplicity and overall quality of the generations. compute efficiency analysis shows that enumera-tion and iteration are well-suited to a single gpu and can reduce redundancy without specialized search machinery. we hope our work will motivate further investigation of simple non-independent strategies for diverse candidate sampling.