On the Efficacy of Sampling Adapters (2023)

Abstract: Sampling-based decoding strategies are widely employed for generating text from probabilistic models, yet standard ancestral sampling often results in text that is degenerate or incoherent. To alleviate this issue, various modifications to a model’s sampling distribution, such as top-p or top-k sampling, have been introduced and are now ubiquitously used in language generation systems. We propose a unified framework for understanding these techniques, which we term sampling adapters. Sampling adapters often lead to qualitatively better text, which raises the question: From a formal perspective, how are they changing the token-level distributions of language generation models? And why do these local changes lead to higher-quality text? We argue that the shift they enforce can be viewed as a trade-off between precision and recall: while the model loses its ability to produce certain strings, its precision rate on desirable text increases. While this trade-off is not reflected in standard metrics of distribution quality (such as perplexity), we find that several precision-emphasizing measures indeed indicate that sampling adapters can lead to probability distributions more aligned with the true distribution. Further, these measures correlate with higher sequence-level quality scores, specifically, Mauve.

## introduction the vast majority of natural language generation systems take a probabilistic approach. the backbone of such an approach is a probability distribution over strings p θ for a specific target domain. while modern language models have achieved remarkable performance on standard measures of distribution quality, e.g., perplexity @xcite @xcite , they often fall short when applied out of the box for language generation tasks-both sampling directly from them and searching for the maximum-probability string under them can lead to dull, incoherent, and degenerate text @xcite @xcite . surprisingly, applying a post-hoc modification to p θ (• | @xmath0. in this paper, we give a name to these methods, dubbing them sampling adapters. a sampling adapter can be formally defined as a simplex-to-simplex map α : ∆ |v|- @xmath1, thus creating another language model α(p θ (• | @xmath2, e.g., it may only give non-zero probability to items assigned high probability under the original model. sampling adapters often require little to no fine-tuning and can be implemented in just a few lines of code. presumably due to their simplicity, sampling adapters have become a default tool in text generation pipelines, serving as the core component of baseline decoding strategies in various tasks @xcite @xcite . the fact that sampling adapters often lead to qualitatively better text, however, evokes a simple question: how do they change our language generation models such that the distribution p θ (• | @xmath3. moreover, standard evaluation measures 1 do not immediately shed light on why sampling adapters work well because most sampling adapters make language generation models substantially worse according to these measures, e.g., they often reduce the probability assigned to certain strings to zero, which can yield a perplexity of ∞. in this paper, we posit that the change of distribution induced by sampling adapters can be analyzed in terms of a precision-recall trade-off, using the generalizations of these terms to the field of generative modeling @xcite @xcite . while a model loses its ability to produce certain strings, its ability to produce desirable text increases. we experiment with various sampling adapters that have been proposed @xcite @xcite and find that, while the use of these adapters negatively affects recall-emphasizing performance measures, certain choices of hyperparameters increase performance in terms of measures that balance between precision and recall or that are precision-emphasizing. comparing trends in these measures, we see evidence of a precision-recall trade-off, which offers a quantitative motivation for the efficacy of sampling adapters. we further find that precision-emphasizing measures correlate most highly with sequence-level quality metrics, offering a potential avenue for efficiently choosing sampling adapter hyperparameter values. the formal framework and empirical analysis presented here should pave the way for the development of theoretically motivated sampling adapters, and provide a straightforward means for both analysis of and comparison between adapters. ## a precision-recall hypothesis we begin by reviewing generalizations of the concepts of precision and recall in the field of generative modeling. we then discuss the shortcomings of current language generation models and how sampling adapters may address these shortcomings. ## experiments to test the hypothesis that the operations performed by sampling adapters are akin to a re-prioritization of precision over recall in the output of the model, we evaluate the effects of sampling adapters on measures that emphasize recall, precision or a balance of the two, as outlined in §4.1. we then observe how these measures vary as a function of the sampling adapters' hyperparameters. further, we also look at these measures' spearman correlations with mauve, a sequence-level quality metric. we consider five different adapters: temperature, η (eta), top-π, top-k and locally typical sampling, each over a wide range of hyperparameters. note that for the latter three adapters, a smaller hyperparameter value corresponds to a larger shift between p θ and p θ . for η-sampling, the reverse is true, and for temperature sampling, hyperparameter values farther from 1 imply a larger shift. for reproducibility, we leverage the hugging face framework @xcite and its implementation of sampling adapters for all but η-sampling, for which we rely on the original authors' implementation. 12 error bars for all plots indicate 95% confidence intervals for the observed values; note that bars are often small enough that they are not visible. ## conclusion in this work, we offer a formal treatment of sampling adapters and provide an analysis that aims to uncover why they are effective when used with probabilistic models for language generation. to this end, we first introduce a general framework that encompasses most of the transformations performed by previously proposed sampling adapters. we then offer an intuition as to why sampling adapters may lead to better language generators. using the notions of precision and recall proposed for generative models, which can be quantified in terms of standard probabilistic measures, we perform an empirical analysis. we find evidence that the application of sampling adapters increases the precision of a distribution at the expense of its recall; this observation is robust across several experimental design choices. we further find a high correlation between sequence-level quality metrics and reverse kl divergence of the generation model with a reference model.