What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability (2023)

Abstract: In Natural Language Generation (NLG) tasks, for any input, multiple communicative goals are plausible, and any goal can be put into words, or produced, in multiple ways. We characterise the extent to which human production varies lexically, syntactically, and semantically across four NLG tasks, connecting human production variability to aleatoric or data uncertainty. We then inspect the space of output strings shaped by a generation system’s predicted probability distribution and decoding algorithm to probe its uncertainty. For each test input, we measure the generator’s calibration to human production variability. Following this instance-level approach, we analyse NLG models and decoding strategies, demonstrating that probing a generator with multiple samples and, when possible, multiple references, provides the level of detail necessary to gain understanding of a model’s representation of uncertainty.

## introduction humans display great variability in language production, in particular when the context or the task are open-ended, such as in storytelling or in dialogue. given a story prompt, for example, there are many plausible ways in which different humans (or a single writer, if asked multiple times) may tell the story @xcite . we refer to this phenomenon as production variability. production variability in humans has two main sources. first, when situated in a context, speakers may entertain variable communicative goals @xcite @xcite , and the number and variety of plausible communicative goals depends on the production task @xcite . translation, for instance, defines the communicative goal almost unequivocally while a dialogue context might allow for a wide variety of communicative goals (expressed, e.g., as a request, an assertion, or a yes-no question). the second source of variability is the fact that even when context and communicative goal are fixed, speakers' linguistic realisations of the communicative goal may vary @xcite . both sources of variability apply to individuals as well as to populations: if an expert is asked to simplify a complicated sentence multiple times, they may perform different rewriting transformations (e.g., paraphrasing, reordering, or sentence splitting) and produce different texts @xcite ; the same is true if multiple experts are asked to perform a task @xcite . if we are to regard a natural language generation (nlg) system (or text generator) as a good model of human production, it should capture the variability observed in humans. text generators combine two mechanisms: (i) an underlying statistical model-typically, an autoregressive factorisation of the probability of sequences, with conditional token probabilities predicted by a neural network; and (ii) an iterative decoding algorithm that chains samples from next token distributions into a complete production. to-gether these two mechanisms specify a probability distribution over sequences of tokens, which can be regarded as a representation of the model's uncertainty about productions for a given generation context (see @xcite for a detailed discussion). in this work, we assess whether this representation of uncertainty is in compliance with production variability exhibited by a population of humans-which in turn, we argue, can be regarded as an expression of aleatoric uncertainty, i.e., irreducible uncertainty due to the stochastic nature of the data generating process @xcite . in other words, we compare the distribution over productions of a text generator against the distribution over the productions of a population of human speakers, given the same context (figure 1 ). quantifying the closeness in distribution between a text generator and a human population is difficult: we only have an iterative view into the generator's distribution; the 'human distribution' is an implicit or even hypothetical object; and in both cases, the sample space is large or even unbounded. we can, however, compare these two objects via the samples they produce and assess their statistical distance-which is what we propose here. for each individual generation context, we compare scalar properties of generations (through repeated model sampling) and human productions (using multireference nlg datasets). in particular, we probe for lexical, syntactic, and semantic distance between productions, thus allowing for a quantitative and interpretable assessment of uncertainty. we find that the uncertainty of neural text generators is higher than justified by human production variability in open-ended tasks, like story generation and open-domain dialogue; and that it is lower on more constrained tasks, like machine translation and text simplification. popular decoding algorithms, which bias away from the distribution of the generator's underlying statistical model (e.g., top-k, top-p, or locally typical, rather than ancestral sampling), have a limited impact on the generator's ability to faithfully represent human variability. we complement our quantitative assessments with a detailed analysis of individual generation contexts, which sheds light on whether a generator has robustly learned to reproduce degrees and aspects of human variability plausible for the communicative task. beyond the experimental results obtained on our selection of models and tasks, our work has important implications for nlg evaluation and data collection. multiple samples and, when possible, multiple references, should be used to assess the statistical fit of text generators. our approach, complementary to other types of automatic evaluation, makes model assessments particularly insightful and trustworthy because it does not judge a model only by a single output but also, intuitively, by what it could have generated-and it does so for each individual input in the test set. we therefore hope our framework will be used by the community as an evaluation criterion for nlg systems, especially to assess them in more open-ended tasks. ## related work automatic approaches to the evaluation of nlg systems are of high practical importance: they allow for model selection at scale and power qualityaware decoding algorithms @xcite @xcite . in spite of their known limitations @xcite , they are a necessary complement to human evaluation @xcite . reference-based evaluation. the most common way of automatically evaluating text generators is via metrics that estimate the similarity between candidate generations and references, such as bleu @xcite , rouge @xcite , comet @xcite , bleurt @xcite , and bertscore @xcite . reference-based metrics are less suited for open-ended tasks such as story generation and dialogue, where a single reference (or even a handful) cannot be representative of the large space of plausible communicative goals and realisations. et al., 2022), yet they have been criticised for being biased (toward models similar to the evaluator) and thus limited in their ability to evaluate generated text @xcite . ## probing language processes for production variability we interpret language production, by humans or nlg systems, as captured by a probability distribution over natural language strings (productions), a random variable y , given a linguistic context @xmath0. the context x can be a source sentence in translation, a story prompt in story generation, or more generally the input to a language process. in turn, a production is a piece of text y such as a single translation, a story, or more generally the output of a language process. foot_0 ## human production variability across nlg tasks consider p y |@xmath1, and define the following special case for human production variability: estimating this probe by drawing pairs of human productions provides an interpretable view on plausible variability-i.e., aleatoric uncertainty-along the dimension captured by k. figure 2 shows h k (x) marginalised over inputs for the four nlg tasks. we use unigram distance for the lexical probe, pos bigram distance for the syntactic probe, and cosine distance for the semantic probe. high distance indicates high variability, and vice versa. translation and text simplification. humans show low production variability in these two tasks. while translations of a given source sentence are more lexically and semantically varied, simplifications exhibit a higher degree of syntactic variability, probably as a result of the instructions used during data collection (writers were asked to use varying rewriting transformations). overall, low levels of variability are to be expected as, in both tasks, content preservation is part of the communicative goal. ## qualitative instance-level analysis we now qualitatively analyse individual inputs for which a generator's uncertainty is miscalibrated to human variability-as detected by d w 1 . for each task, we use up to 10 human productions (5 for dialogue) and 10 generations. figures accompanying the examples in this section are in appendix e. while it is not a replacement for more standard nlg evaluation procedures, we argue that this level of analysis is complementary and crucial to gain deeper understanding of a generator's representation of uncertainty. variability underestimation in translation and simplification. we have seen that in translation and simplification, generators' self-variability is lower than human variability ( § 6.1). we now zoom in on examples from these two tasks, inspecting instances that show inadequate model fit on all linguistic levels (i.e., d w 1 (m k (x), h k (x)) is high for all k). the most severe cases of miscalibration for opusmt are all instances of variability underestimation. 12 for most of these, generations are virtually or completely identical, while a few present slightly higher but still substantially lower variability than human productions. for example, ten humans translated the phrase 'reacted cautiously' in the english source sentence 'several companies have thus far reacted cautiously when it comes to hiring' in six different ways ('vorsichtig reagiert', 'zurückhaltend reagiert', 'mit vorsichtsmaßnahmen reagiert', 'reagierten mit zurückhaltung', 'mit vorsicht reagiert', 'reagierten verhalten') while all ten generated samples contain the german phrase 'vorsichtig reagiert', signalling that the generator's lexical rephrasing abilities do not generalise to this input instance. for text simplification, we focus on instances where flan-t5's uncertainty is not calibrated to human syntactic variability. 13 we observe that simplifications sampled from the generator are always syntactically more similar to each other than humans', indicating that the generator struggles to capture an important aspect of text simplification: that many semantically equivalent rewritings are possible if a text's syntactic structure is altered. variability overestimation in dialogue. according to our estimates of human variability ( § 5), dialogue is the most open-ended task on all linguistic levels. we have hypothesised that this is due to the large variety of communicative act types plausible given any dialogue context. we have also seen that dialogpt generally overestimates production variability ( § 6.1)-figure 1 is one such example. now we further inspect instances where cross-variability is miscalibrated with respect to human outputs. 14 we find that the generator's bad fit can be due to very short and generic responses (e.g., 'well...', 'haha', 'ahem', 'well done!'), but is more often due to the presence of fluent yet very diverse and often inadequate samples. for such instances, not only is the generator's crossvariability miscalibrated-self-variability, too, is overestimated on all linguistic levels. in particular, the generator's poor calibration to lexical and syntactic variability is related to its inability to choose the correct dialogue acts (or favouring an excessive variety of dialogue acts). in an example instance where the last dialogue turn goes 'i've got a business call that i really need to take', humans all reply with short affirmative responses ('okay! please.', 'well! go on.', 'sure, why not!', 'sure! go ahead.', 'yes! sure.') while the model's responses are mostly lengthy statements, sometimes not particularly coherent ones (e.g., 'you don't need a business call. you need a friend'). ## conclusion variability is an intrinsic property of human language production. text generators, if they are to be considered as good statistical models of human written production, should exhibit plausible levels of variability. however, in nlg, the widespread practice is (i) collecting only one 'reference' production for each input and (ii) evaluating only a single generation. to appreciate the impact of this incongruity empirically, we analyse multiplereference datasets for four nlg tasks, and show that each task has its own plausible levels of lexical, syntactic, and semantic variability. we connect production variability to aleatoric uncertainty, the irreducible uncertainty of the language production process, and evaluate neural text generators in terms of whether their representation of uncertainty is calibrated to the levels of variability observed 15 dw 1 (ck(x), hk(x)) > 0.3; k is cosine distance. in humans. we find that nlg models overestimate production variability in open-ended tasks and underestimate it in more constrained tasks, and that most popular decoding algorithms all have a similar, limited effect on the generators' ability to reproduce human variability. we advocate for more widespread usage of instance-level probing of nlg systems as a way to evaluate their statistical fit, not just along the dimensions we cover in this study but with respect to any other quality of interest. this approach contrasts with corpus-level analyses of nlg systems (e.g., @xcite @xcite and thanks to its greater interpretability, it builds trust in the ability of generators to reproduce human-like statistics when situated in specific linguistic contexts rather than 'globally', over a possibly heterogeneous corpus. in the future, we plan to devise new ways of improving the calibration of models' uncertainty @xcite , e.g., steering generators with sequence-level decoding algorithms @xcite , and to investigate the relation between uncertainty and perceived generation quality (e.g., kuhn et al., 2022): while we use human levels of variability as a target, desirable levels of variability may deviate from human statistics for specific applications. future work should also study production variability as a function of a more complex notion of discourse context @xcite and attempt to disentangle uncertainty over communicative goals and realisations @xcite . this is an important avenue not only toward more practically useful generators but also toward reliable computational models of language production.