Coherence boosting: When your pretrained language model is not paying enough attention (2022)

Abstract: Long-range semantic coherence remains a challenge in automatic language generation and understanding. We demonstrate that large language models have insufficiently learned the effect of distant words on next-token prediction. We present coherence boosting, an inference procedure that increases a LM’s focus on a long context. We show the benefits of coherence boosting with pretrained models by distributional analyses of generated ordinary text and dialog responses. It is also found that coherence boosting with state-of-the-art models for various zero-shot NLP tasks yields performance gains with no additional training.

## introduction language models (lms) are commonly evaluated for their ability to generate, rank, or classify coherent spans of text. long-range semantic coherence is a unifying feature of modern nlp benchmarks and applications, whether they are about producing short answers to questions, ranking answer choices by their consistency with world knowledge, or generating long responses. large nonspecialized lms, such as gpt-2 and -3 @xcite , sometimes fail to understand or use the semantic link between a text and its prompt or long-range context (fig. 1 ). samples from these lms have an unnaturally low density of words that require many tokens of context to predict ( §4.1), and the scores that the models give to completions of prompts indicate that they are oversensitive to recent context ( §5). we hypothesize that these failures arise from modeling choices and distribution shift. specifically, autoregressive lms are typically fit to a multi-objective problem: simultaneously maximizing token likelihoods conditioned on many lengths of truncated context ( §2.1). yet, at generation or code: github.com/zhenwang9102/coherence-boosting. this paper proposes coherence boosting, a simple inference-time procedure that increases the effect of distant words on predicted token distributions and is applicable in both generation and ranking settings. a pretrained model is viewed as an ensemble of experts that produce token distributions conditioned on varying lengths of context. these experts are log-linearly mixed to form a predictor that is superior to the base model ( §2). coherence boosting greatly improves prediction of words that depend on a long context, as evidenced by state-of-the-art results on tasks specially meant to assess models' attention to distant words ( §3). in generation of generic text and dialog responses, we show that coherence boosting brings the frequency of occurrence of such words close to that seen in natural text ( §4). beyond generation, we study diverse multiple-choice tasks ( §5), in which examples are known to be highly coherent. coherence boosting does not modify the base model and depends on a single parameter than can be estimated in one pass through a validation set, yet is a competitive adaptation algorithm. ## coherence boosting in this section, is an autoregressive lm over a vocabulary with learnable parameters , taking as input a variable number of tokens (up to a maximum context length ) and producing a vector of next-token likelihoods: we denote by the model evaluated on only the last input tokens, ignoring earlier tokens: ## experiments: lambada the lambada dataset @xcite tests lms' understanding of long-range dependencies by measuring the prediction of the final words in passages of several sentences. the task explicitly requires reasoning over a broad context: humans can reliably guess the last word when given a whole passage, but not when given only the last sentence. we perform experiments with the gpt family of models, closely replicating the evaluation setting of @xcite . 3 we predict the final word as the top-ranked token under the boosted model max , where max is the model taking the full available context and , are the chosen length and coefficient of the short context. to choose and , we do a grid search on the validation set and apply the best values to the testing set. results. ## experiments: language generation 4.1 generic text the experiment in this section extends that of @xcite . a selection of 5000 articles from webtext @xcite is taken as a reference corpus of human-written text. a language model (for us, gpt-2 large) is prompted to generate text conditioned only on the first sentence of each of these articles, up to a maximum of 200 tokens, yielding 5000 machine-generated texts. the human-written and machine-generated texts are compared by four automatic metrics: perplexity under the base lm, self-bleu-4 @xcite ; the mean bleu-4 score of a generated text with respect to all other generated texts as references), zipf coefficient (the linear regression coefficient between log-rank and log-frequency of generated tokens) and repetition (the fraction of generated texts that end in a repeating sequence of tokens). it is desirable for a model and inference procedure to produce text that is as close as possible in these metrics to the human-written reference. to measure long-range semantic coherence in the generated text, we define three new metrics: long-range repetition (lr ): for a whole number and document , let ( ) be the number of distinct tokens in , and let ( ) be the number of distinct tokens for which the distance between their first and last occurrence in is at least positions. the long-range repetition score lr of a corpus { 1 , . . . , 5000 } is a macro-average: this simple measure of lexical coherence favors repetition of words long after they are first used, but gives lower weight to documents that degenerate into repetition of a short span. long-dependent token frequency (ltf): a long-dependent token is one to which the base lm assigns a likelihood of at least 20% given its full context, but a likelihood of less than 5% given only the 20 tokens of context preceding it. we compute the frequency of long-dependent tokens among all generated tokens. long-short likelihood difference ( ): the mean difference in likelihoods assigned to tokens by the base lm conditioned on full context and conditioned on 20 tokens of context. although some choices of constants are needed to define ltf and , we intend them to be intuitive summaries of long-range coherence in the absence of established metrics. in particular, 20 tokens is close to the length of one sentence in typical english text. we sample 5000 document completions from gpt-2 large following sampling procedures with a range of boosting schemes. we consider models of the form 1max , @xmath0, 16, 32, 64} @xmath1.4, -0.2, -0.1, -0.05, -0.025, 0}. (such a parametrization of boosting parameters was chosen to ensure that when the context has length less than -or the distant context has very little effect on the next word -the boosted model becomes equivalent to the untempered max .) top-truncation @xmath2.95 is applied to all models. results. metrics of two of the best models, @xmath3, = -0.05 @xmath4, = -0.1, are shown in fig. 3 shows the dependence of selected metrics on and . coherence boosting brings all metrics closer to those of human text. as increases, the optimal grows in magnitude. this is expected: the predictive effect of tokens more than positions away decreases with ( approaches max ). we also note that a simple sampling with temperature 0.9 performs better than top-sampling in most of the coherence metrics. this suggests that the improvements accomplished by top-truncation come at the cost of introducing a bias towards tokens that are predictable from a short context. coherence boosting corrects this bias without sacrificing the gains in other measures. table d.1. in addition to the decoding algorithms considered by @xcite ) -beam search and greedy decoding -we consider greedy decoding with a coherence boosting model. as long and short predictors, we use dialogpt conditioned on the full conversation context and on only the (context-free) response generated so far. that is, if the conversation context is and the text generated so far is 1 . . . , then +1 is predicted using the model max +1 , evaluated on the string sep 1 . . . , where sep is the turn separa- ## experiments: language understanding we evaluate coherence boosting on zero-shot language understanding and inference tasks, where examples are expected to be highly coherent. we study 15 datasets in 5 categories of tasks. (1) cloze tasks: storycloze @xcite , hellaswag @xcite , and copa @xcite . (2) question answering: commonsenseqa (csqa) @xcite , openbookqa (obqa) (mihaylov et al., 5 galley et al. (2019) argue that nist and diversity metrics are more informative measures than bleu for multi-reference scoring, since bleu favors systems that often produce responses with little relation to the prompt (e.g., "i don't know"). 2018), arc easy / challenge (arc-e/c) @xcite , and piqa @xcite . (3) text classification: sst-2/5 @xcite , trec (voorhees and tice, 2000), agnews @xcite . (4) natural language inference: rte @xcite , cb (de marneffe et al., 2019), and boolq @xcite . (5) fact knowledge retrieval: lama @xcite . all tasks except lama are formulated as multiple-choice problems. we convert text classification and inference tasks to multiple-choice tasks by choosing meaningful answer words, e.g., "true"/"false". the prediction is made by selecting the choice with the highest lm likelihood. for in-context learning of gpt models, prompt formats greatly impact performance. we follow previous work @xcite @xcite to create natural prompts to enlarge the effectiveness of in-context learning, but we do not aim to optimize the full and context-free prompt format: our goal is to evaluate coherence boosting models with a fixed prompt. the prompt formats we use are listed in table e .1. as described in §2, within each prompt we identify a premise-free context, which is used as the context for the short-range model in coherence boosting. for each dataset, we pick the optimal value * of the parameter on the validation set and report the accuracy on testing set. (if no testing set is publicly available, we choose on a subset of the training set and report the final number on the validation set.) across all experiments, we do not put any few-shot examples in the prompt. for the knowledge retrieval task, we follow @xcite 's data split of lama and evaluate gpt models on facts whose missing answers are at the end of the sentence (to fit the nature of autoregressive language models). we limit the prompt length to be larger than 5 tokens and rerun the model from @xcite on the new data. results: multiple-choice tasks. results of three representative base models on all multiplechoice tasks are presented in by comparing the third column with the first two columns within each model in we observe that the optimal depends on tasks and models (fourth column within each model), which means that cannot be heuristically set to 0 or -1 as in past work. this finding suggests the necessity of searching for an optimal . we visualize the accuracy curve by varying in the testing set of all datasets. we show the curve for storycloze in fig. 4 and present similar figures for all tasks in figs. f.1 and f.2. consistent with the results on lambada ( §3), the optimal is usually negative, and its absolute value tends to decrease with the model size. we selected the optimal by the validation set, but future work may explore automatic and adaptive methods for setting this parameter. notice that all experiments required only a single pass through the data to compute answer likelihoods conditioned on full and premise-free contexts -no iterative gradient-based finetuning was applied. results: knowledge retrieval. unlike lam-bada, where long contexts are required for inferring the last word, lama contains much shorter sentences for knowledge facts, i.e., (subject, relation, object). a recent study @xcite shows that the prediction is biased by the relation in the short context, i.e., the answer to a prompt (e.g., "dante was born in ___") can be induced by the relation ("was born in") without the subject. coherence boosting mitigates the influence of those short contexts by making the prediction dependent on a longer context containing the subject. we present results for all models on lama in ## extensions and future work we suggest three promising research directions: coherence tuning. the need to evaluate the base lm with multiple contexts in coherence boosting introduces cost and complexity at inference time. it may be desirable instead to modify the weights of the base model to improve long-range coherence properties. in §b, we describe a 'self-tuning' algorithm that achieves this without training on any data created for this purpose. new domains and architectures. in this paper, we mainly considered coherence boosting with decoder-only transformer lms trained on generic text, but future work should consider other architectures and target domains. in §c, we give preliminary results on the text summarization domain. although we expect recency bias to be less pronounced in lms that use separate attention modules to process the prompt and the output -such as encoder-decoder models for translation or summarization -procedures inspired by coherence boosting may prove effective in domains where a strong causal link between prompt and output is known to exist. such domains include language generation conditioned on structured data @xcite @xcite and model-guided reasoning in formal languages, such as proof or program synthesis @xcite @xcite . efficient search proposals. procedures that force lms to be more focused on a prompt, or a specific part of it, when generating or ranking tokens can benefit algorithms that search for combinations of words through sampling. it would be interesting to use coherence boosting in non-autoregressive text generation algorithms, such as to accelerate the mixing of mcmc methods for constrained text generation @xcite @xcite . ## conclusion we have illustrated the hyposensitivity of pretrained language models to long-range context and proposed a simple inference-time remedy. we hope to see coherence boosting used as a simple alternative or complement to finetuning procedures in zero-shot applications of pretrained lms.