Look-back Decoding for Open-Ended Text Generation (2023)

Abstract: Given a prefix (context), open-ended generation aims to decode texts that are coherent, which do not abruptly drift from previous topics, and informative, which do not suffer from undesired repetitions. In this paper, we propose Look-back, an improved decoding algorithm that leverages the Kullback–Leibler divergence to track the distribution distance between current and historical decoding steps. Thus Look-back can automatically predict potential repetitive phrase and topic drift, and remove tokens that may cause the failure modes, restricting the next token probability distribution within a plausible distance to the history. We perform decoding experiments on document continuation and story generation, and demonstrate that Look-back is able to generate more fluent and coherent text, outperforming other strong decoding methods significantly in both automatic and human evaluations.

## introduction despite the impressive success on generating fluent and accurate sentences for low-entropy tasks such as summarization or translation, large-scale language models (llms) still suffer from serious degeneration problems, such as undesired repetitions @xcite and unnatural topic drifts, under open-ended settings @xcite . open-ended neural text generation aims to generate coherent and diverse text from llms, given contextual prefix @xcite , and has spawned a wide range of natural language applications, including contextual text completion @xcite , story generation @xcite , and review generation @xcite . to alleviate the degeneration problem in openended text generation, a number of techniques have emerged over the recent years, which can be categorized into two directions: i) improved learning proposing new learning objectives, e.g., unlikelihood training @xcite , contrastive training @xcite and sequence likelihood calibration @xcite , to compensate for the rooted deficiency of the conventional maximum likelihood estimation (mle) foot_1 ; ii) improved decoding remedying tedious and repetitive generations in decoding search @xcite , or combating topic drifts in sampling procedures @xcite . in this work, we propose a new decoding algorithm, named look-back , which pays particular attention to the probability distribution disparity between continuation and history text. unlike contrastive search @xcite which uses cosine similarity between the hidden representation, look-back leverages the kullback-leibler (kl) divergence to track the distribution distance between current and historical decoding steps. the main motivation of look-back is that kl divergence defines a distance between the probability distributions of decoding steps, which arguably better aligns with the decoding practice. as shown in figure 1 (a), as the greedy algorithm repeatedly outputs single sentences, the distance with the closest past token distribution decreases towards 0. besides, when the continuation switches to another topic in figure 1 (b), the distribution distance of continuation with prefix obtains much higher levels compared with topic-relevant human continuation. based on our prior observations, for informative and coherent generation, the probability distribution should not be too close to history to guarantee diversity, but relatively close to prefix to maintain coherence. experimentally, through two tasks of openended text generation, including document continuation and story generation, we demonstrate that look-back outperforms a variety of open-ended decoding algorithms under different scales of pretrained llms (gpt2-xl and opt-6.7b) by producing much more coherent texts -high mauve score compared with human continuation and high similarity score measured against prefix, while maintaining similar level of diversity. ## proposed method: look-back as presented in algorithm 1, look-back first leverages probability distribution distance between current and prior steps to avoid repetitions ( §4.1), then incorporates reference from given prefix to mitigate topic drifts ( §4.2). ## experiments in this section, we first introduce the datasets ( §5.1) and automatic metrics ( §5.2) used to evaluate the generation quality of the proposed look-back and other strong decoding baselines ( §5.3). we then analyze experimental results evaluated by automatic metrics ( §5.5) and human evaluators ( §5.6). lastly, we show effectiveness of different techniques used in look-back through detailed analyses ( §5.7). ## conclusion the distance between output distributions signals potential failure modes of text generation, such as dull repetition and incoherence. we propose look-back , a novel decoding algorithm that utilizes the kl divergence between the current and historic decoding steps, to steer the output distribution into a plausible subspace. look-back can generate higher-quality text and outperforms several strong decoding algorithms in both automatic and human evaluation. however, kl divergence may not be the optimal measure for text output distributions and we leave the investigation of other measures to future work. in addition, the idea pro-posed in this work can also be used for other specialized constrained decoding scenarios, such as preventing hallucination.