Understanding Neural Abstractive Summarization Models via Uncertainty (2020)

Abstract: An advantage of seq2seq abstractive summarization models is that they generate text in a free-form manner, but this flexibility makes it difficult to interpret model behavior. In this work, we analyze summarization decoders in both blackbox and whitebox ways by studying on the entropy, or uncertainty, of the model’s token-level predictions. For two strong pre-trained models, PEGASUS and BART on two summarization datasets, we find a strong correlation between low prediction entropy and where the model copies tokens rather than generating novel text. The decoder’s uncertainty also connects to factors like sentence position and syntactic distance between adjacent pairs of tokens, giving a sense of what factors make a context particularly selective for the model’s next output token. Finally, we study the relationship of decoder uncertainty and attention behavior to understand how attention gives rise to these observed effects in the model. We show that uncertainty is a useful perspective for analyzing summarization and text generation models more broadly.

## introduction recent progress in abstractive summarization has been fueled by the advent of large-scale transformers pre-trained on autoregressive language modeling objectives @xcite @xcite . despite their strong performance on automatic metrics like rouge @xcite , abstractive models are not as straightforward and interpretable as their extractive counterparts. free-form generation in these models also leads to serious downstream errors, such as factual inconsistencies with the input document @xcite @xcite @xcite . although the interpretability of nlu models has been extensively studied @xcite @xcite , summarization models specifically have not received similar attention, with analysis efforts often focused on datasets and evaluation @xcite . in this work, we focus on interpreting and understanding abstractive summarization models through the lens of decoder uncertainty, or the entropy of decisions during generation. while uncertainty in generation has been studied from the perspective of data @xcite , sampling @xcite , and training @xcite , it is underutilized as a technique for analysis and inspection of generation systems. we study two prominent summarization models, pegasus @xcite and bart @xcite , fine-tuned on two english summarization datasets, cnn/daily mail @xcite and xsum @xcite , to understand model behavior in each setting. first, by comparing n-grams between the input document and generated summaries, we establish two coarse types for decoded tokens, copy and generate @xcite . we find that the entropy of the generation decision correlates with whether the model is copying or generating, as well as where in the sentence the token is. this paints a picture of certain contexts being more restrictive from the standpoint of generation, particularly early in sentences where a model has not "decided" what to copy yet, and illustrates the interaction of content selection and lexical choice. second, we extend this analysis by looking at how uncertainty relates to the syntax of the generated sentence: whether uncertainty connects to syntactic notions of surprisal @xcite and how the entropy varies across certain syntactic productions. finally, we derive a way to quantify decoder attention by aggregating distinct self-attention heads, revealing the correlation between the attention entropy and prediction entropy, and investigating the correspondence between the prediction entropy and the fraction of the past and future decoded tokens. taking this analysis together, we find that the abstractiveness of reference summaries fundamentally changes model behavior: the extractive nature of cnn/dm makes most of its decisions low entropy and copy-oriented while the model maintains higher uncertainty on xsum, yielding more abstractive summaries. more broadly, we show that uncertainty is a simple but effective tool to characterize decoder behavior in text generation. ## model and experimental setup our experiments use pegasus @xcite and bart @xcite , two state-ofthe-art seq2seq pre-trained models. we use the large version of these two models, which have 16 and 12 transformer layers, respectively. both models have pre-training objectives tailored somewhat to this problem domain: seq2seq modeling for denoising (bart) or infilling of masked-out sentences (pegasus). we directly use the pre-trained models from @xcite . 2 as reported in the original papers and measured by rouge-1/2/l @xcite , pegasus achieves 44.17/21.47/41.11 on cnn/dm @xcite and 47.21/24.56/39.25 on xsum @xcite , and bart achieves 44.16/21.28/40.90 and 45.14/22.27/37.25. entropy. entropy is a standard measure of uncertainty in a probabilistic distribution. given a discrete random variable x with all possible outcomes x 1 , • • • , x n , the entropy of x is defined as h(x) = -n @xmath0. for pre-trained transformers, the domain of the predictions (the vocabulary) is large and also differs between models. the vocabulary sizes for pegasus and bart are 96,103 and 50,265, 3 and the prediction distribution is usually long-tailed. to combat this, nucleus sampling @xcite is used to sample from only the top 1 -p most probable outcomes (the nucleus) to avoid generating very unlikely tokens. to more fairly compare models with different vocabulary sizes, and to better reflect the actual sampling distribution, we therefore compute all entropy values in this work over the nucleus distribution. that is, we sort the prediction distribution p (x i ) in descending order and get a minimal set of tokens where v @xmath1. then we renormalize the distribution as follows: ## model uncertainty during generation in this section, we analyze and compare the prediction uncertainty from different models and different datasets by inspecting entropy values during generation, allowing us to localize uncertainty to certain positions in a decoded sentence. a principle factor that past work has investigated is the amount of copying in abstractive summarization models @xcite . we first aim to understand how decisions to copy document content or generate new text are reflected in the model's uncertainty. one complicating factor is that while bart and pegasus both exhibit a mix of copying and novel generation, they do not have an explicit copy operation like in past models and so these behaviors are more difficult to define. we first separate generation decisions by bigrams that appear in the input document (existing bigrams) or whether they are free-form generations (novel bigrams). 4 figure 1 shows a histogram of model entropies broken down by these two categories. most notably, there is a strong correlation between copy-like behavior and the entropy of the model's prediction distribution. on cnn/dm, we see that low entropy decisions are largely those generating existing bigrams, and conversely, existing bigrams are usually generated with low entropy. new bigrams are generated with a broad range of high entropy values, and are much more frequent on xsum. these results align with our manual analysis of these summaries: pegasus cnn/dm and bart cnn/dm summaries largely consist of spans from the input document with minor compression while pegasus xsum and bart xsum summaries involve stitching together disparate concepts and paraphrasing key details. this reflects a corresponding divergence in the gold summaries, where cnn/dm summaries are far more extractive than those in xsum. critically, though the entropy distributions are dissimilar across the two datasets, we see regularities among the approximate copy and generate operations: on cnn/dm and xsum, the median entropy values of using existing bigrams are 0.95 and 1.20, respectively, and for generating new bigrams, 2.27 and 1.75. with this connection between entropy and copying behavior, we make the following additional observations based on figures 1 and 2 : entropy varies across token positions, especially on cnn/dm. in figure 2 , we depict a different view of entropy, looking at the decoding process as it progresses through each sentence. across both cnn/dm and xsum, models are most uncertain at the beginning of the sentence and least uncertain at the end of the sentence. however, the rate at which entropy drops off is quite different: on cnn/dm, the entropy after decoding 20% of tokens falls below 2, while the entropies for example, 0.0 indicates the first 10% of tokens in a sentence, and 0.9 is the last 10% of tokens. pegasus cnn/dm and bart cnn/dm make highly uncertain decisions to start, but then entropy decreases, suggesting that these models may be copying based on a sentence prefix. entropies on xsum are more constant across the sentence. on xsum only begin to considerably drop after decoding 80% of tokens. our manual analysis suggests the following characterization: to generate each sentence on cnn/dm, the model makes some high-entropy decisions to identify a sentence and begin to copy its prefix, followed by a series of low entropy decisions to copy that sentence's content. on xsum, which is highly abstractive and features single sentence summaries, content planning and generation are less clearly decoupled. pegasus copies and generates more tokens with @xmath2. bart and pegasus report similar rouge results on cnn/dm, but these models do not place the same distributions over summaries. pegasus has more low-entropy copying decisions, and its start-of-sentence entropies are also significantly lower (figure 2 ). this suggests that it is more confident than bart in selecting content to discuss next. there are also more low-entropy generation decisions, particularly on xsum. ## entropies of syntactic productions having observed connections between sentence position and entropy, we now flesh out this analysis from the lens of syntax, focusing in particular on uncertainty at constituent boundaries. from our pegasus generations on cnn/dm and xsum, we obtain constituency parses for each summary sentence using the berkeley neural parser @xcite and explore connections between syntax and uncertainty in more depth. low and high entropy decisions can be localized to constituent span boundaries. parsing has long been used to explain psycholinguistic notions of surprisal @xcite , which are in turn related to uncertainty under a language model. in our case, uncertainty about generating a text is a different notion than uncertainty when a reader is processing it. hence, rather than looking at an incremental parser's behavior, we instead look at a simpler notion of syntactic distance @xcite , or the number of left and right parentheses between w t and w t+1 in a linearized constituency tree. our hypothesis is that when these words exhibit high syntactic distance, this word boundary is a "choice point" where the model may be less restricted in what it can choose to generate next. figure 3 shows the correlation between syntactic distance and the percent change in entropy between the adjacent tokens. on both cnn/dm and xsum, we see two patterns emerge: generating a token within the same immediate parent constituent (i.e., zero syntactic distance) is typically a certain decision, while generating a token belonging to a new constituent is an increasingly uncertain decision. from these results, we can draw a parallel to the copy vs. generate behavior established in section 3; for example, generating york after new might be straightforward, perhaps due to a direct copy from the document, but generating a prepositional phrase might be more challenging due to the large search space of possible constructions or the higher chance that the model might delete this constituent. low entropy spans are often short, specific units of information. we also investigate the average entropy of spans within a rule production to uncover what types of spans are likely to elicit certainty or uncertainty during generation. in ## understanding decoder self-attention while we have analyzed the model's predictions, we have not yet determined how the different behaviors we see emerge from the context. our goal is to explore what the encoder attention places its emphasis during generation and how it correlates with the prediction entropy. 5 blocking low-information tokens. analyzing the inner workings of attention in transformers is challenging @xcite , particularly because many heads are useless, redundant, or noisy, and they frequently attend to low-information tokens such as end-of-sentence markers or periods. inspired by tf-idf @xcite , we propose a method to compute a set of tokens most meaningfully attended to by the model. if a token in the encoding document is attended to across many time steps (like a word appearing in many documents in tf-idf), we want to disregard it in our analysis. let t denote the number of decoder timesteps and l be the length of the source document. we compute an aggregate attention matrix @xmath3. we then compute a count of how often each token is attended to above a threshold q: f @xmath4. in practice we discard 5% of tokens from the source document. ## conclusion this work analyzes pre-trained summarization models via uncertainty, or the entropy of decoding decisions. we pursue several lines of inquiry: uncertainty can help us understand copying document spans vs. generating novel text, the behavior of models in different syntactic environments, and coarse properties of the model's attention distribution. all of these give insight into what conditions most heavily restrict the model's generation: generating an observed bigram (copying), low syntactic distance, and attention which can easily identify decoder context in the source document. we believe this approach can power future analyses of pre-trained text generation systems.