S umma R eranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization (2022)

Abstract: Sequence-to-sequence neural networks have recently achieved great success in abstractive summarization, especially through fine-tuning large pre-trained language models on the downstream dataset. These models are typically decoded with beam search to generate a unique summary. However, the search space is very large, and with the exposure bias, such decoding is not optimal. In this paper, we show that it is possible to directly train a second-stage model performing re-ranking on a set of summary candidates. Our mixture-of-experts SummaReranker learns to select a better candidate and consistently improves the performance of the base model. With a base PEGASUS, we push ROUGE scores by 5.44% on CNN- DailyMail (47.16 ROUGE-1), 1.31% on XSum (48.12 ROUGE-1) and 9.34% on Reddit TIFU (29.83 ROUGE-1), reaching a new state-of-the-art. Our code and checkpoints will be available at https://github.com/ntunlp/SummaReranker .

## introduction in recent years, sequence-to-sequence neural models have enabled great progress in abstractive summarization @xcite . in the news domain, they have surpassed the strong lead-3 extractive baseline. with the rise of transfer learning since bert @xcite , leading approaches typically fine-tune a base pretrained model that either follows a general text generation training objective like t5 @xcite , bart @xcite , ernie @xcite and prophetnet @xcite , or an objective specifically tailored for summarization like in pegasus @xcite . most of these sequence-to-sequence models are history-based, where an output sequence is represented as a sequence of decisions and the probabil-* equal contribution. ity of the sequence is computed as a product of decision probabilities. this is also known as the autoregressive factorization. to transform the sequence of probabilities into summaries, beam search is commonly used. while auto-regressive decoding with beam search is simple and has many advantages, it can be difficult to encode global constraints such as grammaticality, coherence and factual consistency within this framework, properties that are believed to be useful in discriminating among candidate outputs. if the model starts decoding in a bad direction, mistakes might propagate, carry over the mistake of previous tokens to the generation of new ones, and the model has no way to know that it should adjust the decoding. furthermore, these models are typically trained with teacher forcing @xcite , which leads to an inherent discrepancy between training time and inference time known as the exposure bias problem @xcite . decoding methods such as beam search maintain a list of top-k best candidates, and output a single best one. in the case of beam search, candidates are sorted by decreasing log-probability, and the last (k -1) hypotheses are discarded. however, these (k -1) other hypotheses often contain considerably better sequences in terms of different evaluation measures. this observation holds over other decoding methods: diverse beam search @xcite , top-k sampling @xcite and top-p sampling @xcite . in given this assessment, we investigate whether it is possible to train a second-stage summarization model which learns to select the best summary among a set of candidates obtained from a base model and with a decoding process, which itself can potentially involve a set of decoding methods (e.g., beam search variants). this way, the model would recover the gap that separates it with the oracle. this raises the question of what makes a summary candidate the optimal one? admittedly, summarization has been an underconstrained task and its evaluation is complex and remains an active research area @xcite @xcite . to build a flexible approach, we use a multi-task learning framework based on a mixture-of-experts architecture in order to optimize jointly over several measures. to design a robust re-ranker, we systematically explore the dimensions of summary re-ranking: base model, decoding process, and evaluation measure. our system, named summareranker, is flexible and multi-task: it can be trained with any set of evaluation metrics. it is considerably less computationnally expensive to train than the singlestage summarization models that it is plugged on. we apply our system across three different datasets {cnn-dailymail, xsum, reddit tifu} and two base models {pegasus, bart}. optimizing rouge metrics leads to relative performance improvements from 1.31% to 9.34% depending on the dataset. it outperforms recently proposed second-stage summarization approaches refsum @xcite and simcls @xcite and sets a new state-of-the-art on cnn-dailymail and xsum @xcite . we present extensive quantitative results coupled with a qualitative human evaluation. ## related work re-ranking has been adopted in several branches of nlp for long. in syntactic parsing, @xcite were the first to employ a re-ranker on the outputs of a base parser, followed by @xcite , who used a maximum entropy re-ranker. passage re-ranking is used as the first stage of question-answering systems, to retrieve relevant passages where the answer might lay @xcite . some recent question-answering models also propose to perform answer re-ranking, to refine the answer selection @xcite . re-ranking has also been used in neural machine translation. checkpoint reranking @xcite generates several translation candidates with multiple model checkpoints, based on the observation (similar to the one we made in §1) that the oracle across checkpoints is of higher quality than just the last checkpoint. @xcite use an energy-based model on top of bert to select translation candidates with higher bleu score. in abstractive summarization, second-stage approaches such as re-ranking remain underexplored. recently, refsum @xcite defined a second-stage summarization framework which helps address the problem of the train-test distribution mismatch in second-stage models. with a base gsum model @xcite , the authors reach a 46.18 state-of-the-art rouge-1 on cnn-dailymail. in simcls @xcite , the authors train a second-stage model with contrastive learning, using a ranking loss to select the best summary candidate from a pool of 16 diverse beam search candidates, reaching 46.67 rouge-1 on cnn-dailymail. our approach differs from ref-sum and simcls in terms of model architecture and loss function, as well as summary candidate generation process. in contrast with refsum, we use a single base model, but mix several decoding methods, as our goal is single-model improvement. unlike simcls, we do not use a ranking loss, but directly model the probability that a summary candidate is the best one. to the best of our knowl-edge, we are the first ones to propose a multi-task re-ranking system for abstractive summarization. this enables practitioners to leverage the recent rich literature in automatic abstractive summarization evaluation @xcite @xcite . ## discussion abstractiveness given that we are not modifying the base model nor its training procedure, we analyze whether our re-ranking system favors more abstractive candidates. in fig. 6 , we display the percentage of novel n-grams for n in {1,2,3,4}, for a base pegasus with beam search (blue) and diverse beam search (purple) decoding, and when adding summareranker in both cases (green and red, respectively). as first raised in @xcite , summary candidates are much less abstractive than ground truth summaries on cnn/dm. yet, our re-ranker selects more abstractive candidates according to all n-grams metrics, even more so with diverse beam search, which is already more abstractive than beam search. this observation also holds on reddit tifu and xsum (other than 1-grams). xsum summary candidates are already almost as abstractive as the ground truth and it is harder to obtain significant abstractiveness gains through our re-ranking. speed/performance trade-off on top of base model training and candidate generation, summareranker inference cost is linear in the number of candidates. a single candidate takes on average 38ms to be scored. as seen in further work to encode the source jointly with the summary candidate, we need to truncate the source to a fixed number of tokens. thus, we are limited by the maximum context window of the language model encoder (512 in the case of roberta-large). applying summareranker to long-document summarization, such as scientific articles summarization @xcite would need better long-range modeling. in §3, we weighted metric-dependent losses uniformly. we leave to further work the exploration of more complex weight balancing or multi-task learning objectives @xcite . ## conclusion we introduced summareranker, the first multi-task re-ranking framework for abstractive summarization. encoding the source with the candidate, our model predicts whether the summary candidate maximizes each of the metrics optimized for. sum-mareranker works well across diverse datasets, models, decoding methods and summarization evaluation metrics. summaries selected by summareranker improve the rouge state-of-the-art on cnn/dm and xsum. in addition, we also show that they are more abstractive and more likely to be preferred by human evaluators over base model outputs. in