Improving abstractive summarization with energy-based re-ranking (2022)

Abstract: Current abstractive summarization systems present important weaknesses which prevent their deployment in real-world applications, such as the omission of relevant information and the generation of factual inconsistencies (also known as hallucinations). At the same time, automatic evaluation metrics such as CTC scores (Deng et al., 2021) have been recently proposed that exhibit a higher correlation with human judgments than traditional lexical-overlap metrics such as ROUGE. In this work, we intend to close the loop by leveraging the recent advances in summarization metrics to create quality-aware abstractive summarizers. Namely, we propose an energy-based model that learns to re-rank summaries according to one or a combination of these metrics. We experiment using several metrics to train our energy-based re-ranker and show that it consistently improves the scores achieved by the predicted summaries. Nonetheless, human evaluation results show that the re-ranking approach should be used with care for highly abstractive summaries, as the available metrics are not yet sufficiently reliable for this purpose.

## introduction in recent years, abstractive methods have greatly benefited from the development and widespread availability of large-scale transformer-based language generative models @xcite @xcite , which are capable of generating text with unprecedented fluency. despite the recent progress, abstractive summarization systems still suffer from problems that hamper their deployment in real-world applications. omitting the most relevant information from the source document is one of such problems. additionally, factual inconsistencies (also known as hallucinations) were estimated to be present in around 30% of the summaries produced by abstractive systems on the cnn/dailymail dataset @xcite . this observation has motivated a considerable amount of research on strategies to mitigate the hallucination problem @xcite @xcite , but the improvements achieved so far are mild. this is partly due to the difficulty of evaluating the quality of summaries automatically, leading to the adoption of metrics that are often insufficient or even inappropriate. despite its limitations, rouge @xcite is still the de facto evaluation metric for summarization, mostly due to its simplicity and interpretability. however, not only does it correlate poorly with human-assessed summary quality @xcite , but it is also unreliable whenever the reference summary contains hallucinations, which unfortunately is not an uncommon issue in widely adopted summarization datasets @xcite . for these reasons, the development of more reliable evaluation metrics with a stronger correlation with human judgment is also an active area of research @xcite @xcite . in this work, we propose a new approach to abstractive summarization via an energy-based model. in contrast to previous approaches, which use reinforcement learning to train models to maximize rouge or bert scores @xcite , our ebm is trained to re-rank the candidate summaries the same way that the chosen metric would rank them -a much simpler problem which is computationally much more efficient. this way, we are distilling the metric, which presents as a by-product an additional advantage: a quality estimation system that can be used to assess the quality of the summaries on the fly without the need of reference summaries. it should be remarked that any reference-free metric, can be used at inference time for re-ranking candidates from any abstractive summarization system, hence improving the 1 quality of the generated summaries. our re-ranking model can therefore leverage the advantages of recently proposed evaluation metrics over traditional ones, which are essentially two-fold: i) being able to better capture high-level semantic concepts, and ii) in addition to the target summary, these metrics take into account the information present on the source document, which is crucial to detect hallucinations. we demonstrate the effectiveness of our approach on standard benchmark datasets for abstractive summarization @xcite , and xsum, @xcite ) and use a variety of summarization metrics as the target to train our model on, showing the versatility of the method. we also conduct a human evaluation experiment, in which we compare our re-ranking model trained to maximize recent transformer-based metrics that aim to measure factual consistency and relevance @xcite ). our proposed model yields improvements over the usual beam search on a baseline model and demonstrates the ability to distill target metrics. however, the human evaluation results suggest that re-ranking according to these metrics, while competitive, may yield lower quality summaries than those obtained by state-of-the-art abstractive systems trained with augmented data and contrastive learning. the remainder of the paper is organized as follows: in section 2, we discuss the related work; in section 3, we do a brief high-level description of neural abstractive summarization systems and how different candidate summaries can be generated from them; in section 4, we describe our methodology in detail, as well as the summarization metrics that we shall use to train our re-ranking model; section 5 presents the experimental results of our model and baselines, which include both automatic and human evaluation; in section 6, we discuss the limitations of our approach and point some directions for future work, and we conclude this work with some final remarks in section 7. ## related work in the context of natural language generation, the idea of re-ranking candidates has been studied extensively for neural machine translation @xcite @xcite @xcite , but only seldom explored for abstractive summarization. among the former, the approach by @xcite is the most similar to ours as they also resort to an energy-based model to re-rank the candidates. however, they do not apply their method to abstractive summarization and their training objective is different than the one we shall define for our model: at each training step, they sample a pair of candidates, and the model is trained so that the difference between the energies of the two candidates is at least as large as the difference of their bleu scores @xcite . thus, their approach only exploits the information of two candidates at each training step. recently, improved learning objectives such as contrastive losses have been proposed to enhance the quality of the predicted summaries, especially their factual consistency. @xcite , @xcite , and @xcite used data augmentation to generate both factual consistent and inconsistent sentences and used these in a contrastive learning objective to regularize the transformer learned representations. in a different line of work, @xcite and @xcite trained separate models on the task of correcting factual inconsistencies in the predicted summaries. @xcite presented a model that learns to extract a knowledge graph from the source document and uses it to condition the decoding step. @xcite trained a model to detect non-factual tokens and used it to identify and discard these tokens from the training data of the summarizer. @xcite modified the output distribution of the model to put more focus on the vocabulary tokens that are similar to the attended input tokens. despite being sensible ideas, these techniques mostly focus on redefining the training objective of the model and disregard the opportunity to improve the summary quality at inference time, either by redesigning the sampling algorithm or using re-ranking. in a somewhat similar direction to ours, a contemporary work @xcite proposes using a ranking objective as an additional term on the usual negative log-likelihood loss. similar to us, @xcite and @xcite propose to use a trained re-ranker in as post-generation step. the former use a contrastive objective to learn a re-ranker that mimics rouge scores. the latter employs a mixture of experts to train a re-ranker on the combination of rouge, bert and bart scores. ## abstractive summarization systems a typical abstractive summarization model approximates the conditional distribution p(y | x), of summaries y given source documents x, and works auto-regressively, exploiting the chain rule of probability: in essence, finding the optimal training objective and decoding algorithm to obtain the best summary remains an open problem. we take a step in this direction by sampling a set of candidate summaries {ŷ 1 , ŷ2 , . . . , ŷk } and then using a re-ranking model to choose the best one. to ensure diverse candidates, we experiment with diverse beam search @xcite , a modification of traditional beam search including a term in the scoring function that penalizes for repetitions across different beams. ## limitations and future work despite the improvements attained by our ebr model, its applicability is fundamentally dependent on the availability of reliable automatic evaluation metrics. unfortunately, the correlation of these metrics with human judgment is still imperfect, especially for highly abstractive summaries. in addition, transformer-based metrics are currently only available for english. finally, their backbone models are trained on news data, which hampers the reliability of these metrics in other domains. it is, therefore, crucial to continue the pursuit for more reliable metrics and to extend them to more languages and domains. ## conclusion we proposed an energy-based re-ranking model that can be trained to rank candidate summaries according to a pre-specified metric, leveraging the recent advancements in automatic summarization metrics to enhance the quality of the generated summaries. the experiments show that the proposed re-ranking model succeeds at distilling the target metrics, consistently improving the scores of the generated summaries. however, these improvements not always agree with the human evaluation, especially in the more abstractive setting (xsum), due to flaws of the adopted target metrics (ctc scores). nonetheless, the proposed approach is flexible in the sense that we can train it with any target metric and apply it in conjunction with virtually any abstractive summarization system. liu zhuang, lin wayne, shi ya, and zhao jun. 2021. a robustly optimized bert pre-training approach with post-training. in proceedings of the 20th chinese national conference on computational linguistics, pages 1218-1227, huhhot, china. chinese information processing society of china.