S im CLS : A Simple Framework for Contrastive Learning of Abstractive Summarization (2021)

Abstract: In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i.e., quality estimation) assisted by contrastive learning. Experimental results show that, with minor modification over existing top-scoring systems, SimCLS can improve the performance of existing top-performing models by a large margin. Particularly, 2.51 absolute improvement against BART and 2.50 over PEGASUS w.r.t ROUGE-1 on the CNN/DailyMail dataset, driving the state-of-the-art performance to a new level. We have open-sourced our codes and results: https://github.com/yixinL7/SimCLS . Results of our proposed models have been deployed into ExplainaBoard platform, which allows researchers to understand our systems in a more fine-grained way.

## introduction sequence-to-sequence (seq2seq) neural models @xcite have been widely used for language generation tasks, such as abstractive summarization @xcite and neural machine translation @xcite . while abstractive models @xcite have shown promising potentials in the summarization task, they share the widely acknowledged challenges of seq2seq model training. specifically, seq2seq models are usually trained under the framework of maximum likelihood estimation (mle) and in practice they are commonly trained with the teacher-forcing (williams and zipser, 1989) algorithm. this introduces a gap between the objective function and the evaluation metrics, as the objective function is based on local, token-level predictions while the evaluation metrics (e.g. rouge @xcite ) would compare the holistic similarity between the gold references and system outputs. furthermore, during the test stage the model needs to generate outputs autoregressivelly, which means the errors made in the previous steps will accumulate. this gap between the training and test has been referred to as the exposure bias in the previous work @xcite . a main line of approaches @xcite proposes to use the paradigm of reinforcement learning (rl) to mitigate the aforementioned gaps. while rl training makes it possible to train the model with rewards based on global predictions and closely related to the evaluation metrics, it introduces the common challenges of deep rl. specifically, rl-based training suffers from the noise gradient estimation @xcite problem, which often makes the training un-stable and sensitive to hyper-parameters. minimum risk training, as an alternative, has also been used in the language generation tasks @xcite . however, the accuracy of the estimated loss is restricted by the number of sampled outputs. other methods @xcite @xcite aim to extend the framework of mle to incorporate sentence-level scores into the objective functions. while these methods can mitigate the limitations of mle training, the relation between the evaluation metrics and the objective functions used in their methods can be indirect and implicit. among this background, in this work we generalize the paradigm of contrastive learning @xcite to introduce an approach for abstractive summarization which achieves the goal of directly optimizing the model with the corresponding evaluation metrics, thereby mitigating the gaps between training and test stages in mle training. while some related work @xcite have proposed to introduce a contrastive loss as an augmentation of mle training for conditional text generation tasks, we instead choose to disentangle the functions of contrastive loss and mle loss by introducing them at different stages in our proposed framework. specifically, inspired by the recent work of zhong et al. (2020); liu et al. (2021b) on text summarization, we propose to use a two-stage model for abstractive summarization, where a seq2seq model is first trained to generate candidate summaries with mle loss, and then a parameterized evaluation model is trained to rank the generated candidates with contrastive learning. by optimizing the generation model and evaluation model at separate stages, we are able to train these two modules with supervised learning, bypassing the challenging and intricate optimization process of the rl-based methods. our main contribution in this work is to approach metric-oriented training for abstractive summarization by proposing a generate-then-evaluate twostage framework with contrastive learning, which not only put the state-of-the-art performance on cnn/dailymail to a new level (2.2 rouge-1 improvement against the baseline model), also demonstrates the great potentials of this two-stage framework, calling for future efforts on optimizing seq2seq models using methods beyond maximum likelihood estimation. ## contrastive learning framework for abstractive summarization given a source document d and a reference summary ŝ, the goal of an abstractive summarization model f is to generate the candidate summary @xmath0, ŝ) assigned by an evaluation metric m . in this work, we break down the holistic generation process into two stages which consist of a generation model g for generating candidate summaries and a evaluation model h for scoring and selecting the best candidate. fig 1 illustrates the general framework. ## conclusion in this work, we present a contrastive summarization framework that aims to optimize the quality of generated summaries at summary-level, which mitigates the discrepancy between the training and test