Alleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization (2023)

Abstract: Most Transformer based abstractive summarization systems have a severe mismatch between training and inference, i.e., exposure bias. From diverse perspectives, we introduce a simple multi-level contrastive learning framework for abstractive summarization (SimMCS) and a tailored sparse decoder self-attention pattern (SDSA) to bridge the gap between training and inference to improve model performance. Compared with previous contrastive objectives focusing only on the relative order of probability mass assigned to non-gold summaries, SimMCS additionally takes their absolute positions into account, which guarantees that the relatively high-quality (positive) summaries among them could be properly assigned high probability mass, and further enhances the capability of discriminating summary quality beyond exploiting potential artifacts of specific metrics. SDSA simulates the possible inference scenarios of deviation in the training phase to get closer to the ideal paradigm. Our approaches outperform the previous state-of-the-art results on two summarization datasets while just adding fairly low overhead. Further empirical analysis shows our model preserves the advantages of prior contrastive methods and possesses strong few-shot learning ability.

## introduction automatic text summarization @xcite is the task of condensing a piece of text into a shorter version while preserving its most salient information and overall meaning. there are two main research directions for text summarization: extractive summarization and abstractive summarization @xcite . extractive summarization involves selecting salient text spans from source documents, while abstractive summarization generates concise summaries in a sequence-to-sequence † corresponding author. xiaofan.zhang@sjtu.edu.cn manner @xcite . recently, large pre-trained neural models, typically based on encoder-decoder transformer @xcite , have shown promising performance in abstractive summarization. these models are generally optimized using maximum likelihood estimation (mle) in teacher forcing form @xcite to maximize the predictive probability of the reference output given its prior gold sub-sequence. however, during inference, the models produce output based on the generated sub-sequence, which may contain errors. this mismatch between training and inference can negatively impact model performance and is known as exposure bias @xcite . to alleviate the ubiquitous issue while maintaining reasonable performance, a variety of methods from various perspectives have been proposed. among them, sentence-level training @xcite @xcite aims to train the model using sentence-level evaluation metrics (e.g., rouge). another direction of attempts involves forging rational schemes to inject noise or perturb reference output during the decoding stage of training @xcite . the model trained in the noisy environment can perceive more states of inference. notably, recent contrastive learning methods in abstractive summarization combine the strengths of these approaches. their contrastive objectives guide the model to distinguish between summaries with different metric scores at the sentence level. while the strong contrastive method @xcite achieves good results by correlating the relative order of probabilities of system-generated summaries with their evaluation metrics, considering these summaries have considerably high relevance to their gold summary, low probabilities of highquality summaries can result in a low probability being assigned to their gold summary. since the ## related work transformer-based pre-trained model recently, large seq2seq transformers @xcite , which contain encoder self-attention, decoder self-attention, and decoder cross-attention, have achieved promising performance in the nlp domain, including text summarization @xcite . these models are pretrained using a variety of self-supervised objectives and fine-tuned with structured losses in downstream tasks. for example, bart @xcite , a denoising autoencoder, is pre-trained to reconstruct original text spans corrupted with an arbitrary noising function such as text infilling. pe-gasus @xcite is distinguished by its specifically tailored self-supervised pre-training objective for the summarization task. in pegasus, salient text spans are removed or masked from the original document, and the model aims to restore the remaining text spans to their original form. we use these models as backbones in our work. mitigating exposure bias for abstractive summarization in the nlg domain, exposure bias is widespread and has received much attention from researchers @xcite @xcite @xcite . in abstractive summarization, @xcite introduces a reinforcement learning method with the rouge metric as a reward to encourage the generation of novel phrases. inspired by generative adversarial networks, @xcite proposes a novel approach for sequence generation, in which the discriminator is integrated into a beam search @xcite @xcite . contrastive learning contrastive learning has been widely confirmed to effectively boost model performance by allowing the model to distinguish between the quality of diverse samples @xcite . recently the method has shown promising performance in natural language generation tasks such as text summarization @xcite and machine translation @xcite . in fact, these contrastive examples can be constructed using rule-or modelbased methods, with the latter able to produce text examples closer to human-generated ones and forge more natural contrastive schemes. on the other hand, contrastive learning can be performed in latent or discrete space. for instance, @xcite introduces a contrastive learning framework into the representation of sentence embeddings and greatly advances state-of-the-art results. @xcite adopts the discriminative re-ranking over generated summaries in discrete space like other works @xcite @xcite . bootstrap re-sampling the bootstrap approach is a collection of sample reuse techniques designed to estimate sampling variances, confidence intervals, and other properties of statistics @xcite @xcite . compared to traditional standard approaches, these techniques have fewer requirements and assumptions while achieving better performance and providing insight into many problems. ## methodology our complete design consists of a generative objective, multi-level contrastive learning framework simmcs, and a sparse decoder self-attention pattern sdsa. during training, we train the abstractive model according to the training pipeline shown in fig. 1 . the reference summary is only used in the generative objective, while other types of summaries are used for contrastive learning. at the inference stage, the optimized model generates summaries in a conventional manner, using only the source document as input to its encoder. ## experiments here we provide a brief overview of the datasets, baselines, implementation, and evaluation. more experimental details are provided in appendix a. ## discussion we evaluate two variants of our contrastive framework simmcs: (1) simmcs-std uses standard attention modules, and (2) simmcs-sdsa instead leverages our sdsa pattern as an alternative during training. specifically, we select bart on cnndm ## conclusion and future work we introduce simmcs, a simple multi-level contrastive learning framework for abstractive summarization that simultaneously considers the relative order and absolute position of probabilities assigned to high-quality summaries and further discriminates semantic discrepancy of summaries at different quality levels. furthermore, we propose a simple yet empirically effective decoder self-attention pattern to alleviate exposure bias and improve model performance by a large margin. all our methods are not restricted to specific tasks or models and demonstrate strong generalization in conditional text generation tasks. there are several directions to further exploit the potential power of our approaches. first, we believe that the margin estimation with bootstrap re-sampling could be more accurate and robust if given more probability mass. second, it is also feasible to explore more sparse decoder self-attention mechanisms with diverse strategies. finally, our methods could be extended to other text generation tasks such as machine translation.