Quantifying Appropriateness of Summarization Data for Curriculum Learning (2021)

Abstract: Much research has reported the training data of summarization models are noisy; summaries often do not reflect what is written in the source texts. We propose an effective method of curriculum learning to train summarization models from such noisy data. Curriculum learning is used to train sequence-to-sequence models with noisy data. In translation tasks, previous research quantified noise of the training data using two models trained with noisy and clean corpora. Because such corpora do not exist in summarization fields, we propose a model that can quantify noise from a single noisy corpus. We conduct experiments on three summarization models; one pretrained model and two non-pretrained models, and verify our method improves the performance. Furthermore, we analyze how different curricula affect the performance of pretrained and non-pretrained summarization models. Our result on human evaluation also shows our method improves the performance of summarization models.

## introduction sequence-to-sequence models have led to the great advancement of summarization. these models require appropriate pairs of source texts and summaries. however, much research has reported summarization datasets contain inappropriate pairs @xcite @xcite . sequenceto-sequence summarization models leverage titles as summaries. in theory, summaries should reflect what is written in the source texts, but in fact, the titles can be too general or contain information not written in the source texts. there is a growing need to deal with these noisy datasets. one way to train with noisy data is curriculum learning @xcite . curriculum learning is a method to change the order of training data and improves convergence speed and the performance of models. in translation tasks, previous studies estimate noise of data using likelihoods of two generative models trained with clean and noisy data, and then applied it to curriculum learning @xcite @xcite . because there is no such datasets in the summarization field, we propose appropriateness estimator, a noise-estimating model that can be trained from a single noisy corpus. the model distinguishes pairs of a source and target text in the original summarization dataset from randomly assigned pairs. the randomly assigned pairs are clearly inappropriate pairs; the target texts do not reflect the information on the source texts. by distinguishing the obvious inappropriate pairs, the model learns to predict appropriateness of data. we apply the appropriateness to curriculum learning; when training a summarization model, we gradually change the training data from inappropriate data to appropriate ones. we experiment with two datasets; enron subject dataset @xcite , and reddit tifu title dataset @xcite . both have noisy training data, but the enron dataset has manually cleansed validation and evaluation datasets, whereas the validation and the evaluation datasets of the reddit dataset are raw datasets that include noise. as summarization models, we employ bart as a pretrained model, and transformer and sequenceto-sequence with attention (seq2seqatt) as nonpretrained models. the result shows our appropriateness estimator improves both pretrained models and non-pretrained models. also, we analyze how three different curricula affect the result and conclude training with small fine data in the last phase is important for pretrained models and generalization with various data in the beginning phase is important for non-pretrained models. also, we conduct human evaluation and verify curriculum learning using our appropriateness estimator improves the performance of summarization models. the contributions of this paper are as follows. ## related works curriculum learning is a method to change the order of training data to improve convergence speed and accuracy @xcite . @xcite applied this to language generation, and introduced two types of curriculum learning: baby step curriculum and one-pass curriculum, and concluded the former is more effective to language generation. many of the later works applied baby step curriculum to translation tasks @xcite , but research of curriculum learning on summarization is yet to be conducted. curriculum learning was originally a method to sort training data by difficulty, but recent research proposed methods to sort data by noise. @xcite proposed a method to quantify the noise in data using two models; one trained on clean data and the other on noisy data. using the same algorithm, @xcite applied reinforcement learning to choose which subset is most appropriate for training. however, it is not possible to apply this to summarization tasks, because clean and noisy versions of the same corpus are not available. sequence-to-sequence summarization models generally use headlines, titles or subjects as summaries. however, it is reported that those datasets are noisy. @xcite introduced a task to generate subjects of emails, but because the original subjects were noisy, they prepared new validation and evaluation datasets on their own. @xcite used rules and a classification model to filter noisy data of review summarization. ## method appropriateness estimator we propose appropriateness estimator, a noise estimator model that can be trained from a single noisy corpus. the overview of the model is described in figure 1 . we label pairs of source and target texts in summarization training data as positive. we assign randomly sampled target texts to source texts and label the pairs as negative. the training task of the model is to predict the labels of the pairs. pairs in summarization training data are all labeled positive, but as explained in introduction, it includes inappropriate pairs. following @xcite , we conduct early stopping to prevent the model from overfitting to noisy data. the probability p(c|s i , t k ) of the model indicates appropriateness of pairs. here s i is a source text, and t k is a target text. c is a binary class; @xmath0, and @xmath1. we sort summarization training data by the appropriateness and conduct curriculum learning. cirik et al. (2016) introduced two curricula: one-pass curriculum and baby step curriculum. the overview of these curricula is described in figure 2 . in both settings, we first sort data by a chosen metric (e.g. appropriateness or target length) in ascending order. next, we split the data into segments. ## result and discussion the result on ## conclusion in this research, we proposed appropriateness estimator that quantifies noise of training data for sequence-to-sequence models from a single noisy corpus. we conducted experiments of curriculum learning on summarization tasks. we experimented on two datasets, enron subject and reddit title datasets and three summarization models: bart, transformer, and sequence-to-sequence with attention. the result showed our method improved the performance of the models. we also conducted experiments with three types of curriculum learning (one-pass, baby step, and noise-annealing curricula), and concluded that choosing small data for fine-tuning in the last phase of the training was important for pretrained models and generalization with various data in the beginning phase was important for non-pretrained models. for future work, we seek for more effective methods to find data for fine-tuning.