Active Learning for Abstractive Text Summarization (2022)

Abstract: Construction of human-curated annotated datasets for abstractive text summarization (ATS) is very time-consuming and expensive because creating each instance requires a human annotator to read a long document and compose a shorter summary that would preserve the key information relayed by the original document. Active Learning (AL) is a technique developed to reduce the amount of annotation required to achieve a certain level of machine learning model performance. In information extraction and text classification, AL can reduce the amount of labor up to multiple times. Despite its potential for aiding expensive annotation, as far as we know, there were no effective AL query strategies for ATS. This stems from the fact that many AL strategies rely on uncertainty estimation, while as we show in our work, uncertain instances are usually noisy, and selecting them can degrade the model performance compared to passive annotation. We address this problem by proposing the first effective query strategy for AL in ATS based on diversity principles. We show that given a certain annotation budget, using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores. Additionally, we analyze the effect of self-learning and show that it can additionally increase the performance of the model.

## introduction abstractive text summarization (ats) aims to compress a document into a brief yet informative and readable summary, which would retain the key information of the original document. state-of-theart results in this task are achieved by neural seq-toseq models @xcite @xcite @xcite based on the transformer architecture @xcite . training a model for ats requires a dataset that contains pairs of original documents and their short summaries, which are usually writ-ten by human annotators. manually composing a summary is a very tedious task, which requires one to read a long original document, select crucial information, and finally write a small text. each of these steps is very time-consuming, resulting in the fact that constructing each instance in annotated corpora for text summarization is very expensive. active @xcite ) is a well-known technique that helps to substantially reduce the amount of annotation required to achieve a certain level of machine learning model performance. for example, in tasks related to named entity recognition, researchers report annotation reduction by 2-7 times with a loss of only 1% of f1-score @xcite . this makes al especially important when annotation is expensive, which is the case for ats. al works iteratively: on each iteration, (1) a model is trained on the so far annotated dataset; in this work, we present the first effective query strategy for al in ats, which we call in-domain diversity sampling (idds). it is based on the idea of the selection of diverse instances that are semantically dissimilar from already annotated documents but at the same time similar to the core documents of the considered domain. the empirical investigation shows that while techniques based on uncertainty cannot overcome the random sampling baseline, idds substantially increases the performance of summarization models. we also experiment with the self-learning technique that leverages a training dataset expanded with summaries automatically generated by an ats model trained only on the human-annotated dataset. this approach shows improvements when one needs to generate short summaries. the code for reproducing the experiments is available online foot_0 . the contributions of this paper are the following: active learning in natural language generation. while many recent works leverage al for text classification or sequence-tagging tasks @xcite @xcite , little attention has been paid to natural language generation tasks. among the works in this area, it is worth mentioning @xcite @xcite . these works focus on neural machine translation (nmt) and suggest several uncertainty-based query strategies for al. @xcite successfully apply al in the interactive machine translation. @xcite exploit reinforcement learning to train a policy-based query strategy for nmt. although there is an attempt to apply al in ats @xcite , to the best of our knowledge, there is no published work on this topic yet. ## uncertainty-based active learning for text generation in this section, we give a brief formal definition of the al procedure for text generation and uncertainty-based query strategies. here and throughout the rest of the paper, we denote an input sequence as @xmath0. . . x m ) and the output sequence as @xmath1. . . y n ), with m and n being lengths of x and y respectively. let @xmath2, y (k) )} k @xmath3, summaries). consider a probabilistic model p w (y | x) parametrized by a vector w. usually, p w (y | x) is a neural network, while the parameter estimation is done via the maximum likelihood approach: many al methods are based on greedy query strategies that select instances for annotation, optimizing a certain criterion a(x | d, ŵ) called an acquisition function: the selected instance x * is then annotated with a target value y * (document summary) and added to the training dataset: d := d ∪ (x * , y * ). subsequently, the model parameters w are updated and the instance selection process continues until the desired model quality is achieved or the available annotation budget is depleted. the right choice of an acquisition function is crucial for al. a common heuristic for acquisition is selecting instances with high uncertainty. below, we consider several measures of uncertainty used in text generation. normalized sequence probability (nsp) was originally proposed by @xcite and has been used in many subsequent works @xcite @xcite . this measure is given by usually, the exact computation of the posterior is intractable, and to perform training and inference, a family of distributions q θ (w) parameterized by θ is introduced. the parameter estimate θ minimizes the kl-divergence between the true posterior π(w | d) and the approximation q θ(w). given such an approximation, several uncertainty measures can be constructed. expected normalized sequence probability (ensp) is proposed by @xcite and is also used in @xcite : in practice, the expectation is approximated via monte carlo dropout @xcite , i.e. averaging multiple predictions obtained with activated dropout layers in the network. expected normalized sequence variance (ensv; @xcite ) measures the variance of the sequence probabilities obtained via monte carlo dropout: bleu variance (bleuvar) is proposed by @xcite . it treats documents as points in some high dimensional space and uses the bleu metric @xcite for measuring the difference between them. in such a setting, it is possible to calculate the variance of generated texts in the following way: the bleu metric is calculated as a geometric mean of n-grams overlap up to 4-grams. consequently, when summaries consist of less than 4 tokens, the metric is equal to zero since there will be no common 4-grams. this problem can be mitigated with the sacrebleu metric @xcite , which smoothes the n-grams with zero counts. when we use this query strategy with the sacre-blue metric, we refer to it as sacrebleuvar. unlabeled instances labeled instances out-of-domain instances idds queries idds: far from labeled, close to unlabeled on average uncertainty-based methods: far from both labeled and unlabeled figure 1: the visualization of the idea behind the idds alogrithm on the synthetic data: select instances located far from labeled data while close on average to unlabeled data. ## conclusion in this work, we convey the first study of al in ats and propose the first active learning query strategy that outperforms the baseline random sampling. the query strategy aims at selecting for annotation the instances with high similarity with the documents in the unlabeled pool and low similarity with the already annotated documents. it outperforms the random sampling in terms of rouge metrics on all considered datasets. it also outperforms random sampling in terms of the consistency score calculated via the summac model on the aeslc dataset. we also demonstrate that uncertainty-based query strategies fail to outperform random sampling, resulting in the same or even worse performance. finally, we show that self-learning can improve the performance of an ats model in terms of both the rouge metrics and consistency. this is especially favorable in al since there is always a large unlabeled pool of data. we show that combining al and self-learning can give an improvement of up to 58% in terms of rouge metrics. in future work, we look forward to investigating idds in other sequence generation tasks. this query strategy might be beneficial for tasks with the highly variable output when uncertainty estimates of model predictions are unreliable and cannot outperform the random sampling baseline. idds facilitates the representativeness of instances in the training dataset without leveraging uncertainty scores.