Active Learning for Abstractive Text Summarization via LLM -Determined Curriculum and Certainty Gain Maximization (2024)

Abstract: For abstractive text summarization, laborious data annotation and time-consuming model training become two high walls, hindering its further progress. Active Learning, selecting a few informative instances for annotation and model training, sheds light on solving these issues. However, only few active learning-based studies focus on abstractive text summarization and suffer from low stability, effectiveness, and efficiency. To solve the problems, we propose a novel LLM-determined curriculum active learning framework. Firstly, we design a prompt to ask large language models to rate the difficulty of instances, which guides the model to train on from easier to harder instances. Secondly, we design a novel active learning strategy, i.e., Certainty Gain Maximization, enabling to select instances whose distribution aligns well with the overall distribution. Experiments show our method can improve stability, effectiveness, and efficiency of abstractive text summarization backbones.

## introduction abstractive text summarization (ats) aims to condense a lengthy document into a concise yet informative summary, preserving the essential information of the original document @xcite . benefiting from the transformer @xcite and pre-training strategies @xcite , a new paradigm, i.e., pretraining and fine-tuning transformer-based large language models (llms), achieves state-of-the-art results in ats @xcite . however, as the number of parameters in llms increases, processing llms for ats becomes impractical due to the huge time and computational resources required. moreover, similar to other natural language generation (nlg) tasks, fine-tuning llms for ats requires large-scale datasets, which always necessitate labor intensive cost and expert knowledge, further impeding ats @xcite . active learning (al) sheds light on alleviating these issues @xcite . to reduce human labor and time consumption, al aims to select a few unlabeled yet informative instances for annotation and model training @xcite . specifically, bas @xcite and idds @xcite are recent best-performing al methods for ats. bas measures the uncertainty of a model's prediction for each instance and annotates an instance with the highest uncertainty. given a model and an input sentence, bas randomly drops certain parameters in the model to generate multiple outputs. the variance of the bleu scores @xcite between the input and each output is considered as the uncertainty of the given input instance. in contrast, idds @xcite aims to acquire representative instances for model training. it selects instances that are dissimilar to those already selected but are the most representative among unselected instances. despite the great success of the al methods in ats, there are still three issues. i) unstable performance. as shown in figure 1 hibit unstable fluctuations on model performance. this instability often prevents the model from being fully optimized, thereby hindering its performance. because the distribution of selected instances differs from that of the entire dataset, performance improvement is hampered. iii) high computational cost. in figure 1(b ), among all al methods, bas is the most time-consuming one, because it visits the remaining unselected instances in each iteration. although idds visits all instances in advance without inference at each iteration, its time complexity is o(n 2 ) with n being the number of instances, which is a huge overhead for large-scale datasets. to address the above issues, we make two contributions. firstly, to address unstable performance, we propose llm-determined curriculum active learning (ldcal). we found that the al-based training process is highly sensitive to the training order of instances, which easily leads to the model's unstable performance. drawing inspiration from the way humans realize learning stability by gradually increasing the difficulty of learning content, we employ curriculum learning by gradually increasing the complexity of instances from easy to hard during training. unlike previous studies that always determine document difficulty based on traditional curriculum, we design a prompt to directly ask llms for document difficulty. by following the curriculum determined by llms, stable performance can be achieved. secondly, to address the misaligned distribution and high computational cost, we propose a novel al strategy named certainty gain maximization. certainty measures how well unselected instances are represented by se-lected ones, and we select instances that maximize the certainty gain for the unselected instances. by using certainty gain maximization, we can select instances whose distribution aligns well with the overall distribution in figure 2(d ). in addition, the time complexity of certainty gain maximization is o(nk) (k ≪ n), where n and k represent the number of unselected and selected instances, respectively. it is significantly lower than that of existing methods in figure 1(b ). the main contributions of this study are summarized as follows: ## methodologies we first introduce the detailed process of the proposed ldcal framework in section 3.1. then, we introduce the llm-determined curriculum learning strategy for difficulty measurement in section 3.2. finally, in section 3.3, we describe the novel al strategy, i.e., the certainty gain maximization. ## conclusion this research presented the first study of the curriculum active learning framework for ats. extensive experiments showed that llm-determined cl helps to improve model's stability and performance. our al strategy, i.e., certainty gain maximization, could select diverse instances in uneven distribution scenes, further enhancing model's effectiveness and efficiency. in future work, we want to design more effective prompts for cl and investigate ldcal in other nlg tasks beyond ats.