Dynamic Augmentation Data Selection for Few-shot Text Classification (2022)

Abstract: Data augmentation has been a popular method for fine-tuning pre-trained language models to increase model robustness and performance. With augmentation data coming from modifying gold train data (in-sample augmentation) or being harvested from general domain unlabeled data (out-of-sample augmentation), the quality of such data is the key to successful fine-tuning. In this paper, we propose a dynamic data selection method to select effective augmentation data from different augmentation sources according to the model’s learning stage, by identifying a set of augmentation samples that optimally facilitates the learning process of the most current model. The method firstly filters out augmentation samples with noisy pseudo labels through a curriculum learning strategy, then estimates the effectiveness of reserved augmentation data by its influence scores on the current model at every update, allowing the data selection process tightly tailored to model parameters. And the two-stage augmentation strategy considers in-sample augmentation and out-of-sample augmentation in different learning stages. Experiments with both kinds of augmentation data on a variety of sentence classification tasks show that our method outperforms strong baselines, proving the effectiveness of our method. Analysis confirms the dynamic nature of the data effectiveness and the importance of model learning stages in utilization of augmentation data.

## introduction finetuning pre-trained language models (plms), such as bert @xcite and roberta @xcite , requires a large amount of high-quality labeled data to reach high performance. recently, few-shot learning with plms @xcite has become a popular research topic, where approaches have been proposed to help fine-tuned plms achieve good performance in tasks where gold training data is scarce. one significant challenge in fine-tuning plms in low-resource settings is that the model can easily overfit to the training data of a small size and have a hard time generalizing to unseen data @xcite . data augmentation (da) is a popular method in preventing plms from overfitting by creating new training data by either modifying existing training samples with external resources such as a machine translation engine, or finding new training samples with pseudo-labels using tools and heuristics @xcite . da has been successfully applied to various tasks such as image classification @xcite , object detection @xcite ), text classification @xcite , and neural machine translation @xcite @xcite to improve model performance. the quality of augmentation data is crucial to the success of the application of da to a task, therefore data selection (e.g. @xcite @xcite is necessary where augmentation instances are chosen for their effectiveness in improving model performance on the task. this is especially true for silver training instances harvested from general domain sources such as wikipedia. such silver instances can be used to enhance model generalizability due to their out-of-sample nature, whereas data augmentation through data modification @xcite has limited capability. however, out-of-sample augmentation training instances tend to be noisier and more numerous than in-sample augmentation instances, exacerbating the problem of selecting the effective augmentation instances @xcite . for both kinds of augmentation data, the effectiveness for improving model performance is highly dynamic and changes as training progresses. this can be observed in figure 1 where 1,000 silver instances are chosen for their oracle effectiveness in reducing the test error for a stance detection task @xcite with a randomly initialized model. after training with gold data for a different number of epochs, the number of effective augmentation data among the 1,000 instances changes drastically, showcasing how the effectiveness of the silver instances changes dynamically at different learning stages. however, previous research either does not pay attention to the interaction between the learning stage of the model and silver instance effectiveness @xcite , or only considers representation similarity dynamically, such as choosing augmentation samples generated from the current batch @xcite @xcite while missing the arguably more important aspect of dynamic effectiveness. in this paper, we address the augmentation instance selection problem for both in-sample and out-of-sample data augmentation with a unified framework centered around data effectiveness. the framework, named dynselector, leverages the influence function @xcite to directly estimate the effectiveness of each augmentation sample dynamically based on the current learning stage. this ensures the selected augmentation samples always have a positive influence on model performance. when coupled with a curriculum learning-based strategy for mitigating the noisy supervision from augmentation data, our augmentation data selection method can effectively utilize and combine different augmentation sources to improve few-shot text classification performance. when it comes to augmentation data selection, @xcite empirically proves that the main bottleneck of data augmentation for few-shot learning comes from the corrupted label in the augmen-tation data. @xcite applies contrastive learning to enforce augmentation data with various weights to be separable, according to the assigned weight, in the embedding space. @xcite implements two filters to detect effective augmentation samples from data candidates generated by a language generative model. one of the filter is based on influence function to evaluate the change of generalization error if accepting one augmentation sample and another filter evaluates the diversity property of augmentation data. our method leverages the influence function to estimate the effectiveness of augmentation data, and only considers augmentation data that has a positive influence to the model's performance. ## method we propose an algorithm which dynamically selects augmentation instances based on the effectiveness on a model learning stage from pools of silver training instances generated by in-sample or outof-sample augmentation methods. at a high level, the algorithm selects a batch of augmentation instances after each model update with gold training instances, based on the influence scores of the augmentation instances to ensure their effectiveness for the most current model. when both in-sample and out-of-sample silver instances are available, the algorithm first selects in-sample silver instances to ensure the model is well-trained, and then expands into out-of-sample silver instances to boost the model's generalizability and robustness once the model has approached convergence. we specify in detail each part of the algorithm below. ## experiments we conduct extensive experiments with the proposed algorithm and a variety of classification tasks. we adopt bert @xcite as our base modeland one fully-connected layer as the classifier for all experiments, with details about the training hyperparameters described in appendix a.1. all the models are implemented with pytorch @xcite . ## analysis three ablation studies are conducted to explore the effect of training data size, hessian matrix update frequency and augmentation strategy on the effectiveness of the algorithm. ## conclusion in this paper, we propose an augmentation data selection method to improve plms-based few-shot text classifications through dynamically choosing effective in-sample and out-of-sample augmentation data in accordance to the learning stage of classification models. the experimental results confirm the effectiveness of our method. given these two augmentation sources, our method achieves large performance gain by using the two-stage augmentation training strategy together with the dynamic data selection algorithm.