Mini- DA : Improving Your Model Performance through Minimal Data Augmentation using LLM (2024)

Abstract: When performing data augmentation using large language models (LLMs), the common approach is to directly generate a large number of new samples based on the original dataset, and then model is trained on the integration of augmented dataset and the original dataset. However, data generation demands extensive computational resources. In this study, we propose Mini-DA, a minimized data augmentation method that leverages the feedback from the target model during the training process to select only the most challenging samples from the validation set for augmentation. Our experimental results show in text classification task, by using as little as 13 percent of the original augmentation volume, Mini-DA can achieve performance comparable to full data augmentation for intent detection task, significantly improving data and computational resource utilization efficiency.

## introduction data is the lifeblood of deep learning models, and the availability of high-quality data is crucial for achieving strong model performance. however, acquiring such data can be a challenge, particularly in scenarios where data is limited or unavailable. moreover, human annotation, a common method for obtaining labeled data, is known to be financially expensive and time-consuming. as such, data augmentation techniques have become increasingly important, especially in scenarios where data is limited. data augmentation has been studied for a long time in various domains, with rule-based method, data interpolation techniques, and model based approaches explored @xcite . while these traditional data augmentation methods have shown effectiveness, the rapidly * these authors contributed equally to this work evolving field of large language models (llms) has ushered in a new era of augmentation methods for natural language processing tasks. with their remarkable ability to generate human-like text, llms have enabled generative data augmentation techniques that can create more diverse and realistic synthetic samples, potentially leading to improved model performance. however, as highlighted in the comprehensive survey by @xcite , the generation of extensive augmented datasets can cause significant expenses due to the demands of considerable computational resources, especially for sota models. to address the limitation of data augmentation with llms, we propose mini-da, a novel framework that aims to maximize the benefits of llmbased data augmentation while minimizing the associated costs. the key innovation of mini-da lies in its ability to leverage the prediction result of the target model on the validation set during kfold cross-validation to identify "challenging samples" that the model struggles to predict correctly. then, for these difficult samples, a instructiontuned large language model is used to generate synthesized data based on a given query and its label. this process is repeated iteratively, collecting augmented data until the model's performance on the test set stabilizes. through our experiments on two datasets for the intent detection task, we demonstrate that by focusing augmentation efforts on a limited number of difficult samples, mini-da significantly reduces the augmentation volume compared to full data augmentation, leading to substantial savings in computational resources while still producing comparable performance. ## method in the following section, we describe our proposed iterative llm-in-the-loop data augmentation approach mini-da, as illustrated in figure 1 . at each iteration, we leverage the feedback from the target model to identify difficult examples from validation set and instruct llm to only augment these selected samples. the mini-da process can be broken down into the following steps: ## conclusion in this work, we present mini-da to efficiently augment intent detection data with llms. we design a iterative llms-in-the-loop framework that incorporates feedback from fine-tuning stage of target model to generate an augmented dataset. the results demonstrate that with as little as 13% of the augmented data generated, we can achieve comparable performance to full data augmentation on intent detection task in data-limited scenarios. overall, mini-da presents a promising solution for data augmentation which significantly reducing computational costs and improving data efficiency. for future work, our plan involves conducting comprehensive experiments across various tasks, including but not limited to question answering, text generation, and text retrieval. we believe this approach can be effective in improving model performance across a wide range of tasks in a dataefficient manner.