Empowering Large Language Models for Textual Data Augmentation (2024)

Abstract: With the capabilities of understanding and executing natural language instructions, Large language models (LLMs) can potentially act as a powerful tool for textual data augmentation. However, the quality of augmented data depends heavily on the augmentation instructions provided, and the effectiveness can fluctuate across different downstream tasks. While manually crafting and selecting instructions can offer some improvement, this approach faces scalability and consistency issues in practice due to the diversity of downstream tasks. In this work, we address these limitations by proposing a new solution, which can automatically generate a large pool of augmentation instructions and select the most suitable task-informed instructions, thereby empowering LLMs to create high-quality augmented data for different downstream tasks. Empirically, the proposed approach consistently generates augmented data with better quality compared to non-LLM and LLM-based data augmentation methods, leading to the best performance on 26 few-shot learning tasks sourced from a wide range of application domains.

## introduction large language models (llms) have recently demonstrated their potential in performing data augmentation on text data @xcite @xcite . serving as a semantic-preserving transformation function, llms transform original texts based on instructions to create diverse and informative data augmentations. with the augmented data, users can further train a spreadable and affordable model (e.g. opt @xcite ) to perform specific tasks. unlike traditional heuristic-based methods such as word swapping @xcite and modelbased methods like back-translation @xcite how many revolutions does the moon make around our planet during spring? how many revolutions does the moon make around the earth during spring? question answering: openbookqa improvement of clarity. pronoun replace sentence 1: her putt for birdie was three feet short but she saved par without difficulty. ## proposed approach -self-llmda to reduce the human efforts in designing augmentation instructions and selecting a task-specific instruction for a given task, we propose self-llmda depicted in fig. 2 . the process begins with the llm generating a diverse set of potential instructions @xmath0: a selection model s then scores these generated instructions against the dataset d to identify the most suitable instruction i * : based on the selected instruction i * , the llm performs data augmentation on d, producing an enhanced augmented dataset d ′ for training the target model more effectively. ## conclusion in this work, we introduced self-llmda, a novel framework that leverages the capabilities of llms for textual data augmentation. our approach addresses the challenges associated with traditional data augmentation methods and the limitations of manual instruction generation in llm-based augmentation. self-llmda automates the generation and selection of augmentation instructions, thereby significantly enhancing the quality and applicability of augmented data across diverse downstream tasks. tested across 26 diverse few-shot learning tasks, self-llmda consistently outperforms both non-llmda and manual-llmda methods, showcasing its effectiveness and applicability.