On LLM s-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024)

Abstract: Within the evolving landscape of deep learning, the dilemma of data quantity and quality has been a long-standing problem. The recent advent of Large Language Models (LLMs) offers a data-centric solution to alleviate the limitations of real-world data with synthetic data generation. However, current investigations into this field lack a unified framework and mostly stay on the surface. Therefore, this paper provides an organization of relevant studies based on a generic workflow of synthetic data generation. By doing so, we highlight the gaps within existing research and outline prospective avenues for future study. This work aims to shepherd the academic and industrial communities towards deeper, more methodical inquiries into the capabilities and applications of LLMs-driven synthetic data generation.

## introduction the game-changing emergence of large language models (llms) instigated a significant paradigm shift in the field of deep learning @xcite @xcite . despite these advancements, a large amount of high-quality data remains the foundation for building robust nlp models @xcite . to be more specific, here high-quality data typically refers to diverse data that carries rich supervision signals (generally in the form of labels) closely aligned with human intent. however, fulfilling such data reliance with human data can be challenging or even unrealistic sometimes, due to high costs, data scarcity, privacy concerns, etc. @xcite . moreover, several studies @xcite @xcite have highlighted that human-generated data, being inherently susceptible to biases and errors, may not even be optimal for model training or evaluation. these considerations necessitate a more serious inquiry * corresponding author. ## generic workflow existing studies on llms-driven synthetic data generation generally incorporate three main topics: generation, curation, and evaluation. various approaches are employed within these aspects to collaboratively achieve optimal data generation. ## conclusion in this paper, we present a systematic review of advancements in synthetic data generation propelled by large language models (llms). we aim to offer guidance to enterprises and organizations on effectively building their domain-specific datasets using llms. in the meantime, we endeavor to provide insights into the challenges and opportunities within this field, while also proposing potential directions for future research. we hope that our work can promote the rapid production of large amounts of data in various fields and push the limits of data-centric ai. we also envision a fantastic future, where an llms community, endowed with human-like abilities such as bionics and communication, may be constructed to generate data for its own self-improvement.