DPGA - T ext S yn: Differentially Private Genetic Algorithm for Synthetic Text Generation (2025)

Abstract: Using large language models (LLMs) has a potential risk of privacy leakage since the data with sensitive information may be used for fine-tuning the LLMs. Differential privacy (DP) provides theoretical guarantees of privacy protection, but its practical application in LLMs still has the problem of privacy-utility trade-off. Researchers synthesized data with strong generation capabilities closed-source LLMs (i.e., GPT-4) under DP to alleviate this problem, but this method is not so flexible in fitting the given privacy distributions without fine-tuning. Besides, such methods can hardly balance the diversity of synthetic data and its relevance to target privacy data without accessing so much private data. To this end, this paper proposes DPGA-TextSyn, combining general LLMs with genetic algorithm (GA) to produce relevant and diverse synthetic text under DP constraints. First, we integrate the privacy gene (i.e., metadata) to generate better initial samples. Then, to achieve survival of the fittest and avoid homogeneity, we use privacy nearest neighbor voting and similarity suppression to select elite samples. In addition, we expand elite samples via genetic strategies such as mutation, crossover, and generation to expand the search scope of GA. Experiments show that this method significantly improves the performance of the model in downstream tasks while ensuring privacy.

## introduction using large language models (llms) in sensitive textual domains @xcite poses significant privacy risks, where attackers may hack the fine-tuned llms to extract privacy information in the llms' fine-tuning data @xcite . to prevent privacy data from being extracted, researchers apply differential privacy (dp) (dwork, * corresponding author 2006) to llms, providing a theoretical guarantee for protection by adding calibrated noises. however, its core challenge is the privacy-utility tradeoff: while dp offers strong privacy guarantees, model utility often suffers from overly conservative noise foot_0 . so, enhancing llms' utility under dp is a key topic. researchers explored different ways to implement dp, mainly in two categories. (1) model-free methods add calibrated noise to the input/output to anonymize text @xcite , which does not require model fine-tuning and can adapt to closed-source llms like gpt-4. however, since the noise must be added to each sample, the noise scale increases with the length and number of samples; and the increasing of noise reduces the model utility. (2) model-based methods feed noises into the model training @xcite . the advantage is that once the model training is accomplished, privacy loss is fixed (stopping adding additional noise). therefore, the strength of privacy protection and utility does not decrease as the users feed new samples to the model for inference. however, these methods require sufficient domain-specific supervised training samples, which are often hard to obtain @xcite in data-sensitive fields such as medicine. additionally, since dp fine-tuning adds noise to gradients, the privacy cost increases with more training data. ensuring reasonable privacy guarantees requires adding more noise, thus degrading model utilities. to further enhance model-based methods, researchers used llms' generative abilities to synthesize training samples under dp. these methods construct the dp-based generators by supervised fine-tuning or prompt-tuning llms to produce samples for downstream tasks. according to the dp post-processing property @xcite , once synthetic data are generated, no additional noise needs to be required to account for privacy loss. so, downstream tasks can use almost infinite samples from the generator without slashing privacy protection. there are two main categories of work using dp to synthesize text. (1) someone fine-tune llms under dp to synthetic data @xcite . this method inherits dp fine-tuning limitations, including requiring many supervised privacy samples. moreover, they require trainable llms, so it is unsuitable for closed-source llms like gpt-4 foot_1 . (2) methods in the second category use llms apis to synthesize text similar to privacy data @xcite . this method does not require large-scale domain-specific data and is adaptable to closed-source llms. compared with fine-tuned llms, closed-source llms are typically stronger in synthesizing text but are not so flexible in fitting the given privacy distributions due to the lack of direct fine-tuning. it is hard to balance the diversity of synthetic data and its relevance to target privacy data. low relevance leads to a gap between synthetic data and data required by downstream tasks (i.i.d. with the privacy set); low diversity makes the model hard to cover the whole privacy distribution and thus the downstream models may overfit partial privacy distribution. to synthesize high-quality samples, we should ensure the synthesized samples are both diverse and relevant to the privacy data. genetic algorithm can effectively evolve populations, which is suitable for our scenarios: our model may inherit the "gene" information from the privacy data and evolve to obtain new data maintaining the "gene" while keeping the diversity. in this paper, we propose differential privacy genetic algorithm for text synthesis method (dpga-textsyn) 3 to protect the privacy of sensitive data, which imports the idea of genetic evolution to llms to generate high-utility synthetic text under dp constraints. the core innovation is to synthesize samples according to "gene" of the privacy data and expand more samples via genetic evolution mechanism. specifically, to make synthetic samples consistent with the privacy data, we extract the "gene" (i.e. metadata) of privacy data under dp protection to obtain the initial synthetic samples ( §3.2). to achieve survival of the fittest and avoid homogeneous per generation, we select distinct samples to construct an elite set for each generation. here, we construct a dp histogram based on nearest neighbor voting to select elite samples and employ similarity suppression to avoid samples being homogeneous (more and more similar) as iterations ( §3.3). to evolve to obtain more samples, we design three genetic strategies including mutation, crossover, and generation, to expand the elite set, which ensures the quality and diversity in the next generation. ( §3.4). experiments show that dpga-textsyn excels baselines. our contributions are: (1) we propose dpga-textsyn, a method that implements genetic algorithms to synthesize data under dp, it uses llms apis to iteratively generate synthetic text data that is close to privacy data, addressing privacy-utility trade-offs in data-scarce scenarios. (2) we propose to extract privacy metadata "gene" to generate better initial synthetic data with a low privacy cost, which instantiates svt to obtain statistics and conduct privacy voting to obtain meta descriptions. (3) experiments show our model excels strong baselines in downstream tasks. dp synthetic text is generated by training models to produce new text similar to real data while protecting personal information @xcite . @xcite using dp-sgd to fine-tune pre-trained language models like gpt-2 for generating synthetic text datasets @xcite . mattern and kurakin et al. demonstrated that training downstream models on dp synthetic text can achieve performance comparable to dp training directly on real data @xcite . however, achieving a good trade-off between fidelity and privacy requires large batch sizes and long training iterations @xcite . the rise of closed-source llms like gpt-4 has made dp fine-tuning infeasible, driving research on apibased generation methods @xcite . a new approach uses api access to pre-trained models and applies dp to select samples similar to privacy data @xcite , generating variations of these samples @xcite . the aug-pe algorithm improves generation and selection via apis to produce high-quality dp synthetic text @xcite . some additional related work is added in app. a. ## privacy analysis we defer dp definitions and lemmas to app. b and only give high-level ideas of the analysis. ## conclusion in summary, we propose dpga-textsyn, a method for privacy text synthesis under dp based on genetic algorithm and llms. first, we integrate privacy metadata information into the initialization prompt to generate better initial synthetic samples. then, we construct a dp histogram based on nearest neighbor voting and a self-similarity matrix to select distinct elite samples. finally, we use three different genetic strategies including mutation, crossover, and generation to expand the elite set. experiments show that our method excels in text synthesis, outperforming all baseline methods.