Differentially Private Knowledge Distillation via Synthetic Text Generation (2024)

Abstract: Large Language models (LLMs) are achieving state-of-the-art performance in many different downstream tasks. However, the increasing urgency of data privacy puts pressure on practitioners to train LLMs with Differential Privacy (DP) on private data. Concurrently, the exponential growth in parameter size of LLMs necessitates model compression before deployment of LLMs on resource-constrained devices or latency-sensitive applications. Differential privacy and model compression generally must trade off utility loss to achieve their objectives. Moreover, simultaneously applying both schemes can compound the utility degradation. To this end, we propose DistilDP: a novel differentially private knowledge distillation algorithm that exploits synthetic data generated by a differentially private teacher LLM. The knowledge of a teacher LLM is transferred onto the student in two ways: one way from the synthetic data itself– the hard labels, and the other way by the output distribution of the teacher evaluated on the synthetic data– the soft labels. Furthermore, if the teacher and student share a similar architectural structure, we can further distill knowledge by aligning the hidden representations between both. Our experimental results demonstrate that DistilDP can substantially improve the utility over existing baselines, at least 9.0 PPL on the Big Patent dataset, with strong privacy parameters, \epsilon=2 . These promising results progress privacy-preserving compression of autoregressive LLMs. Our code can be accessed here: https://github.com/james-flemings/dp_compress.

## introduction large language models (llms) have accomplished amazing feats, from generating humanlike text to improving human productivity through ai-powered assistants. however, recent work has demonstrated that practical privacy attacks are possible due to memorization of training data from llms @xcite . differential privacy (dp), a rigorous mathematical framework, is widely used to mitigate this type of information leakage @xcite . to guarantee dp for machine learning models, dp is operationalized within the learning algorithm, known as dp-sgd @xcite . the standard training paradigm of llms is to first pre-train an llm on large publicly available data, then fine-tune it on a specific downstream task using private data. recent works in differentially private training have taken advantage of this paradigm by initializing an llm using pre-trained weights and then applying dp-sgd in the fine-tuning process, which can significantly boost its performance @xcite @xcite , albeit with some reduction in model utility. due to the huge computational and memory demands of llms, it becomes necessary to consider model compression in this training paradigm. for example, model compression is required when deploying llms on client devices where memory resources are scarce, or real-time systems such as sentence completion in text editors where inference latency is crucial @xcite . one popular technique for model compression that our work focuses on is task-specific knowledge distillation (kd) @xcite , which distills the knowledge of a larger teacher model onto a compressed student model. although knowledge distillation in llms has been quite successful @xcite @xcite , few works have explored compressing llms with dp @xcite . since achieving dp or kd for an llm already trade-offs utility loss, accomplishing both can result in substantial utility loss, which highlights the challenge of this problem. a naive solution to this problem would be to apply dp-sgd when training the teacher model, then apply dp-sgd again during the knowledge distillation between the teacher and student. since the teacher model is a function of the entire private corpus, applying dp-sgd during the fine-tuning of the teacher is seemingly unavoidable. despite the additive computational and memory costs due to an additional run of dp-sgd, this solution has also been shown to have worst utility compared to differentially private fine-tuning the student without knowledge distillation @xcite . the goal of our work is to enable efficient differentially private compression of llm deployment while minimizing model utility degradation. to this end, we introduce distildp: a simple, but effective, differentially private knowledge distillation framework, shown in figure 1 . the innovation of our approach is that it avoids running an additional dp-sgd during the knowledge distillation by utilizing dp synthetic data. our frameworks proceeds as follows: (1) first, dp-sgd is employed to finetune a teacher model on a training dataset, where the teacher accurately captures the distributional properties of the training data. (2) then we generate synthetic text closely resembling the original data distribution without incurring additional privacy loss. (3) lastly, the student model is trained using a non-private optimizer on the differentially private synthetic data while the teacher distills its knowledge to the student. in summary, our key contributions are the following: (1) we introduce distildp, a novel differentially private knowledge distillation algorithm for autoregressive llms using synthetic text generation. our framework utilizes two forms of knowledge transfer from the teacher: one from the synthetic text data, and the other from aligning the teacher output distribution with the student. (2) we comprehensively demonstrate that our algorithm significantly improves the utility of the student model over existing baselines with strong privacy parameters, ϵ = 2; in particular, the alignment step between the output distribution of the teacher and student is a crucial component to boosting the performance of the student. (3) we perform extensive experiments exploring different hyperparameters to provide insight into our approach. additionally, if the teacher and student share a similar architectural structure, we can align the hidden representation of the teacher and student to further boost the utility of our framework. dp-sgd @xcite is the most widely used technique to guarantee dp for llms. specifically, a pre-trained llm trained on large amounts of public text data scraped from the internet is fur-ther fine-tuned on a private corpus d using dp-sgd @xcite @xcite ). an orthogonal approach investigated differentially private decoding of llms @xcite @xcite , thereby avoiding the use of dp-sgd. @xcite first introduced dp model compression and gave a framework for dp knowledge distillation. however, their framework requires using dp-sgd twice: once for the teacher and once for the student. this procedure leads to suboptimal results for text classification tasks. wang et al. (2019) also explored private model compression, but is limited by the number of queries to the teacher for knowledge distillation. @xcite explores dp model compression during the pre-training stage; our work focuses solely on compression during fine-tuning. another line of work similar to our approach is data-free distillation @xcite where a generative model synthesizes data to substitute training data. since no private data is used, this results in no privacy loss, ϵ = 0. our approach uses a private corpus to generate synthetic data in a differentially private manner. another problem setup broadly related to dp knowledge distillation is private ensemble learning via pate @xcite ). an ensemble of teacher models is trained on disjoint private data, then produces noisy labels on unlabeled public data, which comes from a similar distribution as the private data, for the student to train on. lyu and chen (2020) used pate to achieve differentially private knowledge distillation for mobile analytics. our method does not require unlabeled public data. ## conclusion in this work, we introduced distildp, a novel dp knowledge distillation algorithm by utilizing dp synthetic data generation. we presented comprehensive experimental results that demonstrate dis-tildp obtains the best utility over all existing baselines for strict privacy settings. more generally, we showed that differentially private knowledge distillation for autoregressive large language models can be effective. the provided ablation study highlights that both the synthetic text as the hard label and the output distribution of the teacher as the soft label are crucial to boosting the performance of the student. we believe further improvements to our framework are feasible by generating more synthetic text data and including additional knowledge distillation from the hidden representations.