Can Public Large Language Models Help Private Cross-device Federated Learning? (2024)

Abstract: We study (differentially) private federated learning (FL) of language models. The language models in cross-device FL are relatively small, which can be trained with meaningful formal user-level differential privacy (DP) guarantees when massive parallelism in training is enabled by the participation of a moderate size of users. Recently, public data has been used to improve privacy-utility trade-offs for both large and small language models. In this work, we provide a systematic study of using large-scale public data and LLMs to help differentially private training of on-device FL models, and further improve the privacy-utility tradeoff by techniques of distillation. Moreover, we propose a novel distribution matching algorithm with theoretical grounding to sample public data close to private data distribution, which significantly improves the sample efficiency of (pre-)training on public data. The proposed method is efficient and effective for training private models by taking advantage of public data, especially for customized on-device architectures that do not have ready-touse pre-trained models.

## introduction federated learning (fl) @xcite @xcite is designed to collaboratively train a global model on decentralized data across user clients while protecting data privacy. fl emerged as an effective privacy-preserving solution of training (language) models, as rich text data are generated by users, which may contain sensitive and personal information. after @xcite proposed to train on-device recurrent neural networks, fl has been widely used in various natural language processing applications and products, including next-word prediction @xcite , keyword spotting @xcite , and out-ofvocabulary word discovery @xcite . to further protect user privacy, differential privacy (dp) @xcite @xcite is introduced to provide formal privacy guarantees of models trained by federated learning. dp for deep learning explicitly adds random noise with bounded sensitivity to a training process (e.g., dp-sgd @xcite ), ensuring a quantifiable similarity in output model distributions when the training dataset changes. when combining dp with fl, a variant of dp-sgd called dp-fedavg @xcite ) is applied to guarantee user-level dp @xcite . current research primarily focuses on applying user-level dp to small on-device models with fewer than 10 million parameters @xcite @xcite . the model size is limited due to challenges such as significant dp noise required to preserve privacy @xcite and the communication costs in cross-device fl. recent advances in large language models (llms) @xcite @xcite @xcite have revolutionized natural language processing (nlp) and achieved unprecedented performance on various tasks such as text generation, machine translation, and sentiment analysis. however, their success comes at a cost of requiring massive amounts of computational resources, making them difficult to deploy on resource-constrained devices such as smartphones, tablets, or other edge devices. additionally, there are concerns regarding the user privacy in various aspects such as memorizing personal information in training, and exposing private query in inference. recent work explore incorporating public information to improve privacy-utility trade-off in applying dp for (large) lms @xcite . public data @xcite or other side information @xcite are also studied for (dp) fl. in non-dp fl settings, @xcite studies the effect of initializing from a pretrained model. however, it is an open question on how to leverage the power of pre-trained llms to facilitate private fl for on-device lms. in this work, we answer the question through systematic study aimed at enhancing private federated learning for on-device lms with public pretrained lms. specifically, our approach involves leveraging both public data and pre-trained llms to improve differentially private federated learning for on-device models by techniques of public pre-training and distillation. additionally, we propose a novel distribution matching algorithm, which is backed by theoretical analysis, to sample public data closely resembling the private data distribution, which significantly increases sample efficiency in public training. moreover, our extensive empirical results align with our theoretical predictions, further substantiating our approach. our work complements existing research by utilizing llms to improve public training through knowledge distillation for private cross-device federated learning, and achieve a strong privacy-utility trade-off with substantial improvements on sampling efficiency for public data. our method points to a novel direction of efficiently enhancing private fl with public pretraining data and llms. we summarize our contributions as follows: • we focus on improving private federated learning for language modeling tasks and explore ways to leverage public data and pre-trained llms for tokenizers, training protocols, and data (sub)sampling. ## differentially private federated learning for on-device lms in this section, we walk through the preliminaries of differentially private federated learning of language models following the cross-device federated learning literature @xcite @xcite . we also introduce the experimental setup used throughout this paper. @xcite introduce federated learning to collaboratively train lms for next-word prediction from decentralized user data on a large number of mobile devices without directly sharing the private data. a common training algorithm of federated learning is @xcite , where each client downloads the current model from the centralized server, computes an update by performing local computation on their dataset (e.g., running sgd) and sends the update back to the server. the server aggregates the updates across clients to update the global model and send the updated model back to local clients to achieve the goal of collaborative learning without directly accessing the training data on each user's mobile device. ## inspiration from llms the success of publicly pre-trained llms motivate us to have retrospective views on further improving private on-device lms. in this section, we explore inpiration from llms: the use of subword tokenizers and a large public corpus for pre-training. we apply them to on-device lms, and observe that both techniques bring significant performance improvement for private fl. ## distillation from public llm on one hand, the cost of public pre-training for on-device lms is still expensive on a large public corpus (around a week of gpu time). on the other hand, existing llms are well pre-trained and demonstrate promising performance across a variety of downstream tasks. this motivates us to explore on whether we can leverage existing llms to improve the sample efficiency of pre-training ondevice lms. in this section, we answer the question above with systematic studies and show that we can improve the sample efficiency by using only 1% of pre-training data and distillation from llms, achieving similar or even better performance than using 100% of pretrianing data without distillation. ## distribution matching in the previous section, we achieve compelling performance by employing llm distillation using only 1% of the randomly sampled pre-training corpus. now we further investigate the possibility of improving sample efficiency by selectively identifying public samples that align with the distribution of private samples. to this end, we propose a novel distribution matching method to sample public records for pre-training with a novel theoretical analysis jointly considering public-private distribution shift and dp mechanism. we demonstrate that by carefully selected 0.08% of public samples, we can pre-train on-device lms that perform as well as using 1% of public samples with distillation. this approach significantly improves sample efficiency, providing an additional knob of using public pre-training for private on-device models. ## conclusion in this work, we propose to improve private federated learning by using llms in public training. we leverage llms to aid public training of on-device lms via distribution matching to sample public data close to private data distribution, which further improves the effectiveness and efficiency of public training, demonstrating strong private learning accuracy while minimizing the need for large amounts of public training data. our work sheds light on a promising direction to improve private federated learning with public llms.