Differentially Private Natural Language Models: Recent Advances and Future Directions (2024)

Abstract: Recent developments in deep learning have led to great success in various natural language processing (NLP) tasks. However, these applications may involve data that contain sensitive information. Therefore, how to achieve good performance while also protecting the privacy of sensitive data is a crucial challenge in NLP. To preserve privacy, Differential Privacy (DP), which can prevent reconstruction attacks and protect against potential side knowledge, is becoming a de facto technique for private data analysis. In recent years, NLP in DP models (DP-NLP) has been studied from different perspectives, which deserves a comprehensive review. In this paper, we provide the first systematic review of recent advances in DP deep learning models in NLP. In particular, we first discuss some differences and additional challenges of DP-NLP compared with the standard DP deep learning. Then, we investigate some existing work on DP-NLP andpresent its recent developments from three aspects: gradient perturbation based methods, embedding vector perturbation based methods, and ensemble model based methods. We also discuss some challenges and future directions.

## introduction the recent advances in deep neural networks have led to significant success in various tasks in natural language processing (nlp), such as sentiment analysis, question answering, information retrieval, and text generation. however, such applications always involve data that contains sensitive information. for example, a model of aid typing on a keyboard which trained from language data might contain sensitive information such as passwords, text messages, and search queries. moreover, language data can also identify a speaker explicitly by name or implicitly, for example, via a rare or unique phrase. thus, one often encountered challenge in nlp is how to handle this sensitive information. to overcome the challenge, privacy-preserving nlp has been intensively studied in recent years. one of the commonly used approaches is based on text anonymization @xcite , which identifies sensitive attributes and then replaces these sensitive words with some other values. another approach is injecting additional words into the original text without detecting sensitive entities in order to achieve text redaction @xcite . however, removing personally identifiable information or injecting additional words is often unsatisfactory, as it has been shown that an adversary can still infer an individual's membership in the dataset with high probability via the summary statistics on the datasets @xcite . moreover, recent studies claim that deep neural networks for nlp tasks often tend to memorize their training data, which makes them vulnerable to leaking information about training data @xcite @xcite . one way that takes into account the limitations of existing approaches by preventing individual re-identification and protecting against any potential data reconstruction and side-knowledge attacks is designing differentially private (dp) algorithms. dp @xcite provides provable protection against identification and is resilient to arbitrary auxiliary information that might be available to attackers. thanks to its formal guarantees, dp has become a de facto standard tool for private statistical data analysis. although there are numerous studies on dp machine learning and dp deep learning, such as @xcite @xcite @xcite , most of them mainly focus on either the continuous tabular data or image data, and less attention has been paid to adapting variants of dp algorithms to the context of nlp and the text domain. on the other side, while there are several surveys on dp and its applications, such as @xcite @xcite @xcite , none of them study its applications to the nlp domain. recently, @xcite gave a brief introduction to applications of dp in nlp, but the reviewed work is not exhaustive, and it lacks a technical and systematic view of dp-nlp. thus, to fill in this gap, in this paper, we provide the first technical overview of the recent developments and challenges of dp in language models. specifically, we give a survey on the most recent 70 foot_0 papers on deep learning based approaches for nlp tasks under dp constraints. first, we show some specificities of dp-nlp compared with the general deep learning with dp. then we discuss current results from three perspectives via the ways of adding randomness to ensure dp: the first one is gradient perturbation based methods which includes dp-sgd and dp-adam; the second one is embedding vector perturbation based methods which includes dp auto-encoder; the last one is ensemble model based methods which includes pate. for each type of approach, we also consider its applications to different nlp tasks. finally, we present some potential challenges and future directions. due to space limits, in appendix c, we give a preliminary introduction to dp to readers who are unfamiliar with dp. ## specificities of nlp with dp we first discuss some specificities for dp-nlp compared with the standard dp deep learning. generally speaking, there are two aspects: one is privacy notations, and another is privacy levels. ## gradient perturbation based methods generally speaking, a gradient perturbation method is based on adding noises to gradients of the loss during training the network to ensure dp. as the baseline and canonical algorithm for this type of approach, differentially private stochastic gradient descent (dp-sgd) @xcite is a dp version of sgd. its main idea is to use the noisy and clipped subsampled gradient g t to approximate the whole gradient ∇l(θ t , d). in fact, besides sgd, we can use this idea for any optimizer, such as adam (kingma and ba, 2015), whose private version dp-adam is proposed and applied in bert by @xcite . in the past few years, there has been a long list of work on dp-sgd from different perspectives, such as the subsampling strategy, faster clipping procedures, private clipping parameter tuning, and the selection of batch size. in the following, we will only discuss the previous work on using dp-sgd-based methods for variants of nlp tasks. see appendix a for an introduction to dp-sgd. ## embedding vector perturbation based methods generally speaking, this type of approach considers privatizing the embedding vector for each token. specifically, in this framework, the text data is first transformed into a vector (text representation) via some word embedding method such as word2vec @xcite and bert. then we use some dp mechanism to privatize each representation and train nlp models based on these privatized text representations. due to the postprocessing property of dp, we can see the main strength of this approach is any further training on these private embeddings also preserves the dp property, while gradient perturbation based methods heavily rely on the network structure. we can see that the main step of this method is to design the best private text representation. note that since we need to privatize each embedding representation separately, the whole algorithm could be considered as an ldp algorithm, and thus, it can also be used in the ldp setting. it is also notable that different studies may consider different notions and levels of privacy. in fact, most of the existing work considers the word level of privacy.