DP - NMT : Scalable Differentially Private Machine Translation (2024)

Abstract: Neural machine translation (NMT) is a widely popular text generation task, yet there is a considerable research gap in the development of privacy-preserving NMT models, despite significant data privacy concerns for NMT systems. Differentially private stochastic gradient descent (DP-SGD) is a popular method for training machine learning models with concrete privacy guarantees; however, the implementation specifics of training a model with DP-SGD are not always clarified in existing models, with differing software libraries used and code bases not always being public, leading to reproducibility issues. To tackle this, we introduce DP-NMT, an open-source framework for carrying out research on privacy-preserving NMT with DP-SGD, bringing together numerous models, datasets, and evaluation metrics in one systematic software package. Our goal is to provide a platform for researchers to advance the development of privacy-preserving NMT systems, keeping the specific details of the DP-SGD algorithm transparent and intuitive to implement. We run a set of experiments on datasets from both general and privacy-related domains to demonstrate our framework in use. We make our framework publicly available and welcome feedback from the community.

## introduction privacy-preserving natural language processing (nlp) has been a recently growing field, in large part due to an increasing amount of concern regarding data privacy. this is especially a concern in the context of modern neural networks memorizing training data that may contain sensitive information @xcite . while there has been a body of research investigating privacy for text classification tasks @xcite and language models @xcite , there has not been as much focus on text generation tasks, in 1 https://github.com/trusthlt/dp-nmt particular neural machine translation (nmt). however, nmt is particularly worrying from a privacy perspective, due to a variety of machine translation services available online that users send their personal data to. this includes built-in nmt services to existing websites, e-mail clients, and search engines. after data has been sent to these systems, it may be further processed and used in the development of the nmt system @xcite , which has a significant risk of being memorized if trained in a non-private manner. one of the most popular methods for tackling this privacy issue is differential privacy (dp), being a formal framework which provides probabilistic guarantees that the contribution of any single data point to some analysis is bounded. in the case of nlp and machine learning (ml), this means that a data point associated with some individual which is included in the model's training data cannot stand out 'too much' in the learning process of the model. the dp-sgd algorithm @xcite ) is one of the most standard methods to achieve this for ml systems, yet implementations of dp-sgd often lack some technical details on the specifics of the algorithm. in particular, this includes the privacy amplification method assumed for calculating the privacy budget ε when composed over all training iterations of the model. this means that the exact strength of the privacy protection that the resulting systems provide is not clear, with the 'standard' random shuffling method for iterating over batches providing a weaker privacy guarantee for the training data than poisson sampling. with different implementations using different software libraries, the community currently does not have a consistent platform for conducting experiments for scalable differentially private systems, such as nmt. to tackle this problem, we develop a modular framework for conducting research on private nmt in a transparent and reproducible manner. our pri-mary goal is to allow for a deeper investigation into the applications of dp for nmt, all while ensuring that important theoretical details of the dp-sgd methodology are properly reflected in the implementation. following previous work on dp-sgd @xcite , we implement our framework in the jax library @xcite , which provides powerful tools that help to reduce the significant computational overhead of dp-sgd, allowing for scalability in implementing larger systems and more extended training regimes. our primary contributions are as follows. first, we present dp-nmt, a framework developed in jax for leading research on nmt with dp-sgd. it includes a growing list of available nmt models, different evaluation schemes, as well as numerous datasets available out of the box, including standard datasets used for nmt research and more specific privacy-related domains. second, we demonstrate our framework by running experiments on these nmt datasets, providing one of the first investigations into privacy-preserving nmt. importantly, we compare the random shuffling and poisson sampling methods for iterating over training data when using dp-sgd. we demonstrate that, in addition to the theoretical privacy guarantee, there may indeed be differences in the model performance when utilizing each of the two settings. ## dp-sgd and subsampling we describe the main ideas of differential privacy (dp) and dp-sgd in appendix a. we refer to @xcite ; @xcite ; @xcite ; @xcite for a more comprehensive explanation. a key aspect of the dp-sgd algorithm (see alg. 1 in the appendix) is privacy amplification by subsampling, in which a stronger privacy guarantee can be obtained for a given dataset x when a subset of this dataset is first randomly sampled @xcite . if the sampling probability is q, then the overall privacy guarantee can be analyzed as being approximately qε. a key point here is the nature of this sampling procedure and the resulting privacy guarantee. the moments accountant of @xcite , which is an improvement on the strong composition theorem @xcite for composing multiple dp mechanisms, assumes poisson sampling. un-der this procedure, each data point is included in a mini-batch with probability @xmath0, with l being the lot size and n the size of the dataset. an alternative method to poisson sampling is uniform sampling, in which mini-batches of a fixed size are independently drawn at each training iteration @xcite . in practice, however, many modern implementations of dp-sgd utilize random shuffling, with the dataset split into fixed-size mini-batches. several training iterations thus form an epoch, in which each training data point appears exactly once, in contrast to poisson sampling for which the original notion of 'epoch' is not quite suitable, since each data point can appear in any training iteration and there is no "single passing of the training data through the model". in @xcite , the term epoch is redefined as n l lots, being essentially an expectation of the number of batches when utilizing n data points for training the model. while simply shuffling the dataset can indeed result in privacy amplification @xcite , the nature of the corresponding privacy guarantee is not the same as the guarantee achieved by poisson sampling, generally being weaker. we refer to @xcite 3) for further details. ## description of software the aim of our system is to offer a reliable and scalable approach to achieve differentially private machine translation. figure 1 illustrates the central structure of our system. the user can upload a translation dataset that is either accessible on the huggingface datasets hub foot_2 or is provided by us out of the box, and integrate it seamlessly for both training and efficient privacy accounting, utilizing huggingface's datasets library @xcite . accelerated dp-sgd with jax and flax our goal is to accelerate dp-sgd training through the use of a transformer model implemented with jax and flax @xcite . the speed of training dp-sgd in the framework can be considerably enhanced through vectorization, just-in-time (jit) compilation, and static graph optimization @xcite . jit compilation and automatic differentiation are defined and established on the xla compiler. jax's main transformation methods of interest for fast dp-sgd are grad, vmap, and pmap, offering the ability to mix these operations as needed @xcite . in the dp-sgd scenario, combining grad and vmap facilitates efficient computation of per-example gradients by vectorizing the gradient calculation along the batch dimension @xcite . additionally, our training step is decorated by pmap to leverage the xla compiler on multiple gpus, significantly accelerating training speed. the framework offers to conduct experiments with multiple encoder-decoder models preprocessor dataloader dataset for training selected model (e.g. mt5) nmt model model training with dp-sgd nmt model initialize model for experiment different preprocessing procedures depending on model type optimizer (e.g. adam) poisson sampling and type of dataloader privacy parameters ( ) other options (lot size, learning rate, sequence length, steps) dataset for evaluation preprocessor dataloader model inference outputs (model predictions, evaluations) private released model and integrate new seq2seq models, in addition to existing ones, such as mbart @xcite , t5 @xcite , and mt5 @xcite . when selecting a model, the corresponding preprocessor will prepare the dataset accordingly. this allows the software to be flexible and modular, enabling researchers to exchange models and datasets to perform a range of private nmt experiments. ## experiments to demonstrate our framework in use, fill the gaps on current knowledge of the privacy/utility tradeoff for the task of nmt, as well as examine the effects of using random shuffling vs. poisson sampling, we run a series of experiments with dp-sgd on several nmt datasets, using a variety of privacy budgets. ## conclusion we have introduced dp-nmt, a modular framework developed using the jax library, with the goal of leading research on neural machine translation with dp-sgd. to demonstrate our framework in use, we have presented several experiments on both general and privacy-related nmt datasets, comparing two separate approaches for iterating over training data with dp-sgd, and facilitating in filling the research gap on the privacy/utility trade-off in this task. we are continuing to actively expand the framework, including the integration of new models and nmt datasets. we hope that our framework will help to expand research into privacy-preserving nmt and welcome feedback from the community. the current knowledge on how to effectively apply differential privacy to data used in nmt systems. however, applications of differential privacy to textual data are still at an early research stage, and should not currently be used in actual services that handle real sensitive data of individuals. the primary reason for this is that our understanding of what is private information in textual data is still very limited. applications of differential privacy in the machine learning setting provide a privacy guarantee to each individual data point. in the context of dp-sgd, this means that if any single data point is removed from the dataset, the impact on the resulting model parameter update is bounded by the provided multiplicative guarantee in eqn. 1. in other words, it does not stand out 'too much' in its contribution to training the model. for textual data, a single data point will often be a sentence or document. however, this does not mean that there is a one-to-one mapping from individuals to sentences and documents. for instance, multiple documents could potentially refer to the same individual, or contain the same piece of sensitive information that would break the assumption of each data point being independent and identically distributed (i.i.d.) in the dp setting. thus, we require further research on how to properly apply a privacy guarantee to individuals represented within a textual dataset. we refer to @xcite ; @xcite ; @xcite for a more comprehensive discussion on this.