We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes "large step" updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset. 1. We apply differential privacy to model training using the notion of user-adjacent datasets, leading to formal guarantees of user-level privacy, rather than privacy for single examples. Published as a conference paper at ICLR 2018 2. We introduce a noised version of the federated averaging algorithm (McMahan et al., 2016) in §2, which satisfies user-adjacent differential privacy via use of the moments accountant (Abadi et al., 2016a) first developed to analyze differentially private stochastic gradient descent (SGD) for example-level privacy. The federated averaging approach groups multiple SGD updates together, enabling large-step model updates. 3. We demonstrate the first high quality LSTM language model trained with strong privacy guarantees in §3, showing no significant decrease in model accuracy given a large enough dataset. For example, on a dataset of 763,430 users, baseline (non-private) training achieves an accuracy of 17.5% in 4120 rounds of training, where we use the data from 100 random users on each round. We achieve this same level of accuracy with (4.6, 10 −9 )-differential privacy in 4980 rounds, processing on average 5000 users per round-maintaining the same level of accuracy at a significant computational cost of roughly 60×. 1 Running the same computation on a larger dataset with 10 8 users would improve the privacy guarantee to (1.2, 10 −9 ). We guarantee privacy and maintain utility despite the complex internal structure of the LSTM-with per-word embeddings as well as dense state transitions-by using the federated averaging algorithm. We demonstrate that the noised model's metrics and qualitative behavior (with respect to head words) does not differ significantly from the non-private model. To our knowledge, our work represents the most sophisticated machine learning model, judged by the size and the complexity of the model, ever trained with privacy guarantees, and the first such model trained with user-level privacy.
In terms of defenses, REF study how to train LSTM models with differential privacy guarantees at a user-level.
Learning Differentially Private Recurrent Language Models
ICLR
Mathematics Computer Science