Understanding Impacts of Differential Privacy: A Unified Framework with Two-Layer Neural Networks

Authors: Ruichen Xu, Kexin Chen
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: differential privacy, two-layer neural networks, disparate impact

Abstract:
With the growing demand for data and the increasing awareness of privacy, differentially private learning has been widely applied in various deep models. Experiments have observed several side effects of differentially private learning, including bad learning features (performance), disparate impact, and worse adversarial robustness that hurt the trustworthiness of the trained models. Recent works have expected pre-training to mitigate these side effects. It is valuable to theoretically understand the impact of differential privacy on the training process. However, existing theoretical research only explained parts of the phenomena and failed to extend to non-convex and non-smooth neural networks. To fill this gap, we propose a unified framework to explain all the above phenomena by studying the feature learning process of differentially private stochastic gradient descent in two-layer ReLU convolutional neural networks. By analyzing the test loss, we find both its upper and lower bound decrease with feature-to-noise ratios (FNRs). We then show that disparate impact comes from imbalanced FNRs among different classes and subpopulation groups. Additionally, we show that the suboptimal learned features and reduced adversarial robustness are caused by the randomness of privacy-preserving noise introduced into the learned features. Moreover, we demonstrate that pre-training cannot always improve the model performance, especially with increased feature differences in the pre-training and fine-tuning datasets. Numerical results on both synthetic and real-world datasets validate our theoretical analyses.

Decision: reject

Reviews and discussion:

Metareview: