Privacy-preserving Learning via Deep Net Pruning
Authors: Yangsibo Huang, Xiaoxiao Li, Yushan Su, Sachin Ravi, Zhao Song, Sanjeev Arora, Kai Li
Keywords: Neural Network Pruning, Differential Privacy
Abstract: Neural network pruning has demonstrated its success in significantly improving the computational efficiency of deep models while only introducing a small reduction on final accuracy. In this paper, we explore an extra bonus of neural network pruning in terms of enhancing privacy. Specifically, we show a novel connection between magnitude-based pruning and adding differentially private noise to intermediate layers under the over-parameterized regime. To the best of our knowledge, this is the first work that bridges pruning with the theory of differential privacy. The paper also presents experimental results by running the model inversion attack on two benchmark datasets, which supports the theoretical finding.