Abstract. Several recent studies in privacy-preserving learning have considered the trade-o between utility or risk and the level of dierential privacy guaranteed by mechanisms for statistical query processing. In this paper we study this trade-o in private Support Vector Machine (SVM) learning. We present two ecient mechanisms, one for the case of nite-dimensional feature mappings and one for potentially innite-dimensional feature mappings with translation-invariant kernels. For the case of translation-invariant kernels, the proposed mechanism minimizes regularized empirical risk in a random Reproducing Kernel Hilbert Space whose kernel uniformly approximates the desired kernel with high probability. This technique, borrowed from large-scale learning, allows the mechanism to respond with a nite encoding of the classier, even when the function class is of innite VC dimension. Dierential privacy is established using a proof technique from algorithmic stability. Utilitythe mechanism's response function is pointwise -close to non-private SVM with probability 1 − δis proven by appealing to the smoothness of regularized empirical risk minimization with respect to small perturbations to the feature mapping. We conclude with a lower bound on the optimal dierential privacy of the SVM. This negative result states that for any δ, no mechanism can be simultaneously ( , δ)-useful and β-dierentially private for small and small β.
REF used γ-uniform stability to calibrate the level of noise when using the Laplace mechanism to achieve differential privacy for the SVM.
Learning in a Large Function Space: Privacy-Preserving Mechanisms for SVM Learning
ArXiv
ArXiv
Mathematics Computer Science