Title: Privacy Amplification by Iteration with Projected Alternating Direction Method
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: projected alternating direction method, differential privacy, privacy amplification, exact feasibility

Abstract: Alternating direction method of multipliers (ADMM) is a common approach for privacy amplification and utility guarantees in various machine learning tasks, especially those require cooperation between private and public users (or servers). However, this approach cannot achieve exact feasibility constraint throughout the learning process, and even has a large feasibility gap at the early iterative stage, which cannot handle the small-sampled situations. To solve these problems, we propose a projected alternating direction method that achieves exact feasibility and enables each user to supervise the objective value throughout the learning process. Moreover, it allows both Gaussian and Laplace noise for variable masking and privacy amplification. Third, it does not require the Markov operator condition or double-iterations to achieve one-step privacy and utility guarantees. Fourth, it achieves the same order of privacy-utility tradeoff as that of the existing ADMM methods. In summary, the proposed methodology requires fewer conditions but solves more general privacy amplification problems and enjoys more favorable properties than the existing ADMM methods.