Learning with User-Level Privacy

√ m ) as users provide more samples . In contrast , when increasing the number of users n , the privacy cost decreases at a faster O ( 1/n ) rate . We complement these results with lower bounds showing the minimax optimality of our algorithms for mean estimation and stochastic convex optimization . Our algorithms rely on novel techniques for private mean estimation in arbitrary dimension with error scaling as the concentration radius τ of the distribution rather than the entire range . 1 Introduction . Releasing seemingly innocuous functions of a data set can easily compromise the privacy of individuals , whether the functions are simple counts [ 35 ] or complex machine learning models like deep neural networks [ 52 , 30 ] . To protect against such leaks , Dwork et al . proposed the notion of differential privacy ( DP ) . Given some data from n participants in a study , we say that a statistic of the data is differentially private if an attacker who already knows the data of n− 1 participants can not reliably determine from the statistic whether the n-th remaining participant is Alice or Bob . With the recent explosion of publicly available data , progress in machine learning , and widespread public release of machine learning models and other statistical inferences , differential privacy has become an important standard and is widely adopted by both industry and government [ 32 , 5 , 21 , 55 ] . The standard setting of DP described in [ 22 ] assumes that each participant contributes a single data point to the dataset , and preserves privacy by “ noising ” the output in a way that is commensurate with the maximum contribution of a single example . This is not the situation faced in many applications of machine learning models , where users often contribute multiple samples to the model—for example , when language and image recognition models are trained on the users ’ own data , or in federated learning settings [ 37 ] . As a result , current techniques either provide privacy guarantees that degrade with a user ’ s increased participation or naively add a substantial amount of noise , relying on the group property of differential privacy , which significantly harms the performance of the deployed model . To remedy this issue , we consider user-level DP , which instead of guaranteeing privacy for individual samples , protects a user ’ s entire contribution ( m ≥ 1 samples ) . This is a more stringent but more realistic privacy desideratum . To hold , it requires that the output of our algorithm does not significantly ∗Equal contribution . Work was done during an internship at Google Research . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . change when changing user ’ s entire contribution—i.e . possibly swapping up to m samples in total . We make this formal in Definition 1 . Very recently , for the reasons outlined above , there has been increasing interest in user-level DP for applications such as estimating discrete distributions under user-level privacy constraints [ 46 ] , PAC learning with user-level privacy [ 31 ] , and bounding user contributions in ML models [ 4 , 26 ] . Differentially private SQL with bounded user contributions was proposed in [ 59 ] . User-level privacy has been also studied in the context of learning models via federated learning [ 49 , 48 , 58 , 6 ] . In this paper , we tackle the problem of learning with user-level privacy in the central model of DP . In particular , we provide algorithms and analyses for the tasks of mean estimation , empirical risk minimization ( ERM ) , stochastic convex optimization ( SCO ) , and learning hypothesis classes with finite metric entropy . Our utility analyses assume that all users draw their samples i.i.d . from related distributions , a setting we refer to as limited heterogeneity . On these tasks , naively applying standard mechanisms , such as Laplace or Gaussian , or using the group property with item-level DP estimators , both yield a privacy error independent of m. We first develop novel private mean estimators in high dimension with statistical and privacy error scaling with the ( arbitrary ) concentration radius rather than the range , and apply these to the statistical query setting [ SQ ; 41 ] . Our algorithms then rely on ( privately ) answering a sequence of adaptively chosen queries using users ’ samples , e.g. , gradient queries in stochastic gradient descent algorithms . We show that for these tasks , the additional error due to privacy constraints decreases as O ( 1/ √ m ) , contrasting with the naive rate—independent of m. Interestingly , increasing n , the number of users , decreases the privacy cost at a faster O ( 1/n ) rate . Importantly , our results imply concrete practical recommendations on sample collection , regardless of the level of heterogeneity . Indeed , increasing m will yield the most value in the i.i.d . setting and will yield no improvement when the users ’ distributions are arbitrary . As the real-world will lie somewhere in between , our results exhibit a regime where , for any heterogeneity , it is strictly better to collect more users ( increasing n ) than more samples per user ( increasing m ) . 1.1 Our Contributions and Related Work . We provide a theoretical tool to construct estimators for tasks with user-level privacy constraints and apply it to a range of learning problems . Optimal private mean estimation and uniformly concentrated queries ( Section 3 ) We show that for a random variable in [ −B , B ] concentrated in an unknown interval of radius τ ( made precise in Definition 2 ) , we can privately estimate its mean with error proportional to τ rather than B , as we would obtain using standard private mean estimation techniques such as Laplace mechanism [ 24 ] . When data is concentrated in ` ∞-norm , several papers show that one can achieve an error scaling with τ rather than B , either asymptotically [ 53 ] , for Gaussian mean-estimation [ 40 , 38 ] , for sub-Gaussian symmetric distributions [ 18 , 17 ] or for distributions with bounded p-th moment [ 39 ] . We propose a private mean estimator ( Algorithm 2 ) with error scaling with τ that works in arbitrary dimension when data is concentrated in ` 2-norm ( Theorem 2 ) . In Corollary 1 , we show it ( optimally ) solves mean estimation under user-level privacy constraints for random vectors bounded in ` 2-norm . In Appendix D.6 , we show that for uniformly concentrated queries ( see Definition 3 ) , sequentially applying Algorithm 2 privately answers K adaptively chosen queries with privacy cost Õ ( τ √ K/nε ) . Our conclusions relate to the growing literature in adaptive data analysis . While a sequence of work [ 25 , 9 , 27 , 28 ] use techniques from differential privacy and their answers are ( ε , δ ) -DP with ε = Θ ( 1 ) , our work guarantees privacy for arbitrary ε with the additional assumption of uniform concentration . Empirical risk minimization ( Section 4 ) An influential line of papers studies ERM under itemlevel privacy constraints [ 19 , 42 , 8 ] . Importantly , these papers assume arbitrary data , i.e. , not necessarily samples from users ’ distributions . The exact analog of ERM in the user-level setting is consequently less interesting as , for n data points { z1 , . . . , zn } , in the worst case , each user u ∈ [ n ] contributes m copies of zu and the problem reduces to the item-level setting . Instead , we consider the ( related ) problem of ERM when users contribute points sampled i.i.d . Assuming some regularity ( A3 and A4 ) , we develop and analyze algorithms for ERM under user-level DP constraints for convex , strongly-convex , and non-convex losses ( Theorem 3 ) . Optimal stochastic convex optimization ( Section 5 ) Under item-level DP ( or equivalently , userlevel DP with m = 1 ) , a sequence of work [ 19 , 8 , 10 , 11 , 29 ] establishes the constrained minimax risk as Θ̃ ( 1/ √ n + √ d/ ( nε ) ) . In this paper , with the additional assumptions that the losses are individually smooth2 and the gradients are sub-Gaussian random vectors , we prove matching upper ( Theorem 4 ) and lower bounds ( Theorem 5 ) of order Θ̃ ( 1/ √ nm + √ d/ ( n √ mε ) ) in a regime we make precise . We leave closing the gap outside of this regime to future work . Limit of learning with a fixed number of users ( Appendix B ) Finally , we resolve a conjecture of [ 4 ] and prove that with a fixed number of users , even in the limit m → ∞ ( i.e. , each user has an infinite number of samples ) , we can not reach zero error . In particular , we prove that for all the learning tasks we consider , the risk under user-level privacy constraints is at least Ω ( e−εn ) regardless of m. Note that this does not contradict the results above since they require n = Ω ( ( logm ) /ε ) . Finally , we provide results in Appendix A for learning under pure user-level DP for function classes with finite metric entropy . We apply these to SCO with ` ∞ constraints ( Remark 1 ) and achieve ( near ) -optimal rates . 2 Preliminaries . Notation . Throughout this work , d denotes the dimension , n the number of users , and m the number of samples per user . Generically , σ will denote the sub-Gaussian parameter , τ the concentration radius , ν the variance of a random vector and P a data distribution . We denote the optimization variable with θ ∈ Θ ⊂ Rd , use z ( or Z when random ) to denote the data sample supported on a space Z , and ` : Θ × Z → R for the loss function . Gradients ( denoted ∇ ) are always taken with respect to the optimization variable θ . For a convex set C , ΠC denotes the euclidean projection on C , i.e . ΠC ( y ) : = argminz∈C ‖y − z‖2 . We use A to refer to ( possibly random ) private mechanisms and Xn as a shorthand for the dataset ( X1 , . . . , Xn ) . For two distributions P and Q , we denote by ‖P −Q‖TV their total variation distance and Dkl ( P ||Q ) their Kullback-Leibler divergence . For a random vector X ∼ P supported on Rd , we use Var ( P ) or Var ( X ) to denote E [ ‖X − E [ X ] ‖22 ] , which is equal to the trace of the covariance matrix of X . Next , we consider differential privacy in the most general way , which only requires specifying a dataset space S and a distance d on S. Definition 1 ( Differential Privacy ) . Let ε , δ ≥ 0 . Let A : S → Θ be a ( potentially randomized ) mechanism . We say that A is ( ε , δ ) -DP with respect to d if for any measurable subset O ⊂ Θ and all S , S′ ∈ S satisfying d ( S , S′ ) ≤ 1 , P ( A ( S ) ∈ O ) ≤ eεP ( A ( S′ ) ∈ O ) + δ . ( 1 ) If δ = 0 , we refer to this guarantee as pure differential privacy . For a data space Z , choosing S = Zn and d ( S , S′ ) = dHam ( S , S′ ) = ∑n i=1 1 { zi 6= z′i } recovers the canonical setting considered in most of the literature—we refer to this as item-level differential privacy . When we wish to guarantee privacy for users rather than individual samples , we instead assume a structured dataset into which each of n users contributes m > 1 samples . This corresponds to S = ( Zm ) n such that for S ∈ S , we have S = ( S1 , . . . , Sn ) , where Su = { z ( u ) 1 , . . . , z ( u ) m } and duser ( S , S ′ ) : = n∑ u=1 1 { Su 6= S′u } , which means that , in this setting , two datasets are neighboring if at most one of the user ’ s contributions differ . We henceforth refer to this setting as user-level differential privacy . Distributional assumptions . In the case of user-level privacy with n users each providing m samples , we assume existence of a collection of distributions { Pu } u∈ [ n ] over Z . One then observes the following user-level dataset3 S = ( S1 , . . . , Sn ) where Su iid∼ Pu . ( 2 ) 2We note that the results only require Õ ( n3/2 ) -smooth losses . For large n—keeping all other problem parameters fixed—this is a very weak assumption . More precisely , when n > poly ( d , m , 1/ε ) , our algorithm on a smoothed version ˜̀of ` ( e.g. , using the Moreau envelope [ 33 ] ) yields optimal rates for non-smooth losses . Whether the smoothness assumption can be removed altogether is an open question . 3For simplicity , we assume that |Su| = m but our guarantees directly extend to the setting where users have different number of samples with m replaced by median ( m1 , . . . , mn ) using techniques from [ 46 ] . We leave eliciting the optimal rates in settings when mu is an arbitrary random variable to future work . In this paper , we consider the limited heterogeneity setting , i.e . when the users have related distributions . This setting is more reflective of practice , especially in light of growing interest towards federated learning applications [ 37 , 60 ] . Assumption A1 ( Limited heterogeneity setting ) . There exists a distribution P0 over Z such that all the user distributions are close to P0 in total variation distance , i.e . max u∈ [ n ] ‖Pu − P0‖TV ≤ ∆ , where ∆ ≥ 0 quantifies the level of heterogeneity . Note that ∆ = 0 corresponds to assumption A2 . Note that our TV-based definition is natural in this setting as it is closely related to the notion of discrepancy ( or dA distance ) which plays a key role in domain adaption scenarios [ 47 , 12 ] . Lower bound results have been given in terms of the discrepancy measure ( see [ 13 ] ) , which further justify the adoption of this definition in the presence of multiple distributions . In the case that ∆ = 0 , A1 reduces to the standard homogeneous setting . Many fundamental papers choose this setting when explicating minimax rates under constraints ( e.g . in distributed optimization and federated learning [ 61 ] or under communication constraints [ 63 , 15 ] ) . Assumption A2 ( Homogeneous setting ) . The distributions of individual users are equal , meaning there exists P0 such that for all u ∈ [ n ] , Pu = P0 . In this paper , we develop techniques and provide matching upper and lower bounds for solving learning tasks in the homogeneous setting . In Appendix C , we prove that our techniques naturally apply to the heterogeneous setting in a black-box fashion , and for all considered problems provide meaningful guarantees under Assumption A1 . Moreover , the algorithm achieves almost optimal rate whenever ∆ is ( polynomially ) small . See the detailed statement in Theorem 9 .

Summary: The paper studies learning problems under so-called user-level differential privacy: in this setting, each one of n users holds m data points of the training set. This type of privacy then requires that under a change of *any number of data points belonging to a single user*, the distributions over the outputs of the algorithm are close in the sense of (epsilon, delta)-DP. A trivial way this could be achieved is group-privacy, where the privacy parameters would degrade linearly with the number of samples m.   The paper first gives a private mean estimation algorithm for the 1-dimensional case, which, given an input dataset that is both bounded in [-B,B] and (tau, gamma)-concentrated (that is, with probability gamma all points are within tau of a center point), returns an estimate that is close to the empirical mean plus a Laplace noise term of order tau/epsilon*n. This is then extended to the d-dimensional setting, using a random rotation of the data to ensure that each dimension is bounded appropriately in ell-2 norm rather than the ell-infty norm, and using the 1-dimensional solution on each dimension. Both these algorithms are *not* user-level DP but are used in the rest of the paper as building blocks of user-level DP algorithms.  The first application of these is a user-level DP d-dimensional mean estimation algorithm which, given samples from a distribution P supported on the euclidean ball with radius B, has squared ell-2 error in the order of sqrt{Var(P)/(m*n)}+\sqrt{d}*B/(\sqrt{m}*n*epsilon). The error due to privacy in this result degrades with 1/\sqrt{m} rather than being independent of m which is what the trivial approach would give us.  Using these tools, the authors then present user-level DP algorithms for empirical risk minimization with smooth convex, strongly convex, and non-convex losses as well as for stochastic convex optimization (SCO) with subgaussian gradients, demonstrating this improvement of 1/sqrt{m} in the error bounds.  They finally prove lower bounds that show that both the latter result on SCO (although not in all generality) as well as the d-dimensional user-level DP mean estimation bound are tight.