Learning with User-Level Privacy

√ m ) as users provide more samples . In contrast , when increasing the number of users n , the privacy cost decreases at a faster O ( 1/n ) rate . We complement these results with lower bounds showing the minimax optimality of our algorithms for mean estimation and stochastic convex optimization . Our algorithms rely on novel techniques for private mean estimation in arbitrary dimension with error scaling as the concentration radius τ of the distribution rather than the entire range . 1 Introduction . Releasing seemingly innocuous functions of a data set can easily compromise the privacy of individuals , whether the functions are simple counts [ 35 ] or complex machine learning models like deep neural networks [ 52 , 30 ] . To protect against such leaks , Dwork et al . proposed the notion of differential privacy ( DP ) . Given some data from n participants in a study , we say that a statistic of the data is differentially private if an attacker who already knows the data of n− 1 participants can not reliably determine from the statistic whether the n-th remaining participant is Alice or Bob . With the recent explosion of publicly available data , progress in machine learning , and widespread public release of machine learning models and other statistical inferences , differential privacy has become an important standard and is widely adopted by both industry and government [ 32 , 5 , 21 , 55 ] . The standard setting of DP described in [ 22 ] assumes that each participant contributes a single data point to the dataset , and preserves privacy by “ noising ” the output in a way that is commensurate with the maximum contribution of a single example . This is not the situation faced in many applications of machine learning models , where users often contribute multiple samples to the model—for example , when language and image recognition models are trained on the users ’ own data , or in federated learning settings [ 37 ] . As a result , current techniques either provide privacy guarantees that degrade with a user ’ s increased participation or naively add a substantial amount of noise , relying on the group property of differential privacy , which significantly harms the performance of the deployed model . To remedy this issue , we consider user-level DP , which instead of guaranteeing privacy for individual samples , protects a user ’ s entire contribution ( m ≥ 1 samples ) . This is a more stringent but more realistic privacy desideratum . To hold , it requires that the output of our algorithm does not significantly ∗Equal contribution . Work was done during an internship at Google Research . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . change when changing user ’ s entire contribution—i.e . possibly swapping up to m samples in total . We make this formal in Definition 1 . Very recently , for the reasons outlined above , there has been increasing interest in user-level DP for applications such as estimating discrete distributions under user-level privacy constraints [ 46 ] , PAC learning with user-level privacy [ 31 ] , and bounding user contributions in ML models [ 4 , 26 ] . Differentially private SQL with bounded user contributions was proposed in [ 59 ] . User-level privacy has been also studied in the context of learning models via federated learning [ 49 , 48 , 58 , 6 ] . In this paper , we tackle the problem of learning with user-level privacy in the central model of DP . In particular , we provide algorithms and analyses for the tasks of mean estimation , empirical risk minimization ( ERM ) , stochastic convex optimization ( SCO ) , and learning hypothesis classes with finite metric entropy . Our utility analyses assume that all users draw their samples i.i.d . from related distributions , a setting we refer to as limited heterogeneity . On these tasks , naively applying standard mechanisms , such as Laplace or Gaussian , or using the group property with item-level DP estimators , both yield a privacy error independent of m. We first develop novel private mean estimators in high dimension with statistical and privacy error scaling with the ( arbitrary ) concentration radius rather than the range , and apply these to the statistical query setting [ SQ ; 41 ] . Our algorithms then rely on ( privately ) answering a sequence of adaptively chosen queries using users ’ samples , e.g. , gradient queries in stochastic gradient descent algorithms . We show that for these tasks , the additional error due to privacy constraints decreases as O ( 1/ √ m ) , contrasting with the naive rate—independent of m. Interestingly , increasing n , the number of users , decreases the privacy cost at a faster O ( 1/n ) rate . Importantly , our results imply concrete practical recommendations on sample collection , regardless of the level of heterogeneity . Indeed , increasing m will yield the most value in the i.i.d . setting and will yield no improvement when the users ’ distributions are arbitrary . As the real-world will lie somewhere in between , our results exhibit a regime where , for any heterogeneity , it is strictly better to collect more users ( increasing n ) than more samples per user ( increasing m ) . 1.1 Our Contributions and Related Work . We provide a theoretical tool to construct estimators for tasks with user-level privacy constraints and apply it to a range of learning problems . Optimal private mean estimation and uniformly concentrated queries ( Section 3 ) We show that for a random variable in [ −B , B ] concentrated in an unknown interval of radius τ ( made precise in Definition 2 ) , we can privately estimate its mean with error proportional to τ rather than B , as we would obtain using standard private mean estimation techniques such as Laplace mechanism [ 24 ] . When data is concentrated in ` ∞-norm , several papers show that one can achieve an error scaling with τ rather than B , either asymptotically [ 53 ] , for Gaussian mean-estimation [ 40 , 38 ] , for sub-Gaussian symmetric distributions [ 18 , 17 ] or for distributions with bounded p-th moment [ 39 ] . We propose a private mean estimator ( Algorithm 2 ) with error scaling with τ that works in arbitrary dimension when data is concentrated in ` 2-norm ( Theorem 2 ) . In Corollary 1 , we show it ( optimally ) solves mean estimation under user-level privacy constraints for random vectors bounded in ` 2-norm . In Appendix D.6 , we show that for uniformly concentrated queries ( see Definition 3 ) , sequentially applying Algorithm 2 privately answers K adaptively chosen queries with privacy cost Õ ( τ √ K/nε ) . Our conclusions relate to the growing literature in adaptive data analysis . While a sequence of work [ 25 , 9 , 27 , 28 ] use techniques from differential privacy and their answers are ( ε , δ ) -DP with ε = Θ ( 1 ) , our work guarantees privacy for arbitrary ε with the additional assumption of uniform concentration . Empirical risk minimization ( Section 4 ) An influential line of papers studies ERM under itemlevel privacy constraints [ 19 , 42 , 8 ] . Importantly , these papers assume arbitrary data , i.e. , not necessarily samples from users ’ distributions . The exact analog of ERM in the user-level setting is consequently less interesting as , for n data points { z1 , . . . , zn } , in the worst case , each user u ∈ [ n ] contributes m copies of zu and the problem reduces to the item-level setting . Instead , we consider the ( related ) problem of ERM when users contribute points sampled i.i.d . Assuming some regularity ( A3 and A4 ) , we develop and analyze algorithms for ERM under user-level DP constraints for convex , strongly-convex , and non-convex losses ( Theorem 3 ) . Optimal stochastic convex optimization ( Section 5 ) Under item-level DP ( or equivalently , userlevel DP with m = 1 ) , a sequence of work [ 19 , 8 , 10 , 11 , 29 ] establishes the constrained minimax risk as Θ̃ ( 1/ √ n + √ d/ ( nε ) ) . In this paper , with the additional assumptions that the losses are individually smooth2 and the gradients are sub-Gaussian random vectors , we prove matching upper ( Theorem 4 ) and lower bounds ( Theorem 5 ) of order Θ̃ ( 1/ √ nm + √ d/ ( n √ mε ) ) in a regime we make precise . We leave closing the gap outside of this regime to future work . Limit of learning with a fixed number of users ( Appendix B ) Finally , we resolve a conjecture of [ 4 ] and prove that with a fixed number of users , even in the limit m → ∞ ( i.e. , each user has an infinite number of samples ) , we can not reach zero error . In particular , we prove that for all the learning tasks we consider , the risk under user-level privacy constraints is at least Ω ( e−εn ) regardless of m. Note that this does not contradict the results above since they require n = Ω ( ( logm ) /ε ) . Finally , we provide results in Appendix A for learning under pure user-level DP for function classes with finite metric entropy . We apply these to SCO with ` ∞ constraints ( Remark 1 ) and achieve ( near ) -optimal rates . 2 Preliminaries . Notation . Throughout this work , d denotes the dimension , n the number of users , and m the number of samples per user . Generically , σ will denote the sub-Gaussian parameter , τ the concentration radius , ν the variance of a random vector and P a data distribution . We denote the optimization variable with θ ∈ Θ ⊂ Rd , use z ( or Z when random ) to denote the data sample supported on a space Z , and ` : Θ × Z → R for the loss function . Gradients ( denoted ∇ ) are always taken with respect to the optimization variable θ . For a convex set C , ΠC denotes the euclidean projection on C , i.e . ΠC ( y ) : = argminz∈C ‖y − z‖2 . We use A to refer to ( possibly random ) private mechanisms and Xn as a shorthand for the dataset ( X1 , . . . , Xn ) . For two distributions P and Q , we denote by ‖P −Q‖TV their total variation distance and Dkl ( P ||Q ) their Kullback-Leibler divergence . For a random vector X ∼ P supported on Rd , we use Var ( P ) or Var ( X ) to denote E [ ‖X − E [ X ] ‖22 ] , which is equal to the trace of the covariance matrix of X . Next , we consider differential privacy in the most general way , which only requires specifying a dataset space S and a distance d on S. Definition 1 ( Differential Privacy ) . Let ε , δ ≥ 0 . Let A : S → Θ be a ( potentially randomized ) mechanism . We say that A is ( ε , δ ) -DP with respect to d if for any measurable subset O ⊂ Θ and all S , S′ ∈ S satisfying d ( S , S′ ) ≤ 1 , P ( A ( S ) ∈ O ) ≤ eεP ( A ( S′ ) ∈ O ) + δ . ( 1 ) If δ = 0 , we refer to this guarantee as pure differential privacy . For a data space Z , choosing S = Zn and d ( S , S′ ) = dHam ( S , S′ ) = ∑n i=1 1 { zi 6= z′i } recovers the canonical setting considered in most of the literature—we refer to this as item-level differential privacy . When we wish to guarantee privacy for users rather than individual samples , we instead assume a structured dataset into which each of n users contributes m > 1 samples . This corresponds to S = ( Zm ) n such that for S ∈ S , we have S = ( S1 , . . . , Sn ) , where Su = { z ( u ) 1 , . . . , z ( u ) m } and duser ( S , S ′ ) : = n∑ u=1 1 { Su 6= S′u } , which means that , in this setting , two datasets are neighboring if at most one of the user ’ s contributions differ . We henceforth refer to this setting as user-level differential privacy . Distributional assumptions . In the case of user-level privacy with n users each providing m samples , we assume existence of a collection of distributions { Pu } u∈ [ n ] over Z . One then observes the following user-level dataset3 S = ( S1 , . . . , Sn ) where Su iid∼ Pu . ( 2 ) 2We note that the results only require Õ ( n3/2 ) -smooth losses . For large n—keeping all other problem parameters fixed—this is a very weak assumption . More precisely , when n > poly ( d , m , 1/ε ) , our algorithm on a smoothed version ˜̀of ` ( e.g. , using the Moreau envelope [ 33 ] ) yields optimal rates for non-smooth losses . Whether the smoothness assumption can be removed altogether is an open question . 3For simplicity , we assume that |Su| = m but our guarantees directly extend to the setting where users have different number of samples with m replaced by median ( m1 , . . . , mn ) using techniques from [ 46 ] . We leave eliciting the optimal rates in settings when mu is an arbitrary random variable to future work . In this paper , we consider the limited heterogeneity setting , i.e . when the users have related distributions . This setting is more reflective of practice , especially in light of growing interest towards federated learning applications [ 37 , 60 ] . Assumption A1 ( Limited heterogeneity setting ) . There exists a distribution P0 over Z such that all the user distributions are close to P0 in total variation distance , i.e . max u∈ [ n ] ‖Pu − P0‖TV ≤ ∆ , where ∆ ≥ 0 quantifies the level of heterogeneity . Note that ∆ = 0 corresponds to assumption A2 . Note that our TV-based definition is natural in this setting as it is closely related to the notion of discrepancy ( or dA distance ) which plays a key role in domain adaption scenarios [ 47 , 12 ] . Lower bound results have been given in terms of the discrepancy measure ( see [ 13 ] ) , which further justify the adoption of this definition in the presence of multiple distributions . In the case that ∆ = 0 , A1 reduces to the standard homogeneous setting . Many fundamental papers choose this setting when explicating minimax rates under constraints ( e.g . in distributed optimization and federated learning [ 61 ] or under communication constraints [ 63 , 15 ] ) . Assumption A2 ( Homogeneous setting ) . The distributions of individual users are equal , meaning there exists P0 such that for all u ∈ [ n ] , Pu = P0 . In this paper , we develop techniques and provide matching upper and lower bounds for solving learning tasks in the homogeneous setting . In Appendix C , we prove that our techniques naturally apply to the heterogeneous setting in a black-box fashion , and for all considered problems provide meaningful guarantees under Assumption A1 . Moreover , the algorithm achieves almost optimal rate whenever ∆ is ( polynomially ) small . See the detailed statement in Theorem 9 .

Summary: This paper provides algorithms for various learning tasks under the constraint of user-level differential privacy. Specifically, they provide solutions for problems like mean estimation, empirical risk minimisation, stochastic convex optimisation, and learning hypothesis classes with finite metric entropy. They also provide lower bounds showing optimality of their algorithms for mean estimation and stochastic convex optimisation for different settings.  For mean estimation, they consider 1D distributions (and extend to higher dimensions, too) that are bounded within an interval, but are concentrated in some smaller sub-interval. Using this, they show that for uniformly concentrated queries, they can adaptively answer $K$ queries with a small privacy cost depending polynomially in the concentration parameter ($\tau$) and $K$.  For empirical risk minimisation and stochastic convex optimisation, they assume that each user's data is drawn i.i.d. from different distribution, such that all the distributions are close in TV distance.  They also show that even when each user contributes infinite samples, there would still be some error.