Renyi Differential Privacy of The Subsampled Shuffle Model In Distributed Learning

1 Introduction . As learning moves towards the edge , there is a need to collaborate to build learning models1 , such as in federated learning [ 36 , 44 , 33 ] . In this framework , the collaboration is typically mediated by a server . In particular , we want to collaboratively build a learning model by solving an empirical risk minimization ( ERM ) problem ( see ( 2 ) in Section 2 ) . To obtain a model parametrized by ✓ using ERM , the commonly used mechanism is Stochastic Gradient Descent ( SGD ) [ 12 ] . However , one needs to solve this while enabling strong privacy guarantees on local data from the server , while also obtaining good learning performance , i.e. , a suitable privacy-learning performance operating point . Differential privacy ( DP ) [ 18 ] is the gold standard notion of data privacy that gives a rigorous framework through quantifying the information leakage about individual training data points from the observed interactions . Though DP was originally proposed in a framework where data resides centrally [ 18 ] , for distributed learning the more appropriate notion is of local differential privacy ( LDP ) [ 35 , 17 ] . Here , each client randomizes its interactions with the server from whom the data is to be kept private ( e.g. , see industrial implementations [ 23 , 31 , 16 ] ) . However , LDP mechanisms suffer 1This is because no client has access to enough data to build rich learning models locally and we do not want to directly share local data . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . from poor performance in comparison with the central DP mechanisms [ 17 , 35 , 32 ] . To overcome this , a new privacy framework using anonymization has been proposed in the so-called shuffled model [ 22 , 25 , 6 , 26 , 5 , 15 , 7 , 8 ] . In the shuffled model , each client sends her private message to a secure shuffler that randomly permutes all the received messages before forwarding them to the server . This model enables significantly better privacy-utility performance by amplifying DP through this shuffling . Therefore , in this paper we consider the shuffle privacy framework for distributed learning . In solving ( 2 ) using ( distributed ) gradient descent , each exchange leaks information about the local data , but we need as many steps as possible to obtain a good model ; setting up the tension between privacy and performance . The goal is to obtain as many such interactions as possible for a given privacy budget . This is quantified through analyzing the privacy of the composition of privacy mechanisms . Abadi et al . [ 1 ] developed a framework for tighter analysis of such compositions , and this was later reformulated in terms of Rényi Differential Privacy ( RDP ) [ 37 ] , and mapping this back to DP guarantee [ 38 ] . Therefore , studying RDP is important to obtaining strong composition privacy results , and is the focus of this paper . In distributed ( and federated ) learning , a fraction of the data samples are sampled ; for example , with random client participation and stochastic gradient descent ( SGD ) , which can be written as ✓t+1 ✓t ⌘t 1 |I| X i2I R ( rfi ( ✓t ) ) , where R is the local randomization mechanism and I are the indices of the sampled data . This is a subsampled mechanism that enables another privacy amplification opportunity ; which , in several cases , is shown to yield a privacy advantage proportional to the subsampling rate ; see [ 35 , 42 ] . The central technical question addressed in this paper is how to analyze the RDP of an arbitrary discrete mechanism for the subsampled shuffle privacy model . This enables us to answer the overall question posed in this paper , which is an achievable privacy-learning performance trade-off point for solving ( 2 ) in the shuffled privacy model for distributed learning ( see Figure 1 ) . Our contributions are : • We analyze the RDP of subsampled mechanisms in the shuffle framework by developing a novel bound applicable to any discrete ✏0-LDP mechanism as a function of the RDP order , subsampling rate , the LDP parameter ✏0 , and the number of clients n ; see Theorem 1 . The bound is explicit and amenable to numerics , including all constants.2 Furthermore , the bounds are valid for generic LDP mechanisms and all parameter regimes.3 We also provide a lower bound for the RDP in Theorem 2 . We prove our upper bound ( Theorem 1 ) using the following novel analysis techniques : First , we reduce the problem of computing the RDP of sub-sampled shuffle mechanisms to the problem of computing ternary | |↵-DP [ 43 ] of shuffle ( non sub-sampled ) mechanisms ; see Lemma 2 . Then we reduce the computation of the ternary | |↵-DP of shuffle mechanisms for a generic triple of neighboring datasets to those that have a special structure ( see Theorem 5 ) – this reduction step is one of the core technical results of this paper . Then we bound the ternary | |↵-DP of the shuffle mechanisms for triples of neighboring datasets having special structures by bounding the Pearson-Vajda divergence [ 43 ] using some concentration properties ( see Theorem 6 ) . • Using the core technical result in Theorem 1 , we analyze privacy-convergence trade-offs of the CLDP-SGD algorithm ( see Algorithm 1 ) for Lipschitz convex functions in Theorem 3 . This partially resolves an open question posed in [ 27 ] , to extend their privacy analysis to RDP and significantly strengthening their privacy guatantees . • Numerically , we save a factor 14⇥ in privacy ( ✏ ) over the best known results for approximate DP for shuffling [ 24 ] combined with strong composition [ 34 ] for T = 105 , = 0.001 , n = 106 , and a factor of 2.5⇥ better than the best known RDP for shuffling bound [ 29 ] combined with the sub-sampling result in [ 43 ] . Translating these to privacy-performance operating point in distributed 2As emphasized in [ 43 ] , “ in differential privacy , constants matter ” . 3Some of the best known approximate DP bounds for the shuffle model [ 7 , 24 ] are restricted to certain parameter regimes in terms of n , , ✏0 , etc . optimization , over the MNIST data set with ` 1 clipping we numerically show gains : For the same privacy budget of ✏ = 1.4 , we get a test performance of 80 % whereas using strong composition the test performance of [ 24 ] is 70 % ; furthermore , we achieves 90 % accuracy with the total privacy budget ✏ = 2.91 , whereas , [ 24 ] ( with strong composition ) achieves the same accuracy with a total privacy budget of ✏ = 4.82 . See Section 4 and the supplementary material for more results . Related work : We give a more complete literature review in Appendix A , and focus here on the works that are closest to the results presented in this paper . Private optimization in the shuffled model : Recently , [ 21 ] and [ 27 , 28 ] have proposed differentially private SGD algorithms for federated learning , where at each iteration , each client applies an LDP mechanism on the gradients with the existence of a secure shuffler between the clients and the central server . However , the privacy analyses in these works developed approximate DP using advanced composition theorems for DP ( e.g. , [ 20 , 34 ] ) , which are known to be loose for composition [ 1 ] . To the best of our knowledge , analyzing the private optimization framework using RDP and subsampling in the shuffled model is new to this paper . Subsampled RDP : The works [ 38 , 43 , 45 ] have studied the RDP of subsampled mechanisms without shuffling . They demonstrated that this provides a tighter bound on the total privacy loss than the bound that can be obtained using the standard strong composition theorems . The RDP analysis of subsampled mechanisms in the shuffled privacy framework has not been studied before,4 and is new to this paper . The RDP of the shuffled model was very recently studied in [ 29 ] , but without incorporating subsampling , which poses new technical challenges , as directly bounding the RDP of subsampled shuffle mechanisms is non-trivial . We overcome this by reducing our problem of computing RDP to bounding the ternary | |↵-DP , and bounding the latter is a core technical contribution of our paper . Paper organization : We give preliminaries and problem formulation in Section 2 , main results ( upper and lower bounds , and privacy-convergence tradeoff ) in Section 3 , numerical results in Section 4 , proof of the upper bound in Section 5 , and proof of the ternary DP of the shuffle model in Section 6 . Omitted details/proofs from this paper are given in the supplementary material . 2 Preliminaries and Problem Formulation . We use several privacy definitions throughout this paper . Among these , the local and central differential privacy definitions are standard and we defer them to Appendix B . The other privacy definitions ( Rényi DP and ternary | |↵-DP ) are relatively less standard and we define them below . We say that two datasets D = { d1 , . . . , dn } 2 Xn and D0 = { d01 , . . . , d0n } 2 Xn are neighboring ( and denoted by D ⇠ D0 ) if they differ in one data point , i.e. , there exists an i 2 [ n ] such that di 6= d0i and for every j 2 [ n ] , j 6= i , we have dj = d0j . Definition 1 ( ( , ✏ ) -RDP ( Rényi Differential Privacy ) [ 37 ] ) . A randomized mechanism M : Xn ! Y is said to have ✏-Rényi differential privacy of order 2 ( 1,1 ) ( in short , ( , ✏ ( ) ) -RDP ) , if for any neighboring datasets D , D0 2 Xn , the Rényi divergence of order between M ( D ) and M ( D0 ) is upper-bounded by ✏ ( ) , i.e. , D ( M ( D ) ||M ( D0 ) ) = 1 1 log E✓⇠M ( D0 ) `` ✓ M ( D ) ( ✓ ) M ( D0 ) ( ✓ ) ◆ # ! ✏ ( ) , ( 1 ) where M ( D ) ( ✓ ) denotes the probability that M on input D generates the output ✓ . Definition 2 ( ⇣-Ternary | |↵-differential privacy [ 43 ] ) . A randomized mechanism M : Xn ! Y is said to have ⇣-ternary-| |↵-DP , if for any triple of mutually adjacent datasets D , D0 , D00 2 Xn ( i.e. , they mutually differ in the same location ) , the ternary-| |↵ divergence of M ( D ) , M ( D0 ) , M ( D0 ) is upper-bounded by ( ⇣ ( ↵ ) ) ↵ for all ↵ 1 ( where ⇣ is a function from R+ to R+ ) , i.e. , D| |↵ ( M ( D ) , M ( D0 ) ||M ( D00 ) ) : = EM ( D00 ) M ( D ) M ( D0 ) M ( D00 ) ↵ ( ⇣ ( ↵ ) ) ↵ . 4One naive approach is to plug in the RDP analysis of shuffle model [ 29 ] into the results of [ 43 ] ; however , our direct analysis of subsampled mechanisms yields better results in several interesting regimes ; see Section 4 . The ternary | |↵-DP was proposed in [ 43 ] to characterize the RDP of the sub-sampled mechanism without shuffling . In this work , we analyze the ternary | |↵-DP of the shuffled mechanism to bound the RDP of the sub-sampled shuffle model . We can use the following result for converting the RDP guarantees of a mechanism to its central DP guarantees . To the best of our knowledge , this result gives the best conversion . Lemma 1 ( From RDP to DP [ 13 , 4 ] ) . Suppose for any > 1 , a mechanism M is ( , ✏ ( ) ) -RDP . Then , the mechanism M is ( ✏ , ) -DP , where > 0 is arbitrary and ✏ is given by ✏ = min ✓ ✏ ( ) + log ( 1/ ) + ( 1 ) log ( 1 1/ ) log ( ) 1 ◆ . Problem formulation : We consider a distributed private learning setup comprising a set of n clients , where the ith client has a data point di drawn from a universe X for i 2 [ n ] ; see also Figure 1 . Let D = ( d1 , . . . , dn ) denote the entire training dataset . The clients are connected to an untrusted server in order to solve the following empirical risk minimization ( ERM ) problem min ✓2C ⇣ F ( ✓ , D ) : = 1 n nX i=1 f ( ✓ , di ) ⌘ , ( 2 ) where C ⇢ Rd is a closed convex set , and f : C ⇥ D ! R is the loss function . Our goal is to construct a global learning model ✓ via stochastic gradient descent ( SGD ) while preserving privacy of individual data points in the training dataset D by providing strong DP guarantees . Algorithm 1 Acldp : CLDP-SGD Input : Datasets D = ( d1 , . . . , dn ) , LDP privacy parameter ✏0 , gradient norm bound C , and learning rate schedule { ⌘t } . 1 : Initialize : ✓0 2 C 2 : for t 2 [ T ] do 3 : Client sampling : A random set Ut of k clients is chosen . 4 : for clients i 2 Ut do 5 : Compute gradient : gt ( di ) r✓tf ( ✓t , di ) 6 : Clip gradient : g̃t ( di ) gt ( di ) /max n 1 , kgt ( di ) kpC o 7 : Client i sends Rp ( g̃t ( di ) ) to the shuffler . 8 : end for 9 : Shuffling : The shuffler sends random permutation of { Rp ( g̃t ( di ) ) : i 2 Ut } to the server . 10 : Aggregate : gt 1k P i2Ut Rp ( g̃t ( di ) ) 11 : Descent Step : ✓t+1 Q C ( ✓t ⌘tgt ) , where Q C is the projection operator onto the set C. 12 : end for Output : The model ✓T and the privacy parameters ✏ , . We revisit the CLDP-SGD algorithm presented in [ 27 ] and described in Algorithm 1 to solve the ERM ( 2 ) . In each step of CLDP-SGD , we choose uniformly at random a set Ut of k n clients out of n clients . Each client i 2 Ut computes and clips the ` p-norm of the gradient r✓tf ( ✓t , di ) to apply the LDP mechanism Rp , where Rp : Bdp ! { 0 , 1 } b is an ✏0-LDP mechanism when inputs come from an ` p-norm ball . In [ 27 ] , the authors proposed different ✏0-LDP mechanisms for general ` p-norm balls . After that , the shuffler randomly permutes the received k gradients { Rp ( g̃t ( di ) ) } i2Ut and sends them to the server . Finally , the server takes the average of the received gradients and updates the parameter vector . Our main contribution in this work is to present a stronger privacy analysis of the CLDP-SGD algorithm by characterizing the RDP of the sub-sampled shuffle model .

Summary: This paper follows the line of research aiming at tightening the privacy analysis of existing differentially private learning schemes. The authors consider the distributed learning framework, where a group of clients collaborate within a parameter-server architecture to learn a joint ML model under privacy (against the server). In this context, the paper analyzes the differential privacy guarantees obtained by  combining (i) subsampling of the clients (ii) local randomization at the client level and (iii) shuffling clients’ responses before sending them to the server. The authors provide new (upper and lower) bounds on the privacy guarantee of the above scheme for discrete local randomizers.  To do so, they rely on some well-known result from differential privacy (especially Renyi differential privacy), but also devise some interesting new proof schemes. The numerical comparison with existing works show that in some parameter regimes, the proposed bound considerably improves existing bounds studying the same framework.