Renyi Differential Privacy of The Subsampled Shuffle Model In Distributed Learning

1 Introduction . As learning moves towards the edge , there is a need to collaborate to build learning models1 , such as in federated learning [ 36 , 44 , 33 ] . In this framework , the collaboration is typically mediated by a server . In particular , we want to collaboratively build a learning model by solving an empirical risk minimization ( ERM ) problem ( see ( 2 ) in Section 2 ) . To obtain a model parametrized by ✓ using ERM , the commonly used mechanism is Stochastic Gradient Descent ( SGD ) [ 12 ] . However , one needs to solve this while enabling strong privacy guarantees on local data from the server , while also obtaining good learning performance , i.e. , a suitable privacy-learning performance operating point . Differential privacy ( DP ) [ 18 ] is the gold standard notion of data privacy that gives a rigorous framework through quantifying the information leakage about individual training data points from the observed interactions . Though DP was originally proposed in a framework where data resides centrally [ 18 ] , for distributed learning the more appropriate notion is of local differential privacy ( LDP ) [ 35 , 17 ] . Here , each client randomizes its interactions with the server from whom the data is to be kept private ( e.g. , see industrial implementations [ 23 , 31 , 16 ] ) . However , LDP mechanisms suffer 1This is because no client has access to enough data to build rich learning models locally and we do not want to directly share local data . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . from poor performance in comparison with the central DP mechanisms [ 17 , 35 , 32 ] . To overcome this , a new privacy framework using anonymization has been proposed in the so-called shuffled model [ 22 , 25 , 6 , 26 , 5 , 15 , 7 , 8 ] . In the shuffled model , each client sends her private message to a secure shuffler that randomly permutes all the received messages before forwarding them to the server . This model enables significantly better privacy-utility performance by amplifying DP through this shuffling . Therefore , in this paper we consider the shuffle privacy framework for distributed learning . In solving ( 2 ) using ( distributed ) gradient descent , each exchange leaks information about the local data , but we need as many steps as possible to obtain a good model ; setting up the tension between privacy and performance . The goal is to obtain as many such interactions as possible for a given privacy budget . This is quantified through analyzing the privacy of the composition of privacy mechanisms . Abadi et al . [ 1 ] developed a framework for tighter analysis of such compositions , and this was later reformulated in terms of Rényi Differential Privacy ( RDP ) [ 37 ] , and mapping this back to DP guarantee [ 38 ] . Therefore , studying RDP is important to obtaining strong composition privacy results , and is the focus of this paper . In distributed ( and federated ) learning , a fraction of the data samples are sampled ; for example , with random client participation and stochastic gradient descent ( SGD ) , which can be written as ✓t+1 ✓t ⌘t 1 |I| X i2I R ( rfi ( ✓t ) ) , where R is the local randomization mechanism and I are the indices of the sampled data . This is a subsampled mechanism that enables another privacy amplification opportunity ; which , in several cases , is shown to yield a privacy advantage proportional to the subsampling rate ; see [ 35 , 42 ] . The central technical question addressed in this paper is how to analyze the RDP of an arbitrary discrete mechanism for the subsampled shuffle privacy model . This enables us to answer the overall question posed in this paper , which is an achievable privacy-learning performance trade-off point for solving ( 2 ) in the shuffled privacy model for distributed learning ( see Figure 1 ) . Our contributions are : • We analyze the RDP of subsampled mechanisms in the shuffle framework by developing a novel bound applicable to any discrete ✏0-LDP mechanism as a function of the RDP order , subsampling rate , the LDP parameter ✏0 , and the number of clients n ; see Theorem 1 . The bound is explicit and amenable to numerics , including all constants.2 Furthermore , the bounds are valid for generic LDP mechanisms and all parameter regimes.3 We also provide a lower bound for the RDP in Theorem 2 . We prove our upper bound ( Theorem 1 ) using the following novel analysis techniques : First , we reduce the problem of computing the RDP of sub-sampled shuffle mechanisms to the problem of computing ternary | |↵-DP [ 43 ] of shuffle ( non sub-sampled ) mechanisms ; see Lemma 2 . Then we reduce the computation of the ternary | |↵-DP of shuffle mechanisms for a generic triple of neighboring datasets to those that have a special structure ( see Theorem 5 ) – this reduction step is one of the core technical results of this paper . Then we bound the ternary | |↵-DP of the shuffle mechanisms for triples of neighboring datasets having special structures by bounding the Pearson-Vajda divergence [ 43 ] using some concentration properties ( see Theorem 6 ) . • Using the core technical result in Theorem 1 , we analyze privacy-convergence trade-offs of the CLDP-SGD algorithm ( see Algorithm 1 ) for Lipschitz convex functions in Theorem 3 . This partially resolves an open question posed in [ 27 ] , to extend their privacy analysis to RDP and significantly strengthening their privacy guatantees . • Numerically , we save a factor 14⇥ in privacy ( ✏ ) over the best known results for approximate DP for shuffling [ 24 ] combined with strong composition [ 34 ] for T = 105 , = 0.001 , n = 106 , and a factor of 2.5⇥ better than the best known RDP for shuffling bound [ 29 ] combined with the sub-sampling result in [ 43 ] . Translating these to privacy-performance operating point in distributed 2As emphasized in [ 43 ] , “ in differential privacy , constants matter ” . 3Some of the best known approximate DP bounds for the shuffle model [ 7 , 24 ] are restricted to certain parameter regimes in terms of n , , ✏0 , etc . optimization , over the MNIST data set with ` 1 clipping we numerically show gains : For the same privacy budget of ✏ = 1.4 , we get a test performance of 80 % whereas using strong composition the test performance of [ 24 ] is 70 % ; furthermore , we achieves 90 % accuracy with the total privacy budget ✏ = 2.91 , whereas , [ 24 ] ( with strong composition ) achieves the same accuracy with a total privacy budget of ✏ = 4.82 . See Section 4 and the supplementary material for more results . Related work : We give a more complete literature review in Appendix A , and focus here on the works that are closest to the results presented in this paper . Private optimization in the shuffled model : Recently , [ 21 ] and [ 27 , 28 ] have proposed differentially private SGD algorithms for federated learning , where at each iteration , each client applies an LDP mechanism on the gradients with the existence of a secure shuffler between the clients and the central server . However , the privacy analyses in these works developed approximate DP using advanced composition theorems for DP ( e.g. , [ 20 , 34 ] ) , which are known to be loose for composition [ 1 ] . To the best of our knowledge , analyzing the private optimization framework using RDP and subsampling in the shuffled model is new to this paper . Subsampled RDP : The works [ 38 , 43 , 45 ] have studied the RDP of subsampled mechanisms without shuffling . They demonstrated that this provides a tighter bound on the total privacy loss than the bound that can be obtained using the standard strong composition theorems . The RDP analysis of subsampled mechanisms in the shuffled privacy framework has not been studied before,4 and is new to this paper . The RDP of the shuffled model was very recently studied in [ 29 ] , but without incorporating subsampling , which poses new technical challenges , as directly bounding the RDP of subsampled shuffle mechanisms is non-trivial . We overcome this by reducing our problem of computing RDP to bounding the ternary | |↵-DP , and bounding the latter is a core technical contribution of our paper . Paper organization : We give preliminaries and problem formulation in Section 2 , main results ( upper and lower bounds , and privacy-convergence tradeoff ) in Section 3 , numerical results in Section 4 , proof of the upper bound in Section 5 , and proof of the ternary DP of the shuffle model in Section 6 . Omitted details/proofs from this paper are given in the supplementary material . 2 Preliminaries and Problem Formulation . We use several privacy definitions throughout this paper . Among these , the local and central differential privacy definitions are standard and we defer them to Appendix B . The other privacy definitions ( Rényi DP and ternary | |↵-DP ) are relatively less standard and we define them below . We say that two datasets D = { d1 , . . . , dn } 2 Xn and D0 = { d01 , . . . , d0n } 2 Xn are neighboring ( and denoted by D ⇠ D0 ) if they differ in one data point , i.e. , there exists an i 2 [ n ] such that di 6= d0i and for every j 2 [ n ] , j 6= i , we have dj = d0j . Definition 1 ( ( , ✏ ) -RDP ( Rényi Differential Privacy ) [ 37 ] ) . A randomized mechanism M : Xn ! Y is said to have ✏-Rényi differential privacy of order 2 ( 1,1 ) ( in short , ( , ✏ ( ) ) -RDP ) , if for any neighboring datasets D , D0 2 Xn , the Rényi divergence of order between M ( D ) and M ( D0 ) is upper-bounded by ✏ ( ) , i.e. , D ( M ( D ) ||M ( D0 ) ) = 1 1 log E✓⇠M ( D0 ) `` ✓ M ( D ) ( ✓ ) M ( D0 ) ( ✓ ) ◆ # ! ✏ ( ) , ( 1 ) where M ( D ) ( ✓ ) denotes the probability that M on input D generates the output ✓ . Definition 2 ( ⇣-Ternary | |↵-differential privacy [ 43 ] ) . A randomized mechanism M : Xn ! Y is said to have ⇣-ternary-| |↵-DP , if for any triple of mutually adjacent datasets D , D0 , D00 2 Xn ( i.e. , they mutually differ in the same location ) , the ternary-| |↵ divergence of M ( D ) , M ( D0 ) , M ( D0 ) is upper-bounded by ( ⇣ ( ↵ ) ) ↵ for all ↵ 1 ( where ⇣ is a function from R+ to R+ ) , i.e. , D| |↵ ( M ( D ) , M ( D0 ) ||M ( D00 ) ) : = EM ( D00 ) M ( D ) M ( D0 ) M ( D00 ) ↵ ( ⇣ ( ↵ ) ) ↵ . 4One naive approach is to plug in the RDP analysis of shuffle model [ 29 ] into the results of [ 43 ] ; however , our direct analysis of subsampled mechanisms yields better results in several interesting regimes ; see Section 4 . The ternary | |↵-DP was proposed in [ 43 ] to characterize the RDP of the sub-sampled mechanism without shuffling . In this work , we analyze the ternary | |↵-DP of the shuffled mechanism to bound the RDP of the sub-sampled shuffle model . We can use the following result for converting the RDP guarantees of a mechanism to its central DP guarantees . To the best of our knowledge , this result gives the best conversion . Lemma 1 ( From RDP to DP [ 13 , 4 ] ) . Suppose for any > 1 , a mechanism M is ( , ✏ ( ) ) -RDP . Then , the mechanism M is ( ✏ , ) -DP , where > 0 is arbitrary and ✏ is given by ✏ = min ✓ ✏ ( ) + log ( 1/ ) + ( 1 ) log ( 1 1/ ) log ( ) 1 ◆ . Problem formulation : We consider a distributed private learning setup comprising a set of n clients , where the ith client has a data point di drawn from a universe X for i 2 [ n ] ; see also Figure 1 . Let D = ( d1 , . . . , dn ) denote the entire training dataset . The clients are connected to an untrusted server in order to solve the following empirical risk minimization ( ERM ) problem min ✓2C ⇣ F ( ✓ , D ) : = 1 n nX i=1 f ( ✓ , di ) ⌘ , ( 2 ) where C ⇢ Rd is a closed convex set , and f : C ⇥ D ! R is the loss function . Our goal is to construct a global learning model ✓ via stochastic gradient descent ( SGD ) while preserving privacy of individual data points in the training dataset D by providing strong DP guarantees . Algorithm 1 Acldp : CLDP-SGD Input : Datasets D = ( d1 , . . . , dn ) , LDP privacy parameter ✏0 , gradient norm bound C , and learning rate schedule { ⌘t } . 1 : Initialize : ✓0 2 C 2 : for t 2 [ T ] do 3 : Client sampling : A random set Ut of k clients is chosen . 4 : for clients i 2 Ut do 5 : Compute gradient : gt ( di ) r✓tf ( ✓t , di ) 6 : Clip gradient : g̃t ( di ) gt ( di ) /max n 1 , kgt ( di ) kpC o 7 : Client i sends Rp ( g̃t ( di ) ) to the shuffler . 8 : end for 9 : Shuffling : The shuffler sends random permutation of { Rp ( g̃t ( di ) ) : i 2 Ut } to the server . 10 : Aggregate : gt 1k P i2Ut Rp ( g̃t ( di ) ) 11 : Descent Step : ✓t+1 Q C ( ✓t ⌘tgt ) , where Q C is the projection operator onto the set C. 12 : end for Output : The model ✓T and the privacy parameters ✏ , . We revisit the CLDP-SGD algorithm presented in [ 27 ] and described in Algorithm 1 to solve the ERM ( 2 ) . In each step of CLDP-SGD , we choose uniformly at random a set Ut of k n clients out of n clients . Each client i 2 Ut computes and clips the ` p-norm of the gradient r✓tf ( ✓t , di ) to apply the LDP mechanism Rp , where Rp : Bdp ! { 0 , 1 } b is an ✏0-LDP mechanism when inputs come from an ` p-norm ball . In [ 27 ] , the authors proposed different ✏0-LDP mechanisms for general ` p-norm balls . After that , the shuffler randomly permutes the received k gradients { Rp ( g̃t ( di ) ) } i2Ut and sends them to the server . Finally , the server takes the average of the received gradients and updates the parameter vector . Our main contribution in this work is to present a stronger privacy analysis of the CLDP-SGD algorithm by characterizing the RDP of the sub-sampled shuffle model .

Summary: This paper studies the problem of bounding the privacy loss of the sub-sampled shuffle model, motivated by the real-world application of federated learning with differential privacy.  The broad type of mechanism studied in this paper is as follows:\ Setup: - Consider dataset $(d_1, \ldots, d_n)$ where client #$i$ has the datapoint $d_i \in \mathcal{X}$. - Consider any $\epsilon_0$-locally differentially private mechanism $\mathcal{R} : \mathcal{X} \to [B]$.  (One round of) the mechanism is as follows: - $k$ out of $n$ clients are chosen. Each chosen client $i$ send $\mathcal{R}(d_i)$ to the _shuffler_. - The shuffler randomly permutes the received responses from the $k$ chosen clients and sends it to a central server, which can post-process the same in an arbitrary manner.  The goal is to know, for any given $\delta$, what is the smallest $\epsilon$ for which the above mechanism is $(\epsilon, \delta)$-differentially private (DP) : this $\epsilon$ can depend on $\epsilon_0$, $k$, $n$ and $\delta$.  One approach for analyzing the privacy loss is to compose (in some way) the privacy loss guarantees of each of the two steps : sub-sampling and shuffling. One such way is to compose the Renyi differential privacy guarantees of the two steps (since Renyi-DP usually gives better bounds than the standard "advanced composition"). This paper instead takes a direct approach to bounding the Renyi-DP loss, and also demonstrates empirically that doing so, does better than performing any kind of composition of the two steps.  The paper uses the recently introduced technique of _ternary-$|\chi|^{\alpha}$-differential-privacy_ to prove the theorems.