Bit-aware Randomized Response for Local Differential Privacy in Federated Learning

1 INTRODUCTION . Recent data privacy and security regulations ( GDPR , 2018 ; Regulation , 2018 ; Cybersecurity Law , 2016 ) pose major challenges in collecting and using personally sensitive data in different places for machine learning ( ML ) applications . Federated Learning ( FL ) ( McMahan et al. , 2017 ; Kairouz et al. , 2019 ) is a promising way to address these challenges , enabling clients to jointly train ML models by sharing and aggregating gradients computed from clients ’ local data through a coordinating server for model updates . However , recent attacks ( Zhu et al. , 2019 ; Y. et al. , 2021 ; Zhao et al. , 2020a ) have shown that clients ’ training samples , each of which includes an input x and a ground-truth label yx , can be extracted from the shared gradients . These attacks underscore the implicit privacy risk in FL . Our main goal is to provide a strong guarantee that the shared gradients protect the privacy of clients ’ local data without undue sacrifice in model utility . Local differential privacy ( LDP ) has emerged as a crucial component in various FL applications ( Yang et al. , 2019 ; Kairouz et al. , 2019 ) . To achieve our goal , we focus on preserving LDP in cross-device FL , i.e. , in which clients jointly train an FL model ( Kairouz et al. , 2019 ) . In cross-device FL , existing LDP-preserving approaches can be categorized into three lines : Clients ( 1 ) add noise into local gradients derived from their local training samples , e.g. , using Gaussian mechanism ( Abadi et al. , 2016 ) , to protect membership information at the training sample level with DP guarantees ( Zheng et al. , 2021 ; Dong et al. , 2019 ; Malekzadeh et al. , 2021 ; Geyer et al. , 2017 ; Huang et al. , 2020 ) , ( 2 ) add noise to local gradients using Randomized Response ( RR ) mechanisms to protect the values of the local gradients with LDP guarantees ( Sun et al. , 2021 ; Liu et al. , 2020 ; Zhao et al. , 2020b ; Wang et al. , 2019a ) , and ( 3 ) add noise into each training sample , e.g. , embedded features and labels , using RR mechanisms to protect the value of each training sample with LDP guarantees ( Arachchige et al. , 2019 ; Lyu et al. , 2020a ) , then the clients use LDP-preserved training samples to derive local gradients . For all three approaches , in each training round , clients send DP or LDP-preserved local gradients to the coordinating server for model updates , which will be sent back to the clients for the next training round . In this paper , we focus on protecting clients ’ training data at the value level with LDP guarantees . Existing RR mechanisms to preserve LDP in FL suffer from the curse of privacy composition , in which excessive privacy budgets are consumed proportionally to the large dimensions of input or embedded features ( Arachchige et al. , 2019 ) , gradients ( Zhao et al. , 2020b ; Wang et al. , 2019a ) , and training rounds ( Zhao et al. , 2020b ; Wang et al. , 2019a ) , causing loose privacy protection or inferior model accuracy ( Wagh et al. , 2021 ) . Addressing the curse of privacy composition is non-trivial . Existing approaches , such as anonymizers ( assumed to be trusted ) , i.e. , shuffler ( Erlingsson et al. , 2019 ; Sun et al. , 2021 ; L. et al. , 2020 ; Wang et al. , 2019b ; Cheu et al. , 2019 ; Balle et al. , 2019 ) or anonymity approaches ( e.g. , faking source IP , VPN , Proxy , etc . ( Sun et al. , 2021 ; Cormode et al. , 2018 ) ) , and dimension reduction ( Liu et al. , 2020 ; Zhao et al. , 2020b ; Shin et al. , 2018 ; Xu et al. , 2019 ) , mitigate the problem but also have limitations . In the real world , it is possible that the anonymizers can either be compromised or collude with the coordinating server to extract sensitive information from observing LDP-preserved local gradients ( Erlingsson et al. , 2019 ) . Meanwhile , applying RR mechanisms on reduced sets of embedded features or gradients using dimension reduction techniques can work well with lightweight models , such as logistic regression and SVM ( Liu et al. , 2020 ; Zhao et al. , 2020b ; Wang et al. , 2019a ) . However , it is challenging for these techniques to achieve good model utility under tight LDP guarantees given complex models and tasks , such as DNNs , since the dimensions of reduced embedded features , gradients , and training rounds still need to be sufficiently large ( Zhao et al. , 2020b ; Liu et al. , 2020 ) . Hence , the curse of privacy composition in preserving LDP by applying RR mechanisms in FL remains a largely open problem . Orthogonal to this , preserving LDP to protect ground-truth labels yx in FL has not been well-studied . Two known approaches for centralized training are 1 ) injecting Laplace noise into the labels ( Phan et al. , 2020 ) and 2 ) applying RR mechanisms on the labels to achieve DP at the label level ( Ghazi et al. , 2021 ) . However , centralized training in Ghazi et al . ( 2021 ) has not been designed for FL with LDP guarantees since they require centralized and trusted databases . The model utility in Phan et al . ( 2020 ) is notably affected by the number of model outcomes . Key Contributions . To mitigate the curse of privacy composition and optimize the trade-off between privacy and model utility , our paper is structured around the following contributions : 1 ) We propose BitRand , which is a combination of a novel bit-aware f -RR mechanism and label-RR mechanism , to preserve LDP at both levels of embedded features and labels in FL . In f -RR , we encode embedded features ( extracted from x ) into a binary-bit string , which will be adaptively randomized such that bits with a more substantial impact on model utility will have smaller randomization probabilities and vice-versa under the same privacy budget . To preserve LDP on yx , we develop a generalized randomization , in which the probability of randomizing label yx from one to another class is a function of the number of model outcomes C. By doing that , we can optimize the trade-off between model utility and privacy loss with significantly tighter expected error bounds . 2 ) By incorporating sensitivities of binary encoding bits into a generalized privacy loss bound , we show that increasing the dimensions of embedded features r , encoding bits l , and model outcomes C marginally affect the randomization probabilities in BitRand under the same privacy budget . This dimension-elastic property is crucial to evade the curse of privacy composition by retaining a high value of data transmitted correctly through our randomization given large dimensions of r , l , and C. 3 ) These bit-aware and dimension-elastic properties allow us to work with complex models and tasks with formal LDP guarantees for training samples ( x , yx ) while retaining high model utility . Extensive theoretical analysis and experimental results conducted on fundamental FL tasks , i.e. , text and image classification , using benchmark datasets and our collected Security and Exchange Commission financial contract dataset show that our BitRand significantly outperforms a variety of baseline approaches in terms of model utility under the same privacy budget . 2 BACKGROUND . LDP-preserving mechanisms ( Erlingsson et al. , 2014 ; Duchi et al. , 2018 ; Wang et al. , 2017 ; Acharya et al. , 2019 ; Bassily & Smith , 2015 ) generally build on the ideas of randomized response ( Warner , 1965 ) , which was initially introduced to allow survey respondents to provide their correct inputs while maintaining their confidentiality . -LDP is presented as follows : Definition 1 . -LDP . A randomized algorithmM fulfills -LDP , if for any two inputs x and x′ , and for all possible outputs O ∈ Range ( M ) , we have : Pr [ M ( x ) = O ] ≤ e Pr [ M ( x′ ) = O ] , where is a privacy budget and Range ( M ) denotes every possible output of the algorithmM . The privacy budget controls the amount by which the distributions induced by inputs x and x′ may differ . A smaller enforces a stronger privacy guarantee . We revisit RR mechanisms for LDP preservation in Appendix A . Our approach is a binary encoding-based approach , similar to ( Arachchige et al. , 2019 ; Lyu et al. , 2020a ) , since it has the potential to overcome the curse of privacy composition . In binary encoding , x is converted into an l-bit vector v consisting of 1 sign bit , m bits for the integer part , and l−m−1 bits for the fraction part ( Figure 1 ) , as follows : ∀i ∈ [ 0 , l − 1 ] : vi = b2i−m|x|c mod 2 ( 1 ) Each bit in v is randomized by applying a RR mechanism , e.g. , ( Erlingsson et al. , 2014 ; Bassily & Smith , 2015 ; Wang et al. , 2017 ) , to generate a perturbed l-bit vector v′ , which preserves LDP . However , in our theoretical reassessment ( Appendices I and J ) , directly applying RR mechanisms on binary encoded vectors as in existing mechanisms , i.e. , LATENT ( Arachchige et al. , 2019 ) and OME ( Lyu et al. , 2020a ) , consumes huge privacy budgets since each binary encoding bit can not be treated as a bit in a hash . Each binary encoding bit i has a different sensitivity , i.e. , ∆i = 2m−i for the integer and fraction parts or ∆i = 2m+1 for the sign bit ( Lemma 1 ) , compared with a bit Bi in a hash , i.e. , ∀Bi : ∆Bi = 1 . Our mechanism does not suffer from this problem , thanks to our bit-aware randomization in binary encoding ( Theorem 2 ) . 3 BITRAND ALGORITHM . Let us now present our FL setting , threat model , and BitRand algorithm ( Figure 2 and Alg . 1 , Appendix B ) , and privacy guarantees . Then , we will show that our algorithm is dimension-elastic and the ability to opti- mize the randomization probabilities with expected error bounds in our theoretical analysis . Federated Learning Given N clients , each client u ∈ [ 1 , N ] has a set of training samples Du = { ( x , yx ) } nu where x ∈ Rd is the input features , and its associated ground-truth label yx ∈ ZC is one-hot encoded with C categorical model outcomes { yx,1 , . . . , yx , C } , and nu is the number of training samples . In a pre-processing step , each client u extracts r numerical embedded features from x , denoted ex , by using a pre-trained model fpre . In practice , fpre could be trained on large-scale and publicly available datasets without introducing any extra privacy cost to the clients ’ local data ( He et al. , 2016 ; Devlin et al. , 2018 ) . N clients jointly train the model fθ : Rr → RC by minimizing a loss function L ( fθ ( ex ) , yx ) on their local training samples in Du that penalizes mismatching between the prediction fθ ( ex ) and the ground-truth label yx , given the model ’ s parameters θ . In each training round t , each client u receives the most updated model parameters θt from the coordinating server and then computes local gradients5uθt = ∑ ( vx , yx ) ∈Du 5θtL ( f ( ex ) , yx ) /nu , which are sent to the coordinating server for aggregating and model updating : θt+1 = θt − ηt ∑ u∈ [ 1 , N ] 5uθt/N . Threat Model . The coordinating server strictly follows the training procedure but curious about the training data Du . This is a practical threat model in the real world since service providers always aim at providing the best services to the clients ( Haeberlen et al. , 2011 ; Truex et al. , 2019 ; Lyu et al. , 2020b ) . Given the observed gradients 5uθt , the coordinating server can extract the clients ’ data { ex , yx } nu by using recently developed attacks ( Carlini et al. , 2020 ; Fredrikson et al. , 2015 ) . In a defense-free environment , { ex , yx } nu can be used to infer the sensitive training data Du = { ( x , yx ) } nu using the pre-trained model fpre , since fpre ( x ) = ex ( Song & Raghunathan , 2020 ) . This poses a severe privacy risk to the sensitive data Du . BitRand Algorithm . To protect the sensitive training data Du against the threat model , in our algorithm , we preserve LDP on both embedded features ex and labels yx . Each of the r embedded features in ex is encoded into l binary bits following Eq . 1 . Binary encoded features are concatenated together into a vector vx consisting of rl binary bits to represent the embedded features ex ( Alg . 1 , line 16 ) . Each bit i ∈ [ 0 , rl − 1 ] in vx is randomized by our f -RR mechanism ( Alg . 1 , line 17 ) with a bit-aware term i % ll optimizing randomization probabilities : ( f -RR mechanism ) ∀i ∈ [ 0 , rl − 1 ] : P ( v′x ( i ) = 1 ) = pX = 1 1 + α exp ( i % l l X ) , if vx ( i ) = 1 qX = α exp ( i % l l X ) 1 + α exp ( i % l l X ) , if vx ( i ) = 0 ( 2 ) where vx ( i ) ∈ { 0 , 1 } is the value of vx at the bit i , v′x is the perturbed vector created by randomizing all the bits in vx , X is a privacy budget , and α is a parameter bounded in Theorem 2 . From Eq . 2 , we also have that P ( v′x ( i ) = 0 ) = 1− pX if vx ( i ) = 1 , and P ( v′x ( i ) = 0 ) = 1− qX if vx ( i ) = 0 . We use the bit-aware term i % ll to indicate the location of bit i , which is associated with the sensitivity of the bit at that location , in its l-bit binary encoded vector among rl concatenated binary bits . One of the key differences between our mechanism and existing works ( Sun et al. , 2021 ; Zhao et al. , 2020b ; Wang et al. , 2019a ; Liu et al. , 2020 ; Arachchige et al. , 2019 ; Lyu et al. , 2020a ) is the bit-aware randomization probabilities . By introducing the bit-aware term i % ll , we are able to : 1 ) Derive significantly tighter privacy loss and expected error bounds compared with existing approaches ( Sections 4 and 5 ) ; and 2 ) Adaptively control the randomization probabilities across bits , such that bits with a stronger influence on the model utility , e.g. , sign bits and integer bits , have smaller randomization probabilities qX , and vice-versa ( Section 5 ) . These advantages are crucial to evade the curse of privacy composition enabling us to work with complex tasks and models ( Section 6 ) . In addition , inspired by the LabelDP ( Ghazi et al. , 2021 ) , we randomize yx using the following label-RR mechanism ( Alg . 1 , line 18 ) : ( label-RR mechanism ) P ( y′x = ȳx ) = pY = exp ( β ) 1 + exp ( β ) , if ȳx = yx qY = 1 ( 1 + exp ( β ) ) ( C − 1 ) , if ȳx 6= yx , ȳx ∈ Z C ( 3 ) where ȳx is one-hot encoded , and β is a parameter bounded in Theorem 3 under a privacy budget Y . Randomizing the label yx provides a complete LDP protection to each local training sample ( ex , yx ) . All the perturbed training samples ( e′x , y ′ x ) are included in a local dataset D ′ u , which will be used to train the model fθ : Rr → RC , i.e. , e′x = E ( v′x ) where E ( · ) is a decoding function , ( Alg . 1 , lines 20-22 ) . The training will never access the original data Du . All other operations remain the same with our aforementioned FL setting .

Summary: The importance of rigorously taking care of privacy leakage in a DP algorithm on the bit level is well-know, see e.g.  Mironov, Ilya. On significance of the least significant bits for differential privacy. In: Proceedings of the 2012 ACM conference on Computer and communications security. 2012. p. 650-661.  This paper considers locally differentially private (LDP) mechanisms for binary representations of floating point numbers. The technique seems like a natural thing to do: the individual bits are flipped with a probability that scales with their importance.