Photonic Differential Privacy with Direct Feedback Alignment

1 Introduction . The widespread use of machine learning models has created concern about their release in the wild when trained on sensitive data such as health records or queries in data bases [ 7 , 15 ] . Such concern has motivated a abundant line of research around privacy-preserving training of models . A popular technique to guarantee privacy is differential privacy ( DP ) , that works by injecting noise in an deterministic algorithm , making the contribution of a single data-point hardly distinguishable from the added noise . Therefore it is impossible to infer information on individuals from the aggregate . While there are alternative methods to ensure privacy , such as knowledge distillation ( e.g . PATE [ 25 ] ) , a simple and effective strategy is to use perturbed and quenched Stochastic Gradient Descent ( SGD ) [ 1 ] : the gradients are clipped before being aggregated and then perturbed by some additive noise , finally they are used to update the parameters . The DP property comes at a cost of decreased utility . These biased and perturbed gradients provide a noisy estimate of the update direction and decrease utility , i.e . end-task performance . We revisit this strategy and develop a private-by-design learning algorithm , inspired by the implementation of an alternative training algorithm , Direct Feedback Alignment [ 23 ] , on Optical Processing Units [ 20 ] , photonic co-processors dedicated to large scale random projections . The analog nature of the photonic co-processor implies the presence of noise , and while this is usually minimized , in this case we propose to leverage it , and tame it to fulfill our needs , i.e to control the level of privacy of the learning process . The main sources of noise in Optical Processing Units can be modeled as additive Poisson noise on the output signal , and approach a Gaussian distribution in the operating regime of the device . In particular , these sources can be modulated through temperature control , in order to attain a desired privacy level . Finally , we test our algorithm using the photonic hardware demonstrating solid performance on the goal metrics . To summarize , our setting consists in OPUs performing the multiplication by a fixed random matrix , with a different realization of additive noise for every random projection . ∗Equal contribution . Corresponding authors : ruben @ lighton.ai and hj.medinaruiz @ criteo.com 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1.1 Related work . The amount of noise needed to guarantee differential privacy was first formalized in [ 9 ] . Later , a training algorithm that satisfied Renyi Differential Privacy was proposed in [ 1 ] . This sparked a line of research in differential privacy for deep learning , investigating different architecture and clipping or noise injection mechanisms [ 2 , 3 ] . The majority of these works though rely on backpropagation . An original take was offered in [ 19 ] , that evaluated the privacy performance of Direct Feedback Alignment ( DFA ) [ 23 ] , an alternative to backpropagation . While Lee et al . [ 19 ] basically extend the gradient clipping/Gaussian mechanism approach to DFA , our work , while applied to the same DFA setting , is motivated by a photonic implementation that naturally induces noise that we exploit for differential privacy . As such , we provide a new DP framework together with its theoretical analysis . 1.2 Motivations and contributions . We propose a hardware-based approach to Differential Privacy ( DP ) , centered around a photonic co-processor , the OPU . We use it to perform optical random projections for a differentially private DFA training algorithm , leveraging noise intrinsic to the hardware to achieve privacy-by-design . This is a significant departure from the classical view that such analog noise should be minimized , instead leveraging it as a key feature of our approach . Our mechanism can be formalized through the following ( simplified ) learning rule at layer ` : δW ` = −η [ ( B ` +1e︸ ︷︷ ︸ scaled DFA learning signal + Gaussian hardware noise︷︸︸︷ g ) φ′ ` ( z ` ) ] ( h ` −1︸︷︷︸ neuron-wise clipped activations ) > ( 1 ) Photonic-inspired and tested . The OPU is used as both inspiration and an actual platform for our experiments . We demonstrate theoretically that the noise induced by the analog co-processor makes the algorithm private by design , and we perform experiments on a real photonic co-processor to show we achieve end-task performance competitive with DFA on GPU . Differential Privacy beyond backpropagation . We extend previous work [ 19 ] on DP with DFA both theoretically and empirically . We provide new theoretical elements for noise injected on the DFA learning signal , a setting closer to our hardware implementation . Theoretical contribution . Previous works on DP and DFA [ 19 ] proposes a straightforward extension of the DP-SGD paradigm to direct feedback alignment . In our work , by adding noise directly to the random projection in DFA , we study a different Gaussian mechanism [ 22 ] with a covariance matrix depending on the values of the activations of the network . Therefore the theoretical analysis is more challenging than in [ 19 ] . We succeed to upper bound the Rényi Divergence of our mechanism and deduce the ( , δ ) -DP parameters of our setup . 2 Background . Formally the problem we study the following : the analysis of the built-in Differential Privacy thanks to the combination of DFA and OPUs to train deep architectures . Before proceeding , we recall a minimal set of principles of DFA and Differential Privacy . From here on { xi } Ni=1 are the training points belonging to Rd , { yi } Ni=1 the target labels belonging to R. The aim of training a neural network is to find a function f : Rd → R that minimizes the true L-risk EXY∼DL ( f ( X ) , Y ) , where L : R × R → R is a loss function and D a fixed ( and unknown ) distribution over data and labels ( and the ( xi , yi ) are independent realizations of X , Y ) , and to achieve that , the empirical risk 1N ∑N i=1 L ( f ( xi ) , yi ) is minimized . 2.1 Learning with Direct Feedback Alignment ( DFA ) . DFA is a biologically inspired alternative to backpropagation with an asymmetric backward pass . For ease of notation , we introduce it for fully connected networks but it generalizes to convolutional networks , transformers and other architectures [ 17 ] . It has been theoretically studied in [ 21 , 27 ] . Note that in the following , we incorporate the bias terms in the weight matrices . Forward pass . In a model with L layers of neurons , ` ∈ { 1 , . . . , L } is the index of the ` -th layer , W ` ∈ Rn ` ×n ` −1 the weight matrix between layers ` − 1 and ` , φ ` the activation function of the neurons , and h ` their activations . The forward pass for a pair ( x , y ) writes as : ∀ ` ∈ { 1 , . . . , L } : z ` = W ` h ` −1 , h ` = φ ` ( z ` ) , ( 2 ) where h0 .= x and ŷ .= hL = φ ( zL ) is the predicted output . Backpropagation update . With backpropagation [ 28 ] , leaving aside the specifics of the optimizer used ( learning rate , etc . ) , the weight updates are computed using the chain-rule of derivatives : δW ` = − ∂L ∂W ` = − [ ( ( W ` +1 ) > δz ` +1 ) φ′ ` ( z ` ) ] ( h ` −1 ) > , δz ` = ∂L ∂z ` , ( 3 ) where φ′ ` is the derivative of φ ` , is the Hadamard product , and L ( ŷ , y ) is the prediction loss . DFA update . DFA replaces the gradient signal ( W ` +1 ) > δz ` +1 with a random projection of the derivative of the loss with respect to the pre-activations δzL of the last layer . For losses L commonly used in classification and regression , such as the squared loss or the cross-entropy loss , this will amount to a random projection of the error e ∝ ŷ − y . With a fixed random matrix B ` +1 of appropriate shape drawn at initialization of the learning process , the parameter update of DFA is : δW ` = − [ ( B ` +1e ) φ′ ` ( z ` ) ] ( h ` −1 ) > , e = ∂L ∂zL ( 4 ) Backpropagation vs DFA training . Learning using backpropagation consists in iteratively applying the forward pass ( 2 ) on batches of training examples and then applying backpropagation updates ( 3 ) . Training with DFA consists in replacing the backpropagation updates by DFA ones ( 4 ) . An interesting feature of DFA is the parallelization of the training step , where all the random projections of the error can be done at the same time , as shown in Figure 1 . 2.2 Optical Processing Units . An Optical Processing Unit ( OPU ) 2 is a co-processor that multiplies an input vector x ∈ Rd by a fixed random matrix B ∈ RD×d , harnessing the physical phenomenon of light scattering through a diffusive medium [ 20 ] . The operation performed is : p = Bx ( 5 ) If the coefficients of B are unknown , they are guaranteed to be ( independently ) distributed according to a Gaussian distribution [ 20 , 24 ] . An additional interesting characteristics of the OPU is its low energy consumption compared to GPUs for high-dimensional matrix multiplication [ 24 ] . 2Accessible through LightOn Cloud : https : //cloud.lighton.ai . A central feature we will rely on is that the measurement of the random projection ( 5 ) may be corrupted by an additive zero-mean Gaussian random vector g , so as for an OPU to provide access to p = Bx+ g. If g is usually negligible , its variance can be modulated by controlling the physical environment around the OPU . We take advantage of this feature to enforce differential privacy . In the current versions of the OPUs , however , modulating the analog noise at will is not easy , so we will simulate the noise numerically in the experiments . 2.3 Differential Privacy ( DP ) . Differential Privacy [ 9 , 10 ] sets a framework to analyze the privacy guarantees of algorithms . It rests on the following core definitions . Definition 2.1 ( Neighboring datasets ) . Let { X j } Nj=1 ( e.g . X j = Rd ) be a domain andD . = ∪Nj=1 X j . D , D′ ∈ D are neighboring datasets if they differ from one element . This is denoted by D ∼ D′ . Definition 2.2 ( ( ε , δ ) -differential privacy [ 10 ] ) . Let ε , δ > 0 . LetA : D → RangeA be a randomized algorithm , where RangeA is the range of D through A . A is ( ε , δ ) -differentially private , or ( ε , δ ) -DP , if for all neighboring datasets D , D′ ∈ D and for all sets O ∈ Im A , the following inequality holds : P [ A ( D ) ∈ O ] ≤ eεP [ A ( D′ ) ∈ O ] + δ where the probability relates to the randomness of A. Mironov [ 22 ] proposed an alternative notion of differential privacy based on Rényi α-divergences and established a connection between their definition and the ( ε , δ ) -differential privacy of Definition 2.2 . Rényi-based Differential Privacy is captured by the following : Definition 2.3 ( Rényi α-divergence [ 29 ] ) . For two probability distributions P and Q defined over R , the Rényi divergence of order α > 1 is given by : Dα ( P‖Q ) . = 1 α− 1 logEx∼Q ( P ( x ) Q ( x ) ) α ( 6 ) Definition 2.4 ( ( α , ε ) -Rényi differential privacy [ 22 ] ) . Let ε > 0 and α > 1 . A randomized algorithm A is ( α , ε ) -Rényi differential private or ( α , ε ) -RDP , if for any neighboring datasets D , D′ ∈ D , Dα ( A ( D ) ‖A ( D′ ) ) ≤ ε. Theorem 1 ( Composition of RDP mechanisms [ 22 ] ) . Let { Mi } ki=1 be a set of mechanisms , each satisfying ( α , i ) -RDP . Then their combination is ( α , ∑ i i ) -RDP . Going from RDP to the Differential Privacy of Definition 2.2 is made possible by the following theorem ( see also [ 4 , 6 , 30 ] ) . Theorem 2 ( Converting RDP parameters to DP parameters [ 22 ] ) . An ( α , ε ) -RDP mechanism is ( ε+ log 1/δα−1 , δ ) -DP for all δ ∈ ( 0 , 1 ) . For the theoretical analysis , we will need the following proposition , specific to the case of multivariate Gaussian distributions , to obtain bounds on the Rényi divergence . Proposition 1 ( Rényi divergence for two multivariate Gaussian distributions [ 26 ] ) . The Rényi divergence for two multivariate Gaussian distributions with means µ1 , µ2 and respective covariances Σ1 , Σ2 is given by : Dα ( N ( µ1 , Σ1 ) ‖N ( µ2 , Σ2 ) ) = α 2 ( µ1 − µ2 ) > ( αΣ2 + ( 1− α ) Σ1 ) −1 ( µ1 − µ2 ) − 1 2 ( α− 1 ) log [ det ( αΣ2 + ( 1− α ) Σ1 ) ( det Σ1 ) 1−α ( det Σ2 ) α ] ( 7 ) with det ( Σ ) the determinant of the matrix Σ . Note that ( αΣ2 + ( 1 − α ) Σ1 ) −1 must be definitepositive3 , otherwise the Rényi divergence is not defined and is equal to +∞ . 3Note that here α > 1 and the combination αΣ2 + ( 1− α ) Σ1 is not a convex combination ; extra-case must be given to ensure that the resulting matrix is positive Remark 2.1 . A standard method to generate an ( R ) DP algorithm from a deterministic function f : X → Rd is the Gaussian mechanismMσ acting asMσf ( · ) = f ( · ) + v where v ∼ N ( 0 , σ2Id ) . If f has ∆f - ( or ` 2- ) sensitivity ∆f . = max D∼D′ ‖f ( D ) − f ( D′ ) ‖2 , thenMσ is ( α , α∆2f 2σ2 ) -RDP .

Summary: The manuscript entitled “Photonic Differential Privacy with Direct Feedback Alignment” extends their previous work - Differential Privacy with Direct Feedback Alignment. In this work, they leverage the intrinsic noise (which might be controlled by temperature) of optical random projections to build a differentially private DFA mechanism, making Optical Processing Units (OPUs) a solution of choice to provide a private-by-design training. This work is motivated by a photonic implementation that naturally induces noise that we exploit for differential privacy. The proposed method is demonstrated by simulations showing that adding noise does not decrease the performance significantly. In other words, this approach is robust to increasing DP noise. This paper also provides its theoretical analysis.