Photonic Differential Privacy with Direct Feedback Alignment

1 Introduction . The widespread use of machine learning models has created concern about their release in the wild when trained on sensitive data such as health records or queries in data bases [ 7 , 15 ] . Such concern has motivated a abundant line of research around privacy-preserving training of models . A popular technique to guarantee privacy is differential privacy ( DP ) , that works by injecting noise in an deterministic algorithm , making the contribution of a single data-point hardly distinguishable from the added noise . Therefore it is impossible to infer information on individuals from the aggregate . While there are alternative methods to ensure privacy , such as knowledge distillation ( e.g . PATE [ 25 ] ) , a simple and effective strategy is to use perturbed and quenched Stochastic Gradient Descent ( SGD ) [ 1 ] : the gradients are clipped before being aggregated and then perturbed by some additive noise , finally they are used to update the parameters . The DP property comes at a cost of decreased utility . These biased and perturbed gradients provide a noisy estimate of the update direction and decrease utility , i.e . end-task performance . We revisit this strategy and develop a private-by-design learning algorithm , inspired by the implementation of an alternative training algorithm , Direct Feedback Alignment [ 23 ] , on Optical Processing Units [ 20 ] , photonic co-processors dedicated to large scale random projections . The analog nature of the photonic co-processor implies the presence of noise , and while this is usually minimized , in this case we propose to leverage it , and tame it to fulfill our needs , i.e to control the level of privacy of the learning process . The main sources of noise in Optical Processing Units can be modeled as additive Poisson noise on the output signal , and approach a Gaussian distribution in the operating regime of the device . In particular , these sources can be modulated through temperature control , in order to attain a desired privacy level . Finally , we test our algorithm using the photonic hardware demonstrating solid performance on the goal metrics . To summarize , our setting consists in OPUs performing the multiplication by a fixed random matrix , with a different realization of additive noise for every random projection . ∗Equal contribution . Corresponding authors : ruben @ lighton.ai and hj.medinaruiz @ criteo.com 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1.1 Related work . The amount of noise needed to guarantee differential privacy was first formalized in [ 9 ] . Later , a training algorithm that satisfied Renyi Differential Privacy was proposed in [ 1 ] . This sparked a line of research in differential privacy for deep learning , investigating different architecture and clipping or noise injection mechanisms [ 2 , 3 ] . The majority of these works though rely on backpropagation . An original take was offered in [ 19 ] , that evaluated the privacy performance of Direct Feedback Alignment ( DFA ) [ 23 ] , an alternative to backpropagation . While Lee et al . [ 19 ] basically extend the gradient clipping/Gaussian mechanism approach to DFA , our work , while applied to the same DFA setting , is motivated by a photonic implementation that naturally induces noise that we exploit for differential privacy . As such , we provide a new DP framework together with its theoretical analysis . 1.2 Motivations and contributions . We propose a hardware-based approach to Differential Privacy ( DP ) , centered around a photonic co-processor , the OPU . We use it to perform optical random projections for a differentially private DFA training algorithm , leveraging noise intrinsic to the hardware to achieve privacy-by-design . This is a significant departure from the classical view that such analog noise should be minimized , instead leveraging it as a key feature of our approach . Our mechanism can be formalized through the following ( simplified ) learning rule at layer ` : δW ` = −η [ ( B ` +1e︸ ︷︷ ︸ scaled DFA learning signal + Gaussian hardware noise︷︸︸︷ g ) φ′ ` ( z ` ) ] ( h ` −1︸︷︷︸ neuron-wise clipped activations ) > ( 1 ) Photonic-inspired and tested . The OPU is used as both inspiration and an actual platform for our experiments . We demonstrate theoretically that the noise induced by the analog co-processor makes the algorithm private by design , and we perform experiments on a real photonic co-processor to show we achieve end-task performance competitive with DFA on GPU . Differential Privacy beyond backpropagation . We extend previous work [ 19 ] on DP with DFA both theoretically and empirically . We provide new theoretical elements for noise injected on the DFA learning signal , a setting closer to our hardware implementation . Theoretical contribution . Previous works on DP and DFA [ 19 ] proposes a straightforward extension of the DP-SGD paradigm to direct feedback alignment . In our work , by adding noise directly to the random projection in DFA , we study a different Gaussian mechanism [ 22 ] with a covariance matrix depending on the values of the activations of the network . Therefore the theoretical analysis is more challenging than in [ 19 ] . We succeed to upper bound the Rényi Divergence of our mechanism and deduce the ( , δ ) -DP parameters of our setup . 2 Background . Formally the problem we study the following : the analysis of the built-in Differential Privacy thanks to the combination of DFA and OPUs to train deep architectures . Before proceeding , we recall a minimal set of principles of DFA and Differential Privacy . From here on { xi } Ni=1 are the training points belonging to Rd , { yi } Ni=1 the target labels belonging to R. The aim of training a neural network is to find a function f : Rd → R that minimizes the true L-risk EXY∼DL ( f ( X ) , Y ) , where L : R × R → R is a loss function and D a fixed ( and unknown ) distribution over data and labels ( and the ( xi , yi ) are independent realizations of X , Y ) , and to achieve that , the empirical risk 1N ∑N i=1 L ( f ( xi ) , yi ) is minimized . 2.1 Learning with Direct Feedback Alignment ( DFA ) . DFA is a biologically inspired alternative to backpropagation with an asymmetric backward pass . For ease of notation , we introduce it for fully connected networks but it generalizes to convolutional networks , transformers and other architectures [ 17 ] . It has been theoretically studied in [ 21 , 27 ] . Note that in the following , we incorporate the bias terms in the weight matrices . Forward pass . In a model with L layers of neurons , ` ∈ { 1 , . . . , L } is the index of the ` -th layer , W ` ∈ Rn ` ×n ` −1 the weight matrix between layers ` − 1 and ` , φ ` the activation function of the neurons , and h ` their activations . The forward pass for a pair ( x , y ) writes as : ∀ ` ∈ { 1 , . . . , L } : z ` = W ` h ` −1 , h ` = φ ` ( z ` ) , ( 2 ) where h0 .= x and ŷ .= hL = φ ( zL ) is the predicted output . Backpropagation update . With backpropagation [ 28 ] , leaving aside the specifics of the optimizer used ( learning rate , etc . ) , the weight updates are computed using the chain-rule of derivatives : δW ` = − ∂L ∂W ` = − [ ( ( W ` +1 ) > δz ` +1 ) φ′ ` ( z ` ) ] ( h ` −1 ) > , δz ` = ∂L ∂z ` , ( 3 ) where φ′ ` is the derivative of φ ` , is the Hadamard product , and L ( ŷ , y ) is the prediction loss . DFA update . DFA replaces the gradient signal ( W ` +1 ) > δz ` +1 with a random projection of the derivative of the loss with respect to the pre-activations δzL of the last layer . For losses L commonly used in classification and regression , such as the squared loss or the cross-entropy loss , this will amount to a random projection of the error e ∝ ŷ − y . With a fixed random matrix B ` +1 of appropriate shape drawn at initialization of the learning process , the parameter update of DFA is : δW ` = − [ ( B ` +1e ) φ′ ` ( z ` ) ] ( h ` −1 ) > , e = ∂L ∂zL ( 4 ) Backpropagation vs DFA training . Learning using backpropagation consists in iteratively applying the forward pass ( 2 ) on batches of training examples and then applying backpropagation updates ( 3 ) . Training with DFA consists in replacing the backpropagation updates by DFA ones ( 4 ) . An interesting feature of DFA is the parallelization of the training step , where all the random projections of the error can be done at the same time , as shown in Figure 1 . 2.2 Optical Processing Units . An Optical Processing Unit ( OPU ) 2 is a co-processor that multiplies an input vector x ∈ Rd by a fixed random matrix B ∈ RD×d , harnessing the physical phenomenon of light scattering through a diffusive medium [ 20 ] . The operation performed is : p = Bx ( 5 ) If the coefficients of B are unknown , they are guaranteed to be ( independently ) distributed according to a Gaussian distribution [ 20 , 24 ] . An additional interesting characteristics of the OPU is its low energy consumption compared to GPUs for high-dimensional matrix multiplication [ 24 ] . 2Accessible through LightOn Cloud : https : //cloud.lighton.ai . A central feature we will rely on is that the measurement of the random projection ( 5 ) may be corrupted by an additive zero-mean Gaussian random vector g , so as for an OPU to provide access to p = Bx+ g. If g is usually negligible , its variance can be modulated by controlling the physical environment around the OPU . We take advantage of this feature to enforce differential privacy . In the current versions of the OPUs , however , modulating the analog noise at will is not easy , so we will simulate the noise numerically in the experiments . 2.3 Differential Privacy ( DP ) . Differential Privacy [ 9 , 10 ] sets a framework to analyze the privacy guarantees of algorithms . It rests on the following core definitions . Definition 2.1 ( Neighboring datasets ) . Let { X j } Nj=1 ( e.g . X j = Rd ) be a domain andD . = ∪Nj=1 X j . D , D′ ∈ D are neighboring datasets if they differ from one element . This is denoted by D ∼ D′ . Definition 2.2 ( ( ε , δ ) -differential privacy [ 10 ] ) . Let ε , δ > 0 . LetA : D → RangeA be a randomized algorithm , where RangeA is the range of D through A . A is ( ε , δ ) -differentially private , or ( ε , δ ) -DP , if for all neighboring datasets D , D′ ∈ D and for all sets O ∈ Im A , the following inequality holds : P [ A ( D ) ∈ O ] ≤ eεP [ A ( D′ ) ∈ O ] + δ where the probability relates to the randomness of A. Mironov [ 22 ] proposed an alternative notion of differential privacy based on Rényi α-divergences and established a connection between their definition and the ( ε , δ ) -differential privacy of Definition 2.2 . Rényi-based Differential Privacy is captured by the following : Definition 2.3 ( Rényi α-divergence [ 29 ] ) . For two probability distributions P and Q defined over R , the Rényi divergence of order α > 1 is given by : Dα ( P‖Q ) . = 1 α− 1 logEx∼Q ( P ( x ) Q ( x ) ) α ( 6 ) Definition 2.4 ( ( α , ε ) -Rényi differential privacy [ 22 ] ) . Let ε > 0 and α > 1 . A randomized algorithm A is ( α , ε ) -Rényi differential private or ( α , ε ) -RDP , if for any neighboring datasets D , D′ ∈ D , Dα ( A ( D ) ‖A ( D′ ) ) ≤ ε. Theorem 1 ( Composition of RDP mechanisms [ 22 ] ) . Let { Mi } ki=1 be a set of mechanisms , each satisfying ( α , i ) -RDP . Then their combination is ( α , ∑ i i ) -RDP . Going from RDP to the Differential Privacy of Definition 2.2 is made possible by the following theorem ( see also [ 4 , 6 , 30 ] ) . Theorem 2 ( Converting RDP parameters to DP parameters [ 22 ] ) . An ( α , ε ) -RDP mechanism is ( ε+ log 1/δα−1 , δ ) -DP for all δ ∈ ( 0 , 1 ) . For the theoretical analysis , we will need the following proposition , specific to the case of multivariate Gaussian distributions , to obtain bounds on the Rényi divergence . Proposition 1 ( Rényi divergence for two multivariate Gaussian distributions [ 26 ] ) . The Rényi divergence for two multivariate Gaussian distributions with means µ1 , µ2 and respective covariances Σ1 , Σ2 is given by : Dα ( N ( µ1 , Σ1 ) ‖N ( µ2 , Σ2 ) ) = α 2 ( µ1 − µ2 ) > ( αΣ2 + ( 1− α ) Σ1 ) −1 ( µ1 − µ2 ) − 1 2 ( α− 1 ) log [ det ( αΣ2 + ( 1− α ) Σ1 ) ( det Σ1 ) 1−α ( det Σ2 ) α ] ( 7 ) with det ( Σ ) the determinant of the matrix Σ . Note that ( αΣ2 + ( 1 − α ) Σ1 ) −1 must be definitepositive3 , otherwise the Rényi divergence is not defined and is equal to +∞ . 3Note that here α > 1 and the combination αΣ2 + ( 1− α ) Σ1 is not a convex combination ; extra-case must be given to ensure that the resulting matrix is positive Remark 2.1 . A standard method to generate an ( R ) DP algorithm from a deterministic function f : X → Rd is the Gaussian mechanismMσ acting asMσf ( · ) = f ( · ) + v where v ∼ N ( 0 , σ2Id ) . If f has ∆f - ( or ` 2- ) sensitivity ∆f . = max D∼D′ ‖f ( D ) − f ( D′ ) ‖2 , thenMσ is ( α , α∆2f 2σ2 ) -RDP .

Summary: The paper proposes a hardware-based approach to Differential Privacy (DP) using OPU to perform optical random projections for a differentially private DFA training algorithm, leveraging noise intrinsic to the hardware to achieve privacy-by-design. The proposed approach was analyzed mathematically in detail. The proposed methods were analyzed on the vanilla Fashion MNIST dataset.    The paper is very well written in general. The proposed methods/approach are very interesting, and mathematical results are solid.