Photonic Differential Privacy with Direct Feedback Alignment
Authors: Ruben Ohana
Hamlet Jesse Medina Ruiz
Julien Launay
Alessandro Cappelli
Iacopo Poli
Liva Ralaivola
Alain Rakotomamonjy
Venue: NeurIPS 2021 Poster (2021)
Keywords: differential privacy
direct feedback alignment
optical computing
deep learning
alternative training methods

Abstract: Optical Processing Units (OPUs) -- low-power photonic chips dedicated to large scale random projections -- have been used in previous work to train deep neural networks using Direct Feedback Alignment (DFA), an effective alternative to backpropagation. Here, we demonstrate how to leverage the intrinsic noise of optical random projections to build a differentially private DFA mechanism, making OPUs a solution of choice to provide a \emph{private-by-design} training. We provide a theoretical analysis of our adaptive privacy mechanism, carefully measuring how the noise of optical random projections propagates in the process and gives rise to provable Differential Privacy. Finally, we conduct experiments demonstrating the ability of our learning procedure to achieve solid end-task performance. 

Photonic Differential Privacy with Direct Feedback Alignment
Ruben Ohana Hamlet J. Medina Ruiz*2 Julien Launay" 1,3, Alessandro Cappelli', Iacopo Poli', Liva Ralaivola Alain Rakotomamonjy"
LightOnn Paris, France
Critto Al Lab, Paris, France
LPENS, École Normale Supérieure, Paris, France
Abstract
Optical Processing Units (OPUs) -ow-power photonic chips dedicated to large scale random projections have been used in previous work train deep neural networks using Direct Feedback Alignment (DFA), an effective alternative to backpropagation. Here, we demonstrate how leverage the intrinsic noise of optical random projections to build a differentially private DFA mechanism, making OPUs solution of choice provide a private-by-deste theoretical analysis of our adaptive privacy mechanism, carefully measuring how the noise of optical random projections propagates in the process and gives rise provable Differential Privacy. Finally, we conduct experiments demonstrating the ability our learning procedure to achieve solid end-task performance.
training. We provide a
Introduction
The widespread use of machine learning models has created concern about their release in the wild when trained on sensitive data such health records or queries in data bases [5]. Such concern has motivated a abundant line of research around popular technique to guarantee privacy differential privacy (DP), that works by injecting noise in an deterministic algorithm, making the contribution of a single data-point hardly distinguishable from the added noise. Therefore impossible infer information on individuals from the aggregate.
training of models. A
While there are alternative methods to ensure privacy, such as knowledge distillation (e.g. PATE (5]), simple and effective strategy to use perturbed and quenched Stochastic Gradient Descent (SGD) [I: the gradients are clipped before being aggregated and then perturbed by some additive noise, finally they used to update the parameters. The DP property comes at cost of decreased utility. These biased and perturbed gradients provide noisy estimate of the update direction and decrease utility, i.c. end-task performance.
We revisit this strategy and develop private-by-desip learning algorithm. inspired by the implemen- tation of alternative training algorithm, Direct Feedback Alignment [23] Optical Processing
Units 200 photonic co- processors dedicated to large scale random projections. The analog nature of the photonic co-processor implies the presence of noise, and while this is usually minimized, in this case we propose to leverage it, and tame it to fulfill our needs, i. to control the level of privacy of the learning process. The main sources of noise in Optical Processing Units can be modeled as additive Poisson noise on the output signal, and approach a Gaussian distribution in the operating regime of the device. In particular, these sources can be modulated through temperature control, in order to attain a desired privacy level.
Finally, we test our algorithm using the photonic hardware demonstrating solid performance on the goal metrics. To summarize, our setting consists OPUs performing the multiplication by fixed random matrix, with different realization of additive noise for every random projection.
Equaal contribution. Corresponding authors: ruben@lighton.as ai and hj meddnarruzlorrtee.ccom
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

1.1 Related work
The amount of noise needed to guarantee differential privacy was first formalized in 19. Later, training algorithm that satisfied Renyi Differential Privacy was proposed in [I] This sparked a line of research differential privacy for deep learning, investigating different architecture and clipping or noise injection mechanisms 22 3 The majority of these works though rely on backpropagation. An original take was offered in 19 that evaluated the privacy performance of Direct Feedback Alignment (DFA) 23 alternative to backpropagation. While Lee et al. [19] basically extend the gradient Slipping/Gaussinn mechanism approach to DFA, our work, while applied to the same DFA setting, is motivated by photonic implementation that naturally induces noise that exploit for differential privacy. As such, we provide a new DP framework together with its theoretical analysis.
1.2 Motivations and contributions
We propose a hardware-based approach to Differential Privacy (DP), centered around a photonic co-processor, the OPU. We use it to perform optical random projections for differentially private DFA training algorithm, leveraging noise intrinsic to the hardware to achieve privacy-by- This is significant departure from the classical view that such analog noise should be minimized, instead leveraging it as a key feature of our approach. Our mechanism can be formalized through the following (simplified) learning rule at layer &
Gaussian hardware
ow -nl(B
0' (h
9
scaled learning signal
neuron-wise elipped activations
Photonic-inspired and tested. The OPU is used as both inspiration and actual platform for experiments. We demonstrate theoretically that the noise induced by the analog co- co-processor makes the algorithm private by design, and perform experiments we achieve end-task performance competitive with DFA on GPU.
real photonic co-processor to show
Differential Privacy beyond backpropagation. We extend previous work [19] on DP with DFA both theoretically and empirically. We provide new theoretical elements for noise injected on the DFA learning signal, setting closer our hardware implementation.
Theoretical contribution. Previous works on DP and DFA 199 proposes a straightforward extension of the DP-SGD paradigm to direct feedback alignment. In our work, by adding noise directly to the random projection in DFA, we study a different Gaussian mechanism [22] with a covariance matrix depending on the values of the activations of the network. Therefore the theoretical analysis is more challenging than in [19]. We succeed to upper bound the Rényi Divergence of our mechanism and deduce the (c, DP parameters of our setup.
2 Background
Formally the problem we study the following: the analysis of the built-in Differential Privacy thanks to the combination of DEA and OPUs to train deep architectures. Before proceeding, we recall a minimal set of principles of DFA and Differential Privacy.
From here on {xx N are the training points belonging to Rd,, {ur) the target labels belonging to E. The aim of training neural network is to find function f Rd + IR that minimizes the true C-risk Exy~D (X((), Y), where L : R x R + RR is loss function and D a fixed (and unknown) distribution over data and labels (and the (æ,, yy) are independent realizations of X, Y), and to achieve that, the empirical risk N
N L( (xx)) ys) is minimized.
2.1 Learning with Direct Feedback Alignment (DFA)
DFA biologically inspired alternative to backpropagation with an asymmetric backward pass. For ease of notation, we introduce it for fully connected networks but it generalizes to convolutional networks, transformers and other architectures [77. been theoretically studied in 21 27 Note that in the following, we incorporate the bias terms in the weight matrices.

---

forward
backpropagation
direct feedback alignment
sample
layer
layer
layer
layer
loss
random projection the
Figure Schematic comparison of backpropagation and direct feedback alignment. The two approaches differ in how the loss impacts each layer of the model. While in backpropagation, the loss propagated sequentially backwards, in DFA, it directly acts on each layer after random projection.
Forward pass. model with L layers of neurons, E fI...., L} is the index of the f-th layer, w gnexne-1 the weight matrix between layers L and l, Φe the activation function of the neurons, and hy their activations. The forward pass for pair (æ, y) writes as:
VA E (1....., L) :2 wh h dz((z).
(2)
where h = x and y= = h = p(z²) ") is the predicted output.
Backpropagation update. With backpropagation 28 leaving aside the specifics of the optimizer used (learning rate, etc.), the weight updates are computed using the chain-rule of derivatives:
Ww dC -[[((W++))T& 11 c (z (h(f1))
ac
,88
(3)
ow
Oz
where &y is the derivative de. O is the Hadamard product, and L(y,y) y) is the prediction loss.
DEA update. DFA replaces the gradient signal (W 8z with a random projection of the derivative of the loss with respect to the pre-activations 6z of the last layer. For losses L commonly used in classification and regression, such as the squared loss or the cross-entropy loss, this will amount to random projection of the error appropriate shape drawn at initialization the learning process, the parameter update of DFA is:
& y. With a fixed random matrix B B of
OC
w = ( (+
26 ((( == dzL
(4)
Backpropagation vs DFA training. Learning using backpropagation consists in iteratively ap- plying the forward pass (2) on batches training examples and then applying backpropagation updates 3 Training with DFA consists in replacing the backpropagation updates by DFA ones (4)
An interesting feature DFA the parallelization of the training step, where all the random projections of the error can be done at the same time, as shown in Figure
2.2 Optical Processing Units
Optical Processing Unit (OPU co- processor that multiplies an input vector æ E Rd by fixed random matrix B E Dxd xdd hamesssing the physical phenomenon of light scattering through
diffusive medium 20 The operation performed is:
p Ba
(5)
If the coefficients of B are unknown, they guaranteed to be (independently) distributed according to Gaussian distribution [20] 24). An additional interesting characteristics of the OPU is its low energy consumption compared to GPUs for high-dimensional matrix multiplication 24
Accessible through LightOn Cloud: https://cloud: 1ighttnn.htha

---

A central feature we will rely on is that the measurement of the random projection may be corrupted by an additive zero-mean Gaussian random vector g. so as for an OPU to provide access to Bæ: g. If is usually negligible, its variance can be modulated by controlling the physical environment around the OPU. We take advantage of this feature to enforce differential privacy. the current versions of the OPUs, however, modulating the analog noise at will is not easy, so we will simulate the noise numerically in the experiments.
2.3 Differential Privacy (DP)
Differential Privacy Ø 10 sets framework analyze the privacy guarantees of algorithms. It rests on the following core definitions.
Definition 2.1 (Neighboring datasets). Let { (c.g. X' = IR be a domain and D U Xi D' E are neighboring datasets if they differ from one element. This is denoted by ~ D'.
Definition 2.2 ((s, 3)-differelttal privacy [T). Let 5 > Let A D + RangeA be a randomized algorithm, where Range.A is the range of D through A. A is (e, -ddifferret private, or (s, DDD for all neighboring datasets D, D' E DD and for all sets O E Im A, the following inequality holds:
PIA(D) E 0| S PFP((((D)) E 0]++
where the probability relates to the randomness A.
22 proposed an alternative notion of differential privacy based Rényi a-divergences and established a connection between their definition and the (e, -iffifrrrett privacy of Definition 2.2 Rényi-based Differential Privacy captured by the following:
Mironov
Definition 2.3 (Rényi a-divergence 22 ). For two probability distributions P and Q defined over R. the Rényí divergence of order a > is given by:
P(x) ((x)
Da (PIIQ) = log E E
(6)
Definition 2.4 ((a, =)-ééni differential privacy (22)). Let 0 and a > A randomized algorithm A is (a,ı)-Rényi differential private or (a, E) RD., if for any neighboring datasets D, D' E D,
D. (A(D))LA(D)) s.
Theorem (Composition of RDP mechanisms
22 Let {M) be of mechanisms, each satisfying (a, 4) RDP. Then their combination is (a, E, RDP.
Going from RDP to the Differential Privacy of Definition 2.2 made possible by the following theorem (see also 6 30)).
Theorem 2 (Converting RDP parameters to DP parameters 22 An s)r mechanism is -DP for all ô 10,
(e+ log 1/8 1/8
a-T 0 -
For the theoretical analysis, we will need the following proposition, specific the case of multivariate Gaussian distributions, to obtain bounds on the Rényi divergence.
Proposition 1 (Rényi divergence for two multivariate Gaussian distributions (66). The Rényi diver- gence for two multivariate Gaussian distributions with means P1 Mb and respective covariances E1, Eg is given by:
Da(N(M (.WN(,EE))WWA Ex)) = [11 122) 122 Ta + a)-)) (14 112)
1
det((a22 (1 a1Ei))
log
2( ( 1)
[del En ) (det Σ₂)²
with ())(E) the determinant of the matrix Σ. Note that (aΣ + (1 a 21) must be defiite- p otherwise the Rényi divergence is not defined and is equal to +o0.
Note that here >> 1 and the combination aE + (1 2; is not a convex combination; extra-case must be given to ensure that the resulting matrix is positive

---

Remark 2.1. A standard method to generate an (R)DP algorithm from deterministic function
f: Rd is the Gaussian mechanism M, acting as M, f(-) + where v ~ ~N(00 If f has D (or l2-) sensitivity
AS = max II(D) 1(D')122
D-D
then Mo is aa, asi 20 -RDP.
3 Photonic Differential Privacy
This section explains how to use photonic devices perform DFA and theoretical analysis showing how this combination entails photonic differential privacy.
3.1
Clipping parameters
As usual in Differential Privacy, we will need to clip layers of the network during the backward pass. Given vector E Rd and positive constants C, 8, M, define:
(v) = (sign(v)) min(e, sign)....,ssn((aa))) min(c, Iual))
scale,(v) = min(s /0112) Tellz
offset,(v) (0) + U..... va V)
The weight update with clipping to be considered in place of is given by
1 m
ow
m _
(scale,, (B'e,) g,) d' ('(2)) clip,, (offsett, (h₂))
(8)
For each layer f, we set the 8e, ce and Vs parameters as follows:
Th mar
Ci
Th VE
8e - TB
These choices ensure that:
VTt
T
Timin <
(h.))lla < Tmme
|scale,, (Bfe:)ll2 STB
Moreover, we require the derivatives of the each activation function &e are lower and upper bounded by constants i.e. ?mmán < (() S ma for all scalars L. This is reasonable assumption for activation functions such as sigmoid, tanh, ReLU...
In the following, the quantities should considered clipped/scaled/offset as above and, for sake of clarity, drop the explicit mentions of these operations.
3.2 Photonic Direct Feedback Alignment is a natural Gaussian mechanism
Noise modulation. Mainly due to the photon shot noise of the measurement process 16 Gaussian noise is naturally added to the random projection of (5). This noise is negligible for machine learning purposes, however it can be modulated through temperature control yielding the following projection:
p = Bæ + (O, I²Ip)
(10)
where In is the identity matrix in dimension D. Note that this noise is truly random due to the quantum nature of the photon shot noise. As previously stated, due to experimental constraints, the noise will be simulated in the experiments of Section
Building on that feature, we perform the random projection DFA of (4) using the OPU. Since this equation is valid for any layer & of the network, we allow ourselves, for sake of clarity, to drop the layer indices and study the generic update (the quantities below are all clipped as in Section 3.1
JW
BBe 9.) 0 hz
(clipped quantities in section 3.1
(11)
1
[Be; O (((z,)
hT
+
g. 3 (z
(12)
i=1
5

---

Algorithm Photonic DFA training
Require: training set S {(r,, 8y)1 6 with bounded derivatives, scale parameters s6, clipping
thresholds V/ and Cg, stepsize 17, noise scale o, minibatch of size m, number of iterations T
Ensure: A performing DP model
1: for 8= to L. do
2: Sample B
Note: with OPUs, there is no explicit sampling of B
3: end for
4: for I iiterationn do
5: Create minibatch S [I,...,AN size |S| m (sampling without replacement)
6: for i E S do
7:
for l 1 L do 2 = h
8-1
8:
9:
h ob((z))
10:
end for
y; + d5 (=h 1
11:
12:
end for
13:
for l= to do
14:
Perform B e, with the OPU
15:
Independently sample "ff.....ge ~ ~N0,002
W W Di ((scale, (B)+le;) + g 3 dz(2)) )elip,, (offiset (h₂))
17:
end for
end for
where ~ of is the Gaussian noise added during the OPU process. As stated previously, its variance o² can be modulated to obtain the desired value. The overall training procedure with Photonic DFA (PDFA) is described in Algorithm 3.1
3.3 Theoretical Analysis of method
In the following, the quantities in the DFA update of the weights are always clipped according to and as before clipping/scale/offsett operators are in force but dropped from the text.
To demonstrate that our mechanism Differentially Private, we will use the following reasoning: the noise being added the random projection level in (TO), we decompose the update of the weights as a Gaussian mechanism as in (12). We will compute the covariance matrix of the Gaussian noise, which will depend the data, which is in striking contrast with the standard Gaussian mechanism [1. We will then use Proposition to compute the upper bound the Rényí divergence. The Differential Privacy parameters will be obtained using Theorem 2
In the following, we will consider the Gaussian mechanism applied to the columns of the weight matrix. We consider this case for the following reasons: since our noise matrix has the same realisation of the Gaussian noise (but multiplied by different scalars), it makes sense to consider Differential Privacy parameters of only columns of the weight matrix and then multiply the Rényi divergence by the number of columns. If our noise was i.i.d. we could have used the theorems from 8 to lower the Rényi divergence. Given the update equation of the weights at layer / in (22). the update of column the weight of layer the following Gaussian mechanism:
m
(Be;) o (z())) +
(g e d(z)))aa = fx(D) + + Ex)
(13)
m
VI
where Σ, (2) and (ax)
M me-.. Note that these
expressions are due to the inner product with h; In the following, we will focus on column & and we therefore drop the index is in the notation. Using the clipping of the quantities of interest detailed in (8) we can compute some useful bounds bounds az:
m
min
Th
ag S
m
maz mar
Th
(14)
n2
ng

---

Proposition 2 (Sensitivity of Photonic DFA 19).. For neighboring datasets D and D' (i.e. differing from only one element), the sensitivity A of the function fx described 12 at layer I is given by:
v 2 [l e:) o (1
A = sup [ - S(D)
(15)
D-D
2
mea '
Tt
(16)
m
fg
The following proposition our main theoretical result: we compute the e parameter of Rényi Differential Privacy.
Proposition 3 (Photonic Differential Privacy parameters). Given two probability distributions P (f) Σ) and ~~ (ff() E') corresponding the Gaussian mechanisms depicted in 13
~
on neighboring datasets D and D', the Rényi divergence of order a between these mechanisms is:
(Pll)) < 2mg.a (ymaa maaa TB)²
ml min min)
7g..0
Th
log
m..²2 (ymin m2ng
2( 1)
(m+ 1)((2 T )2 (36 man max 2
man min
EPDFA
(17)
Our mechanism is therefore (a, TEPDA)-RDD with T the number of training epochs. We can deduce that the mechanism on the weight matrix with ng-1 columns ( ( -RDP. Then the mechanism of the whole network composed of lavers is (a, LT -RDP. We can then convert our bound DP parameters using Theorem 2 obtain (LTnę-1ÊpDss + mechanism for all 8 E (0,1).
log 1/6
a-1 8)-DP
Proof. In the following, the variables with prime correspond to the ones built upon dataset D'. According to (13), the covariance matrices Σ and E' diagonal and any of their weighted sum is diagonal, as well as their inverse. Moreover, the determinant of a diagonal matrix is the product of its diagonal elements. Using these elements in (7) yield:
(PII)) Σ
am3 (,(()- 2a anl + (1-
2(ax 1) log
1
(1 a)aa + + 2(1-a) af
dj
Using the fact that we are studying neighboring datasets, the sums composing aj and d' differ by only one element element I. This implies that
a a + -1- a)aa = talloi (1ight))))))
where Th and 12 are taken on the dataset D'. By choosing D and D' such that hrr)) (d)))1)))) and some rearrangement,
upper bound the Rényi divergence by:
D. (PI <
2
A
a.
log
m( Th ming?
20²
m min min minn
2(a - (m + 1)(12 min min )2 (7/ mar
T maryz
Using the bound on the sensitivity f computed in (15). the Rényi divergence. more detailed proof is presented in the Appendix.
we obtain the desired EPDFA, upper bound of
Remark 3.1. This bound is not tight since it assumes that all the activations reach their worst cases in all the layers for upper bounding. However, obtaining tighter bound would very challenging since the values of the covariance matrices depend on the output of the neurons of the Neural network, which are data and architecture dependent.
We believe tighter bounds could be obtained much simpler cases. First we can notice that having equal covariance matrices Σ and E' would cancel the logarithm term. If additionally assume that all the activations saturate to their clipping values, then we would retrieve the formula of e in 19]
Owing to mini-batch training, we believe the privacy parameter could be further improve by consider- ing subsampling mechanism (30) and its properties. However, this would require novel theoretical framework adapted to case and we leave it for future work.
7

---

04
0.3
Aiganeed (ros.
Valicatto
02
6-0
--000
DBA TOTA
0.1
2200
0.1
0.0
Ipocca
Ipochs
Figure 2: Photonic training on FashionMNIST. Left: BP, DFA, and photonic DFA (PDFA) training runs for various degrees of privacy. Dashed runs (--) are non-private. Increasingly transparent runs have increased noise, see Table for details. PDFA always very close DFA performance, and both robust to noise. Right: gradient alignment (cosine similarity between PDFA and BP gradients) for the second layer of the network, at varying degrees of noise. Increasing noise degrades alignment, but alignment values remain high enough to support learning.
4 Experimental results
In this section, we demonstrate that photonic training is robust to Gaussian mechanism. i.c. adding noise in Eq. [8] delivering good end-task performance even under strong privacy constraints. As detailed by our theoretical analysis, we focus on two specific mechanisms:
Clipping and offsetting the neurons with TA
and
to enforce T/ < |
17'112
S
Th as explained in section 3.1
Adding noise g to Be, according to the clipping Be with (1 Be l2 S Ty) scaling of g with (g oN(00o²T)
and the
To make our results easy interpret, we fix ty such that 0.1 implies ||g||2 = 0.01, this means that the noise is roughly 10% of the norm of the DFA learning signal. This is in line with pnrentially-privatt setup using backpropagation, and in fact more demanding as our experimental setup makes it such that this is lower bound on the noise.
Bell 2- At
Photonic DFA. We perform the random projection Be at the core of the DFA training step using the OPU, a photonic co-processor. As the inputs of the OPU are binary, we ternarize the error although binarized DFA, known as Direct Random Target Propagation [11. is possible, its performance is inferior. Ternarization is performed using a tunable treshold I, such that values smaller than -/ are set to -1, values larger than t are set 1, and all in between values positive part e. of this vector using the OPU, obtaining Be+, and then the negative part e obtaining Be_. Finally, we substract the two to obtain the projection of the ternarized error, B(e+ -e.). This line with the setup proposed 18]. We refer thereafter to DFA performed on a ternaiized error GPU ternarized DFA (TDFA), and to DFA performed optically with ternarized error as photonic DEA (PDFA).
set to 0. We then project the
Setting. We run our simulations on cloud servers with single NVIDIA V100 GPU and an OPU, for total estimate of 75 GPU-hours. We perform our experiments on MNST dataset (32). reserving 10% of the data as validation, and reporting test accuracy on held-out set. We use fully-connected network, with two hidden layers of size 512, with tanh activation. Optimization is done over 15 epochs with SGD, using a batch size of 256, learning rate of 0.01 and 0.9 momentum. For TDFA, we use threshold of 0.15. Despite the fundamentally different hardware platform. no specific hyperparameter tuning is necessary for the photonic training: this demonstrates the reliability and robustness of our approach.
baseline. We also apply our DP mechanism to a network trained with backpropagation. The clipping and offsetting of the activations' neurons is unchanged, but adapt the noise element. We

---

Table I: Test accuracy on FashionMNIST with our DP mechanism. We find our approach to be robust to increasing DP noise a. particular, photonic DFA results (PDFA) are always within 1% of the corresponding DFA run.
0
non-private
0.01
0.03
0.05
0.1
0.2
Ty
BP
88.33
75.22
70.71
71.47
71.27
70.28
66.78
DEA
86.80
84.20
84.04
84.15
83.70
83.06
81.66
TDFA
86.63
84.20
84.38
84.04
83,94
82.98
80.80
PDFA
85.85
84.00
83.79
83.69
83.36
82.63
80.94
apply the noise on the derivative of the loss once at the top of the network. We also lower the learning rate 10-1 to stabilize runs.
Results. We fix T for all experiments, and consider a range of a corresponding to noise between 0-200% of the DFA training signal Be. We also compare non-private, vanilla baseline. Results are reported in Table and Figure We find our DFA-basee approach Differential Privacy, with a test accuracy hit contained within 3% of baseline for to a = 0.05 (i.c. noise 50% as large as the training signal). Most of the performance hit can actually be attributed to the aggressive activation clipping, with noise having a limited effect. In comparison, BP is far more sensitive to activation clipping and our noise mechanism. However, our method was devised for DFA and not BP, explaining the under-perforr onder-performance of BP. Finally, photonic training achieves good test accuracy, always within 1% of the corresponding DFA run. This demonstrates the validity of our approach, real photonic co-processor. We note that, usually, demonstrations of neural networks with beyond silicon hardware are mostly limited to simulations [14 12],
be remarkably robust the addition of noise, providing
or that these demonstrations
come with significant end-task performance penalty Additional results with similar conclusions on MNIST and CIFAR-10 are presented in the Appendix.
34 31
Conclusion and Outlooks
We have investigated how the Gaussian measurement noise that goes with the of the photonic chips known as Optical Processor Units, can be taken advantage of to ensure a Differentially Private Direct Feedback Alignment training algorithm for deep architectures. We theoretically establish the features of the so-obtained Photonic Differential Privacy and we feature these theoretical findings with compelling empirical results showing how adding noise does decreases the performance significantly.
At age where both procedures are a must, our contribution addresses both points through our photonic differential privacy framework. As such believe the holistic machine learning contribution positive impacts by reducing energy consumption when learning from large-scale datasets and by keeping those datasets private. On the negative impact side, DP approach is heavily based on clipping, which is well-known to have negative effects on underrepresented classes and groups [5] [3 in machine learning model.
training algorithms and energy-awand machine learning
bring will mostly bring
We plan to extend the present work in two ways. First, we would like to refine the theoretical analysis and exhibit privacy properties that are more in line with the observed privacy; this would give us theoretical grounds to help us set parameters such as the clipping thresholds or the noise modulation. Second, want to expand our training scenario and address wider ranges of applications such as recommendation, federated learning, natural language processing. We also plan to spend efforts so as to mitigate the effect of clipping the fairness of our model 33

---

Acknowledgments
R.O. acknowledges funding from the Région Ilc-de-France. The authors would like thanks Kilian Muller and Gustave Pariente for fruitful discussions regarding the OPU. This work was granted access to the HPC/AI resources of IDRIS under the allocation 2021-A0101012429 made by GENCI. Specifically, we thank the team of the Jean Zay supercomputer for their support.
References
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308-318, 2016.
[2] Nazmiye Ceren Abay, Y. Zhou, Murat Kantarcioglu, B. Thuraisingham, and L. Sweeney, Privacy preserving synthetic data release using deep learning. In ECML/PKDD, 2018.
[3] G. Åcs, Luca Melis, C. Castelluccia, and Emiliano De Cristofaro. Differentially private mixture of generative neural networks. IEEE Transactions on Knowledge and Data Engineering,
3 11:1109-1221, 2019.
[4] Shahab Asoodeh, Jiachun Liao, Flavio P Calmon, Oliver Kosut, and Lalitha Sankar. Three variants of differential privacy: Lossless conversion and applications. arXiv preprint 0rXiv:2008.0653 2020.
[5] Eugene Bagdasaryan, Omid Poursaced, and Vitaly Shmatikov. Differential privacy has disparate impact on model accuracy. Advances in Neural Information Processing Systems, 32:15479- 15488, 2019.
[6] Borja Balle and Yu-Xiang Wang. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In Jennifer Dy and Andreas Krause, editors, Pro- ceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 394-403, Stockholmsmässaan Stockholm Sweden, 10-15 Jul 2018. PMLR.
[7] Bonnie Berger and Hyunghoon Cho. Emerging technologies towards enhancing privacy in genomic data sharing, 2019.
[8] Thee Chanyaswad, Alex Dytso, H Vincent Poor, and Prateck Mittal. Mvg mechanism: Differen- tial privacy under matrix -valued query. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 230-246, 2018.
[9] C. Dwork, F. McSherry, Kobbi Nissim, and A. D. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006.
[10] Cynthia Dwork. Differential privacy: A survey of results. In International conference on theory and applications of models of computation, pages 1-19. Springer, 2008
[11] Charlotte Frenkel, Martin Lefebvre, and David Bol. Learning without feedback: Fixed ran- dom learning signals allow for feedforward training of deep neural networks. Frontiers in neuroscience, 15, 2021.
[12] Xianxin Guo, TD Barrett, 7M Wang, and AT Lvovsky. End-to-end optical backpropagation for training neural networks. 2019.
[13] Sara Hooker. Moving beyond "algorithmic bias is a data problem"". Patterns, 2(4): 2021.
[14] Tyler W Hughes, Momchil Minkov, Yu Shi, and Shanhui Fan. Training of photonic neural networks through situ backpropagation and gradient measurement. Optica, 5(7)8864-771, 2018.
[15] Noah Johnson, Joseph P Near, and Dawn Song. Towards practical differential privacy for sql queries. Proceedings of the VLDB Endowment, 11(5):526-53,, 2018.
[16] Mikhail Konnik and James Welsh. High-level numerical simulations of noise in eed and cmos photosensors: review and tutorial. arXiv preprint arXiv: (al:::41244440 2014.
[17] Julien Launay, Iacopo Poli, François Boniface, and Florent Kızakala. Direct feedback alignment scales to modern deep learning tasks and architectures. arXiv preprint arXiv:2006. 108..,8000..2878, 2020.
10

---

[18] Julien Launay, lacopo Poli, Kilian Müller, Gustave Pariente, Igor Carron, Laurent Daudet, Florent Krzakala, and Sylvain Gigan. Hardware beyond backpropagation: a photonic co- processor for direct feedback alignment. Beyond Backpropagation Workshop, NeurlPS 2020, 2020.
[19] Jaewoo Lee and Daniel Kifer. Differentially private deep learning with direct feedback alignment. CoRR, abs/2010.03701, 2020.
[20]
LightOn. Photonic computing for massively parallel AI A White Paper, v1.0. https:// 2020.
White Paatte--aper--aa - v1. pd.ff May
[21] Timothy P Lillierap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synap- tic feedback weights support error backpropagation for deep learning. Nature communications, 1(1):1-10, 2016.
[22] I. Mironov. Rényi differential privacy. 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263-275, 2017.
[23] Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. In NIPS, 2016.
[24] Ruben Ohana, Jonas Wacker, Jonathan Dong, Sébastien Marmin, Florent Krzakala, Maurizio Filippone, and Laurent Daudet. Kernel computations from large-scale random features obtained by optical processing units. In ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 9294-9298. IEEE, 2020.
[25] Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable private learning with pate. arXiv preprint arXiv: 008.80000888,,. 2018.
[26]
Leandro Pardo. Staristical inference based divergence measures. CRC press, 2018.
[27]
Maria Refinetti, Stéphane d' d'Accoli, Ruben Ohana, and Sebastian Goldt. The dynamics of learning with feedback alignment. arXiv preprint arXiv:201 J. 1.124 2020.
[28] David E. Rumelhart, Geoffrey E. Hinton, and Ronald Williams. Learning representations by back-propagating errors. Nature, 323(6083):533-533, 1986.
[29] Alfréd Rényi. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium Mathematical Statistics and Probability, Volume 1: Contributions the Theory of Statistics, pages 547-561, Berkeley, Calif., 1961. University of California Press.
[30] Yu-Xi Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan. Subsampled rényi differential privacy and analytical moments accountant. In The 22nd International Conference Artificial Intelligence and Statistics, pages 1226-1235. PMLR, 2019.
[31] Gordon Wetzstein, Aydogan Ozcan, Sylvain Gigan, Shanhui Fan, Dirk Englund, Marin Soljačić, Comelia Denz, David AB Miller, and Demetri Psaltis. Inference in artificial intelligence with deep optics and photonics. Nature, 588(7836):39
2020.
[32] Han Xiao, Kashif Rasul, and Roland Vollgraf. fushion-mnist novel image dataset for benchmarking machine learning algorithms. arXiv preprini arXiv: 0777000077444 2017.
[33] Depeng Xu, Wei Du, and Xintao Wu. Removing disparate impact of differentially private stochastic gradient descent model accuracy. arXiv preprint arXiv:2003. (ixXi:2000303699, 2020.
[34] Xingyuan Xu, Mengxi Tan, Bill Corcoran, Jiayang Wu, Andreas Boes, Thach G Nguyen, Sai T Chu, Brent E Little, Damien G Hicks, Roberto Morandotti, al. 11 tops photonic convolutional accelerator for optical neural networks. Nature, 589(7840):44-51) 2021.


Reviews:
[{"review_id": "fxhwTyFH6V0", "reviewer": "Reviewer_bXri", "summary": "The manuscript entitled “Photonic Differential Privacy with Direct Feedback Alignment” extends their previous work - Differential Privacy with Direct Feedback Alignment. In this work, they leverage the intrinsic noise (which might be controlled by temperature) of optical random projections to build a differentially private DFA mechanism, making Optical Processing Units (OPUs) a solution of choice to provide a private-by-design training. This work is motivated by a photonic implementation that naturally induces noise that we exploit for differential privacy. The proposed method is demonstrated by simulations showing that adding noise does not decrease the performance significantly. In other words, this approach is robust to increasing DP noise. This paper also provides its theoretical analysis.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper has some significance in terms of making Photonic Differential Privacy with Direct Feedback Alignment suitable for protecting privacy in training by nature. I have a few comments and hope them can be addressed before its publication:\n\n1.\tOn Line 14 Page 4, what is the relation between χ and χ_n?\n\n2.\tIn definition 2.2 on Page 4, it says A : D ->Im A means Im A is the image of D through A, and O ∈ Im A. How can A(D) which is Im A belong to O, i.e. A(D) ∈ O?\n\n3.\tNeither Figure 1 nor Figure 2 are discussed in any section. The authors should discuss them in the corresponding sections.\n\n4.\tIn Figure 2, the authors use the degree of transparency of the lines to indicate the noise level. However, with more transparency, it becomes quite hard to distinguish the colors of the lines (e.g blue vs black). It will become even worse if we print the paper out. Figure 2 should be revised with other line styles.\n\n5.\tIn Table 1 Page 9, what is the difference between non-private and sigma=0?\n\n6.\tOn Line 215 Page 8, it should be ‘improved’ instead of ‘improve’. On Line 31-32 Page 8, the sentence should be rephrased to avoid grammar mistakes.\n\nIn summary, I recommend its publication in NeurIPS after those commends are addressed.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they are addressed adequately.", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "c-C2RGbNrTm", "reviewer": "Reviewer_BUbz", "summary": "The paper proposes a hardware-based approach to Differential Privacy (DP) using OPU to perform optical random projections for a differentially private DFA training algorithm, leveraging noise intrinsic to the hardware to achieve privacy-by-design. The proposed approach was analyzed mathematically in detail. The proposed methods were analyzed on the vanilla Fashion MNIST dataset.  \n\nThe paper is very well written in general. The proposed methods/approach are very interesting, and mathematical results are solid.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "One of the major issues with the paper is limited experimental analysis of the proposed methods in comparison with baseline DFA and state-of-the-art DP methods on different benchmark datasets. Although mathematical results are interesting, more detailed additional experimental results are needed to examine and justify its merit compared to the state-of-the-art in different tasks.\nAdditional comments and suggestions are provided as follows:\n\n1.\tPlease definite the notation and terms as they are used. For instance, please either define the notation used in (1) or just refer to (4).\n\n2.\tWhat do you mean by “train deep architectures”? I suppose that you refer training deep learning models whose functional structures are identified by the corresponding network architectures, instead of “designing network architectures using data”.\n\n3.\tCould you please provide additional analyses using more complicated datasets such as Cifar-10/100 or variations of ImageNet,  and networks, such as ResNets? Indeed, how would random projection of the error will be applied to networks with bottlenecks, skip connections and dense connections?\n\n4.\tPlease check the consistency of the notation and fix it in the text. For instance, please update the notation used for distribution D, and data spaces/sets X and Y according to that used in Section 2.3, vice versa.\n\n5.\tIn Definition 2.1, please define the record.\n\n6.\t In Theorem 1, please define the mechanism M_i, more precisely.\n\n7.\tPlease define | \\cdot |\n\n8.\tPlease define || || used at line 148, page 5.\n\n9.\tPage 5, line 162: Section 3.1\n\n10.\tAlg. 1, line 15: g_1^l,…,g_m^l\n\n11.\tPlease fix use of gaussian or Gaussian.\n\n12.\tPlease capitalize section names.\n\n13.\t In Proposition 13, what is the difference between .n_l\\alpha and n_l.\\alpha?\n\n14.\tPlease provide a complexity analysis of the proposed method in comparison with non-private and baseline DFA.\n\nAfter Rebuttal: Thank you for your response to my comments. I checked the comments of other reviewers and the response to their comments as well. The authors answered most of the questions in the rebuttal. Therefore, I increase my grade, while I consider that the paper should be improved for a clear acceptance, especially with additional experimental results.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please see the above comments.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "6t7LsY0ipk1", "reviewer": "Reviewer_MLtY", "summary": "This work proposes leveraging the intrinsic noise induced by optical processors performing matrix multiplication to provide differential privacy to neural networks trained with the direct feedback alignment algorithm. The authors demonstrate both theoretically and empirically the viability of this approach by training a neural network on an optical processing unit with direct feedback alignment and providing multiple levels of differential privacy noise.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work demonstrates what may have originally been considered a weakness of analog hardware to be a benefit in the context of data privacy. Furthermore, they demonstrate the real-world applicability of this approach by implementing the proposed learning procedure on an actual optical processing device. But details about the operations performed by the optical processing unit are not reported and what features about this device make it particularly well suited for the direct feedback alignment compared to other neural network learning procedures (most notably the backpropagation algorithm). There is some mention about the inherent parallelizability of the layerwise training with direct feedback alignment as compared to backpropagation, but this is generally a benefit to most neural network hardware solutions including CPUs/GPUs. Therefore, further justification about the chosen learning algorithm with the optical processing unit hardware should be expanded upon. It's also a little unintuitive why the optical processing is only used in the matrix multiplication of the DFA backward pass while a conventional GPU (nvidia v100) is used for the forward pass. Does this not limit the advantages of the proposed procedure since the conventional GPU creates a lower bound in training time?\n\nOverall, the work is an exciting demonstration of the viability of the direct feedback alignment algorithm to a novel hardware domain. But this excitement is somewhat muted by the lack of detail of why combining these two things makes sense and more detail in this respect would further improve this work.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Some suggestions for improvement:\n\n-Further detail on properties of the optical processing unit that make it particularly well suited for direct feedback alignment (the intrinsic noise justifies the differential privacy)\n\n-Some experiments beyond the FashionMNIST dataset would also be helpful to demonstrate whether the performance differences widen or narrow with dataset difficulty (possibly CIFAR10 and MNIST).\n\n-In the context of real-world privacy and what differential privacy implies, it would be helpful to provide a specific example use case. The formulations (Definition 2.2) imply that private information can still be present in the network and only the notion of plausible deniability of what data the model was trained on. This may be beyond the scope of this particular paper but would help make a more complete standalone work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper draws a connection between differentially private machine learning and noise injected by specialty analog hardware - specifically, optical processing units. This noise is normally a downside, but this submission shows that it may, in principle, be useful for privacy.

The reviewers agree that this is an interesting connection. There are some reservations about how well-developed this approach is - e.g. limited experiments, which are simulations rather than actual OPU hardware. However, the results seem acceptable as an initial proof-of-concept for this connection.
Decision: Accept (Poster)