Convergence and Alignment of Gradient Descent with Random Backpropagation Weights
Authors: Ganlin Song
Ruitu Xu
John Lafferty
Venue: NeurIPS 2021 Poster (2021)
Keywords: feedback alignment
deep learning

Abstract: Stochastic gradient descent with backpropagation is the workhorse of artificial neural networks. It has long been recognized that backpropagation fails to be a biologically plausible algorithm. Fundamentally, it is a non-local procedure---updating one neuron's synaptic weights requires knowledge of synaptic weights or receptive fields of downstream neurons. This limits the use of artificial neural networks as a tool for understanding the biological principles of information processing in the brain. Lillicrap et al. (2016) propose a more biologically plausible "feedback alignment" algorithm that uses random and fixed backpropagation weights, and show promising simulations. In this paper we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss. In the overparameterized setting, we prove that the error converges to zero exponentially fast, and also that regularization is necessary in order for the  parameters to become aligned with the random backpropagation weights. Simulations are given that are consistent with this analysis and suggest further generalizations. These results contribute to our understanding of how biologically plausible algorithms might carry out weight learning in a manner different from Hebbian learning, with performance that is comparable with the full non-local backpropagation algorithm.

Convergence and Alignment of Gradient Descent with Random Backpropagation Weights
Ganlin Song"
Ruitu Xu*
John Lafferty
Department of Statisties and Data Science
Wu Tsai Institute
Yale University
(ganlin.song ruitu. xu, Syffer edu
Abstract
Stochastic gradient descent with backpropagation is the workhorse of artificial neural networks. It has long been recognized that backpropagation fails to be a biologically plausible algorithm. Fundamentally, it non-local procedure- updating one neuron's synaptic weights requires knowledge of synaptic weights receptive fields of downstream neurons. This limits the use of artificial neu- ral networks as tool for understanding the biological principles of information processing in the brain. Lillicrap et (2016) propose a more biologically plausi- ble "feedback aligmment" algorithm that uses random and fixed backpropagation weights, and show promising simulations. In this paper we study the mathemati- cal properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss. In the overparameter- ized setting, prove that the error converges to zero exponentially fast, and also that regularization is necessary in order for the parameters to become aligned with the random backpropagation weights. Simulations are given that are consistent with this analysis and suggest further generalizations. These results contribute to our understanding of how biologically plausible algorithms might carry out weight learning in a manner different from Hebbian learning, with performance that is comparable with the full non-local backpropagation algorithm.
Introduction
The roots of artificial neural networks draw inspiration from networks of biological neurons (Rumel- hart et al., 1986a; Elman et al., 1996; Medler, 1998). Grounded in simple abstractions of membrane potentials and firing, neural networks are increasingly being employed
computational tool for better understanding the biological principles of information processing in the brain; examples in- clude Yildirim et al. (2019) and Yamins & DiCarlo (2016). Even when full biological fidelity is required, it can be useful to better align the computational abstraction with neuroscience principles.
Stochastic gradient descent has been a workhorse of artificial neural networks. Conveniently, cal- culation of gradients can be carried out using the backpropagation algorithm, where reverse mode automatic differentiation provides powerful way of computing the derivatives for general archi- tectures (Rumelhart et al., 1986b). Yet it has long been recognized that backpropagation fails to be biologically plausible algorithm. Fundamentally, it is non-local procedure upttating the weight between presynaptic and postsynaptic neuron requires knowledge of the weights between the postsynaptic neuron and other neurons. No known biological mechanism exists for propagating
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

information in this manner. This limits the use of artificial neural networks as tool for understand- ing learning in the brain.
A wide range of approaches have been explored as a potential basis for learning and synaptic plas- ticity. Hebbian learning is the most fundamental procedure for adjusting weights, where repeated stimulation by a presynaptic neuron that results in the subsequent firing of the postsynapic neuron will result in an increased strength in the connection between the two cells (Hebb, 1961; Paulsen & Sejnowski, 2000), Several variants of Hebbian learning, some making connections to principal com- ponents analysis, have been proposed (Oja, 1982; Sejnowski & Tesauro, 1989; Sejnowski, 1999). In this paper, our focus is on formulation of Lillicrap et al. (2016) based on random backprop- agation weights that are fixed during the learning process, called the "feedback alignment" (FA) algorithm. Lillicrap et al. (2016) show that the model can still learn from data, and observe the interesting phenomenon that the error signals propagated with the forward weights align with those propagated with fixed random backward weights during training. Direct feedback alignment (DFA) (Nokland, 2016) extends FA by adding skip connections to send the error signals directly to each hidden layer, allowing parallelization of weight updates. Empirical studies given by Launay et al. (2020) show that DFA can be successfully applied to train a number of modern deep learning mod- els, including transformers. Based on DFA, Frenkel et al. (2021) proposes direct the random target projection (DRTP) algorithm that trains the network weights with a random projection of the target vector instead of the emror, and shows aligmment for linear networks. Related proposals, including methods based on the use of differences of neuron activities, have been made in a series of recent papers (Akrout et al., 2019; Bellec et al., 2019; Lillicrap et al., 2020). A comparison of some of these methods is made by Bartunov et al. (2018).
The use of random feedback weights, which are not directly tied to the forward weights, removes issues of non-locality. However, it is not clear under what conditions optimization of error and learn- ing can be successful. While Lillicrap et al. (2016) give suggestive simulations and some analysis for the linear case, it has been an open problem to explain the behavior of this algorithm for training the weights of a neural network. In this paper, we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss. In the overparameterized setting, prove that the error converges zero exponentially fast. We also show, unexpectedly, that the parameters become aligned with the random backpropa- gation weights only when regularization is used. Simulations given that consistent with this analysis and suggest further generalizations. The following section gives further background and an overview of our results.
2 Problem Statement and Overview of Results
In this section we provide a formulation of the backpropagation algorithm to establish notation and the context for our analysis. We then formulate the feedback aligment algorithm that uses random backpropation weights. A high-level overview of our results is then presented, together with some of the intuition and proof techniques behind these results; we also contrast with what was known previously.
We mainly consider two-layer neural networks in the regression setting, specified by family of functions f TRd -> IR with input dimension d, sample size n, and p neurons in the hidden layer. For an input Rd the network outputs
1 Σ B.V(wTa)= Vp
1
f(x)
- BT (WW),, vp
(2.1)
where W (wi,..,, ) Roxd and ,3 (B1 .... pp) E R represent the feed-forward weights in the first and second layers, and a denotes an element-wise activation function. The scaling by /p simply for convenience in the analysis.
Given n input-response pairs {(x,, y:) Hi) the training objective is to minimize the squared error
n (y - (xx)))
C(W
(2.2)
2 1=1
2

---

Algorithm Feedback Alignment
Input: Dataset {(xi, yi H= step size
I: initialize W, 33 and as Gaussian
2: while not converged do
O
3: Gr 3r vp Di=1 exb(wxx))
4: W0 + wr
= Ei= (wTTaa
vD
5: for r E [p]
W
by
6: end while
Standard backpropagation updates the first layer weights for hidden node with the second layer feedforward weight B,. We study the procedure where the error is backpropagated instead using a fixed, random weight b,-
Figure 1:
Standard gradient descent attempts to minimize (2.2) by updating the feed-forward weights follow- ing gradient directions according to
11 ac (W),,, 3()))
03,
OC
(1++) Dw,
(W),), B()))
for each r E [p]. where > 0 denotes the step size. We initialize ((0) and (0)(0) as standard
Gaussian vectors. We introduce the notation e(t) E R", with fi(() [(x;) denoting the network output on input v; when the weights are (()) and 3(t), and e4(() = fi(() denoting the corresponding prediction error residual. With this notation, the gradients are expressed
dC 1 Σ eeb(wTT)), 0
Oc 1
BaBr (wFT)w
Sw,
BC involves not only the local input Here it is seen that the the gradient of the first-layer weights Twe I; and the change in the response of the r-th neuron, but also the backpropagated error signal e,8, The appearance of 3, is, of course, due the chain rule; but in effect it requires that the forward weights between layers are identical to the backward weights under error propagation. There is no evidence of biological mechanisms that would enable such "synaptic symmetry."
In the feedback alignment procedure of (Lillicrap et al., 2016), when updating the weights wy, the er- signal is weighted, and propagated backward, not by the second layer feedforward weights 3, but rather by a random set of weights E R" that are fixed during the course of training. Equivalently, the gradients for the first layer are replaced by the terms
dC Ow,
VP Σ
1
(2.3)
Note, however, that this update rule does not correspond to the gradient with respect modified loss function. The use of a random weight b, when updating the first layer weights w, does not vio- late locality, and could conceivably be implemented by biological mechanisms; refer Lillicrap et al. (2016); Bartunov et al. (2018); Lillicrap et al. (2020) for further discussion. A schematic of the relationship between the two algorithms is shown in Figure
We can now summarize the main results and contributions of this paper. Our first result shows that the error converges to zero when using random backpropagation weights.
Under Gaussian initialization of the parameters, if the model sufficiently over- parameterized with p > n, then the error converges to zero linearly. Morcover, the pa- rameters satisfy ([,,(t) (t) w. (0)] Ō "p and [t) (0) -())) VT
The precise assumptions and statement of this result are given in Theorem 3.2. The proof shows in the over-parameterized regime that the weights only change by small amount. While related to
3

---

results for standard gradient descent, new methods are required because the "effective kernel" is not positive semi-definite.
We next turn to the issue of alignment of the second layer parameters / with the random back- propagation weights b. Such alignment was first observed in the original simulations of Lilli- crap et al. (2016), With h E Ie denoting the hidden layer of the two-layer network, the term DC 1
->= e represents how the error signals e, are sent backward to update the feed-forward weights. With the use of random backpropagation weights, the error is instead propagated backward as òFA (h) = >> Ei.
(h) = 8h
Lillicrap et al. (2016) notice decreasing angle between ÓвP (B) and ÓFA (h) during training, which sufficient condition to ensure that the algorithm converges. In the case of k-way classification, the last layer has k nodes, /3 and b are p x k matrices, and each error term e₁ is a kevector. In the regression setting,
so the angle between ÓBP (h) and ōFA (h) the same as the angle between 3 and b. Intuitively, the possibility for alignment is seen in the fact that while the updates for W use the error weighted by the random weights b, the updates for 33 indirectly involve W, allowing for the possibility that dependence on b will be introduced into B.
Our first result shows that, in fact, alignment will occur in the over-parameterized setting. (So, while the error may still converge, "feedback alignment" may be bit of a misnomer for the algo- rithm.)
*
The cosine of the angle between the p-dimensional vectors ōFA and dвP satisfies cos (5pA, (t) = cos (bb, (()) = 0 n
However, we show that regularizing the parameters will cause dBp align with ōPA and therefore the parameters 3 to align with b. Since (()) and b are high dimensional Gaussian vectors, they are nearly orthogonal with high probability. The effect of regularization can be seen as shrinking the component of ((0) in the parameters over time. Our next result establishes this preciscly in the linear case.
Supposing that (u) u. then introducing ridge penalty ((1) where A(t) X for S T and X(t) = 0 for > on / causes the parameters to align, with cos Z(b, (t)) 2
c 0 for sufficiently large t.
The technical conditions are given in Theorem 4.6. Our simulations are consistent with this result, and also show alignment with a constant regularization X(I) = X, for both linear and nonlinear activation functions. Finally, we complement this result by showing that convergence is preserved with regularization, for general activation functions. This is presented in Theorem 4.2.
3 Convergence with Random Backpropagation Weights
Due to the replacement of backward weights with the random backpropagation weights, there is no guarantee a priori that the algorithm will reduce the squared error loss C. Lillicrap et al. (2020) study the convergence on two-layer linear networks in a continuous time setting. Through the analysis of a system of differential equations on the network parameters, convergence to the true linear target function is shown, in the population setting of arbitrarily large training data. Among recent studies of aver-parametrized networks under backpropagation, the neural tangent kernel (NTK) is heavily utilized to describe the evolution of the network during training (Jacot et al., 201 8; Chen & Xu, 2020). For any neural network (a, 0) with parameter 0, the NTK defined as
( 9) 0f(z,0) af(y.0)
00
00
Given a dataset {( Po we can also consider its corresponding Gram matrix K (Ky (2,, (2), (xy)))xnn Jacot et al. (2018) show that in the infinite width limit, K2 converges to a con- stant at initialization and does not drift away from initialization throughout training. In the over- parameterized setting, if the Gram matrix K is positive definite, then K will remain close to its ini- tialization during training, resulting in linear convergence of the squared error loss (Du et al., 2018, 2019; Gao & Lafferty, 2020). For the two-layer network f(x, 0) defined in (2.1) with 0 = (8, W),

---

the kernel K, can be written in two parts, G and H,, which correspond to 3 and W respectively:
GG (1 (ff(z,0) 0) af(y,0)
@f(r, 0) of(y,0) Sw,
03
03
Ow,
Under the feedback alignment scheme with random backward weights b, Gy remains the same as for standard backpropagation, while one the gradient terms Bwr
af in Hz changes to 0f(r.0) Owr
vp bya (w,, æ),,, with H replaced by ; positive semi-definite and close to at initialization the network is over-parameterized. How- ever, if G (Gj (z,, xz)))xx is positive definite and H ( mz))) remains small during training, we are still able to show that the loss & will converge to zero exponentially fast.
= - @f(a..0) af(v.0) As result, Hy is no longer Ow,
Assumption 3.1. Define the matrix G E R with entries Gi.- = Ew.,,( (w (w))(()))))))
Then we assume that the minimum eigenvalue satisfies min (G) y. where y is positive constant.
Theorem 3.2. Let (O), 3(0) and b have ii.d standard Gaussian entries. Assume (1) Assump- tion 3.1 holds, (2) vb is smooth, sb, v and v²" are bounded and (3) Wil and ||xe|] are bounded for all E [n]. Then there exists positive constants et cz C2 and C2. such that for any 8 (0, 1), if
2 max (Ci
n² 5,p C2 " loge) then with probability at least 1 - we have that
7t
||e(t + 1)11 S (1- 17 |e(()]l
(3.1)
and
[|w,((t) wo (0)l S Ci
nVlogg
(B,(1) - 3,(0)) s c2
n 7
(3.2)
T
for all r E [p] and t 1> 0.
We note that the matrix G in Assumption 3.1 is the expectation of G with respect to the random initialization, and is thus close to G due to concentration. To justify the assumption, we provide the following proposition, which states that Assumption 3.1 holds when the inputs z; are drawn independently from a Gaussian distribution. The proofs of Theorem 3.2 and Proposition 3.3 are deferred to ??.
Proposition 3.3. Suppose Ii,...,n, In. i.dd N(O, La/d) and the activation function vo is sigmoid or tanh. If = (n)) then Assumption 3.1 holds with high probability.
4 Alignment with Random Backpropagation Weights
The most prominent characteristic of the feedback alignment algorithm is the phenomenon that the error signals propagated with the forward weights align with those propagated with fixed random backward weights during training. Specifically, if we denote h E R2 to be the hidden layer of the ac network, then we write (BP(h) = ah to represent the error signals with respect to the hidden layer that are backpropagated with the feed-forward weights and ōFA (h) as the error signals computed with fixed random backward weights. In particular, the error signals dвP (h) and SpA for the two-layer network 12 are given by
1
1
a
SBr (h)
e and 8PA (N)
c
Lillicrap et al. (2016) notice decreasing angle between dpp (h) and 8Px (h) during training. We formalize this concept of alignment by the following definition.
Definition 4.1. We say two-layer network aligns with the random weights during training if there exists a constant e > 0 and time T. such that cos Z(ōPA, (BP())) = cos L(b, ((t) (b,31)) 2 c
for all t > Tc.
5

---

4.1 Regularized feedback alignment
Unfortunately, alignment between (t) and b is not guaranteed for over overpaarameterizen networks and the loss (2.2). In particular, we control the cosine value of the angle by inequalities (3.2) from Theorem 3.2, i.e.,
K Mi ((O)) (8(0) 3(0)
cos <(b, ((t)
O
n
|[(0)] (18(t) 3(0)
which indicates that (t) and b become orthogonal as the network becomes wider. Intuitively, this can be understood as resulting from the parameters staying near their initializations during training when p is large, where (()) and 6 are almost orthogonal to each other. This motivates regularize the network parameters. We consider in this work the squared error loss with an & regularization term on B:
2 Σ (s(z)) - v:) + 1 2
(t, W3 B)
(4.1)
where {A(() a sequence of regularization rates, which defines a series of loss functions for different training steps f. Thus, the update for w,, remains the same and the update for / changes to
= (1 (1-)) 7 Σ (t)))(,, for E [p].
Comparing to Algorithm 1, an extra contraction factor 1 - A(t) is added in the update of (()), which doesn't affect the locality of the algorithm but helps the alignment by shrinking the component of ((0) in ((t).
Following Theorem 3.2, provide error bound for regularized feedback alignment in Theo- rem 4.2. Since regularization terms X(t) make additional contributions to the error e(t) as well as to the kernel matrix G, upper bound Σi20 X(t) is needed to ensure positivity of the minimal eigenvalue of G during training, in order for the error e(t) be controlled. In particular, if there is no regularization, i.e., X(t) = 0 for all t 2 O, then we recover exponential convergence for the error e(t) as in Theorem 3.2. The proof of Theorem 4.2 is also deferred ??.
Theorem 4.2. Assume all the conditions from Theorem 3.2. Assume ΣE A(t) S 3x =
7²v/
čS ym Vlogp" for some constant čz. Then there exist positive constants C1 and Cz. such that for
any 8 E (0,1), if 2 max (C1 85z; C2 "0 xx
(olee)) then with probability at least 1 5, we have
||e(t + 1)1l < (1- "Y
(P((t))le(( + A(1)llill
(4.2)
4
for all t 2 O.
4.2 Alignment analysis for linear networks
In this section, we focus the theoretical analysis of alignment for linear networks, which is equiv- alent to setting the activation function &p to the identity map. The loss function can be written as
(t. W. B)
1 1 _
1
XWT3 w11²
X(t) 11113
where X (ai...... N. T; this is a form of over-parameterized ridge regression. Before present- ing results alignment, we first provide linear version of Theorem 4.2 that adopts slightly different conditions.
M for some constants M > > 0, and (2) - < Sx = cs nyTM for some constant cs. Then for any 8 (0, 1), if p 22 (ddoo///))), the following inequality holds for all t 2 with probability at 7
Theorem 4.3. Assume (1) Ily/l @(vn), Amia Amin (XXT) > r and >max
Ty5P
least 1 8:
||e(t + 1) $ 1(1- 2
17 +x(t))]l(tt]l++>)) +X0)lllll
(4.3)

---

We remark that in the linear case, the kernel matrix G reduces to the form XWTWXT WXT and its expectation G initialization also reduces to X.X.. Thus, Assumption 3.1 holds if X positive definite, which is equivalent to the 24's ''s being linearly independent. The result of Theorem 4.2 can not be directly applied to the linear case since we assume that şi bounded, which is true for sigmoid or tanh but not for the identity map. This results in a slightly different order for Sx and an improved order for p.
Our results on alignment also rely on an isometric condition on X, which requires the minimum and the maximum eigenvalues of XXT be sufficiently close (cf. Definition 4.4). On the other hand, this condition is relatively mild and can be satisfied when X has random Gaussian entries with a gentle dimensional constraint, as demonstrated by Proposition 4.5, Finally, we show in Theorem 4.6 that under a simple regularization strategy where constant regularization is adopted until a cutoff time T, regularized feedback alignment achieves alignment if X satisfies the isometric condition.
Definition 4.4 ((7, -Isommerr)) Given positive constants 7 and e, we say X is (7,,) -isommerii if Amin (X T) 2 7 and Amax (XXT) S (1++ (1+e)7.
Proposition 4.5. Assume X E 1axd has independent entries drawn from (0. 1/d). For any (0, 1/2 and 5 E (0, if 2 log 8 + n 1og log!), then X is e, -ismmmerte with probability 5.
Theorem 4.6. Assume all conditions from Theorem 4.3 hold and X is (7, -iisometrii with small constant e. Let the regularization weights satisfy
X T 0. t>
x(t)
with X = In and T [Sx/A] for some large constant T.. Then for any 5 E (0, if ((dlog(d/)), with probability at least 6, regularized feedback alignment achieves alignment. Specifically, there exist positive constant c c$ and time To such that cos L(b, ((t)) 2 c ffr all t Te
We defer the proof's of Proposition 4.5, Theorem 4.3 and Theorem 4.6 to ??. In fact, we prove Theorem 4.6 by directly computing 3(t) and the cosine of the angle. Although b doesn' 't show up in the update of (3, it can still propagate to / through W. Since the size of the component of b in 3(t) depends on the inner-produce (e(t), e(t')) for all previous steps t' < t,, the norm bound (4.3) from Theorem 4.3 is insufficient; thus, more careful analysis of e(f) is required.
We should point out that the constant in the lower bound is independent of the sample size n, input dimension d, network width p and learning rate 7/. We also remark that the cutoff schedule of X(l) is just chosen for simplicity. For other schedules such as inverse-suuared decay or exponential decay, one could also obtain the same alignment result as long as the summation of Att) is less than Sx.
Large sample scenario. In Theorems 4.3 and 4.6, we consider the case where the sample size is less than the input dimension d, so that positive definiteness of XXX can be established. However, both results still hold for n >> d. In fact, the squared error loss L can be written
(((x))- - 22)
XW -y1 -''
1 vp
XW 3 5| 18
vl
where ÿ denotes the projection of y onto the column space of X. Without loss of generality, we assume y = y. As result, and the columns of X are all in the same d-dimensional subspace of R" and X is positive definite on this subspace, as long as X has full column rank. Consequently, we can either work on this subspace of R" or project all the vectors onto Rd, and the isometric condition revised to only consider the nonzero eigenvalues of XXT.
Simulations
Our experiments apply the feedback alignment algorithm to two-layer networks, using a range of networks with different widths and activations. The numerical results suggest that regularization essential in achieving alignment, in both regression and classification tasks, for linear and nonlinear models. We implement the feedback alignment procedure in Py PyToch as an extension of the autograd module for backpropagation, and the training is done on V100 GPUs from internal clusters.
7

---

Feedback alignment on synthetic data. We first train two-layer networks on synthetic data, where each network shares the architecture shown in (2.1) and the data are generated by an- other network Jo that has the same architecture but with random Gaussian weights. We present the experiments for both linear and nonlinear networks, where the activation functions are chosen to be Rectified Linear Unit (Rel.U) and hyperbolic tangent (Tanh) for nonlinear case. We set training sample sample size to n 50 and the input dimension d = 150, but vary the hidden layer width p 100 x 2 with & E [7]. During training, take step size 10 for linear networks and 7 = 10-3, 10 for ReLU and Tanh networks, respectively.
0.8
0.4
Alignment
0.6
Aligmmett
Align mentt
0.4
Regibastociin
03
0.3
0.5
0.0
60
Iff
UP
13a
10*
10f
Hidden Layer Width
y Hidden Layer Width
Hidden Layer Width
(a) Alignment on linear network.
(b) Alignment on ReLU network.
(c) Alignment on Tanh network.
322
10"4
Lo85
1013
10
Loss
16-
Repularizatinn
Regelarzatton
Regularization on
0.0
0.0
0.2
0.00
0.021
0.5
0.622
1000
5003
2000
1000
5000
1008
3000
2000
5000
2003
1000
5000
3300
3000
2000
Step
Step
Stop
(d) Loss on linear network.
(c) Loss on ReLU network.
(f) Loss on Tanh network.
Figure 2: Comparisons of alignment and convergence for the feedback alignment algorithm with different levels of lg regularization. In Figs. 2a to 2c, the data points represent the mean value computed across simulations, and the error bars mark the standard deviation out of 50 independent runs. In Figs. 2d to 2f, we show the trajectories of the training loss for networks with p 3200, with the shaded areas indicating the standard deviation over 50 independent runs. The z-axes on the first row and the y-axes on the second row are presented using a logarithmic scale.
In Figs. 2a to 2c, we show how alignment depends on regularization and the degree of overparam- eterization as measured by the hidden layer width p. Alignment is measured by the cosine of the angle between the forward weights 3 and backward weights b. We train the networks until the loss function converges; this procedure is repeated 50 times for each p and X. For all three types of net- works, as increases, alignment vanishes if there is no regularization, and grows with the level of regularization \ for the same network. We complement the alignment plots with the corresponding loss curves, where the training loss converges slower with larger regularization. These numerical results are consistent with our theoretical statements. Due to the regularization, the loss converges to positive number that is of the same order as X.
We remark that using dropout forward and backward weights (Wager et al., 2013). However, our numerical results suggest that dropout regularization fails to keep the alignment away from zero for networks with large hidden layer width. No theoretical result is available that explains the underlying mechanism.
form of regularization can also help the alignment between
Feedback alignment on the MNIST dataset. The MNIST dataset is available under the Creative Commons Attribution-Sha Alike 3.0 license (Deng, 2012). It consists of 60,000 training images and 10,000 test images of dimension 28 by 28. We reshape them into vectors of length d = 784 and
8

---

Regularization
0.9
0.0
0.1
0.8
0.5
0.3
0.7
Alignment
0.4
Accuracy
0.6
0.3
0.5
0.2
0.3
Regularization X
0.0
0.1
0.0
0.3
0.1 5000
10000
15000
20000
25000
30000
10000
1,0000
20000
25000
Step
Step
Figure 3: Comparisons alignment and accuracy for feedback alignment algorithm with À
0, 0.1, 0.3. The left figure shows alignment defined by cos ((pp (h), 8P^ (h)), and right figure shows the accuracy on the test set. The dashed lines and corresponding shaded areas represent the means and the standard deviations over 10 runs with random initialization.
normalize them by their mean and standard deviation. The network structure is 784-1000-10 with ReLU activation the hidden layer and with softmax normalization at output layer. During training, we choose the batch size to be 600 and the step size 7 10 2 The training procedure uses 300 epochs total. We repeat the training 10 times for each choice of X.
Fig. 3 shows the performance of feedback alignment with regularization À = 0, 0.1,0.1..00 0.3. Since the output of the network is not one-dimensional but 10-dimensional, the alignment by cos (ÖBp (h), ŌFA (4)). where ÓBP (h) is the error signal propagated to the hidden neurons through forward weights and (h) the error weighted by the random backward weights b. We observe that both alignment and convergence are improved by adding regularization to the training, and increasing the regularization level can further facilitate alignment, with a small gain test accuracy.
measured
Discussion
In this paper we have analyzed the feedback alignment algorithm of Lillicrap al. (2016), showing convergence of the algorithm. The convergence is subtle, as the algorithm does not directly minimize the target loss function; rather, the error transferred the hidden neurons through random weights that do not change during the course of learning. The supplement to Lillicrap et al. (2016) presents interesting insights on the dynamics of the algorithm, such as how the feedback weights act as pseudoinverse of the forward weights. After giving analysis of convergence in the linear case, the authors state that "a general proof must be radically different from those used to demonstrate convergence for backprop" (Supplementary note 16), observing that the algorithm does not minimize any loss function. Our proof of convergence in the general nonlinear case leverages techniques from the use of neural tangent kernel analysis in the over paraametrreeeee setting, but requires more care because the kernel is not positive semi-definite at initialization. In particular, as a sum two terms G' and H. the matrix G is concentrated around its postive-definite mean, while H is not generally postive-semidefinite However, we show that the entries of both matrices remain close to their initial values, due to over- parameterization, and analyze the error term in Taylor expansion, which establishes convergence.
In analyzing alignment, we unexpectedly found that regularization is essential; without it, the align- ment may not persist as the network becomes wider, as our simulations clearly show. Our analysis in the linear case proceeds by essentially showing that
B(t) (1 -))=-13(0)) 7 ((0) a -1)+ (1) bag 1) VP

---

and controlling a1 while showing that remains sufficiently large; the regularization kills off the first term. Although we see no obstacle, in principle, carrying out this proof strategy in the nonlinear case, the calculations are more complex. While convergence requires analysis of the norm of the error, alignment requires understanding the direction of the error. But our simulations strongly suggest this result will go through.
In terms of future research, technical direction is extend our results to multilayer networks. would be interesting to explore local methods to update the backward weights b, rather than fixing them, perhaps using a Hebbian update rule in combination with the forward weights W. More generally, it is important to study other biologically plausible learning rules that can be implemented deep learning frameworks at scale and without loss of performance. The results presented here offer support for this as fruitful line of research. Biologically plausible computational learning contributes to, and shares societal impact with, a large body of fundamental research that aims to understand the basis for cognition in animals, including humans.
Acknowledgments
Research supported in part by NSF grant CCF- 1839388
References
Akrout, M.. Wilson, C., Humphreys, P., Lillicrap, T., & Tweed, D. B. (2019). Deep learning with- out weight transport. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, & R. Garnett (Eds.), Advances in Neural Information Processing Systems, volume 32: Curran Associates, Inc.
Bartunov, S., Santoro, A., Richards, B., Marris, L., Hinton, E.E., & Lillicrap, T. (2018). Assessing the scalability of sooloically-motiiated in Neural Information Processing Systems (pp. 9368 93788..
motivated deep learning algorithms and architectures. In Advances
Bellec, G., Scherr, F., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2019). Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets.
Chen, L. & Xu, S. (2020). Deep neural tangent kernel and laplace kernel have the same rkhs. arXiv preprint arXiv:2009. 10683.
Deng, L. (2012). The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6), 141-142.
Du, S., Lee, J., Li, H., Wang, L., & Zhai, X. (2019). Gradient descent finds global minima of deep neural networks. In International Conference Machine Learning (pp. 1675-1685).: PMLR.
Du, S. S., Zhai, X., Poczos, B., & Singh, A. (2018). Gradient descent provably optimizes over- parameterized neural networks. arXiv preprint 1rx10:1810.02054.
Elman, I.,, Bates, F. A., Johnson, M. H., Annette Karmiloff-Smith, D. P., & Plunketr, K. (1996). Rethinking Innateness: A connectionist perspective on developmens. Cambridge MA: MIT Press.
Frenkel, C., Lefebvre, M., & Bol, D. (2021). Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks. Frontiers in neuroscience, 15.
Gao, C. Lafferty, (2020). Model repair: Robust recovery of over-parameterized statistical models. arXiv preprint arXiv:2005.099 1291....................
Hebb, D. O. (1961). Distinctive features of learning in the higher animal. J. F. Delafresnaye (Ed.), Brain Mechanisms and Learning: London: Oxford University Press.
Jacot, A., Gabriel, F., & Hongler, C. (2018). Neural tangent kernel: Convergence and generalization in neural networks. arXiv preprint arXiv:1806.077. 07572.
Launay, J., Poli, I., Boniface, F., Krzakala, F. (2020). Direct feedback alignment scales to modern deep learning tasks and architectures. arXiv preprint 27Xiv:2006.12876
10

---

Lillicrap, T. P.. Cownden, D., Tweed, D. B., & Akerman, C. J. (2016). Random synaptic feedback weights support eπror backpropagation for deep learning, Nature communications, 7(1), 1-10.
Lillicrap, P. Santoro, A., Marris, L., Akerman, J. Hinton, G. (2020). Backpropagation and the brain. Nature Reviews Neuroscience, 21(6), 335-346.
Medler, D. A. (1998). A brief history of connectionism. Neural Computing Surveys, 1, 61-101.
Nøkland, A. (2016). Direct feedback alignment provides learning in deep neural networks, arXiv preprint
Oja, E. (1982). A simplified neuron model as principal component analyzer. J. Mathematical Biology, 15, 267-273.
Paulsen, & Sejnowski, J. (2000). Natural patterns of activity and long-term synaptic plasticity. Current Opinion in Neurobiology, 10(2), 172-179.
Rumelhart, D., McClelland, J., & the PDP Research Group (1986a). Parallel Distributed Process- ing: Explorations the Microstructure of Cognition, volume 2: Psychologcal and Biological Models. Cambridge, Massachusetts: MIT Press.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986b). Learning representations by back- propagating errors. Nature, 323(6088), 533-536.
Sejnowski, T. J. (1999). The book of Hebb. Neuron, 24, 773-776.
Sejnowski. T. J. & Tesauro, G. (1989). The hebb rule for synaptic plasticity: Algorithms and imple- mentations. In J. H. Byrne & W. O. Berry (Eds.), Neural Models of Plasticity (pp. 94-103).
Wager, S., Wang. S., & Liang, P. (2013). Dropout training as adaptive regularization. arXiv preprint 100.:1307.4999
Yamins, D. & DiCarlo, J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience, 19(3), 356-365.
Yildirim, I., Wu, J.. Kanwisher, N., & Tenenbaum, J. (2019). An integrative computational archi- tecture for object-driven cortex. J.B. Current Opinion in Neurobiology.


Reviews:
[{"review_id": "_bGejnAdhj", "reviewer": "Reviewer_giv6", "summary": "This paper studies the convergence of Feedback Alignment, and proves that in the over-parameterized setting the error converges to zero exponentially fast. This is made possible by taking inspirations from recent work on the NTK, albeit with specific challenges as some quantities are not obviously positive semi-definite. Furthermore, it makes the somewhat surprising finding that regularization helps alignment, proving this in linear networks, and confirming this finding in simulations on Gaussian data and MNIST.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*For ease of answering, I have annotated my various points with O.1/Q.1/etc...*\n\nThis is a solid and very well written paper, with a clear contribution to the field of alternative training methods. \n\nThe paper could be better positioned in the existing literature, in particular by mentioning all the works that has been done around Direct Feedback Alignment, such as Refinetti et al., 2020. Moreover, I think the surprising finding around regularization & alignment should be discussed more in depth.\n\nAccordingly, **this paper currently stands marginally above the acceptance threshold (6)**. If the two points above were to be addressed, I would be willing to increase my score to a clear accept (8).\n\n### Originality\n\nThe paper is an interesting application of NTK methods to an open-problem in alternative training methods, the convergence of Feedback Alignment. It is well placed in the immediate literature, but misses a number of important papers on the theory of such methods.\n\n**O.1**: Notably, the authors completely ignore the large body of literature around Direct Feedback Alignment (Nøkland, 2016). DFA is an extension of FA, where the random projection of the error is directly sent to each layer, enabling parallelization of weight updates after the loss is calculated. In particular, Refinetti et al., 2020 have done an extensive analysis of the dynamics of convergence & alignment in DFA. This analysis brings a number of important findings around alignment, which it would be interesting to see discussed and compared with the work here done. Furthermore, Frenkel et al. 2019 have also provided a theoretical analysis for a variant of DFA, Direct Random Target Projection. To go a bit further, this could also lead to bridging the gap between some of the theoretical findings made here and the real-world behaviors of FA/DFA: open questions remain, such as FA/DFA failing on convolutions (Bartunov et al., 2018) but working in many other architectures (Launay et al., 2020). A discussion of the impacts of the findings made in this paper in terms of our general understanding of FA/DFA would be very valuable.\n\n### Quality\n\nThe theoretical contributions are sound and well backed. The experiments to confirm the findings around regularization are interesting, although I think the authors do not go in depth enough in their analysis. \n\n**Q.1**: In results both on synthetic data and MNIST (Figure 2 & Figure 3), it is interesting to see that despite lower alignment in non-regularized nets, end-task performance is not necessarily improved by higher alignment values (the lambda = 0.0 of Figure 3 probably overfits). This warrants a more in depth discussion. Could the higher alignment values be an artifact of the weights being pushed closer to zero? (This is noted by the authors l185.) This would motivate disentangling the components of alignment due to L2 regularization, and the \"true alignment\" of the raw gradients. This would be really interesting to measure and plot, and could provide further elements to understand this \"surprise\" of regularization helping alignment. The authors also briefly mention experiments with dropout, highlighting that a study of other regularization methods (dropout, etc.) could also prove interesting.  \n\n### Clarity\n\nThe paper is well written and the mathematical notations clear. The structure is also good, making it an enjoyable paper to read. \n\n### Significance\n\nThe findings of the authors are interesting, and contribute to a broader understanding of feedback alignment methods. \n\nI do feel that the addition of a discussion of the potential impacts of these findings on the current understand of FA/DFA, as well as a more in depth discussion around the \"surprising\" regularization finding would help make this paper more impactful for the community. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors accurately describe the limitations and potential impact of their work. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "S1BVuzYpOZx", "reviewer": "Reviewer_9xDE", "summary": "The paper uses analysis techniques from Neural Tangent Kernels to prove that the feedback alignment algorithm converges in neural networks with a single non-linear hidden layer. In addition, they prove that the forward weights do not converge to alignment with the random backward weights. \n\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a well-written paper that appears to make a significant contribution to the understanding of the feedback aliignment / random backpropagation. Previous work has shown that this algorithm converges for special cases (deep chains of linear neurons, etc.), but this proof adresses a much more general case. However, I was not able to check the proof due to time constraints.\n\nThe second result about the weights failing to align is less surprising, but still interesting. A recent ICML paper by Refinetti, et al. \"Align, then memorise: the dynamics of learning with feedback alignment\" provides an analysis that is complementary to the results presented here. \n\nEDIT after author rebuttal: The reviewers all agree that this is a well-written paper. I maintain that the contribution is significant enough for acceptance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately address this issue.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "Fsb5hg2ctdL", "reviewer": "Reviewer_B3Jb", "summary": "This paper gives a theoretical analysis of feedback alignment (FA), an algorithm\nto train neural networks by approximating the gradient of the loss function\nusing random matrices. More precisely, the weights of the network that is being\ntrained are replaced with random matrices in the back-propagation step.\n\nThe authors study two-layer neural networks in the over-parametrised regime,\nwhere the number of samples n is much larger than the number of neurons in the\nhidden layer, n, and provide two results:\n\n1. A proof of linear convergence of the loss to 0. This proof follows the proofs\n   of Du et al. (2018), Gao & Lafferty (2020) for standard backprop. The added\n   difficulty of analysing DFA in this setting is that the effective kernel of\n   the network is not a priori positive semi-definite.\n\n2. An analysis of the alignment between the second-layer weights of the network\n   and the feedback matrix used in the feedback alignment algorithm, which\n   highlights the important role of regularisation and finds that (the role of) \n   alignment is more complex than previously thought.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths\n\n- The topic is timely - there has been increasing interest in feedback alignment\n  methods and alternatives to back-prop more broadly, as evidenced by the\n  NeurIPS 2020 workshop on this very topic and a number of recent papers (see below). However, an understanding of the\n  power of these alternatives to backprop remains an open problem, so I see a\n  need for theoretical work in this direction, which the paper addresses.\n- The authors give clear and convincing mathematical arguments for two key\n  problems of FA, namely convergence despite training on a surrogate loss, and\n  alignment of the weights to the feedback vectors.\n- The authors clarify the mechanics of alignment between weights and feedback\n  matrix, which has been conjectured to be key to the success of the method.\n- The paper is well-written: the exposition is clear, and I found that the \n   main arguments were explained clearly.\n\n# Weaknesses\n\nWhile I found the paper insightful and the results interesting, the discussion\nin its present version ignores several papers\nthat have recently provided analysis of feedback alignment algorithms. A discussion of the\nfollowing two theoretical papers, and how the results presented here relate to their results,\nseems particularly appropriate:\n\n- Regarding alignment, Frenkel et al. [Fre19] showed that using only the error sign is sufficient to\n  maintain feedback alignment and to provide learning in the hidden layers of\n  *linear* networks.\n- Refinetti et al. [Ref21] analysed the dynamics of FA and DFA for two-layer\n  non-linear networks in the feature learning regime.\n\nFurthermore, \n- the definition of alignment that the authors use (Def. 4.1) has been discussed\n  extensively in the literature, cf. [Cra19, Fre19, Ref21]. This should be\n  acknowledged.\n- ...it might be worth mentioning that despite the problem with convolutions that\n  the authors mention, DFA has recently been used to train modern neural\n  networks such as Transformers [Lau20]\n\nLet me also note that an important variant of feedback alignment is the direct feedback alignment of\n  Nokland [Nok16], where the error vector is injected into each of the layers\n  directly using a random matrix. For two-layer neural networks, the resulting\n  update equations are the same, so the authors might want to mention that their results\n  cover this relevant case, too.\n\n# Some small comments\n\n- What does the \\tilde O notation mean? (l. 94)\n- In the definition of the NTK after l. 137, the average <.> is taken w.r.t. to\n  which distribution?\n\n# References\n\n- [Cra19] Crafton et al, Frontiers in neuroscience (2019). http://arxiv.org/abs/1903.02083\n- [Fre19] Frenkel et al, (2019) http://arxiv.org/abs/1909.01311\n- [Lau20] Launay et al. (2020) http://arxiv.org/abs/2006.12878\n- [Nok16] Nøkland, NeurIPS 2016. http://arxiv.org/abs/1609.01596\n- [Ref21] Refinetti et al, ICML 2021. https://arxiv.org/abs/2011.12428\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- The authors clearly address the range of validity of their theoretical claims. \n- Since this is a theoretical work, there are no immediate societal impacts that need to be addressed imho.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "EkUVQh_tdrV", "reviewer": "Reviewer_4iby", "summary": "The paper provides a theoretical analysis of the feedback alignment algorithm (FA), a biologically plausible approximation to backpropagation. The authors consider a two-layered network, and derive training error bounds and error alignment for a variety of cases, using the neural tangent kernel method. The results show that FA-trained network can achieve zero training error for infinitely wide networks, and have some (strictly positive) degree of alignment between backprop and FA error vectors when proper regularization is used.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The paper leverages the theory used for neural tangent kernels, but adapts it in a non-trivial way for feedback alignment. The results presented in the paper are new, and the related work is adequately cited.\n     \nQuality: The paper is technically sound.\n\nClarity: Overall, the paper is well written. I would suggest explaining spectral properties of $G(0)$ and $H(0)$ (around assumption 3.1) in more detail. One option is to mention the result of lemma A.3, and explain intuitively why $G(0)$ has large eigenvalues, and $H(0)$ -- small, as it is important for the whole method. I also disagree with the last line of the abstract, which says “with performance that is comparable with the full non-local backpropagation algorithm” -- we know that FA doesn’t perform well on hard tasks (see Akrout et.al. 2019 paper cited in this work).\n\nSignificance: The theoretical results of this paper are important for our understanding of feedback alignment. The results are somewhat limited as we know that FA fails on hard tasks (see Akrout et.al. 2019 mentioned above), but the presented method (and the NTK framework in general) might be useful for studying other alternatives to backprop. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "1NSb26xYz8F", "reviewer": "Reviewer_pXMq", "summary": "This paper builds upon the work of Lillicrap et al 2016 of attempting to find biologically plausible methods for implementing backpropagation. They thoroughly describe the issues with non-local information and why there are limits to using backprop-based learning to understand more about biological neural systems.  They describe the algorithm of feedback alignment using random back-propagated weights.  The authors go on to confirm some of the results of the Lillicrap paper, and find a surprising and novel result that regularization is required to produce convergence in over-parameterized settings.\nThe experimental section of the paper shows some work on a more standard ML benchmark task of MNIST classification.  They show the results that regularization is required to get good performance on the task.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I believe this area of research, finding biologically plausible mechanisms, is an important direction for the field.  This will not only help us understand biological systems better, but also aid in the development of technologies that are more scalable and efficient.  The Lillicrap work was the introduction of a new concept, and this paper attempts to explain why and when the concept of random BP weights applies.  However, I don’t feel the authors really did enough in this paper to cross the threshold of a minimum publishable unit of work.  The main result of showing that regularization is required for convergence is an interesting result, but there is no theoretical basis as to why this is happening.  I’m ok with empirical results, but they haven’t demonstrated that this technique can really work on larger problems either.  Most of the paper is devoted to explaining the Lillicrap work and providing a notation.  In my view, either more empirical results or more well-motivated concepts are required to publish this work.\n\nThe quality of the writing is excellent, and the background provides a good platform to build from.  The explanations are clear and concise.  The experimental work is a bit sparse.  For instance, in figure 3 there is no discussion on why the classification performance on MNIST is best with lambda of 0.1 rather than 0.3.  How does the alignment relate to classification performance?  Why might the performance drop when alignment clearly increases?\n\nThe significance of the work is the main issue I have.  If the significance of the concepts presented were of much great magnitude, I would be ok with the level of rigor in the paper.  However, with an extension/clarification of a concept, there must an explanation of what new questions the work opens up.  \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have not addressed this issue directly. They highlighted that a better understanding of biologically plausible algorithms will lead to a better understanding of the brain.  I don't believe there are immediate negative consequences of this work, but the authors could choose to include a statement making this explicit. ", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the convergence of Feedback Alignment (FA), an alternative to backpropagation that has been found empirically to work well, but until very recently did not received any theoretical justification of any sort, as the weights of the network that is being trained are replaced with random matrices in the back-propagation step. 

This paper is an interesting addition in the study of these “biologically”  plausible algorithm. The authors study two-layer neural networks in the over-parametrised regime, and mainly provide two results: a) A proof of linear convergence of the loss to zero error and b) An analysis of the alignment between the second-layer weights of the network and the feedback matrix used in the feedback alignment algorithm.  

The consensus among the reviewer was that, despite FA not being a mainstream algorithm, given its interest as a potential solution to the weight transport problem, the study was of interest to the Neurips community, and that the paper provided a definitely worthy  contribution. Given the rigour and the level of detail in the paper, this work has been judged to provide as well a blueprint for theoretical analysis of other biologically plausible learning rules, and therefore has a potential to impact future research. 

The interaction between the authors and the reviewers during the rebuttal phase have seen some reviewers increasing their grade for the paper. Currently, all reviewers are largely in favor of acceptance, and thus the area chair recommend acceptance as well. 



Decision: Accept (Poster)