Learning the Connections in Direct Feedback Alignment
Authors: Matthew Bailey Webster
Jonghyun Choi
changwook Ahn
Venue:  (2021)
Keywords: Deep Learning
Feedback Alignment
Backpropagation

Abstract: Feedback alignment was proposed to address the biological implausibility of the backpropagation algorithm which requires the transportation of the weight transpose during the backwards pass. The idea was later built upon with the proposal of direct feedback alignment (DFA), which propagates the error directly from the output layer to each hidden layer in the backward path using a fixed random weight matrix. This contribution was significant because it allowed for the parallelization of the backwards pass by the use of these feedback connections. However, just as feedback alignment, DFA does not perform well in deep convolutional networks. We propose to learn the backward weight matrices in DFA, adopting the methodology of Kolen-Pollack learning, to improve training and inference accuracy in deep convolutional neural networks by updating the direct feedback connections such that they come to estimate the forward path. The proposed method improves the accuracy of learning by direct feedback connections and reduces the gap between parallel training to serial training by means of backpropagation.

Under review as a conference paper at ICLR 2021
LEARNING THE CONNECTIONS IN DIRECT FEEDBACK ALIGNMENT
Anonymous authors Paper under double-blind review
ABSTRACT
Feedback alignment was proposed to address the biological implausibility of the backpropagation algorithm which requires the transportation of the weight trans- pose during the backwards pass. The idea was later built upon with the proposal of direct feedback alignment (DFA), which propagates the error directly from the output layer to each hidden layer in the backward path using a fixed random weight matrix. This contribution was significant because it allowed for the parallelization of the backwards pass by the use of these feedback connections. However, just as feedback alignment, DFA does not perform well in deep convolutional networks. We propose to learn the backward weight matrices in DFA, adopting the method- ology of Colen-Pollack learning, to improve training and inference accuracy in deep convolutional neural networks by updating the direct feedback connections such that they come estimate the forward path. The proposed method improves the accuracy of learning by direct feedback connections and reduces the gap be- tween parallel training to serial training by means of backpropagation.
INTRODUCTION
When feedback aligmment was proposed by Lillicrap et al. (2016) it was cited as being biologically plausible alternative to the backpropagation algorithm, but not long after Nøkland (2016) showed that variants of this approach may show tangible benefits during training such mitigating the vanishing gradients issue or enabling parallelization of the backwards pass at the cost of additional memory requirements. Recently, interest in the latter has begun to grow as the memory capacity and compute capability of modern GPUs has continued to observe significant leaps. While many of these recently proposed alternatives have been shown to be just as capable as the backpropagation algorithm in terms of inference accuracy on deep convolutional networks, it should be noted that many of these approaches have not yet been shown to perform well outside of the image classifi- cation task. Direct feedback alignment (DFA), an earlier approach proposed by Nøkland (2016), was shown to perform reasonably well on a number natural language processing tasks with recur- rent neural networks and transformers by Launay et al. (2020). However, direct feedback alignment still shows poor performance on the image classification task due to its inability to effectively train convolutional layers.
We propose a modification to the DFA algorithm to improve its ability in training deep convolu- tional neural networks. Due to its relationship with another approach(Akree (Akrout et al., 2019), we call our method Direct Kolen-Pollack learning or DKP. We empirically show the mechanisms that al- low the improvement in our approach over DFA by measuring DKP's ability better estimate the backpropagation algorithm. We also show this improvement directly by training two deep convo- lutional neural network architectures on the Fashion-MNIST. CIFAR1Ó. CIFAR100, and Tinylm- ageNet200(Le & Yang, 2015) datasets. More so, we recommend training procedures for training with DFA, pointing out the important role batch normalization plays in our experiments. And while a couple of works have shown that direct feedback connections can be viable when connecting to only the output of a block of layers in a network (Ororbia et al., 2020; Han & Yoo, 2019), we show advances in the case of having feedback connections to all layers in deep convolutional neural net- works. While direct feedback connections to all layers for current PC hardware, and also from a software perspective, may not be practical, in the future it may be useful for edge devices, IoT, SOC design, etc.(Frenkct et al., 2019; Han & Yoo, 2019), especially those that involve learning vision tasks. Thus, making advances in the training scenario of direct feedback connections to all layers

---

Under review as a conference paper at ICLR 2021
in neural network minimal computational cost for vision tasks is a primary motivation of this work.
1.1 RELATED WORK
Alternatives to the backpropagation algorithm have been proposed for their heightened biologically plausibility, or often as a means of parallelizing the training process. More recently, a number of algorithms have shown impressive results on large classification datasets such as Srageeett(AAkoon et al., 2019; Kunin et al., 2020; Belilovsky et al., 2019; Xu et al., 2020). To enable further paral- lelization of the training process, these works often focus on tackling three major deficiencies with the backpropagtion algorithm: forward locking, backward locking, and update locking. Forward locking prevents any calculation of gradients until the forward pass has been completed. Backward locking means that the gradients at some layer can not be calculated until the learning signals at all of the downstream layers have been calculated first. Update locking means that the parameters at some layer cannot be updated before the learning signal at the layer upstream of it has been calculated.
Difference Target Propagation (DTP), proposed by Lee et al. (2015), is one such alternative to the backpropagation algorithm that instead of computing gradients at each layer computes targets that are propagated backwards through the network by means of layer-wise autoencoders. In a recent paper by Lillicrap et al. (2020), DTP and methods that use layer-wise wise autoencoders in the backward path to propagate gradients are claimed to be more biologically plausible alternative to backpropa- gation and help to explain how biological neural networks might learn using process similar the backpropagation algorithm. Around the time DTP was first proposed, Lillicrap et al. (2016) demon- strated that artificial neural networks can learn using so-called feedback connections that inspired by the biological feedback connections in the brain, and did so by using fixed random weight ma- trices in place of the weight transpose when calculating each learning signal during the backwards pass. This approach, referred to as feedback alignment (FA), was claimed by the authors to be more biologically plausible than backpropagation as addressed the implausibility of weight transporta- tion in biological neural networks. FA and its derivatives would be further evaluated by Bartunov et al. (2018) and be shown to be very limited in comparison to the backpropagation algorithm on difficult tasks such as ImageNet. However, Moskovitz et al. (2019) would concurrently propose their own variations of the feedback alignment algorithm and make great strides in bringing these bio- logically motivated algorithms closer to the performance of backpropagation in deep convolutional neural networks. Following this initial work on feedback alignment, Nøkland (2016) proposed an alternative approach that connected each layer directly to the error through fixed random weight matrix in the backward path. Called direct feedback alignment (DFA), this contribution was signifi- cant as it leveraged feedback alignment to enable backwards unlocking meaning that during training the gradients for each layer can be calculated in parallel. Unfortunately, just as the original feed- back alignment method, direct feedback alignment has difficulty scaling to more difficult problems and training convolutional layers. In a follow up paper on DFA, Launay et al. (2019) showed that the approach simply failed to train convolutional layers. Later, Han & Yoo (2019) showed that VGG-16 could be trained with DFA if only the full connected layers are trained with DFA while the convolutional layers are trained with backpropagation, and Han et al. (2020) later showed that by only having direct connections to specific layers better performance in accuracy over DFA while training convolutional networks on the CIFAR1O dataset could be made. Despite this shortcoming, various NLP tasks as shown by (Launay et al., 2020), and
DFA shows fairly strong performance been used to enable higher power efficiency in SOC design (Han et al., 2019). Other follow up works DFA helped to reduce the additional memory costs of (Han et al., 2019; Crafton al., 2019), and Frenkel et al. (2019) even showed that propagating targets in place of the gradient at the output can be just as effective. Further recLRA, proposed by Ororbia et al. (2020), showed strong performance on the ResNet architectures with its own biologically inspired derivation of DFA that, similarly to our proposed approach, updates the backward feedback connections, but this perfor- mance was achieved by the more practical method of only have direct feedback connections to some layers. More recently, Akrout et al. (2019) and Kunin et al. (2020) have shown that credit assign- ment approaches similar to FA can scale to larger problems by training the backward weights and even come close to matching the performance of backpropagation on the ImageNet classification task. Akrout et al. (2019) proposed weight mirroring (WM) which trained the backward weights to mirror their forward counterparts using the transposing rule and proposed another method, referred to Kolen-Pollack learning, based on the research of Kolen & Pollack (1994), that updates the
2

---

Under review as a conference paper at ICLR 2021
backward matrices with the same gradient the forward weights and uses weight decay on both the forward and backward matrices to encourage symmetry between the two, As a follow up to this, Kunin et al. (2020) proposed a set of regularization primitives with which to update the backward weights and combined these primitives into various configurations that showed improved perfor- mance and stability over WM. One such configuration, information alignment (IA), uses only local learning rules that follow what the authors consider key biological constraints to train various deep ResNet architectures on the ImageNet data set.
Since the introduction of DFA, a number of notable works have proposed their own unique ap- proaches for enabling the parallelization of the forward and backward passes during training. Among these approaches, one of the most notable advances was made by Jaderberg et al. (2017) in their work on decoupled neural interfaces (DNI) which trains modules at each layer to predict synthetic gra- dients given the layer's s input and some context information, such as the global target, as the input into the module, and then waiting for the true gradients in order to update each module's parame- ters. Nøkland & Eidnes (2019) used sub-networks at each layer to produce local loss functions with which to update the weights of the primary network. They also showed that a backprop free version of this approach can train with only minor loss in accuracy. More recently, based on the earlier greedy layer-wise learning (DGL) (Benio et al., 2007), Belilovsky et al. (2019) proposed synchronous and asynchronous variants of DGL for addressing the issues of update locking and for- ward locking, and showed that these variants can match the performance of DGL and are capable of training deep convolutional neural networks on the ImageNet classification task. In the same vein as prior work which decoupled the backpropagation algorithm using delayed gradients(Huo et al., 2018), Xu et al. (2020) also recently introduced another novel training method, diversely stale parameters (DSP), which achieved comparable results to backpropagation on the ImageNet task.
2 APPROACH
2.1
BACKPROPAGATION AND DIRECT FEEDBACK ALIGNMENT
The propapropagation algorithm has long proven to be a robust, well performing approach for credit assignment in artificial neural networks. However, in recent years, there has been a search for alternatives that overcome the constraints of backwards and forwards locking which are found in backpropagation. One such alternative is direct feedback alignment (DFA), bio-inspired credit as- signment algorithm that enables backwards unlocking and the foundational approach for the method are proposing- For simple comparison of BP to DFA and our approach, feed-forward neural network with / layers, ignoring the biases. Then the learning signal 6ş for the output layer and the learning signal & for some layer l < k as prescribed by the backpropagation algorithm, where ag are our activations and f' the derivative of some non-linearity such as sigmoid function, are as follows.
define a simple
5G = error ④ f'(ax).
(1)
6, = de W @ (ar)
(2)
In the equations above and from here on out, is the dot product operation between two matrices, and @ is the Hadamard product, or also known as the element-wise product. In DFA, B, is a fixed random weight matrix that projects the gradient 6y at the output of network to the output of layer f - 1. Thus, the learning signal &g for any layer l upstream of output layer can be computed in the following way where og are the activations and f' the derivative of the non-linearity used at layer f.
8z on Br+1 O ('(a))
Then, every other aspect of the network is calculated in the same way as backpropagation. Thus, the weights We layer would be updated in the following way, where nw is the learning rate for the forward parameters, just as it is with backpropagation.
VW.
51 af
Because the backward weights are fixed, it is thought that the forward weights are aligning them selves with the backward weights such that the backward weights become useful for providing
3

---

Under review as a conference paper at ICLR 2021
O
W:
W
W
B2
W/
O
O
O
OOO
W, 1
W
W
B
W,
B
[OO
OOO
O
W,
Wo
Wo
1000
00O
x |00
(a) BP
(b) FA/KP
(c) DFA/Proposed DKP
Figure 1: From left to right, the three figures above depict the forward(-) and backward(-) paths when training with backpropagation(a) feedback alignment Kolen-Pollack learning(b), and direct feedback alignment or direct Mienn-oollack learning (c) respectively.
meaningful updates the network during training. However, Launay et al. (2019) showed that DFA, while showing results that nearly match backpropagation the MNIST dataset with fully connected (FC) layers, fails to scale to harder problems such as CIFAR 100 and shows significantly worse performance while training convolutional neural networks.
2.2
LEARNING
For the purpose of overcoming the weight transportation problem, both Kolen-Pollack learning and the method it is based on, feedback alignment (FF). use backward matrix at each layer, which shares the same dimensions place of the weight transpose while calculating the learning signal at the upstream layer. While the backward path consists of fixed random weight matrices in feedback alignment, the backward matrices Kolen-Pollack learning are updated with the transpose of the weight updates made to their forward counterparts, and weight decay is used on both the forward and backward weight matrices Kolen-Pollack learning. Following our prior notation, the learning signal & for some layer < k as prescribed by Kolen-Pollack learning and feedback alignment are as follows:
the transpose of the corresponding forward matrix, that takes the
oz = dg Be+1 + f'(a)).
(5)
For Kolen-Pollack learning, the update directions for Bg and We with weight decay, where À is number between 0 and 1, are as follows:
B = o a ABr, VW, = -n of AW
(6)
In later sections we will discuss the learning dynamic of KP and how it applies to our proposed method.
2.3 DIRECT KOLEN-POLLACK
To address these issues with DFA, we propose a method for updating the backward matrices inspired by the work of Akrout et al. (2019) and Kolen & Pollack (1994) which call direct Kolen-Pollack (DKP) learning. For DKP, the prior rules for DFA as stated in the previous section remain the same, however Bg is no longer fixed matrix. Rather, batch using the following update rule where 7/ is the learning rate for the backward parameters and òy is the learning signal at the output layer k.
will adjust the backward matrices after each
VB = - a1-1-
Similarly to the Kolen-Pollack approach used by Akrout et al. (2019), we also use weight decay on both the forward and backward parameters during training. Thus, the update directions for B, and We with weight decay would be as follows where the hyper-parameter is ranged between 0 and 1.
V B -oooo a ABrE VW, -n of AWe-
(8)
4

---

Under review as a conference paper at ICLR 2021
In experiments, we found that weight decay both the forward and backward matrices was crucial for maintaining the stability of the network during training. but we also try to substantiate this mathematically in the following sections.
While clear parallels cxist between DKP and KP. the underlying learning dynamics between the two approaches are different we will discuss in section 2.5 though assert that the end result of these approaches are the same. This result being that the forward and backward connections align estimate the same function.
2.4 KOLEN-POLLACE LEARNING AND CONVERGENCE THROUGH WEIGHT DECAY
Kolen-Pollack learning works the principal of convergence through weight decay. Akrout et al. (2019) show that two synapses receiving the same arbitrary updates with an equal amount weight decay will eventually converge on the same value. Before showing how this may apply direct Kolen-Pollack learning, let us first reiterate how two synapses, and by extension two weight matrices that share the same dimensions, will converge
Kolen-Pollace learning.
We first consider a pair of discrete-valued forward and backward weight matrices, W and B, that share the same dimensions such that all i element pairs Was and Bag are different at time step t = O. Then, at every time step we update each pair of elements (t) and (t) with some arbitrary adjustment value (t) along with an equal amount of weight decay on both elements where 0 < x < 1. Thus, at time step the updates made to (t, and B (t) are given as follows:
Vl (1)(() (W) Vl, Ai,(() (()
Then
Wey(t + 1)= (t, (t),
(t)(tt)
(10)
For the sake of algebraically showing why the forward and backward weight matrices in Kolen- Pollack learning come to mirror one another, let us now consider the difference between W Wt and (t (t- + 1) as
Wty(t 1) - (t + 1) = + [W. [B,, (+)
=W,,(()+ + (0] Bi,( [Ac)(1) (l),
(1-A))Wa,5)) - ((,,,((
(1 - (1_,)*+(We,(0) (o)
(11)
we also find that 0 1 Thus, limy (1 A)+11 0. and by extension (0)] since Wiy and B.g are initialized as discrete values. increases value, the difference between Wi, W and (t, will approach zero. Since this is true for any pair of corresponding j elements between W and B, it follows that the same is true for the two weight matrices themselves, and thus, the following also should hold:
Since 0 < À we
(1- - -X)++( Therefore
lim;
Σ lim (We (t) 1-
(12)
Therefore, after enough time steps, W and will come to approximately equal to each other. Note that while training a neural network with Kolen-Pollack learning, the updates made to the forward and backward weight matrices at each layer are not arbitrary. Thus, each pair of forward and back- ward matrices will come equal each other while also minimizing the global error of the network. Because the forward and backward weight matrices do eventually come to approximately mirror one another, i. W?', expect that the gradients made by Kolen-Pollack learning will eventually closely align with the gradients made by the backpropagation algorithm.
2.5 ALIGNMENT IN DIRECT KOLEN-POOLLACK LEARNING
In direct Golen-Pollach (DKP) learning, it is not immediately clear how the forward and backward paths are aligning since the forward and backward weight matrices each layer do not share the
5

---

Under review as a conference paper at ICLR 2021
90
90
Layer
Layer
75
Layer
75
Layer
Layer
Layer
Angle
Layer
Angle
Layer
45
45
15
15
0
5000
10000
15000
0
5000
10000
15000
Iterations
Iterations
(a) DFA
(b) Proposed DKP
Figure 2: The cosine alignment angle between 0x-1 Bf and hy in degrees for each backwards matrix starting from the first layer(-) to the last layer before the output(-) for a duration of about 18,000 iterations. In these results, lower angles equate to better alignment with the backpropagation algorithm. The value of X used is 10-6, Note that the scales of y-axis in both figures are the same.
same dimensions. The only exception is the second to last layer (k: [k-1) where the forward and back- ward weight matrices do share the same dimensions, and thus, we expect this layer to behave as would in PollnnPollack learning. In Kolen-Pollack learning, we can say that some backward matrix Be is converging on the function fe that maps the input ag-1 of layer to the preactivation hidden state he i.e fe: ag-1 + hg. As we showed in the previous section, this method of convergence works with Kolen-Pollack learning because fe is a linear mapping that uses the same dimensions as B However, with direct Kolen-Pollack learning the function fe is no longer linear function for all layers E < k 1 and instead is the nonlinear function fe: ag-1 H hę which is comprised of layers l through k. Due the highly nonlinear nature of fe alone it would not be reasonably possible for any direct linear function in the backward path to converge with this function.
However, we might expect these direct feedhack connections in DKP to converge with linear function that estimates the nonlinear forward connections. That is to say Be converges on some linear function ge that estimates the nonlinear function f:: dz-1 hg. To substantiate these claims we experimentally show that as a network trains with DKP the cosine alignment angle between the preactivation output of the network h2, to which all functions Se map to, and the dot product of the activations at any layer f < k- 1 and the transpose of the corresponding direct backward weight matrix B& will closely align after sufficient number of training steps has passed. The network we will be using to examine this property of DKP is five layer fully connected neural network trained on MNIST with cross entropy used for the loss function. After each hidden layer, batch normalization is used and followed with rectified linear unit activation function.
In figure 2b, we see that with DKP the cosine alignment angle between a--1 BJ for each layer e < and the output of the network he is much lower than it is with DFA in figure 2a with the exception of the angle measured at the first layer which does not seem to improve much from DFA to DKP. Thus, see that both DFA and DKP seem to exhibit this behavior of estimating the nonlinear forward path with some linear function, however DKP clearly does this to greater effect. We also notice that generally the direct feedback connections further downstream align much more closely with the forward connections than those further upstream. This result likely occurs because the upstream feedback connections are converging to some linear function that has to estimate a forward function that is much more complex and highly nonlinear relative to the downstream functions.
3 EXPERIMENTS
All of our experiments were coded with Python using the PyTorch library. Code will be available on GitHub soon.
3.1 EXPERIMENTAL SET-UP
In our experiments, using batch normalization was necessary to gain stable training for both DFA and DKP, and both benefit significantly from its usage depending on the weight initialization method.

---

Under review as a conference paper at ICLR 2021
This was especially an important for DKP without batch normalization we would often run into an issue of exploding gradients. Additionally, for DFA and DKP we use Kaiming weight initializa- tion(He et al., 2015) on the backward weight matrices as suggested by Launay et al. (2019).
We also found that the optimal hyperparameters and optimizers for the backward weight matrices in DKP seem to vary some from one architecture to the next. For our first set of experiments with convolutional networks we employ a smaller network with only two convolutional layers to train on the CIFARIO and Fashion-MNIST datasets, and for this network, training on CIFAR I0, we found that the best optimizer for the backward weights was simply the same as that used on the forward weights, SGD with Nesterov acceleration and momentum at 0.9, with the only difference being reduced learning rate. On the forward parameters a learning rate of 10 was used, and on the backward parameters learning rate of 10-4 was instead used. Both forward and backward parameters used weight decay with x = 10-6. Additionally, we used a step-wise learning rate decay on the backward parameters with y = 0.85 which further improved the training stability. However, in our second experiment training the AlexNet architecture on the CIFAR 100 dataset and TinyIm- ageNet200 dataset, neither of these previous strategies involving the optimizer and learning rate scheduler were optimal. For training with AlexNet CIFARIO0, the Adam optimizer (Kingma & Ba, 2014) was used on the backward weight matrices with a learning rate of 5 x 10 10-4, while SGD with Nesterov acceleration, learning rate of 0.01, and a momentum of 0.9 was again used on the forward weight matrices. In the case of the AlexNet tests, both the forward and backward param- eters used weight decay where À 10-4 The only difference moving to the TinylmaeeNe2000 dataset was a slight decrease of the forward learning rate to 5 x 10-3 for all approaches except the backpropagation tests.
We believe that the differences in what was optimal for training with DKP between the two networks were the result of the increased difficulty in the problems being solved and potentially the size of the networks themselves. However, the weight initializations, hyperparameters, and optimizers used in our experiments were not found through rigorous search, and thus further improvements to the performance of these learning algorithms could potentially be found. We also note that in all of our experiments, the learning rates seen are about the highest values that one would want to use before rumning the risk of exploding gradients, and that for all methods it is perfectly acceptable to use lower values. However there is a slight exception when training with KP and DKP: if the learning rate on the backward weights are too low, then these methods begin to behave as their static counterparts FA and DFA respectively which leads to reduced performance. Because the change from DFA DKP is analogous to the change from FA to KP, we will be including FA and KP in our experiments for comparison to their direct counterparts. These additional experiments will help to illustrate the difficulty that arises from training with direct feedback connections and that the application of Kolen-Pollie Mooll-oo learning alone is not enough to overcome their shortcomings.
3.2 TRAINING CONVOLUTIONAL NETWORKS
Because DFA has a difficult time training convolutional layers, achieving a more meaningful result with direct feedback connections in these deep convolutional networks is an important step and the primary improvement DKP makes over its predecessor. To demonstrate DKP's capability in training convolutional layers we not only compare it to backpropagation and DFA but also we test each network's ' performance with the training on the convolutional layers frozen such that only the fully connected layers are trained with backpropagation; anything that performs less than or equal to this in terms of accuracy or loss is likely not training the convolutional layers effectively. We also compare to direct random target (Dojection(DRTP)) (Frenkel et al., 2019) which projects the one hot encoding of the target directly to each layer. The loss function used in all of our experiments is the cross entropy loss. All reported numbers are an average of 10 trials.
For our first experiment, we train the CIFAR 10 dataset for 50 epochs using network that consists of just two convolutional layers followed by two fully connected layers, the second being the output layer. The results in Table show that DFA does considerably better than BP with the convolutional layers frozen, and that DKP performs even better than DFA. Of course, we still see that BP is out performing both DFA and DKP by much larger margin.
In our second experiment, we train on the CIFAR100 and TinylmageNet 200 datasets using the AlexNet architecture for 90 epochs, and again, batch normalization is used before the activation
7

---

Under review as a conference paper at ICLR 2021
Table 1: 2-Conv. Layer network results on CIFARIO and Fashion-MNIST with cross entropy loss. Serial/Parallel indicates serial training and parallel training scheme during the backwards pass. Par- allel+ indicates that the method is capable of update unlocking.
Dataset
Method
Serial/Parallel
Inference Accuracy
Train Accuracy
BP (FC Only)
Serial
91.33% + 0.18
99.97% = 0.02
BP (Upperbound)
Serial
92.18% + 0.13
100.00% + 0.00
KP
Serial
91.25% + 0.18
99.47% 0.14
Fashion-MNIS"
FA
Serial
91.12% + 0.39
99.41% + 0.26
DRTP
Parallel+
89.58% + 0.05
94.86% = 0.02
DFA
Parallel
91.54% + 0.14
99.88% + 0.05
DKP (Ours)
Parallel
91.66% = 0.27
99.89% + 0.06
BP (FC Only)
Scrial
60.01% + 1.32
99.34% 0.63
BP (Upperbound)
Serial
70.70% + 0.96
99.82% + 0.49
KP
Serial
70.08% = 0.37
99.98% 0.01
CIFARIO
FA
Serial
60.45% + 1.13
95.36% + 1.46
DRTP
Parallel+
55.32% +: 6.14
72.90% + 0.08
DFA
Parallel
62.70% + 0.36
97.72% = 1.24
DKP (Ours)
Parallel
64.69% = 0.72
99.09% + 0.29
Table 2: AlexNet results on CIFARI00 and TinylmageNet200 with cross entropy loss. Se- rial/Parallel indicates serial training and parallel training scheme during the backwards pass. Paral- lel+ indicates that the method is capable of update unlocking.
Dataset
Method
Serial/Parallel Inference Accuracy
Train Accuracy
BP (FC Only)
Serial
47.72% +: 0.73
39.03% + 0.32
BP (Upperbound)
Serial
65.88% 1.02
64.09% 0.39
KP
Serial
66.78% + 0.47
67.70% = 1.88
CIFARI00
FA
Serial
19.49% + 0.97
12.90% 0.80
DRTP
Parallel+
5.84% = 0.65
5.49% + 0.19
DFA
Parallel
48.03% + 0.61
35.18% 0.43
DKP (Ours)
Parallel
52.62% + 0.48
45.17% + 0.43
BP (FC Only)
Serial
29.78% +: 0.45
24.89% + 0.24
BP (Upperbound)
Serial
49.53% 0.61
49.25% 0.31
KP
Serial
51.36% + 1.50
60.44% + 1.94
TinyImageNet200
FA
Serial
9.98% = 1.44
7.12% + 0.83
DRTP
Parallel+
2.86% 0.59
2.86% + 0.26
DFA
Parallel
32.116% + 0.66
24.75% 0.36
DKP (Ours)
Parallel
35.78% + 1.92
35.97% + 0.37
Layer No.
Layer Configuration
0
Conv(Channels: 3-32, Kernel: 3, Padding: 1). BN., ReLU
Conv(Channels: 32-32, Kernel: 3, Padding: 1). BN, ReLU
2
Max Pool(Kemnel: 2), Linear(6272,, 128), BN, ReLU Linear( (128, 10)
3
Table 3: 2-Conv. Layer network architecture used the first set of experiments. BN stands for batch normalization.
8

---

Under review as a conference paper at ICLR 2021
2.5
2-
-BP (FC Only
90
90
(FC
1.5
80
Accuracy
880
DFA DKP
-BP (FC Only)
DKP
-20
-FA
-70
KP
0.5
60
DKP
0
50
50
30
Epochs
(a) Training Loss (4)
(b) Training Accuracy (1)
(c) Inference Accuracy (1)
Figure 3: From left, the training loss, training accuracy, and test accuracy of the CIFARIO experi- ments using 2-layer CNN. indicates lower the better, 1 indicates higher the better. Dashed lines represent serial training and solid lines represent parallelizable training during the backwards pass.
BP BP [FC Only)
100
100
-BP -BP (FC Only)
-BP IFC Only)
80
FFA
80
Accurayy
DFA
DKP
20
20
20
80
100
20
80
100
60
100
120
Epochs
Epochs
Epochs
(a) Training Loss (L)
(b) Training Accuracy (f)
(c) Inference Accuracy (f)
Figure 4: From left, training loss, training accuracy, and test accuracy on the CIFARIOO 100 experiments using the AlexNet architecture. indicates lower the better, 1 indicates higher the better. Dashed lines represent serial training and solid lines represent parallclizable training during the backwards pass.
functions on all layers except the output layer. For CIFAR100, while we worked to get as close to a direct comparison as we could between BP, DFA, and DKP terms of hyperparameters and weight initializations, we did have to lower the learning rate from 0.1 to 0.01 on the forward parameters with DFA and DKP to achieve more stable training: lowering these values for the backpropagation tests resulted in slightly lower accuracies. Similarly, the Tiny (inylmageNer200 results required slightly lower learning rates. The results in table 2 are mostly consistent with what seen in the CIFARIO test. DKP, while showing solid jump in performance over DFA, still fails to match the performance of BP, and DFA only performs marginally better than BP when only the fully connected layers trained.
We see that the contribution to the results in accuracy made by the fully connected layers dimin- ishes, so too does the results of DFA. Also, we see that as the networks become more reliant on the convolutional layers to perform well, the gap between DFA and DKP in terms of accuracy widens in favor of our approach. We also note that the direct random target projection( (RTPP) approach suffers terribly from the usage of the cross entropy loss despite performing well networks similar to our experiments in table when using mean squared error. So another benefit of DKP is that it is compatible with a more robust loss function, cross entropy.
CONCLUSION
Direct feedback alignment (DFA) enables the parallelization of the backwards pass, called back- wards unlocking, and has shown promising results in NLP tasks(Launay et al., 2020). Despite these clear advantages, DFA fails to effectively train deep convolutional networks on difficult image clas- sifications tasks. We propose direct Kolen-Pollack (DKP) learning by incorporating Kolen-Pollack learning into DFA to more effectively train deep convolutional neural networks with direct feedback connections by updating the backward weight matrices. We empirically show that approach produces gradients that better align with the backpropagation algorithm. We further show that DKP outperforms DFA while training convolutional neural networks on the Fashion-MNISS 10, CIFAR 100, and TinyImageNNt2000 datasets.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap, and Douglas B Tweed. Deep learning without weight transport. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché- Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 976-984. Curran Associates, Inc., 2019. URL http: /papers nips cc/paper/ - we wwweeiittttaassttttppommm
8383- wew-ddee-eaaaann----eeeee-
2,2,4,5
Sergey Bartunov, Adam Santoro, Blake Richards, Luke Marris, Geoffrey E. Hinton, and Timothy Lillicrap. Assessing the scalability of biologically-motivae deep learning algorithms and archi- tectures, In Advances in Neural Information Processing Systems, pp. 9368-9378, 2018. 2
Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Decoupled greedy learning of cnns, 2019. 2,
Y. Bengio, Paseal Lamblin, D. Popovici, Hugo Larochelle, and U. Montreal. Greedy layer-wist training of deep networks. volume 19, 01 2007. 3
Brian Crafton, Abhinav Parihar, Evan Gebhardt, and Arijit Raychowdhury. Direct feedback align- ment with sparse connections for local learning. Frontiers in neuroscience, 13:525, 2019. 2
Charlotte Frenkel, Martin Lefebvre, and David Bol. Learning without feedback: Direct random target projection as a beedback-alignmen algorithm with layerwise feedforward training. arXiv preprint
2019. 2,
Donghyeon Han and Hoi-jun Yoo. Efficient convolutional neural network training with direct feed- back alignment. arXiv preprint arXiv: (etii::900..00 190100186, 2019. 1,2
Donghyeon Han, Jinsu Lee, Jinmook Lee, and Hoi-Jun Yoo. 1.32 tops/w energy efficient deep neural network learning processor with direct feedback alignment based heterogeneous core ar- chitecture. 2019 Symposium on VLSI Circuits, pp. C304-C305. IEEE, 2019.
Donghyeon Han, Gwangtae Park, Junha Ryu, and Hoi-jun Yoo. Extension of direct feedback align- ment to convolutional and recurrent neural network for bio-plausible deep learning. arXiv preprint arXiv:2006. /2830, 2020. 2
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034, 2015. 7
Zhouyuan Huo, Bin Gu, qian Yang, and Heng Huang. Decoupled parallel backpropagation with convergence guarantee. volume 80 of Proceedings of Machine Learning Research, pp- 2098-2106, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. URL http:
/ /pproceedings mlr press/ vv80/ htt.hhulleee 8a html. 3
Max Jaderberg, Wojeiech Marian Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu. Decoupled neural interfaces using synthetic gradients. In Pro- ceedings of the 34th International Conference on Machine Learning Volume 70, ICML'17, pp- 1627-1635. JMLR.org, 2017.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint 1412. 69800, 2014. 7
John Kolen and Jordan Pollack. Backpropagation without weight transport. volume 3, pp. 1375 1380 vol.3, 07 1994. ISBN 0-7803-1901-X doi: (TT1////NN...99..344444. 2,
Daniel Kunin, Aran Nayebi, Javier Sagastuy- Surya Ganguli, Jon Bloom, and Daniel L. K. Yamins. Two routes to scalable credit assignment without weight symmetry, 2020. 2,
Julien Launay, lacopo Poli, and Florent Krzakala. Principled training of neural networks with direct feedback alignment, 2019. 2, 4,
Julien Launay, Iacopo Poli, François Boniface, and Florent Krzakala. Direct feedback alignment scales to modern deep learning tasks and architectures. arXiv preprint arXiv:2006.122 12878, 2020. 1, 2,9
10

---

Under review as a conference paper at ICLR 2021
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 2015. 1
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, Yoshua Bengio, Annalisa Appice, Pedro Pereira Rodrigues, Vítor Santos Costa, Carlos Soares, João Gama, and Alípio Jorge. Difference target propagation. In Machine Learning and Knowledge Discovery in Databases, pp. 498-515, Cham, 2015. Springer International Publishing. ISBN 978-3-319-23528-8. 2
Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin J. Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nar Comman 13276, 2016. 1.2
Timothy P. Lillicrap, Adam Santoro, Luke Marris, Colin J. Akerman, and Geoffrey Hinton. Backpropagation and the brain. Nature Reviews Neuroscience, 21(6))335-346, Jun 2020. ISSN 1471-0048.
doi: 10. 0011008841533--00-0777-33 URL https:/ httpps://do..oe /ttt/////0l.or/1111033//////
s41583- 02000000002222-20 2
Theodore H. Moskovitz, Ashok Litwin-Kumar, and L. F. Abbott. Feedback alignment in deep
convolutional networks, 2019.
Arild Nøkland. Direct feedhack alignment provides learning in deep neural networks. Proceed- ings of the 30th International Conference Neural Information Processing Systems, NIPS' 16, pp. 1045-1053, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819, 1,
Arild Nøkland and Lars Hiller Fidnes. Training neural networks with local error signals, 2019.
Alexander Ororbia, Ankur Mali, Daniel Kifer, and C. Lee Giles. Large-scale gradient-free deep learning with recursive local representation alignment, 2020. 1,2
An Xu, Zhouyuan Huo, and Heng Huang. On the acceleration deep learning model parallelism with staleness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2088-2097, 2020. 2, 3
11


Reviews:
[{"review_id": "j5kXqLsN9qi", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Second Review\n\nI thank authors for taking time and answering my queries. However current manuscript fails to point out key difference between Akrout 19 kolen-pollack method and DKP (proposed method). Combining FA with DKP does not add sufficient novelty. As pointed out by other reviewers, paper should highlight key differences and reasoning for such combination. I am happy to see additional results with DRTP, however it is also important to test your approach based on there methodology.   It is difficult to gauge the significance of your approach, since training protocol varies a lot. I would request authors to add more baselines and training protocols  (future submission) to show that your method is robust and can also train deeper CNNs models. Current submission missed out on many key aspects, despite having promising direction. I hope our reviews help you in strengthening this promising work.   \n## Summary\n\nThis work proposes an approach to update feedback weights in DFA using modification of kolen-pollack method, which helps in training deep CNN network. \n\n## First Review\nCitation missing for key work on assessing the scalability of bio-inspired approaches and highlighting key limitations [Bartunov 18], variants of DFA [ Moskovitz 18, Frenkel 19]  and recently an approach similar to DFA with target projection known as LRA (also has similarity with Direct Kolen-Pollack) showing promising performance on deep CNNs [ Ororbia & Mali 2020].\n\nThe update rule used in recursive-LRA is similar to what proposed in this paper\nDelta_b(update for feedback weights)  = learning rate *( teaching signal(delta_k) * post-activation from layer below (a_(l-1)) \n\nFor LRA Delta_b(update for error weights)  = learning rate *( teaching signal(error_k) * post-activation from layer below (a_(l-1)) \n\nFor weight mirroring [ Akrout 19] Delta_b(updates for feedback weights) = = learning rate *( teaching signal(delta_(l) * delta(updates) from layer above (delta_(l+1)) \n\nOne can see we can derive chain rule formulation with certain assumption in which feedback matrix or error matrix acts like transpose of forward weights (rotated 180 degree).\n\nAs shown in Feedback alignment(lillicrap 16) the updates for FA and LRA lie with 90-degree w.r.t BP. One can provide such plots to show how far way are your updates w.r.t. BP and other bio-inspired approach.\n\n“We also found that the optimal hyperparameters and optimizers for the backward weight matrices in DKP seem to vary greatly from one network to the next”\nCan you provide more detail about your experimental setup? What are the range of hyper-parameters and how does DKP perform w.r.t BP and other variants? It is well known that DFA in its vanilla form suffer whenever tested with deep networks on challenging benchmarks such as imagenet (akrout 19, Bartunov 18). As shown by Moskovitz 18 and Crafton 19, integrating BP or making feedback weights close to forward weights helps in learning for complex benchmarks. So, what different does DKP offer? is it robust, speeds up the convergence, always stays consistent (robust against bad initialization). Current manuscript fails to highlight these points which could make current work stronger.\n\nDo you constraint your feedback weights, if so how? If not, then how does model ensure that feedback weights are respecting forward neural activities and helping it to converge? Won’t feedback weights grow making discrepancy between forward and backward activities, thus slowing the convergence of the network?\n\nComparison against other variants of DFA\nWe would like to see detailed comparison w.r.t various variants or family of FA(Moskovitz 18, Frenkel 19] and LRA (since update rules are similar). \n\n\n\n\n\n[Bartunov 18] Bartunov, S., Santoro, A., Richards, B., Marris, L., Hinton, G.E. and Lillicrap, T., 2018. Assessing the scalability of biologically-motivated deep learning algorithms and architectures. In Advances in Neural Information Processing Systems (pp. 9368-9378).\n\n[Akrout 19] Akrout, M., Wilson, C., Humphreys, P., Lillicrap, T. and Tweed, D.B., 2019. Deep learning without weight transport. In Advances in neural information processing systems (pp. 976-984).\n\n[Moskovitz 18] Moskovitz, T.H., Litwin-Kumar, A. and Abbott, L.F., 2018. Feedback alignment in deep convolutional networks. arXiv preprint arXiv:1812.06488.\n\n[Frenkel 19] Frenkel, C., Lefebvre, M. and Bol, D., 2019. Learning without feedback: Direct random target projection as a feedback-alignment algorithm with layerwise feedforward training. arXiv preprint arXiv:1909.01311.\n\n[Ororbia and Mali 20] Ororbia, A., Mali, A., Kifer, D. and Giles, C.L., 2020. Reducing the Computational Burden of Deep Learning with Recursive Local Representation Alignment. arXiv preprint arXiv:2002.03911.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "g-3t_iMjiRm", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strength:\nAs mentioned in the paper, although DFA is more biologically plausible, it does not work well for deep networks and CNNs. This work proposed a possible improvement for this problem.\n\nWeakness:\n(1) My main concern is the novelty of this paper. To my understanding, the DKP and the convergence of weight decaying are first proposed in (Akrout et al., 2019). However, updating the backward matrix suffers from the problem of different dimensions. I think the main contribution of this work is the experimental demonstration of the lower angle achieved by DKP. Therefore, I hope the authors can make it clear the main contributions of this work. It is also necessary to comment on how the approach in this paper is different from (Akrout et al., 2019).\n\n(2) In the experiments, the authors only compare the DKP with the DFA and BP. It is also necessary to compare the performance with other references. For example, how much the performance can be improved by the proposed method compared to the methods introduced in section 1.1.\n\n(3) In addition, the network sizes are not clearly mentioned in the experiments. It is difficult for readers to make a judgment about the effectiveness of this work.\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "-Ya2TVvHwse", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Thanks for author(s) for their paper. I enjoyed reading it.\n\nThis paper introduces a new method for computing the backward updates of a neural network called Direct Kolen-Pollack learning (DKP). Similar to Direct Feedback Alignment (DFA), the aim of this work is to introduce a viable alternative to back-propagation (BP) that works in parallel while achieving similar performance. Parallelization benefits comes from the fact the the backward path is unlocked. To summarize, DKP shows an interesting improvement over DFA for CNNs but results and analysis are slightly insufficient.\n\nKolen-Pollack learning (KP) suggests “that updates the backward matrices with the same gradient as the forward weights and uses weight decay on both the forward and backward matrices to encourage symmetry between the two.” The idea in DKP is simple: replace error signal from each layer with the final layer’s error signal in KP. ($\\delta_l \\rightarrow \\delta_k$ in Eq. 6 and 8)\nAuthor(s) identified the performance gap between DFA and BP in CNNs as their motivation and tested their method on image classification on Fashion-MNIST, CIFAR10 and CIFAR100 datasets.\n\nNotes:\n1. The writing is easy to understand and clear. Authors are planning to release the code soon.\n2. Please introduce matrix operations used in Eq. 1 and 2 somewhere.\n3. In DKP backward matrices are no longer fixed as in DFA but rather are updated after each batch with their own update rule and learning rate. These updates are still parallelizable but an analysis and/or experimentation on the speed gain with these new updates are needed, specially comparing to BP and DFA. Similarly, in the related work it’s mentioned that “[previous works except DFA] currently show no tangible benefits over backpropagation as they all have larger memory requirements” but I don’t see any discussion on that in the paper.\n4. Following prior works, weight decay on both the forward and backward parameters during training are used. Author(s) mention that weight decay is important to make this method work. As another contribution they provide mathematical justifications for its use. The result of this argument is that following this update rule after enough time, the forward and backward matrices will be close to each other. Please clarify following questions:\n    4. i. This argument depends on $A_{i,j}$ being the same for backward and forward update. But comparing Eq. 8 and 9, doesn’t this require $\\eta_B=\\eta_W$?\n    4. ii. This problem is worse with DKP when comparing Eq. 6 and 9 as ${\\left(-\\eta_B \\delta_k^T . a_{l-1} \\right)}_{i,j} \\neq {\\left(-\\eta_W \\delta_l . a_{l-1}^T \\right)}_{i,j}$.\n5. Description of Kolen-Pollack learning is minimal and a bit too late in the paper despite the proposed method is named after it. In my opinion it would’ve been better to explain it a bit more and earlier. For example the first few paragraphs of section 2.3 could move to earlier sections.\n6. Figure 2: Value of $\\lambda$ is missing. Also if I am not mistaken according to Eq. 11 the rate of convergence of the two should be proportional by $(1-\\lambda)^t$. I am not certain that I see this trend in the graphs, specially layers 0 and 1 that seem to be plateauing. I understand that you have similar observation at the end of section 2. The explanation seems to be that higher layers are easier to be linearly approximated. However, my counterpoint is that Eq. 11 shows an exponential decay to zero and it does not depend on layer l. Could you comment on this please?\n7. Despite the fact that KP does not make any assumptions about the structure and inductive biases of the network, DKP is proposed only for CNNs and image base classifications. Why shouldn’t DKP be used for MLPs, RNNs, etc.? I would really like to see its performance compare to BP and DFA for non-CNN structures.\n8. In related works, author(s) mention prior works that DFA have a hard time with VGG-16 optimization but the experiments are done AlexNet. What is the reason for this mismatch. It would have been much easier to make a direct comparison with previous works.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper investigates an improvement to the direct feedback alignment (DFA) algorithm where the "backward weights" are learned instead of being fixed random matrices. The proposed approach essentially applies the technique of DFA to Kolen-Pollack learning. While reviewers found the paper reasonably clear and thought the experiments were acceptable, there were significant concerns about the novelty of the approach and the fact that the proposed approach was a straightforward combination of existing ideas. Further, the paper could have done a better job situating (and applying) the proposed method to DFA variants that have been proposed since the original DFA paper came out.
Decision: Reject