Activation Sharing with Asymmetric Paths Solves Weight Transport Problem without Bidirectional Connection
Authors: Sunghyeon Woo
Jeongwoo Park
Jiwoo Hong
Dongsuk Jeon
Venue: NeurIPS 2021 Poster (2021)
Keywords: activation sharing
weight transport problem
bidirectional connection
biologically plausible algorithms
neuromorphic algorithms
deep convolutional neural network
biological neural network

Abstract: One of the reasons why it is difficult for the brain to perform backpropagation (BP) is the weight transport problem, which argues forward and feedback neurons cannot share the same synaptic weights during learning in biological neural networks. Recently proposed algorithms address the weight transport problem while providing good performance similar to BP in large-scale networks. However, they require bidirectional connections between the forward and feedback neurons to train their weights, which is observed to be rare in the biological brain. In this work, we propose an Activation Sharing algorithm that removes the need for bidirectional connections between the two types of neurons. In this algorithm, hidden layer outputs (activations) are shared across multiple layers during weight updates. By applying this learning rule to both forward and feedback networks, we solve the weight transport problem without the constraint of bidirectional connections, also achieving good performance even on deep convolutional neural networks for various datasets. In addition, our algorithm could significantly reduce memory access overhead when implemented in hardware.

Activation Sharing with Asymmetric Paths Solves Weight Transport Problem without Bidirectional Connection
Sunghyeon Woo
Jeongwoo Park Seoul National University
Jiwoo Hong
Dongsuk Jeon
(wsh0917, jeffju, micro995, djeon1)@smi 0snu. er.
Abstract
One of the reasons why is difficult for the brain perform backpropagation (BP) is the weight transport problem, which argues forward and feedback neurons cannot share the same synaptic weights during learning in biological neural networks. Recently proposed algorithms address the weight transport problem while providing good performance similar to BP in large-scale networks. However, they require bidirectional connections between the forward and feedback neurons to train their weights, which is observed to be rare in the biological brain. In this work, we propose an Activation Sharing algorithm that removes the need for bidirectional connections between the two types of neurons. this algorithm, hidden layer outputs (activations) are shared across multiple layers during weight updates. By applying this learning rule to both forward and feedback networks, we solve the weight transport problem without the constraint of bidirectional connections, also achieving good performance even on deep convolutional neural networks for various datasets. In addition, our algorithm could significantly reduce memory access overhead when implemented in hardware.
Introdution
Backpropagation (BP) [ is the representative approach to training various deep neural networks. While BP exhibits excellent training performance, similar to
better than that of humans, has been long argued that the structure of biological neural networks does not support the backpropagation of errors 2 4 5]. To resolve this issue, wide range of studies have been conducted to develop
algorithm that is feasible in biological neural networks, One important reason behind the biological implausibility of BP is the weight transport problem 6 BP requires identical forward and feedback paths for reliable training; i.e., the two paths must have the same synaptic weights. While biological neural networks may also implement two separate processing paths (forward and feedback paths), is impossible to explicitly pass the weights between the two paths, as it requires very fast transmission of information along the axon from each synapse output
The Feedback Alignment (FA) algorithm g solves the weight transport problem by propagating errors through the feedback path with random fixed weights. was shown that the feedback weights could be aligned with the forward weights in the course of training, and consequently. the network is
trained similar way to BP. Similarly, Direct Feedback Alignment (DFA) (9] directly propagates errors from the top layer to lower layers using random fixed weights, and Direct Random Target Projection [10] locally creates errors using targets rather than using backpropagated errors. Although these algorithms demonstrate good training performance on simple networks, they exhibit large performance degradation when applied complex networks, especially deep convolutional neural networks 12
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

More recent works suggest training the feedback weights as well as the forward weights instead of using random fixed weights in the feedback path. Training feedback weights through Reinforcement Learning [3] or Spiking Neural Network [14] provides better accuracy in classification tasks than the algorithms using random fixed weights. However, those approaches have been only evaluated in shallow neural networks, and it is questionable whether they would be capable of training deeper neural networks. The Sign-Symmett neural networks by transporting only the signs of the forward weights to the feedback path during training. However, since it still needs to transport the signs of the weights, it is difficult to say that the weight transport problem has been solved entirely.
algorithm (T5) achieves good training performance in deep
Recently, Weight Mirrors and modified Kolen-Pollace algorithms [16 17, which align the feedback weights with the forward weights without explicit weight transport. succeeded in training deep convolutional neural networks on large datasets with outstanding performance close to BP. However, these algorithms require a bidirectional connection: forward and feedback neurons explicitly exchange information with each other through a pair of direct unidirectional paths. While this type of connection is found in some biological neurons [18 19 20 21. it is difficult to generalize them to broader range of biological neural networks that typically only have unidirectional paths between neurons, where feedback must route through multiple neurons in order to reach their destination 12..
In this paper, we propose an Activation Sharing algorithm, which performs weight update of a layer us- ing the ouiput (activation) of a lower layer, not the output of the immediately previous layer. Applying this learning method both forward and feedback paths solves the weight transport problem without using bidirectional paths and also successfully trains deep convolutional networks such as ResNet-34. Furthermore, since activation is shared across multiple layers during training, the algorithm could significantly reduce memory access and is suitable for efficient hardware implementation.
2
Related works
2.1 Backpropagation and weight transport problem
We consider typical training process of a deep neural network. The forward path propagates input data through multiple layers, generating output activation of each layer:
h = Φ W++1 h; + b(+1)
(1)
where h4 is the output activation of the hidden layer I, W; is the weight matrix, and by is the bias vector of layer I in the forward path. In the brain, h could be interpreted as neural firing rates, W as synaptic weights between neurons, b as bias currents, and & as nonlinearities in neurons [16]. To train the model, synaptic weight changes A AW are obtained by
(2) where 8 and denote the error signal that flows through the feedback path and the learning rate, respectively. For calculating synaptic weight changes A W, the error 8 should propagate from the output layer to lower layers in the reverse direction. In backpropagation [I] errors that propagate through the feedback path are represented
AW 8 h2
5, = (h:) W 6:41
where ø' denotes the derivative of the activation function ó in equation 11. For this BP to occur in the brain, the eтror must be propagated in the reverse direction of the forward path, which is not feasible in biological neural networks that only have unidirectional paths. Even if there is separate feedback path, as shown in Fig. [h, the synaptic weights W in equation (T) should be also used to propagate the error in equation (3). In other words, the forward and feedback paths must have identical synaptic weights. Neurons need to exchange their synaptic weights to realize this, but there no proof that this weight transport actually occurs the brain (weight transport problem).
2.2 Training with random fixed feedback weights
The Feedback Alignment algorithm (8 was developed based on the idea that errors could be propa- gated using random fixed weights B rather than the forward weights W in equation (1). As shown in Fig. [p. errors propagate through the feedback path following the equation below.
8, 6' (hi) B 8141
(4)
2

---

weight transport
61s2
2W11
A Alax
Wx+2
Warz'
dW112
1
Wi+3
Bri2"
6111
8241
6,41
Wi+1
AW
Wi+1
Bivs'
6,
hv
(a) Backpropagation
(b) Feedback Alignment
bidirectional connection
6,
6,42
1
6,02
AW
c+2
Bis2"
AW
Wina
Biss"
AB - -noi2
8141
6:
6:+1
huss
Aw
W4+1
Bios"
Wi-1
Biss"
AB
6:
h:
(c) Direct Feedback Alignment
(d) Modified Kolen-Pollack
Figure 1: Backpropagation and biologically plausible algorithms.
Moreover, the Direct Feedback Alignment algorithm [9] showed that the neural network could be trained if the error propagates directly from the output layer to each layer instead of flowing through all hidden layers sequentially (Fig. Ib):
(5) where denotes the output layer of deep neural network. These methods perfectly solve the weight transport problem by using random fixed weights in the feedback path and demonstrate good training performance in simple networks. However, they fail train the model or incur significant performance drop when applied to deep convolutional neural networks
5, 5 (he) B 8L
15 23
2.3 Training with feedback weight update
Since using random fixed weights for error propagation causes poor performance for training deep and complex neural networks, various methods that train the feedback weights simultaneously have been recently proposed [13 14 24. For example, the Weight Mirror (WM) and modified Kolen-Poll Pollack
(KP) algorithms attain very competitive training performance on deep convolutional networks [16]. In WM, the feedback weights updated by
(6) using output activations of layers and + 1 while the error propagates in the same way as FA according to equation (4). This update rule makes A B proportional to WT-1, which aligns the feedback weights with the forward weights during training discussed in [66. On the other hand, the KP algorithm updates the feedback weights as below:
AB -1 h4+1 h
A A A AW41 -n -141 h

---

This method supports updating forward and feedback weights in the same direction, and those two weights could also be aligned during training. These algorithms succeed in training deep convolutional neural networks with similar performance to BP. Both WM and KP solve the weight transport problem by avoiding explicitly exchanging information related to synaptic weights such as weight changes AW and A B between the two paths; instead, a forward neuron and its corresponding neuron in the feedback path exchange activation h and error For example, 0 of the feedback path is needed to update the forward weights W1+1 in equation (2), while fu411 of the forward path is required to update the feedback weights B in equation (6) in WM. KP also needs h1+1 to update the feedback weights Bi+2 in equation (7) However, exchanging information between two neurons constitutes a bidirectional connection, which is not supported in general biological neural network structures consisting of only unidirectional connections.
3 Activation Sharing to solve weight transport problem without bidirectional connections
Prior studies to find biologically plausible algorithms have mainly focused on approximating the forward path without modifying the forward path for training. However, the demand for correct activations to update weights inevitably incurs bidirectional connections when training feedback weights as depicted in Fig. TT. Our study, on the other hand, develops a learning algorithm not relying on accurate activations. In this section, we first review the Feedback A Alignment algorithm, which uses approximate feedback path. Based this, we show that the forward path could also be approximated, and we present an Activation Sharing algorithm. Finally, we show that the algorithm solves the weight transport problem without bidirectional connections in the network.
3.1
Approximate feedback path: Feedback Alignment
For the sake of simplicity, here we consider a linear network (i.c., we assume h441 = Wi+1 h neglecting Φ). As shown in equation ( we need the error 5 and the activation h update the weights in the network. For backpropagation, we could calculate these values very accurately. We use the activations obtained when the input propagates through layers and the errors that back-propagate through the same layers in the reverse order. Since we assume a linear network, equation (3) is simplified to 81+1 WT 2.42 Consequently, the learning process in backpropagation (Fig. 2
follows the equations below.
h141 = W441 hj
(8)
8142 W 0143
(9)
AW1+2 = -1 61-2 h
(10)
However, recent studies suggest that approximate errors could also be used for weight updates. For instance, Feedback Alignment uses approximate errors â that propagate through random fixed weights. The learning process is described in Fig. P2 and is expressed by
h = W41 h
(11)
6 BB4 3.43 # 8142
(12)
AW AW .7 hit1 # -n h
(13)
where 6, = Sy since the error at the output layer due to the loss function is the same for BP and FA. This approximate feedback path successfully trains a neural network by aligning the forward weights W with the random fixed weights B.

---

Approximate feedback path
Approximate forward path
R;e2
8142
hi+
8142
h:4z
Riez
61:2
W
W1+2"
Wx+2
12
Wirz
Bi+2
Ci+z
Wx+2
AW (+2
Weoz"
Rgst
8141
ht
Diel
81+1
h1+1
Aipl
W1+1
aW
WH+1"
Wi+1
141
Wies"
Biu1"
Cro1
aW L+-1
W
h;
8,
h
%
6,
(a) Backpropagation
(b) Feedback Alignment
(c) Forward Alignment
Figure 2: Overview of weight update process in different algorithms.
3.2 Approximate forward path: Forward Alignment Activation Sharing
The Feedback Alignment algorithm showed that approximate errors could be used to train the network. Similarly, we hypothesize that the network could be trained even if we use approximate activations h obtained by propagating the input through random fixed weights C instead of the exact forward path, which we dub Forward Alignment. The flow of the Forward Alignment is described in Fig. 2 and is expressed by the equations below.
h441 = C1+1 by + h
(14)
8142 = W. 8143
(15)
AW142 = 814 h[41 + -n 0142 h
(16)
where ho ho since the network imput is identical in both cases. The Feedback Alignment algorithm was proven to work in simple linear networks through Barbalat's lemma that the loss of the network converges to zero if certain conditions are satisfied [25]. Likewise, it can be shown that the loss converges to zero by Forward Alignment in simple linear networks through Barbalat's lemma. Experimental results confirm that the Forward Alignment algorithm aligns the forward weights with the random fixed weights during training (see Appendix A for details).
Extending this idea, we propose an Activation Sharing algorithm that updates the weights using the activations of lower layer based on two intuitions. First, since the Forward Alignment algorithm works with any random fixed weights, the identity matrix I could also be used for generating approximate activations when the adjacent layers have the same input dimension. Second, propagating the input bog through more layers with random fixed weights to obtain approximate activations h would incur more deviations from the forward activations h since h; C₁ h1-1 II C₁ hg. On the other hand, if we limit the propagation of input or activation through random fixed weights for approximate activation generation within block, which is defined network, this deviation would be significantly reduced, and the training process would more closely follow the backpropagation. For instance, each block consists of two layers, h, = C, C, h1-2- which would be closer to hy than h = Cb hg- Combining these two intuitions, we propose the Activation Sharing algorithm with a block size of follows:
of multiple consecutive layers in the
h C++1 hu hi
(17)
h14 = C142 h141 = C142 C1+1 hu = h
(18)
AW = -1,112 h = -1 8142 h²j
(19)
AW143 = -n h 17 8143 h!
(20)
5

---

By equations (19) and (20), the algorithm updates the weights of all the layers inside a block using
the same activation hts which is the output activation of the previous block (equations (17) and (18). In other words, once the activations to be shared are determined, the weights are updated using the designated activations instead of their own activations for k layers when the block size is k. Fig. 3
depicts the Activation Sharing algorithm when A 2. If k = I, 1, all layers except the first layer share the output activations of the first layer to update the weights.
welght problle
6iss
8iph
Ri+2
ho+2
A
Weo
AWW
81ez
8102
8,22
22
Wi+0 -
1212
AW
ABi
18
Bias
8iss
Wi+i -
aw... A
6:
0,
6:
Ri-2
Activation Sharing (k=2)
(b) Activation Sharing with Asymmetric Paths (k=2)
Figure 3: Activation Sharing & Activation Sharing with Asymmetric Paths.
3.3 How does Activation Sharing solve weight transport problem without bidirectional connections?
Kolen and Pollack 17 showed that when A WW- = AB where W and B are the forward and feedback weights, the network trained similarly a typical artificial neural network with symmetric forward
and feedback paths (W = -B). For proof, the weight changes with weight decay (26) are written as
AW R(t) A (t))
(21)
AB(I)
R(1) - X B(t)
(22)
where R(() denotes the weight change obtained by the learning rule in equation (2), and À denotes
weight decay factor. Then, W and B converge to identical values because W + B + 1)
(W()) + AW(I) (1)) (t) + (B))))
()) B()) (1 X) (W B()))
goes to O when < A < 16 I7].
This logic also applies the Activation Sharing algorithm. As shown in Fig. 3a. the Activation Sharing algorithm itself does not solve the weight transport problem because it still assumes symmetric forward and feedback paths (W B). However, the algorithm could be further modified to avoid the weight transport problem by relaxing this constraint and only enforcing A AW = AB in training. Specifically, assume asymmetric forward and feedback paths (i.c., W # B) and apply the Activation Sharing algorithm to both paths using the same shared activations as shown Fig. 3 Since the forward and feedback neurons share the same activations,
guarantee the following:
A AW AB -1 6142 h
(23)
AW 13 A = 17 h
(24)
Therefore, the forward and feedback weights converge to the same values as discussed above, which enables the training of deep neural networks while solving the weight transport problem. Moreover, by relaxing the constraints that each layer must use its own input activations to update the weights,

---

bidirectional connections between forward and feedback neurons are not required anymore. We name this algorithm Activation Sharing with Asymmetric Paths (ASAP) distinguish from the Activation Sharing algorithm using symmetric paths discussed in Section 3] Code is available at: wwwp://ww.....ooooooooee/////nna -hhhr-higg-ahh--syymma
Paths.
4 Biological implementation of ASAP algorithm
In previous sections, we propose the ASAP algorithm based on the observation that an approximate forward path allows for reliable neural network training. However, one question still remains: the ASAP algorithm actually biologically plausible? In other words, how is this algorithm implemented in biological neural networks? This section discusses the bio-plausibility of the ASAP algorithm in detail.
4.1 Relaxing the constraints of bidirectional connections
Some prior biologically plausible learning algorithms require bidirectional connections between forward and feedback neurons [16. While one-to-one paring between the two neurons is observed in some organisms [8 19 20 21), there exist structural constraints derived from bidirectional
connections, as discussed in
18 21]. Our ASAP algorithm could significantly relax these structural
constraints by removing strict dependency between forward and feedback neurons depicted in Fig. Ti. ASAP also has dependency between forward and feedback neurons through multiple steps. However, this multi-step dependency is more strongly supported by biological observations. Reciprocal con- nections in the brain are formed not through direct bidirectional connections between two neurons, but through multiple intra-inter laminar routes using one-way connections the visual cortex, feedforward neurons of layer cannot directly connect to feedback neurons of layer 3A. but only with the feedback neurons of layer 6 [27]. In addition, reciprocal connections are generally spatially asymmetrical. This is supported by the observation that feedforward con- nections form concentrated terminal arborizations, whereas feedback connections exhibit dispersed Moreover, many unidirectional connections are observed in the feedback direction. In area TEO, neurons receive input from areas 35 and 36, but they do not project themselves into these perirhinal cortical regions (29 30). These observations show that our spatially asymmetric ASAP algorithm makes more sense than previous algorithms using simple and
(27). For example, in
terminal arborizations [27] 28 29
symmetrical bidirectional connections.
In summary, our ASAP algorithm mitigates the structural constraints of strict two-step dependencies by removing the need for direct information exchange between forward and feedback neurons. Furthermore, it can be implemented in biological neural networks using multiple one-way skip connections, which are connections between non-adjacent layers and are frequently found in living This suggests the ASAP algorithm is easier to implement biologically compared to other algorithms relying on bidirectional connections.
organisms [1 32 3
4.2 Learning with neuron-specific signals
The learning rules for biological neural networks (e.g., Hebbian learning) typically use neuron- specific signals; they use the information available in pre- or post-synaptic neurons for updating weights. In ASAP, however, some synapses require other information not directly available in pre- or post-synaptic neurons. For example, Fig- 3 shows that AW1+2 does not use pre-synaptic neuron activity h++1 but uses the neuronal activity of lower layer h;- The post-synaptic neurons may receive these activities as they do for error signals 8,-2 and use them for weight updates, but this necessitates numerous connections at the post-synaptic neuron since each synapse requires synapse-specific signal (i.c.,& a different element of hı). This type of synapse-specifis signal transport is difficult to implement in biological neural networks. This issue could be mitigated by adopting the concept of the mirror mode in the Weight Mirror algorithm [16]. The Weight Mirror algorithm divides the learning process into two phases: engaged mode and mirror mode. In mirror mode, the feedback neuron mimics the forward neuron to generate the same activations.
Our ASAP algorithm could be realized using only neuron-specific signals in a similar way. In Fig. ASAP uses the error signals available in pre-synaptic neurons (81+2 and 8143) and the signals received by post-synaptic neurons (h:) for updating feedback weights (B1+2 and B1+3). To update
7

---

AW = shf
AW
AB
no
k-2
hr
8i+2
81+2
Mimic
Mimic
AW
AB2
AW
ABB
nor
h
A
8ie
Mimic
Mimic
hr
11 (a) Updating feedback weights
h
(b) Updating forward weights with mimicking
Figure 4: ASAP implementation using neuron-specift signals.
forward weights (W142 and W143), forward neurons mimic the shared activation (h:). Then, the pre-synaptic neuron activity (hx) and the signals received by post-synaptic neurons (8142 and 8143) are used for learning as shown in Fig. 4b. Consequently, both forward and feedback weights are updated only using neuron- neuron-specific signals.
Furthermore, the forward and feedback weight updates can occur simultaneously in ASAP. In the Weight Mirror algorithm, the feedback neurons generate error signals 61+1. and the forward neurons receive them to update the forward weights W++1 in engaged mode. Contrarily, in mirror mode, the feedback neurons mimic the forward neurons, and the feedback weights B are updated by equation (6). However, in our ASAP algorithm, the forward weights and feedback weights can be updated simultaneously since the activity of the mimicking neurons (e.g., neurons in layers / 1 and ++2 2 in Fig. 4bb is not used for updating feedback weights. In addition, our ASAP algorithm could be modified to only use neuron-specific signals without introducing mimicry. See Appendix B more details.
Experiments
To verify the effectiveness of the proposed ASAP algorithm, we measured its training performance for classification tasks on different datasets. We also implemented the backpropagation and other biologically plausible algorithms with and without bidirectional connections for comparisons. Since we aim at training deep convolutional networks using biologically plausible algorithms, we choose AlexNet [35], ResNet-18 18, and ResNet-34 (36) for experiments. The target models are trained on MNIST 37 SVHN 38), 11FAR-10, CIFAR-100 (39,, and Tiny ImageNet (40) datasets for image classification. In training, we used the stochastic gradient descent (SGD) with the momentum method 41. In addition, the weight decay method [26 was adopted, and the learning rate was adjusted through cosine annealing (4).
In all experiments, the block size & was set to 2, but we also experimented with A = 4 for ResNet-34 344. Since fully-connected layers can be trained sufficiently well with the Feedback Alignment, ASAP was applied only in convolutional layers, while cunnecco layers are trained using Feedback Alignment. In ASAP, we update the weights of the layers in a block using the shared activations. However, when the channel size increases as it passes through convolutional layer or the feature map size decreases after a maxpool or convolutional layer, the shared activations cannot be used for weight changes in later layers because of the di lifferenee in dimensions. In this case, we apply maxpool shared activation when activation map sizes differ and concatenate along the channel dimension when channel sizes vary, to match its dimension with that of the corresponding layer. The input

---

activation of the first layer. which is the network's 's imput, has much lower number of channels than the following layers, so the input activation cannot be shared with later layers for training. Therefore,
we use random fixed weights in the first layer and apply ASAP to the remaining convolutional layers. More details on the experiments are provided in Appendix
Table I: Test accuracy of BP, KP, FA, DFA, and ASAP (k=2) in classification task
Dataset
Model
BP
KP
FA
DFA
ASAP
MNIST
AlexNet
99.59
99.55
99. 14
99.28
99,32
AlexNet
94.64
93.3
82.21
87.42
88.04
SVHN
ResNet-18 (nsc*)
96.14
95.84
84.91
85.64
93.17
ResNet-18
96.29
96.13
85.08
85.56
94.86
AlexNet
90.58
79.23
67.92
73.85
78,25
ResNet-18 (nsc*)
94.70
94.65
71.38
75.46
83.44
CIFAR-10
ResNet-18
94.93
94.76
72.47
76.0
92.19
ResNet-34
95.18
94.54
66.99
73.02
93.97
AlexNet
63.61
47.43
33.32
35.38
46.99
ResNet-18 (nsc*)
77.34
74.98
37.11
37.48
51.72
CIFAR-100
ResNet-18
77.74
74.51
38.15
38.51
68.86
ResNet- 33
78.41
75.0
33.01
35.6
72.81
ResNet- 18
60.13
58.14
20.54
24.07
48.46
Tiny ImageNet
ResNet-34
62.63
59.45
16.86
21.1
52.25
*no shortcut
Good performance on deep convolutional networks Table T summarizes the performance of training algorithms different datasets and models. In summary, ASAP outperforms other biologi- cally plausible algorithms without bidirectional connections (FA and DFA) in all cases, and shows very competitive performance especially on deep convolutional networks. However, it exhibits some performance degradation compared to KP that employs bidirectional connections. When training the relatively simple AlexNet on the smallest MNIST dataset, FA, DFA, and ASAP all achieve good train- ing performances and closely match BP. For more complex datasets such as CIFAR-10 10, CIFAR- 100, and Tiny ImageNet, ASAP shows significantly better test accuracy than the other algorithms without bidirectional connections. When compared to BP, ASAP shows some performance degradation, but interestingly, the performance gap between ASAP and BP considerably smaller for deeper convolutional networks such as ResNet-18 and ResNet-34. ASAP only shows 1.43% accuracy drop for ResNet-1 18 trained on SVHN, while FA and DFA show more than 10% performance degrada- tion. This trend is confirmed more clearly with complex datasets: for CIFAR- 10, ResNet- 18 and ResNet-34 trained with ASAP demonstrate over 90% test accuracy with less than 3% performance drop compared to BP, whereas FA and DFA only show up to 76% accuracy. For CIFAR-100. ASAP achieves 68.86% and 72.81 test accuracy for ResNet-18 and ResNet-34, respectively, translating to less than 9% performance degradation over BP. Contrarily, FA and DFA exhibit much lower test accuracy in the range of 33-39%. For Tiny ImageNet, ASAP achieves a competitive performance of 48.46% and 52.25% for ResNet-18 and ResNet-34, while FA and DFA exhibit only 16-24% test accuracy, Generally, as a convolutional network gets deeper, its performance gradually improves [43] Nevertheless, the test accuracy of FA and DFA rarely improved by using more convolutional layers in the model. On the other hand, ASAP successfully exploits deep network structures and continues improve test accuracy when having more layers in the model. Through these experiments, confirm that ASAP can train deep convolutional networks on complex datasets such as CIFAR-100 and Tiny ImageNet.
Effeet of shorteut ResNet models employ shorteut connections prevent vanishing and exploding gradients, enabling reliable training on deep convolutional networks [36.. For this reason, when training ResNet-18 18 through BP, KP. FA, and DFA, having shorteut connections generally increases test accuracy in S SVHN, CIFAR- 10, and CIFAR- 100. Interestingly, the performance gap appears larger for ASAP all datasets. This is because ASAP uses activation of lower layer to update weights. In the absence of shortcuts, the activation of a lower layer is only used to update the weights of ASAP and not used all in BP. However, if there is shortcut, the activation of lower layer added to the output activation of the previous layer for weight update in BP. Therefore, the direction

---

of weight changes of ASAP and BP becomes more similar when there is a shortcut. See Appendix E for more details.
100-
100
00
80
Acounsyy (96)
60
60
40
20
20
ASAP3--22
AS#Ftbo4)
ASAPX=4))
RA.
DPA
DPA
00 25
50
75
1.00
125
150
175
200
%
25
50
75
1.0
125
150
175
200
Spach
Epoch
(a) Test accuracy on CIFAR-10
(b) Test accuracy CIFAR- 100
Figure 5: Training performance comparisons for ResNet-34
Effeet of block size ASAP removes bidirectional connections by sharing activations across the layers in block, so the block size itself does not determine biological plausibility. As the block size increases, the number of layers updated using the same shared activations also increases. In other words, the amount of activation required to perform weight updates in training decreases due to data reuse. Therefore, if we perform Activation Sharing with larger block size k, it is possible to reduce memory access when implemented hardware. We compare the test accuracy ResNet-34 for k: = 2 and k in Fig- 5 On CIFAR- 10, there was no performance degradation when using a larger block size (k: 4). The test accuracy is 93.97% when k = 2 and 93.9% when k: = 4. while BP shows 95.18% test accuracy. On 1IFAR-100, the test accuracy is 71.93% when k 4 and 72.81% when = 2, which translates to 0.88% degradation due to increasing block size from However, using the block size of 4 considerably reduces memory access because only about 25% of activations are used compared to BP and 50% compared to for weight updates (see Appendix F for details).
4.
Discussion
has been shown that our algorithm solves the weight transport problem without bidirectional connections by updating forward and feedback weights using shared activation. While ASAP still cxhibits some performance degradation compared BP and KP, it achieves competitive performance in deep convolutional networks complex datasets without using bidirectional connections. Of course, our method not completely biologically plausible. Unlike the brain, ASAP does not use spikes to propagate signals. Also, algorithm still requires paired error pathway (i.e., each forward neuron is paired with a corresponding feedback neuron), which has no biological evidence yet.
Nevertheless, our study is meaningful that it shows that possible to train deep neural networks with approximate activation, while previous biological learning methods require accurate neuronal activity. This approximate learning rule not only climinates bidirectional connections from algorithms that assume paired error pathway, but could also be applied to algorithms assuming single pathway to alleviate the constraint of accurate neural activity, which will be investigated further future work.
When implementing deep learning in hardware, large number of MAC operations and external memory access are the processing bottlenecks 44 In backpropagation, activations calculated the forward path are required in the feedback path to update weights. For large models, it is impossible to store all the activations on chip; hence, it is inevitable to store the activations externally and load them back to the chip later. Contrarily, in our algorithm, not all the activations are required for weight updates due the approximate forward path. Therefore, the ASAP algorithm good candidate for efficient hardware implementation since it could greatly reduce memory access overheads.
10

---

Acknowledgments
This work was supported by the National Research Foundation of Korea (Grant No. NRF. 2019RIC1C100422)) and the KIST Institutional Program (Project No. 2F30610-20-058).
References
[1] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning internal represen- tations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science, 1985.
[2] Yoshua Bengio, Dong-Hyun Lee, Jörg Bornschein, and Zhouhan Lin. Towards biologically plausible deep learning. CoRR, 2015.
[3] Alexander G. Ororbia II, Patrick Haffner, David Reitter, and C. Lee Giles. Learning to adapt by minimizing discrepancy. CoRR, 2017.
[4] Sergey Bartunov, Adam Santoro, Blake A. Richards, Luke Marris, Geoffrey E. Hinton, and Tim- othy P. Lillicrap. Assessing the scalability of bologgially-mottiatte deep learning algorithms and architectures. In Advances in Neural Information Processing Systems, 2018.
[5] James CR Whittington and Rafal Bogacz. Theories of error back-propagation in the brain. Trends in cognitive sciences, 2019.
[6] Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cognitive science, 1987.
[7] Francis Crick. The recent excitement about neural networks. Nature, 337 132, 1989.
[8] Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin Akerman. Random synap- tic feedback weights support error backpropagation for deep learning. Nature communications, 2016.
[9] Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. In Advances in Neural Information Processing Systems, volume 29, 2016.
[10] Charlotte Frenkel, Martin Lefebvre, and David Bol. Learning without feedback: Direct random target projection as feedback-alignmen alignment algorithm with layerwise feedforward training. stat, 2019.
[11] Theodore H. Moskovitz, Ashok Litwin- Litwin-Kumar and L. F. Abbott. Feedback alignment in deep convolutional networks. CoRR, 2018.
[12] Julien Launay, Iacopo Poli, François Boniface, and Florent Krzakala. Direct feedback alignment scales to modern deep learning tasks and architectures. In Advances in Neural Information Processing Systems, volume 33, 2020.
[13] Benjamin James Lansdell, Prashanth Ravi Prakash, and Konrad Paul Kording. Learning to solve the credit assignment problem. In International Conference Learning Representations, 2020.
[14] Jordan Guerguiev, Konrad Kording, and Blake Richards. Spike-based causal inference for weight alignment. In International Conference on Learning Representations, 2020.
(15) Will Xiao, Honglin Chen, Qianli Liao, and Tomaso Poggio. Biologically-plausii learning algorithms can scale to large datasets. In International Conference on Learning Representations, 2019.
[16] Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap. and Douglas B Tweed. Deep learning without weight transport. In Advances in Neural Information Processing Systems, volume 32, 2019.
[17] John F Kolen and Jordan Pollack. Backpropagation without weight transport. International Conference Neural Networks, volume 3. IEEE, 1994.

---

[18] Andrey Gushchin and Ao Tang. Total wiring length minimization of c. elegans neural network: a constrained optimization approach. PloS one, 2015.
[19] Marion Langen, Egemen Agi. Dylan J Altschuler, Lani F Wu, Steven J Altschuler, and Pe- ter Robin Hiesinger. The developmental rules of neural superposition in drosophila. Cell, 2015.
[20] Sanford L Palay and Victoria Chan-Palay. Cerebellar cortex:
cytology and organization. 2012.
[21] João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in Neural Information Processing Systems, 2018.
[22] Kathleen S Rockland. Elements of cortical architecture. In Extrastriate cortex in primates. 1997.
[23] Charlotte Frenkel, Martin Lefebvre, and David Bol. Learning without feedback: Direct random target projection as feedback- -alinmmeel algorithm with layerwise feedforward training. stat, 2019.
[24] Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propaga- tion. In Machine Learning and Knowledge Discovery in Databases, 2015.
[25] Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin Akerman. Random feedback weights support learning in deep neural networks. arXiv preprint 2014.
[26] Anders Krogh and John A Hertz. simple weight decay can improve generalization. In Advances in neural information processing systems, 1992.
[27] Kathleen S Rockland, Jon Kaas, and Alan Peters. Cerebral Cortex: Volume 12: Extrastriate Cortex in Primates, volume 12. Springer Science & Business Media, 2013.
[28] Penelope C Murphy and Adam M Sillito. Functional morphology of the feedback pathway from area 17 of the cat visual cortex to the lateral geniculate nucleus. Journal of Neuroscience, 16(3):1 180- 1192, 1996.
[29] Kathleen Rockland and Agnes Virga. Terminal arbors of individual "feedback" axons projecting from area v2 vl in the macaque monkey: study using immunohistochemistry of anterogradely transported phaseolus vulgaris-leucongglutiii Journal of Comparative Neurology, 1989.
[30] Kathleen S Rockland and Genny W Drash. Collateralized divergent feedback connections that target multiple cortical areas. Journal of Comparative Neurology, 373(4):529 548, 1996.
[31] György Buzsáki and Kenji Mizuseki. The log-dynamic brain: how skewed distributions affect network operations. Nature Reviews Neuroscience, 1544):264-278, 2014.
[32] David Fitzpatrick. The functional organization of local circuits in visual cortex: insights from the study of tree shrew striate cortex. Cerebral cortex, 6(3):329-341, 1996.
[33] Seung Wook Oh, Julie Harris, Lydia Ng, Brent Winslow, Nicholas Cain, Stefan Mihalas. Quanxin Wang, Chris Lau, Leonard Kuan, Alex M Henry, et al. A mesoscale connectome of the mouse brain. Nature, 508(7495):207-25 224,, 2014.
[34] Alex M Thomson. Neocortical layer 6, a review. Frontiers in neuroanatomy. 4:13, 2010.
[35] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 2012.
[36] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
12

---

[37] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
[38] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[39] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[40] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 2015.
[41] Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks, 1999.
[42] Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In International Conference Learning Representations, 2017.
[43] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
[44] Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEFF, 2017.
[45] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on Machine Learning. PMLR, 2015.
[46] Hesham Mostafa, Vishwajith Ramesh, and Gert Cauwenberghs. Deep supervised learning using local errors. Frontiers in neuroscience, 12, 2018.
[47] Arild Nøkland and Lars Hiller Eidnes. Training neural networks with local error signals. In International Conference on Machine Learning,, volume 97, pages 4850- PMLR, 2019.
[48] Roman Pogodin and Peter F. Latham. Kernelized information bottleneck leads to biologically plausible 3-factor hebbian learning deep networks. In Advances in Neural Information Processing Systems, 2020.
[49] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Sur- passing human-level performance on imagenet classification. IEEE international conference on computer vision, 2015.
[50] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2015.
13


Reviews:
[{"review_id": "ictzNfBm5du", "reviewer": "Reviewer_52YK", "summary": "This paper proposes a new learning rule in which the activations of earlier layers (potentially earlier than the presynaptic layer) are used in weight updates -- that is, activations are \"shared\" for use in weight updates of multiple subsequent layers.  This causes weights to partially align with the identity matrix through a feedback alignment-like mechanism, allowing useful updates to be performed without bidirectional connections between feedforward / feedback pathways (as is needed in algorithms like Kolen-Pollack / weight mirror)", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I find the forward alignment idea, and its application (setting C = I) to the activation sharing algorithm interesting.  It is neat that this approach can work reasonably well on tasks like CIFAR.  However, I'm not convinced by the paper's motivation.  Sure, direct bidirectional connections between feedforward / feedback may not be common enough to exactly implement the Kolen-Pollack / weight mirror algorithms, but effective bidirectional connectivity via multisynaptic pathways should do the trick.  This paper's approach induces a lot of extra complexity -- how, mechanistically, are activations shared from earlier layers?  Is there any reason to think this is more biologically plausible than effective bidirectional connectivity?  Moreover, this complexity is introduced at a performance cost -- ASAP clearly lags behind backprop, while weight mirror / kollen-pollack have been shown to be very competitive with backprop (why are they not compared against directly?  Also, see https://arxiv.org/abs/2003.01513, which should probably also be cited / compared to).\n\nOverall, I am not convinced that this paper makes practical ML advances or moves closer to biological plausibility, and hence I don't think it's ready for acceptance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do address limitations, and I don't think that addressing potential negative societal impact is necessary given the subject of this paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "WNb14A8tqti", "reviewer": "Reviewer_i45S", "summary": "Methods like backpropagation require the updates to be propagated along the same pathways and weights as the forward computation step (i.e. the \"weight transport problem\"). To solve this problem, the authors propose activation sharing, which uses random fixed weights both to forwardpropagate activations and to backpropagate error, without bidirectional connections. It is argued that this is more biologically plausible, and can be used to train deep networks (which previous methods could not do).\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The ideas presented in this manuscript are very interesting, and it is nice to see that one can get reasonably good performance in deep neural networks even while propagating through random weights (i.e. the network learns how to learn). This paper is written well, in that it explains the significance and relevance of existing methods, as well as their limitations. The benchmarks were also selected well and were clear. In order to improve the clarity of the mathematical definitions of the ASAP algorithm, it would be preferable to redefine metrics like $\\delta$ and $h$ occasionally, as it is easy to forget what they mean given the large number of different symbols being used.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The biggest real-world limitation is that the method does not perform as well as backprop. This is unfortunate, but also understandable. The authors do mention that ASAP has a lower memory footprint, but there are also other methods that can reduce memory footprints of neural networks trained using backprop.\nGiven that this method is worse than backprop, and it is also not easy to implement, I cannot see any practical use for it.\n\nOn the theoretical side, although the ideas here are interesting, I take issue with the term \"biologically plausible\" and the appeal to biological networks. Given that cognitive neuroscience has not yet proceeded to a point where we understand how patterns and learning occur in brains, it is extremely premature to try and train networks that match biological neurons on the surface, and claim that we can expect better performance because they are biology inspired. To say that these networks behave more similarly to biological neurons is true only on the surface, and the claim that these networks should therefore be better or superior (in any metric, not just predictive performance) is completely unfounded (and in fact, we can see that the more \"inspiration\" we draw from biological neurons, the worse our artificial networks tend to be). In this particular case, humans can perform image classification nearly perfectly, and better than the best CNNs trained with backprop. And these CNNs trained with backprop do better than any networks trained using other methods (including ASAP). To clarify, I do not blame (or penalize) the authors for appealing to biological networks, since I think this is a bigger issue in the theoretical ML community as a whole, but I do implore them to soften the language and recognize the severe limitations that prevent us from claiming homology between artificial neural networks and biological neural networks (at least in this decade). I encourage the authors to explicitly clarify that: 1) biological neurons are not yet understood, so drawing inspiration from the little we know does not improve our chances at building better artificial networks; 2) the artificial networks trained using ASAP (and similar methods) do not improve our understanding of biological neurons at all; and 3) the artificial networks trained using ASAP (and similar methods) do not necessarily resemble biological networks (other than the weight transport problem, which is of arguable importance) more than other techniques like backprop. Again, I do not hold the authors accountable for this, and this does not affect the review I gave.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "ByvWbDpt33O", "reviewer": "Reviewer_Fjkm", "summary": "This work presents a new neural network learning algorithm that removes some of the constraints required by the traditional backpropagation algorithm. Constraints such as explicit bidirectional connections and symmetric weights between the forward and feedback paths are not used with the proposed algorithm. Through experiments on multiple popular datasets, the authors demonstrate their algorithm is competitive or better than several other learning algorithms including Feedback Alignment and Direct Feedback Alignment despite having fewer constraints.\n\nThis work also provides a thorough overview of the signals propagated through the neural network for algorithms like backpropagation, feedback alignment, direct feedback alignment, and weight mirroring. From this review, it is easy to distinguish the differences between how these signals are combined to generate updates for forward and backward paths.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The algorithm proposed in this work appears to be novel, but the benefit of using this particular idea over others seems to be the avoidance of direct bidirectional connections. Beyond the weight transport problem, this paper suggests that bidirectional connections are also biologically unlikely. But there is no clear definition found in the text of what is strictly a bidirectional connection. From a full read-through, it may be defined as both the feedforward and feedback paths having a direct co-dependency with one another over the same set of variables (h, \\delta). But this definition of directionality is rather weak because any chain a 3-step chain of dependency would escape this constraint. The proposed activation sharing algorithm has a 3-step dependency (see Figure 1b h -> \\delta_{l+2} -> \\delta_{l+1} ). It is unclear why a 3-step dependency is more justifiable than a direct two-point dependency.\n\nWhat constitutes a feedforward path vs a feedback pathway in biological systems? Given any randomly selected patch of cortex, what would qualify a neuron as a feedforward as opposed to feedback? Is there any identifiability of these classes of pathways that translate into analogous artificial neural network counterparts? Answers to this are critical to currently justify this algorithm over alternatives.\n\nWhile removing constraints to backpropagation and other algorithms is useful in its own right, an analysis empirical or theoretical of the utility of removing constraints would help justify this algorithm beyond simply being more 'biologically plausible'. \n\nOverall, the Forward Alignment concept proposed in this work is creative and shows another way backpropagation can be modified drastically and still support reasonable efficiency in learning. The work also does well in building upon previous efforts in this direction and adopting aspects like asymmetric paths into the proposed learning method.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Modifications to consider based on the main review:\n\n-Adding a formal definition of what constitutes a feedback path.\n\n-A table showing the direct mapping of how the components of the artificial model proposed map to their respective biological counterparts.\n\n-Justification for why identical weight updates is a more reasonable possibility in biological models as compared to identical weights.\n\n-An ablation to show the impact of weight decay on empirical results because it appears to be theoretically necessary (eq 24, eq 25)\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "AxTTDJTMmlu", "reviewer": "Reviewer_aqbm", "summary": "The paper proposes a novel approximation to backpropagation that solves the need for bidirectional connections between the forward pass neurons, and the backwards pass circuitry. This is done via activation sharing, meaning that the weight update is computed by the backward pass circuitry and activations of some layer prior to the current one.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The activation sharing idea is new, and the overall algorithm (ASAP) combines it with the Kolen-Pollack algorithm to also avoid the weight transport problem. However, the paper might benefit from comparison (in terms of both biological plausibility and performance) of their algorithm with other biologically plausible learning rules that do not approximate backprop, but still achieve competitive performance. Some examples include layer-wise learning rules [1-4] and equilibrium propagation [5]. It might also be a good idea to discuss the predictive coding view on backprop approximations (e.g. [6]).\n\nQuality: The theoretical results in the paper are sound, and the experiments good enough (although again, should be compared with alternative methods mentioned above). \n   \nClarity: Overall, the paper is well-written. However, I think a few claims should be corrected. \nIn the abstract, “also achieving good performance even on deep convolutional neural networks, unlike other biologically plausible algorithms” does not hold as alternative methods (see citations above) have achieved comparable performance too. \nIn line 26, “it is impossible to make their weights identical without explicitly passing the weights between the paths” contradicts the weight mirror and Kolen-Pollack algorithms discussed later, as they do exactly that. Also, I would change dots over variables to tilde, as dots usually denote a time derivative.\n     \nSignificance: The paper proposes an interesting approximation to backprop. However, I have a major concern about the biological plausibility of the proposed algorithm. \nFirst, the bidirectional connections might actually be a good idea, for instance if the backward pass is done within the neuron via apical dendrites [7]. \nSecond, activation sharing makes the final algorithm more biologically implausible that just Kolen-Pollack, in my opinion. In approximations to backprop, the error signal is neuron-specific: each neuron receives its own $\\\\delta$, and uses the pre-synaptic activity to do the weight update. In ASAP, each synapse, not neuron, receives a unique error signal that doesn’t use any information available in either pre- or post-synaptic neurons. The authors should discuss how this can be implemented. \n\n[1] Deep supervised learning using local errors, Hesham Mostafa, Vishwajith Ramesh, and Gert Cauwenberghs\n\n[2] Training neural networks with local error signals, Arild Nøkland and Lars Hiller Eidnes\n\n[3] Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks, Roman Pogodin and Peter Latham\n\n[4] Local plasticity rules can learn deep representations using self-supervised contrastive predictions, Bernd Illing, Jean Ventura, Guillaume Bellec, Wulfram Gerstner\n\n[5] Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing Its Gradient Estimator Bias, Axel Laborieux, Maxence Ernoult, Benjamin Scellier, Yoshua Bengio, Julie Grollier and Damien Querlioz\n\n[6] Predictive Coding Approximates Backprop along Arbitrary Computation Graphs, Beren Millidge, Alexander Tschantz, Christopher L. Buckley\n\n[7] Dendritic cortical microcircuits approximate the backpropagation algorithm, João Sacramento, Rui Ponte Costa, Yoshua Bengio, Walter Senn\n\n**Update: increased the score from 6 to 7 (explanation in the thread below)**\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper the authors present a solution to the problem of bidirectional connections that exist in some solutions to the weight transport problem. More specifically, Akrout et al. (2019) (https://arxiv.org/abs/1904.05391) proposed two algorithmic solutions to the weight transport problem that involved training a separate "error pathway" of neurons, wherein each neuron in the forward processing pathway had a paired error neuron that it was bidirectionally connected to. This is potentially problematic from a biological perspective, as there is no evidence for such a paired error pathway in the neocortex, and moreover, there is no guarantee of bidirectional connections in real neural networks. The present paper solves the bidirectional connection problem (but not the paired error pathway problem) by allowing feedforward neurons to project to multiple error neurons as long as they are equal to or higher in the hierarchy. The authors show that weight alignment and learning can still proceed fairly well under these conditions.

This paper received mixed reviews on the borderline. One of the reviewers was fairly positive, two were borderline accepts, and one felt it was a clear reject. The authors' responses were thorough and clear enough that one of the borderline reviewers updated their score from a 6 to a 7, leading to a final average score of 6 (i.e. a borderline accept). As such, this is a borderline case as an AC. This is especially so because biological plausibility is a tricky concept and sometimes hard to pin down. Generally, it is hard to justify rejecting a paper on biological plausibility alone. And, this paper appears to be technically sound. But, biological plausibility is also not a concept without any meaning, and there are some basic aspects of neurophysiology that we can generally employ when asking about biological plausibility. Moreover, when a paper is framed as having its central contribution be that it is more biologically plausible than previous algorithms the question of biological plausibility necessarily becomes the focus. 

Nonetheless, after all these considerations, and after a lot of discussion, an accept decision was reached. Ultimately, the paper is a sound contribution to the field and deserves to be published.

However, given the reviews, the authors may want to consider the following issues for when they prepare the camera ready version though:

1) The paper is focused 100% on solving a problem that exists for the algorithms proposed in the paper of Akrout et al. (2019). Only this paper assumes bidirectional connections between the backward and forward pathway neurons. Other papers on weight alignment (such as: https://www.nature.com/articles/s41593-021-00857-x, https://openreview.net/forum?id=rJxWxxSYvB, http://proceedings.mlr.press/v119/kunin20a.html and https://arxiv.org/abs/2005.04168) don't use a separate error pathway with paired neurons and bidirectional connections. Thus, at face value, this paper's contribution is really just to address a gap in biological plausibility introduced by a specific paper (Akrout et al. 2019). That's fine, but it makes the potential impact much more limited. Can the authors articulate how their approach may be used more broadly than to just solve the bidirectional connection issue in Akrout et al.?

2) The proposed solution is not clearly all that more biologically plausible than that proposed by Akrout et al., based on well-known facts of neurophysiology. Specifically, it appears that this model still mandates that an error pathway with paired neurons for the feedforward pathway exists (i.e. each feedforward neuron has an error neuron partner), and this was arguably the most problematic aspect of the Akrout et al. proposal, since there is zero empirical evidence for such paired pathways in the brain. Moreover, there are other biological plausibility issues that this paper introduces, as pointed out by the critical reviewers, such as the errors now being synapses specific and the need for additional feedforward pathways with the existing (and as noted) biologically implausible paired error pathway neurons. It would be good for the authors to address these concerns more in the final version.

In summary, this paper is technically sound, and potentially interesting, so it is an accept. But, per some of the reviews, at face value it is focused on solving a problem in a specific model, and there are still issues with biological plausibility, so the authors should consider these issues when preparing the camera ready version.

Decision: Accept (Poster)