Spike-based causal inference for weight alignment
Authors: Jordan Guerguiev
Konrad Kording
Blake Richards
Venue:  (2020)
Keywords: causal
inference
weight
transport
rdd
regression
discontinuity
design
cifar10
biologically
plausible

Abstract: In artificial neural networks trained with gradient descent, the weights used for processing stimuli are also used during backward passes to calculate gradients. For the real brain to approximate gradients, gradient information would have to be propagated separately, such that one set of synaptic weights is used for processing and another set is used for backward passes. This produces the so-called "weight transport problem" for biological models of learning, where the backward weights used to calculate gradients need to mirror the forward weights used to process stimuli. This weight transport problem has been considered so hard that popular proposals for biological learning assume that the backward weights are simply random, as in the feedback alignment algorithm. However, such random weights do not appear to work well for large networks. Here we show how the discontinuity introduced in a spiking system can lead to a solution to this problem. The resulting algorithm is a special case of an estimator used for causal inference in econometrics, regression discontinuity design. We show empirically that this algorithm rapidly makes the backward weights approximate the forward weights. As the backward weights become correct, this improves learning performance over feedback alignment on tasks such as Fashion-MNIST and CIFAR-10. Our results demonstrate that a simple learning rule in a spiking network can allow neurons to produce the right backward connections and thus solve the weight transport problem.

Published as a conference paper at ICLR 2020
SPIKE-BASED CAUSAL INFERENCE ALIGNMENT
FOR WEIGHT
Jordan Guerguiev Konradd P. Kording', Blake A. Richards 4,5677
Department of Biological Sciences, University of Toronto Scarborough, Toronto, ON, Canada
2 Department of Cell and Systems Biology, University of Toronto, Toronto, ON, Canada
3 Department of Bioengineering, University of Pennsylvania, PA, United States
4 Mila, Montreal, QC. Canada
5 Department of Neurology & Neurosurgery, McGill University, Montreal, QC, Canada
School of Computer Science, McGill University, Montreal, QC, Canada
7 Canadian Institute for Advanced Research, Toronto, ON, Canada
Corresponding author, email: blake.richards@men
ABSTRACT
In artificial neural networks trained with gradient descent, the weights used for processing stimuli are also used during backward passes to calculate gradients. For the real brain to approximate gradients, gradient information would have be propagated separately, such that set of synaptic weights is used for pro- cessing and another set is used for backward passes. This produces the so-called "weight transport problem" for biological models of learning, where the backward weights used to calculate gradients need to mirror the forward weights used to process stimuli. This weight transport problem has been considered so hard that popular proposals for biological learning assume that the backward weights are simply random, in the feedback alignment algorithm. However, such random weights do not appear to work well for large networks. Here we show how the dis- continuity introduced in a spiking system can lead to a solution this problem. The resulting algorithm is a special case of an estimator used for causal inference in econometrics, regression discontinuity design. We show empirically that this algorithm rapidly makes the backward weights approximate the forward weights. As the backward weights become correct, this improves learning performance over feedback alignment tasks such as Fashion-MNIST, SVHN, CIFAR-10 and VOC. Our results demonstrate that simple learning rule in a spiking net- work allow neurons produce the right backward connections and thus solve the weight transport problem.
INTRODUCTION
Any learning system that makes small changes to its parameters will only improve if the changes correlated to the gradient of the loss function. Given that people and animals can also show clear behavioral improvements on specific tasks (Shadmehr et al. 2010), however the brain determines its synaptic updates, on average, the changes must also correlate with the gradients of some loss function related to the task (Raman et al. 2019). As such, the brain may have some way of calculating least an estimator of gradients.
To-date, the bulk of models for how the brain may estimate gradients are framed in terms of setting up system where there are both bottom-up. feedforward and top-down, feedback connections. The feedback connections are used for propagating activity that can be used to estimate gradient
Williams 1992, Lillicrap et al. 2016 Akrout et al., 2019 Roelfsema Ooyen! 2005 Lee et al.) 2015 Scellier & Bengio 2017| 7 Sacramento et al.) (018). In all such models, the gradient estimator is less biased the more the feedback connections mirror the feedforward weights. For example, in the REINFORCE algorithm (Williams) 1992), and related algorithms like AGREL (Roelfsema &

---

Published as a conference paper at ICLR 2020
Ooyen 2005).
learning is optimal when the feedforward and feedback connections are perfectly symmetric, such that for any two neurons and j the synaptic weight from to equals the weight from j to i, e.g. W W (Figure TT. Some algorithms simply assume weight symmetry, such as Equilibrium Propagation (Scellier & Bengio 2017). The requirement for synaptic weight symmetry is sometimes referred to the "weight transport problem", since it seems mandate that the values of the feedforward synaptic weights are somehow transported into the feedback weights, which is not biologically realistic (Crick) 1989-01-12] Grossberg 1987). Solving the weight transport problem is crucial to biologically realistic gradient estimation algorithms (Lillicrap et al.) 2016), and is thus an important topic of study.
Several solutions to the weight transport problem have been proposed for biological models, includ- ing hard-wired sign symmetry (Moskovitz et al. 2018), random fixed feedback weights (Lillierap et al. 2016), and learning to make the feedback weights symmetric (Lee et al.] 2015 Saccrmento ct al. 2018| et al. 2019 Kolen& & Pollack] 1994). Learning to make the weights symmetric
promising because it is both more biologically feasible than hard-wired sign symmetry (Moskovitz ct al. 2018) and it leads to less bias in the gradient estimator (and thereby, better training results) than using fixed random feedback weights (Bartunov et al.. 2018 Akrout al. 2019). However, of the current proposals for learning weight symmetry some do not actually work well in practice (Bar-) tunov al. 2018) and others still rely on some biologically unrealistic assumptions, including scalar value activation functions (as opposed to all-or-none spikes) and separate error feedback pathways with one-to-one matching between processing neurons for the forward pass and error propagation neurons for the backward pass Akrout et al. (201)); Sacramento et al. (2018).
Interestingly, learning weight symmetry is implicitly a causal inference problem-the feedback weights need to represent the causal influence of the upstream neuron on its downstream partners. As such, we may look to the causal infererence literature to develop better, more biologically realis- tic algorithms for learning weight symmetry. In econometries, which focuses on eenusiieperimetts, researchers have developed various means of estimating causality without the need to actually ran- domize and control the variables in question (ngrrstt & Pischke (2008): Marinescu et al. (2018) Among such quasi-cxperimental methods, regression discontinuity design (RDD) is particularly promising. It uses the discontinuity introduced by threshold to estimate causal effects. For ex- ample, RDD can be used to estimate the causal impact of getting into a particular school (which is discontinuous, oll-ee-none variable) later earning power. RDD is also potentially promising for estimating causal impact in biological neural networks, because real neurons communicate with discontinuous, all-or-none spikes. Indeed, has been shown that the RDD approach can produce unbiased estimators of causal effects in a system of spiking neurons Lansdell & Kording (2019). Given that learning weight symmetry fundamentally a causal estimation problem, we hypothe- sized that RDD could be used to solve the weight transport problem in biologically realistic, spiking neural networks.
Here, we present a learning rule for feedback synaptic weights that is special case of the RDD algo- rithm previously developed for spiking neural networks (Lansdell & Kording 2019). Our algorithm takes advantage of neuron's spiking discontinuity to infer the causal effect of its spiking on the activity of downstream neurons. Since this causal effect is proportional to the feedforward synaptic weight between the two neurons, by estimating it, feedback synapses can align their weights to be symmetric with the reciprocal feedforward weights, thereby overcoming the weight transport prob- lem. We demonstrate that this leads to the reduction of cost function which measures the weight symmetry (or the lack thereof), that it can lead to better weight symmetry in spiking neural networks than other algorithms for weight alignment (Akrout al. (2019) and it leads to better learning in deep neural networks in comparison to the use of fixed feedback weights (Liliicrap el al.) (0016). Al- together, these results demonstrate a novel algorithm for solving the weight transport problem that takes advantage of discontinuous spiking, and which could be used in future models of biologically plausible gradient estimation.
2 RELATED WORK
Previous work has shown that even when feedback weights in neural network are initialized ran- domly and remain fixed throughout training. the feedforward weights learn to partially align them- selves to the feedback weights, algorithm known as feedback alignment (Lillicrap et al.) 2016).
2

---

Published as a conference paper at ICLR 2020
Outputs
Processing
Gradients
Inputs
Figure Illustration of weight symmetry neural network feedforward and feedback connections. Processing of inputs to outputs mirrored by backward flow of gradient information. Gradient estimation is best when feedhack synapses have symmetric weights to feedforward synapses (illustrated with colored circles).
While feedback alignment is successful at matching the learning performance true gradient de- scent in relatively shallow networks, it does not scale well to deeper networks and performs poorly on difficult computer vision tasks (Bartunov et al. 2018).
The gap in learning performance between feedback alignment and gradient descent can be overcome feedback weights are continually updated to match the sign the reciprocal feedforward weights Moskovitz al.) 2018). Furthermore, learning the feedback weights in order to make them more symmetric to the feedforward weights has been shown to improve learning over feedback alignment (Akrout al. 2019).
To understand the underlying dynamics learning weight symmetry, Kunin et al. 2019 define the symmetric alignment cost function, RSA. one possible cost function that, when minimized, leads to weight symmetry:
RSa == IIW y²1 = 1| + || -WWY)
where W are feedforward weights and Y are feedback weights. The first two terms are simply weight regularization terms that can be minimized using techniques like weight decay. But, the third term is the critical one for ensuring weight alignment.
In this paper we present a biologically plausible method of minimizing the third term. This method is based on the work of Lansdell & Kording (2019), who demonstrated that neurons can estimate their causal effect on global reward signal using the discontinuity introduced by spiking. This accomplished using RDD, wherein a piecewise linear model is fit around discontinuity, and the differences in the regression intercepts indicates the causal impact of the discontinuous variable. In Lansdell & Kording (2019), neurons learn piece-wise linear model of reward signal function of their input drive, and estimate the causal effect of spiking by looking at the discontinuity at the spike threshold. Here, modify this technique to perform causal inference the effect of spiking on downstream neurons, rather than a reward signal. We leverage this to develop a learning rule for feedback weights that induces weight symmetry and improves training.
3

---

Published as a conference paper at ICLR 2020
3 OUR CONTRIBUTIONS
The primary contributions of this paper are as follows:
We demonstrate that spiking neurons can accurately estimate the causal effect of their spik- ing on downstream neurons by using a piece-wise linear model the feedback as function of the input drive to the neuron.
We present learning rule for feedback weights that uses the causal effect estimator to encourage weight symmetry. We show that when feedback weights update using this algo- rithm minimizes the symmetric alignment cost function, RSA-
We demonstrate that this learning weight symmetry rule improves training and test accuracy over feedback alignment, approaching gradient-deseenntleeell performance Fashion-MNIST, SVHN, CIFAR-10 10 and VOC deeper networks.
4 METHODS
4.1 GENERAL APPROACH
In this work, we utilize a spiking neural network model for aligning feedforward and feedback weights. However, due to the intense computational demands of spiking neural networks, we only use spikes for the RDD algorithm. We then use the feedback weights learned by the RDD algorithm for training a non-spiking convolutional neural network. We do this because the goal of our work here is to develop an algorithm for aligning feedback weights in spiking networks, not for training feedforward weights in spiking networks on other tasks. Hence, in the interest of computational expediency, only used spiking neurons when learning align the weights. Additional details on this procedure are given below.
4.2 RDD FEEDBACK TRAINING PHASE
At the start of every training epoch of convolutional neural network, weight training phase, during which all fully-connected sets of feedback weights in the network updated. To perform these updates, (LIF) neurons. LIF neurons incorporate key elements of real neurons such as voltages, spiking thresholds and refractory periods. Each epoch, begin by training the feedback weights in the LIF network. These weights are then transferred to the convolutional network, which is used for training the feedforward weights. The new feedforward weights are then transferred to the LIF net, and another feedback training phase with the LIF net starts the next epoch (Figure 24). During the feedback training phase, the LIF network undergoes training phase lasting 90 of simulated time (30 per set of feedback weights) (Figure B). We find that the spiking network used for RDD feedback training and the convolutional neural network are very closely matched in the activity of the units (Figure SI). which gives us confidence that this approach of using a separate non-spiking network for training the feedforward weights is legitimate.
RDD feedback
simulate a separate network of leaky integrate-and- fire
During the feedback training phase, small subset of neurons in the first layer receive driving input that causes them to spike, while other neurons in this layer receive no input (see Appendix [A.]). The subset of neurons that receive driving input is randomly selected every 100 ms of simulated time. This continues for 30 in simulated time, after which the same process occurs for the subse- quent hidden layers in the network. This protocol enforces sparse, de-correlated firing patterns that improve the causal inference procedure of RDD.
4.3 LIF DYNAMICS
During the RDD feedback training phase, each unit in the network is simulated as a leaky integrate- and-fire neuron. Spiking inputs from the previous layer arrive at feedforward synapses, where they are convolved with temporal exponential kernel to simulate post-synaptic spike responses = [pi [Pt,P2,... pm] (see Appendix [A.]). The neurons can also receive driving input p;, instead of synaptic inputs. The total feedforward input neuron is thus defined as:
4

---

Published as a conference paper at ICLR 2020
A
Time
Conv Net
LIF Net
FC (10)
Train & transfer feedback weights
FC 3 (10)
30
30
FC 3
FC
FC
FC 2 (192)
FC 2 (192)
FC (384)
FC
F
FC 2
FC 1 (384)
Train transfer feedforward weights
Conv2D Pool
FC 1
FC 1
FC 0 (1600)
Conv2D Pool
driving input
FC
FC
FC
Input
C
T
U2
max
U
D
Ui
20
0
0.05
Adj
Bik
9
0.3
Te
par
0.2
Qk
-0.05
0.1
-0.1
40
60
80
100
0.9
1.1
1.4
Time (8)
Figure 2: A. Layers of the convolutional network trained on CIFAR-10 and the corresponding network of LIF neurons that undergoes RDD feedback training. Fully-connected feedforward weights (bluc) and feedback weights (red) are shared between the two networks. Every training epoch consists phase where feedback weights in the LIF net are updated (bold red arrows) and transferred to the convolutional net, and regular training phase where feedforward weights in the convolutional net are updated (bold blue arrows) and transferred back to the LIF net. B. RDD feedback training protocol. Every 30 s, different layer in the LIF network receives driving input and updates its feedback weights (red) using the RDD algorithm. C. Top: Sample voltage (U:, solid line) and input drive (u., dashed line) traces. Whenever vi approaches spiking threshold, an RDD window lasting T ms is triggered. u is the maximum input drive during this window of time. Bottom: Feedback received at synapse, ge. o% is the feedback signal at the start of an RDD window, while o is the average of the feedback signal during the time window. D. Samples of Ag maa are used to update a piece-wise linear function ui=xx and the causal effect B.x. is defined the difference of the left and right limits of the function at the spiking threshold.
RDD feedback training
5

---

Published as a conference paper at ICLR 2020
wp; f=1 W
pf > 0
4
otherwise
where Wi is the feedforward weight to neuron from neuron j in the previous layer, and w is hyperparameter, The voltage of the neuron, V/ evolves
du; = - 9 24)
dt
where 9L and dD are leak and dendritic conductance constants, respectively, The inpul drive to the neuron, Ug is similarly modeled:
du4 dt -1,; 91 u.)
If the voltage vs passes spiking threshold the neuron spikes and the voltage reset to value Ureset 1 (Figure Note that the input drive does not reset. This helps us to perform regressions both above and below the spike threshold.
In addition to feedforward inputs, spiking inputs from the downstream layer arrive at feedback synapses, where they create post-synaptic spike responses q [q1, I2.... 9n). These responses are used in the causal effect estimation (see below).
4.4 RDD ALGORITHM
Whenever the voltage approaches the threshold (ie. |0g - 0 < a where a a constant), an RDD window is initiated, lasting T 30 ms in simulated time (Figure 2C). At the end of this time window, at each feedback synapse, the maximum input drive during the RDD window, u w and the average change in feedback from downstream neuron k during the RDD window, Agk avg recorded. Agj ave is defined the difference between the average feedback received during the RDD avg and the feedback the start of the RDD window, i.
window, 4x
Aq
4s
avg
pre
Importantly, uI provides a measure of how strongly neuron i was driven by its inputs (and whether or not passed the spiking threshold 0), while A is measure of how the input received as feedback from neuron changed after neuron was driven close to its spiking threshold. These two values are then used to piece-wiss linear model of Ag as a function of u. DD) This piece-wise linear model is defined as:
J c
if <<<0
(x(x) =
Gik Gjk
(6)
if 2 0
The parameters ota, c2e oit and e/f are updated to perform linear regression using gradient descent:
= [15(a(u,TT)) max
(7)
2
0L
Ad x
for {1,2,3,4]
(8)
del
An estimate of the causal effect of neuron i spiking on the activity of neuron k. Bu. is then defined as the difference in the two sides of the piece-wise linear function at the spiking threshold:
Bix
lim fa(xx) lim fa((x) x-661
x-8

---

Published as a conference paper at ICLR 2020
Finally, the weight the feedback synapse, Ya., is updated be scaled version of Bas:
Yu Bax o
(10)
where 7 is hyperparameter and o3 is the standard deviation of 3 values for all feedback synapses the layer. This ensures that the scale the full set of feedback weights between two layers in the network remains stable during training.
5 RESULTS
5.1
ALIGNMENT OF FEEDBACK AND FEEDFORWARD WEIGHTS
To measure how well the causal effect estimate at each feedback synapse, 8u. and thus the feed- back weight Ysk, reflects the reciprocal feedforward weight Whi, we can measure the percentage of feedback weights that have the same sign as the reciprocal feedforward weights (Figure When training on CIFAR-10 with no RDD feedback training phase (ie. feedback weights remain fixed throughout training), the feedback alignment effect somewhat increases the sign alignment during training, but it is ineffective aligning the signs of weights in earlier layers the network. Com- pared to feedback alignment, the addition of an RDD feedback training phase greatly increases the sign aligmnent between feedback and feedforward weights for all layers in the network, especially earlier layers. In addition, the RDD algorithm increases sign alignment throughout the hierarchy more than the current state-of- oh-the-art algorithm for weight alignment introduced recently by Akrout al. [Akrout et al. (2019) (Figure A). Furthermore. RDD feedback training changes feedback weights to not only match the sign but also the magnitude of the reciprocal feedforward weights (Figure BB), which makes better for weight alignment than hard-wired sign symmetry (Moskovitz et al. 2018
5.2
DESCENDING THE SYMMETRIC ALIGNMENT COST FUNCTION
The symmetric alignment cost function (Knnin al. 2019) (Equation ! can be broken down as:
Rsa Reeeay + R.ell
(11)
where we define Raeeay and Retf as:
Rdecay = || ||Y
(12)
Reell = WYY)
(13)
Raseay is simply weight regularization term that can be minimized using techniques like weight decay. Ryetts contrast, measures how well aligned in direction the two matrices are. Our learning rule for feedback weights minimizes the Ryet term for weights throughout the network (Figure 4 By comparison, feedback alignment decreases Reel to a smaller extent, and its ability do so diminishes at earlier layers in the network. This helps to explain why our algorithm induces weight alignment, and can improve training performance (see below).
5.3 PERFORMANCE ON FASHION-MNIST, SVHN CIFAR- AND VOC
the Fashion-MNIST, SVHN, CIFAR-10 10 and VOC datasets using standard autograd techniques (backprop). feedback alignment and our RDD feedback training phase. RDD feedback training substantially improved the network's performance over feedback alignment, and led to backprop-level accuracy on the train and test sets (Figure 5)
We trained the same network architecture (see Appendix A.3
7

---

Published as a conference paper at ICLR 2020
A
FC 1
FC 2
FC 3
100
100
100
90
90
90
Sign Alignment (%)
80
80
80
70
70
70
RDD
Akrout algorithm
Feedback alignment
50
50
40
40
40
0
50
75
100
0
50
75
100
50
75
100
Epoch
Epoch
Epoch
B
FC 1
FC 2
FC 3
Feedback weight
-0.05
0.1
-0.05
-0.1
-0.1
-0.1
0
0.1
-0.2
0
0.2
-1.0
1.0
Feedback weight
-4
-4
-4
0
-10
10
-1.0
0.05
Feedback weight
0.02
0.05
-0.02
-0.05
-0.05
-0.05
0
0.05
-0.1
0
-1.0
1.0
Feedforward weight
Feedforward weight
Feedforward weight
Figure 3: A. Evolution of sign alignment (the percent of feedforward and feedback weights that have the sign) for each fully-connected layer the network when trained on CIFAR- 10 using RDD feedback training (blue), using the algorithm proposed by AArouut et al. (2019) (purple), and using feedhack alignment (red). B. Feedforward vs. feedback weights for each fully-connected layer at the end of training, with RDD feedback training (blue), the Akrout algorithm (purple), and feedback alignment (red).
DISCUSSION
In order to understand how the brain learns complex tasks that require coordinated plasticity across many layers of synaptic connections, it is important to consider the weight transport problem. Here, we presented an algorithm for updating feedback weights in a network of spiking neurons that takes advantage of the spiking discontinuity estimate the causal effect between neurons (Figure 2) We showed that this algorithm enforces weight alignment (Figure 3), and identified loss function, Reelt, that is minimized by algorithm (Figure 4h. Finally, demonstrated that algorithm
8

---

Published as a conference paper at ICLR 2020
FC
FC 2
FC
0
0
RDD
-40
Feedback alignment
-10
Reelf
440
-80
-15
-80
20
-120
800
-25
25
50
75
100
0
25
50
75
100
0
25
50
75
100
Epoch
Epoch
Epoch
Figure 4: Evolution of Rear for each fully-connected layer in the network when trained on CIFAR-10 using RDD feedback training (solid lines) and using feedback alignment (dashed lines). RDD feedback training dramatically decreases this loss compared to feedback alignment, especially in earlier layers.
Fashion-MNIST
SVHN
CIFAR-10
VOC
100
100
100
Feedback alignment
RDD
Backprop
aroo (%)
80
60
60
71.1%
40
60
54.0%
18.4%
20
2,4%
5.7%
12.9%
2.0%
100
100
100
100
100
80
Tedt crtor (%)
80
76.3%
70.3%
70
70.5%
40
40
5.2%
23.8%
20
18.1%
50
100
250
Cooch
Epoch
Epoch
Figure 5: Comparison of Fashion-MNIST, SVHN CIFAR-10 and VOC train error (top row) and test error (bottom row). RDD feedback training substantially improves test error performance over feedback alignment in both learning tasks.
allows deep neural networks achieve better learning performance than feedback alignment on Fashion-MNIST and CIFAR-10 (Figure 5). These results demonstrate the potential power of RDD as means for solving the weight transport problem biologically plausible deep learning models.
One aspect of our algorithm that is still biologically implausible is that it does not adhere to Dale's principle, which states that neuron performs the same action on all of its target cells (Strata Har- vey). This means that a neuron's 's outgoing connections cannot include both positive and negative weights. However, even under this constraint, neuron can have an excitatory effect on one down- stream target and an inhibitory effect on another, by activating intermediary inhibitory interneurons. Because our algorithm provides a causal estimate of one neuron's 's impact on another, theoretically, it could capture such polysynaptic effects. Therefore, this algorithm is in theory compatible with Dale's ss principle. Future work should test the effects of this algorithm when implemented in a net- work of neurons that are explicitly excitatory inhibitory.

---

Published as a conference paper at ICLR 2020
REFERENCES
Mohamed Akrout, Collin Wilson, Peter C Humphreys, Timothy Lillicrap. and Douglas Tweed. Us- ing weight mirrors improve feedback alignment. arXiv preprint arXiv:1904.00 1900:1904.00 200405591, 2019.
Joshua Angrist and Jörn-Steffen Pischke. Mostly harmless econometrics: An empiricist's com- panion. Princeton university press, 2008.
Sergey Bartumov, Adam Santoro, Blake Richards, Luke Marris, Geoffrey E Hinton, and Timothy motivated deep learning algorithms and archi- tectures. In Advances Neural Information Processing Systems, pp. 9368-9378, 2018.
Lillicrap. Assessing the scalability siotogicall-mmotiatted
Francis Crick. The recent excitement about neural networks. 33766203) 1989-01-12. doi:
10.1038/337129al http://xx/x doi wtt.ooo..g///../.....oo
Stephen Grossberg, Competitive learning: From interactive activation to adaptive resonance. 11(1): 23-63, 1987. ISSN 1551-6709.
John Kolen and Jordan B Pollack. Backpropagation without weight transport. In Proceedings of 1994 IEEE International Conference Neural Networks (CCN)'94), volume 3, pp. 1375-1380. IEEE, 1994.
Daniel Kumin, Jonathan M Bloom, Alcksandrina Goeva, and Cotton Seed. Loss landscapes of regu- larized linear autoencoders. arXiv preprint 19X.v.1901.08168, 2019.
Benjamin James Lansdell and Konrad Paul Kording. Spiking allows neurons to estimate their causal effect. bioRxiv, pp. 253351, 2019.
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation. In Joint European conference on machine learning and knowledge discovery in databases, pp- 498-515. Springer, 2015.
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications, 7: 13276, 2016.
Ioana E Marinescu, Patrick N Lawlor, and Konrad Kording. Quasi-experimennal causality in neuroscience and behavioural research. Nature human behaviour, pp. 1, 2018.
Theodore H Moskovitz, Ashok Litwin-Kumar, and LF Abbott. Feedback alignment in deep convo- lutional networks. arXiv preprint arXiv:/8/2.06448 2018.
Dhruva Venkita Raman, Adriana Perez Rotondo, and Timothy O'Leary. Fundamental bounds on learning performance in neural circuits. 116221):0537-11546 2019. ISSN 0027-8424.
Pieter R Roelfsema and Arjen van Ooyen. Attention--ated reinforcement learning of internal repre- sentations for classification. Neural computation, 17(10)2217-22114 2005. 2005.
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcir- cuits approximate the backpropagation algorithm. pp. 9735-8746. 2018.
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy- based models and backpropagation. 11:24, 2017. ISSN 1662-5188. doi: 10.3389/fncom.2017 00024. URL http: //ww ncbi mlm. nih. http:///ooomarrrrlocess//////111
Reza Shadmehr, Maurice A Smith, and John W Krakauer. Error correction, sensory prediction, and adaptation in motor control. 33:89-108, 2010. ISSN 0147-006X.
Piergiorgio Strata and Robin Harvey. Dale's principle. 50(5):349-35 ISSN 0361-9230. doi:
10. 121106//$0261--22200990000000000
URLhtttp //w:://wwww.. scí encedi rect. com/sc. eecee
923099001008
Ronald J Williams. Simple statistical gradient- algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-255 1992.
10

---

Published as a conference paper at ICLR 2020
A APPENDIX
A.1 LIF NEURON SIMULATION DETAILS
Post-synaptic spike responses feedforward synapses, p. were calculated from pre-synaptic binary spikes using exponential kernel function K:
Ex(t- Py(t) Σ k(t c tzsk
(14)
&
where lys is the hthh spike time of input neuron and is given by:
x(8) - -T))
(15)
where = 0.003 and Tz = 0.01 represent short and long time constants, and θ is the Heaviside step function. Post-synaptic spike responses at feedback synapses, q, were computed the same way.
A.2 RDD FEEDBACK TRAINING IMPLEMENTATION
A.2.1 WEIGHT SCALING
Weights were shared between the convolutional network and the network of LIF neurons, but feed- forward weights the LIF network were scaled versions of the convolutional network weights:
W bj
WCon 5) (osmn//oo Wooss
(16)
where W/Com is a feedforward weight matrix in the convolutional network, W is the correspond- ing weight matrix in the LIF network, m is the number of units the upstream layer (ie. the number of columns in WCCm),, o uy is the standard deviation of WCon and t is hyperparameter. This rescaling ensures that spike rates in the LIF network stay within an optimal range for the RDD algo- rithm to converge quickly, even if the scale of the feedforward weights in the convolutional network changes during training. This avoids situations where the scale of feedforward weights so small that little no spiking occurs in the LIF neurons.
A.2.2 FEEDBACK TRAINING PARADIGM
The RDD feedback training paradigm is implemented as follows. We start by providing driving input to the first layer in network LIF neurons. create this driving input, we choose subset of 20% of the neurons in that layer, and create unique input spike train for each of these neurons using Poisson process with rate 200 Hz. All other neurons in the layer receive no driving input. Every 100 ms, new set of neurons to receive driving input is randomly chosen. After 30 s, this layer stops receiving driving input, and the process repeats for the next layer in the network.
A.3 NETWORK AND TRAINING DETAILS
The network architectures used to train on Fashion-MNIS" and CIFAR-10 are described in Table
Inputs were randomly cropped and flipped during training, and batch normalization was used each layer. Networks were trained using a minibatch size of 32.
A.4 AKROUT ET AL. (019) ALGORITHM IMPLEMENTATION
In experiments that compared sign alignment using our RDD algorithm with the Akrout et al. (2019) algorithm, kept the same RDD feedback training paradigm (ie. layers were sequentially driven, and a small subset neurons each layer was active at once). However, rather than updating feedback weights using RDD, we recorded the mean firing rates of the active neurons in the upstream
11

---

Published as a conference paper at ICLR 2020
Layer
Fashion-MNIST
SVHN & CIFAR-I0
VOC
Input
28 x 28 x
32 x 32 x
32 x 32 X
Conv2D
64 ReLU
Conv2D 5, 64 ReLU Conv2D) 5, ReL.U
2
MaxPool 2 x 2, stride 2
MaxPool x 2, stride 2
MaxPool x 2, stride 2
3
Conv2D 5 x 5, 64 ReLU
Conv2D 5 x 5, 64 ReLU
Conv2D 5 x 5, 64 ReLU
4
MaxPool x 2, stride 2
MaxPool x 2, stride
MaxPool x stride 2
5
FC 384 ReLU
FC 384 ReLU
FC 384 ReLU
6
FC 192 ReLU
FC 192 ReLU
FC 192 ReLU
FC ReLU
FC 10 ReLU
21 ReLU
Table 1: Network architectures used to train on Fashion-MNIST, SVHN, CIFAR-IO and VOC.
layer, r', and the mean firing rates the downstream layer, r/t!, We then used the following feedback weight update rule:
AY (1+1) AWD Y
(17)
where are the feedback weights between layers + and 1, and 1) and AWD are learning rate and weight decay hyperparameters, respectively.
FC 1
FC
FC 3
250
250
LF neuron spkk rate (H2)
200
150
100
100
100
50
50
0
Conv not unit activity
Conv not unit activity
Conv not unit activity
Figure S1: Comparison of average spike rates in the fully-connected layers of the LIF network vs. activities of the same layers in the convolutional network, when both sets layers were fed same input. Spike rates the LIF network are largely correlated with activities of units in the convolutional network.
12


Reviews:
[{"review_id": "S1eQziLmFB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces a training mechanism for spiking neural nets that employs a causal inference technique, called RDD, for adjustment of backward spiking weights. This technique induces the backward influence strengths to be reciprocal to the forward ones, bringing desirable symmetry properties.\n\nPros:\n * The relationship between causal inference and biologically plausible learning is very interesting. This relationship is also important and impactful for the machine learning community, as we are on the quest of new deep learning technologies.\n\n * Application of the RDD method to spiking neural net training is novel. The reciprocal relationship of the causal effect to the synaptic strength is a very intuitive and elegant solution to the weight transport problem.\n\nCons:\n * From the reported results, it is not possible to decide whether RDD really outperforms Feedback Alignment (FA). The comparison is performed on only two data sets and each algorithm is better on one. Could the authors report results on at least two more data sets (however small or simple) during the rebuttal?\n\n * Fig and Table 1 report the same outcome. One of the two need to be removed.\n\nFurther Questions:\n * The Conv Net illustrated in Fig 2 panel A shares its weights with the biologically plausible net on panel B. Further, these two nets communicate for pre-training. How does the paper then isolate the contribution of the biologically plausible net to the prediction accuracy from the vanilla ConvNet? What would happen if we trained only the LIF net without a contact with the conv net?\n\n * Eq. 1 proposes induction of symmetry to solve the weight transform. At the extreme, this regularizer would make W and Y identical, boiling down to  a vanilla artificial neural net, which the ML community already knows wella nd performs with excellence. Would not having the biologically  implausible artificial neural model as the extreme solution contradict with the goal of biologically plausible learning? This would in the end make one conclude that the biological brain only performs a broken gradient descent.\n\nOverall, this is a decent piece of work with some potential. My initial vote is a weak reject, as I  am at present missing sufficient evidence that the improved symmetry properties introduced by the causal inference scheme also brings an accuracy improvement over the vanilla feedback alignment method. I am open to improve to an accept if this evidence is provided and my aforementioned concerns primarily on the role of ConvNet are properly addressed during rebuttal.\n\n\n--\nPost-rebuttal: My only major concern was the lack of sufficient empirical evidence to support the idea. The updated version of the manuscript has properly addressed this issue by reporting results on additional data sets. The authors have also given enlightening clarifications to some of the open points I have raised earlier. Hence, I'm happy to increase my score.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "H1lpyw_atB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "summary\n\nThis paper considers the \"weight transport problem\" which is the problem of ensuring that the feedforward weights $W_{ij}$ is the same as the feedback weights $W_{ji}$ in the spiking NN model of computation. This paper proposes a novel learning method for the feedback weights which depends on accurately estimating the causal effect of any spiking neuron on the other neurons deeper in the network. Additionally, they show that this method also minimizes a natural cost function. They run many experiments on FashionMNIST and CIFAR-10 to validate this and also show that for deeper networks this approaches the accuracy levels of GD-based algorithms. \n\n\n\ncomments\n\nOverall I find this paper to be well-written and _accessible_ to someone who is not familiar with the biologically plausible learning algorithms. To overcome the massive computational burden, they employ a novel experimental setup. In particular, they use a separate non-spiking neural network to train the feedforward weights and use the spiking neurons only for alignment of weights. They have experimental evidence to show that this method is a legitimate workaround. I find their experimental setup and the results convincing to the best of my knowledge. The experimental results indeed show the claim that the proposed algorithm has the properties stated earlier (i.e., learns the feedback weights correctly and that using this to train deep neural nets provide better performance than weight alignment procedure). I must warn that I am not an expert in this area and thus, might miss some subtleties. Given this, it is also unclear to me why this problem is important and thus, would leave the judgement of this to other reviewers. Here I will score only based on the technical merit of the method used to solve the problem.\n\nI had one minor comment on the arrangement of the writing of the paper. Section 4 starts off with \"Results\" but the earlier sub-sections are not really about the results. I would split section 4 as methodology/algorithm and include the everything until section 4.4. From sub section 4.5 onwards are the actual results.\n\n\noverall decision\n\nWithout commenting on the importance of this problem, I think this paper merits an acceptance based on the technical content. The paper provides convincing experiments to test the properties the author claim the new algorithm has.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "H1lQdqkRKB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strong paper in the direction of a more biologically plausible solution for the weight transport problem, where the forward and the backward weights need to be aligned. Earlier work for feedback alignment has included methods such as hard-coding sign symmetry. In this method, the authors show that a piece-wise linear model of the feedback as a function of the input given to a neuron can estimate the causal effect of a spike on downstream neurons. The authors propose a learning rule based on regression discontinuity design (RDD) and show that this leads to stronger alignment of weights (especially in earlier layers) compared to previous methods. The causal effect is measured directly from the discontinuity introduced while spiking - the difference between the outputs of the estimated piece-wise linear model at the point of discontinuity is used as the feedback.\n\nCompared to feedback alignment, RDD-based pre-training demonstrates stronger alignment between forward and backward weights and better performance on CIFAR-10 and Fashion-MNIST datasets. Overall, the paper is very well written and addresses an important problem. The theoretical foundation, to my knowledge, is well studied.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: All authors agree the paper is well written, and there is a good consensus on acceptance.  The last reviewer was concerned about a lack of diversity in datasets, but this was addressed in the rebuttal.
Decision: Accept (Poster)