Synaptic Dynamics Realize First-order Adaptive Learning and Weight Symmetry
Authors: Yukun Yang
Peng Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: synapses
optimizer
biologically plausible
weight transport
Adam

Abstract: Gradient-based first-order adaptive optimization methods such as the Adam optimizer are prevalent in training artificial networks, achieving the state-of-the-art results.  This work attempts to answer the question whether it is viable for biological neural systems to adopt such optimization methods. To this end, we demonstrate a realization of the Adam optimizer using biologically-plausible  mechanisms in synapses. The proposed learning rule has clear biological correspondence, runs continuously in time, and achieves performance to comparable  Adam's. In addition, we present a new approach, inspired by the predisposition property of synapses observed in neuroscience, to circumvent the biological implausibility of  the weight transport problem in backpropagation (BP). With only local information and no separate training phases, this  method establishes and maintains weight symmetry in the forward and backward signaling paths, and is applicable to the proposed biologically plausible Adam learning rule.  The aforementioned mechanisms may shed light on the way in which biological synaptic dynamics facilitate learning.

Under review as a conference paper at ICLR 2023
SYNAPTIC DYNAMICS REALIZE FIRST-ORDER ADAP- TIVE LEARNING AND WEIGHT S SYMMETRY
Anonymous authors Paper under double-blind review
ABSTRACT
Gradient- first-order adaptive optimization methods such as the Adam opti- mizer are prevalent in training artificial networks, achieving the state-of-the-an re- sults. This work attempts to answer the question whether it is viable for biological neural systems to adopt such optimization methods. To this end, we demonstrate realization of the Adam optimizer using biologically-plausbb mechanisms in synapses. The proposed learning rule has clear biological correspondence, runs continuously in time, and achieves performance dition, we present a new approach, inspired by the predisposition property of synapses observed in neuroscience, to circumvent the biological implausibility of the weight transport problem in backpropagation (BP). With only local informa- tion and no separate training phases, this method establishes and maintains weight symmetry in the forward and backward signaling paths, and is applicable to the proposed biologically plausible Adam learning rule. The aforementioned mecha- nisms may shed light on the way in which biological synaptic dynamics facilitate learning.
comparable Adam's. In ad-
INTRODUCTION
Gradient-based adaptive optimization is widely used in many science and engineering applica- tions including training of artificial neural networks (ANNs) (Rumelhart et al. 1986). In particular, first-order methods are preferred over higher-order methods since their memory overhead is signifi- cantly lower, considering ANNs are often characterized by high dimension feature spaces and large numbers of parameters. Among the most well-known ANN training methods are stochastic gradient descent (SGD) with momentum (Rumelhart et al.) 1986). root mean square propagation (RMSProp) Tieleman & Hinton| 2012). and adaptive moment estimation (Adam) (Kingma & Ba 2014). Dif- ferent from gradient descent, which optimizes an loss function over the complete dataset, SGD runs on a mini-batchh The momentum term accelerates the adjustment of SGD along the direction to a minima, and RMSProp impedes the search in the direction of oscillation. The Adam optimizer can be considered as the combination of the above two ideas; and it is computationally efficient, has fast convergence, works well with noisy/sparse gradient, and achieves the state-of-the-art results many AI applications (Dosovitskiy et al., 2020; Wang et al. 2022).
Given the success of gradient-basee adaptive optimization techniques, particularly Adam, in training ANNs, is natural to ask the question whether it is viable for biological neural systems to adopt such optimization strategies. We attempt answer this question by demonstrating an implementation of the Adam optimizer based on biologically plausible synaptic dynamics and new solution to the well-known weight transport problem. We call our implementation Bio-Adam.
Nevertheless, it is not immediately clear how to realize the Adam optimizer biologically realistically given its intricacies. Comparing to the classical SGD method, Adam has two major new ingredients: use of momentum m to smooth the gradient of multiple batches, and division by smooth estima- tion /v of the root mean square of the gradient to constrain the step size, which is also known as the RMSProp term 1/(vū, e). in which is small number to prevent division by zero. Although signal smoothing is commonly done in biological modeling such as by using the -ntegrnte and fire (LIF) model of spiking neurons (Gerstner al. 2014). we identify that the root mean square calculation V and the existence of the division operator in Adam are biologically problematic. With respect to these difficulties, we define new variable p mimic the dynamics of RMSProp.

---

Under review as a conference paper at ICLR 2023
Biologically, this newly defined variable may be thought as the concentration of certain synaptic substance consumed during the learning process. Essentially, p is designed based on the following biologically motivated ideas. A large weight updating signal accelerates the consumption of the sub- stance, dropping the concentration of the substance, which in turn slows down the pace of weight update. On the other hand, under small weight updating signal, the synapse gradually replenishes the substance and restores the fast weight update pace. This kind of behavior mimics the RMSProp term 1/(vU, + e) in Adam. The overall weight update speed of our biological Adam is proportional to the product of m and p.
Another roadblock to a biologically plausible realization of Adam is the weight transport problem, which is in fact an issue common to the wider family of all BP methods. BP is generally believed to be biologically implausible in the brain for several reasons (Stork) 1989 Illing al., 2019). One key issue is that BP requires symmetrical weights between the forward and backward paths to propagate correct error information. This is also known as the weight transport problem (Grossherg. 1987 Ororbia II et al. 2017), which address by new approach, inspired by the predisposition property of synapses observed in neuroscience. According to predisposition (Chistiakova et al. 2014), weak synapses that have been depressed more prone potentiation, whereas strong ones are more prone to depression. When the forward and backward synapses share local updating signals, potentiation signal potentiates the stronger synapses less than the weaker ones. Conversely, a depression signal depresses the weaker synapses less than the stronger ones. By leveraging the biological predisposition property, the proposed mechanism eventually aligns the forward weights with the backward weights. There are other methods also mentioned to share gradients. like the weight-mirror and the modified KP algorithm presented in (Akrout al. 2019). Yet believe our method is more bio- hio-pluusible for it is based on the well-observed synaptic property. Our solution to the weight transport problem immediately applicable to the proposed Adam learning rule.
To conclude, the presented Bio-Adam learning rule is based on biologically plausible synaptic dy- namics with an underlying biological mechanism addressing the weight transport problem. More- over, our biological Adam optimizer delivers a performance level that is a par with its original implementation (Kingma & Ba 2014). Our findings may shed light on the way in which biological neural systems facilitate powerful learning processes.
BIO-PLAUSIBLE FIRST-ORDER ADAPTIVE OPTIMIZERS
2. PRELIMINARIES
The classical stochastic gradient descent (SGD) follows:
0, < 0 79t,
where f is the weights, y is the learning rate, and is the gradient. SGD with momentum (Rumelhart jet al., 1986) further introduced momentum term m to smooth the gradient g of multiple batches according to the following dynamics:
my - 11m=-1 + (1 31)9t.
where 3₁ is hyperparameter in the range of [0, 1) and is usually close to 1.0. The final updating rule is obtained by changing to m in T.. yielding: O₂ +- 0. 7m,
RMSProp (Tieleman & Hinton 2012) computes a smooth estimate of the root mean square v of the gradient and uses it in the denominator to constrain the step size in the oscillation direction. The dynamic of follows:
1/ - B24- + (1 :92)0
(3)
where 3g is a hyperparamter. Similar to 31 in updating rule of RMSProp is: 0, + 0,- - 79t/ division by zero.
2 32 E [0. 1) typically has a value near 1.0. The
(v + +e), where is a small number prevent
Adam Kingma Ba 2014 can be viewed as a combination of momentum and RMSProp by omitting minor adjustments such as the bias correction and use of the infinity norm. Adam's 's main updating rule is:
0, x- 0.- -7me/( c).
2

---

Under review as a conference paper at ICLR 2023
2.2 BIO-PLAUSIBLE REALIZATION
2.2.1 1 OVERALL DESIGN OF THE BIO-PLAUSIBLE SYNAPTIC DYNAMICS FOR BIO-ADAM
Molecular studies have connected synaptic plasticity to dozens of chemical and biological events, and the direction of adjustment, mang-term-ootetiian (LTP) or long-term depression (LTD), is a crucial determinant of whether and how chemicals are involved (Bliss & Cooke 2011 Miranda et al. 2019: Wiera & Mozrzymas 202I).
It is importantly to know that, when an optimizer like Adam is used, the instant adjusting signal corresponding to the negative gradient -g. may not be completely proportional to the actual weight update Aw. Therefore, we call the direction of the instant adjusting signal potentiation or depres- sion, to distinguish from the LTP/LTD that actually applied on the synapse.
Our synaptic model in the proposed Bio-Adam receives two types of inputs that come from the interactions with other neurons: the potentiation signal and the depression signal, which correspond to the conditions {g(t) signals are readily available on each synapse, and refer interested readers to previous works for
0} and (g(t)lg(t) > 0}, respectively. We assume that these two
more discussions (Ponulak & Kasinski 2010 Lillicrap et al. 2016 Sacramento et al. 2018. Payeur et al. 2021).
As shown in Figure I (A), our design utilizes two types of substances only: m and p. Despite the fact that biological synapses contain various additional types of substance dynamics, the two types of substances we choose here provide enough flexibility to design dynamics akin to those of first-order adaptive optimizers.
The first type contains a group of substances that are modulated complementarity by the poten- tiation and depression stimuli. As detailed in Figure [T (A), we name this type of substances as m. Further, it can be divided into m+ and m depending on whether its plasticity effect is LTP or LTD. The potentiation signal transforms m to m and the depression signal reversely trans- forms m=+ + to m Therefore, the total quantity of m+ and m does not change while the quantity of each changes in complimentary way. Such complementarity is very common in biological synapses. For example, the phosphorylation and de- did-phssshorylation processes, controlled by poten- tiation and depression stimuli respectively, follow such complementary rule and apply on multiple kinase II (CaMKII), cAMP response element-binding
targets such as duliiun/calmoduiinddeeeeeee protein (CREB) and cofilin (Knobloch & Mansuy] 2008). Based on these biological bases, the final dynamics of m in our setting exactly replicates the updating rule of the momentum term, which is the ceaky-integration (exponentially smoothing) of both potentiation and depression stimuli, or the gradient per se. The dynamics of m is illustrated in Figure (B).
The second type of substances p is co-consumed by both potentiation and depression signals. As reflected in Figure | (A), p has a simple biological dynamics, which is balanced by two forces: consumption and supplementation. We assume that the consumption speed is linearly proportional to the current concentration of substances: the more abundant the substances, the faster they are consumed. In addition, we assume that the supplementation speed is linearly proportional to the difference between the saturation value Prest and the current value p: when p is low, supplementation would be fast. While when P reaches saturation, supplementation would slow down and finally stop. This type of dynamics refers to -on-instantaneous chemical/biological processes such translation and transcription. For example, the Brain Derived Neurotrophic Factor (BDNF) is consumed by both LTP and LTD, and supplemented through the transcription in the nucleus (Miranda et al. 2019)
Interestingly, as we will show, with these chosen bio-plausible properties, / exhibits a dynamics that is nearly identical to the RMSProp term 1/(vit e) as depicted in the last subplot of Figure 1 (B).
In conclusion, our synaptic model has three inputs: the potentiation signal, the depression signal, and the supplementation signal. It operates on two types of substances: the complementarily modulated type m and the co-consumed type p. The first two input signals correspond to the negative and positive parts of the gradient g. which are assumed to be readily available. The last input comes from the intraneuron dynamics. Our proposed system adopts the basic bio-plausible leaky-integrate model on both m and p, and linear models are adopted to describe the consumption and supplementation speed of The overall weight update of the proposed Bio-Adam is proportional to the product of the density of the two types of substances m and p:
0, x- 0,1 mmmPt-
(5)
3

---

Under review as a conference paper at ICLR 2023
Receptors
LTD
LTP....
-15
20000
#0010
09000
91300
100000
1.25
& m
0.75
Activate/ Supplement
Consume
1.
49326
66600
00000
100000
1.0
Potentiation
Depression
RMSProp
RMSProp
06
Supplementation
Time/ ms
(A) Dynamics of synaptic substances
(B) dynamics of and p given example gradient
Figure I: First-order optimizers based synaptic substance dynamics. (A) m and m are the densities of two groups of substances that modulates LTP and LTD respectively, and they are com- plementarily controlled by potentiation and depression stimuli. The combination of them forms the momentum term m. (m m= m+)/2 )/2 represents the density of the second types of substances that co-consumed by both potentiation and depression stimuli. The product of and p affects the overall adjustment's 's strength and direction. (B) An example of different variables' dynamics along
time is provided. Please reference the exact dynamics in Section 2.2.2 and 2.2.3 In this example, our assumed gradient is divided into five slices along the 100s total simulation time, and the values are [0, 0.3, -0.7, 0] in each slide respectively. During the simulation, we set Tm Tp = 1000ms, Prest = 1/c = 1. Meanwhile, we set the resting densities of m and m as one. As one may observe from the bottom subplot, The proposed dynamics of converges to the same value as RM- SProp's, and only diverges slightly in the adjusting period
Moreover, all variables in the system have their clear biological correspondences. Therefore, the system yields good bio-plausibility.
Next, we present the detailed design of the proposed biological Adam rule and analyze its dynamical behavior.
2.2.2 BIOLOGICALLY PLAUSIBLE REALIZATION OF MOMENTUM
The concentration of substance is always positive. To deal with this, we implement the momentum term m using two separate concentrations m and m which produce LTP and LTD, respectively. Additionally, we name the potentiation and depression signals as x and x corresponding the absolute value of negative gradient and positive gradient:
(t) lg(()) (o(t) 0),
(t) lg(()) (o(1) 0)
(6)
Using a leaky-integration dynamics to model m and m we have our complementary rule as: Tin (dm + (t)/dt) - (m + (t) mrest) + x (1) x -(), Tin (dm (t)/dt) - (m (t) mzest) x x1) - x++x)) (1).
in which is the time constant, and mrestt is the concentration of m and m at rest, i.c., when no stimulus is received. Noting that m in (2) is the smoothed gradient, which corresponds to the opposite of the updating direction. We define m by m = (m mm -m+)/2 )/2, as the overall impact of
m and Combining the biologically plausible dynamies of (6) and 7 gives the continuous- time dynamics of m. which has form identical to the original momentum implementation of (Rumelhart et al. 1986):
dm(()) m((l) + 9().
Tm dt
The above dynamics can be discretized in time, e.g., by using the first-order forward Euler method:
m, + (1
It-
(9)
Tm
Fm
81 in corresponds to (1 1/7T)): Fm I-3 For example, 3r 0.9 correspond Tm 10.
4

---

Under review as a conference paper at ICLR 2023
Algorithm Bio-Adam, our proposed biologically plausible version of the Adam optimizer. See
Section 2 for details. Good default settings for the tested machine learning problems are = 0.0001.
Tm 10, Tp 1000 and Prest 10
Require: 7:: Stepsize
Require: Tm- T,ho: Time constant of m and p
Require: ʃ(0): Stochastic object function with parameters
Require: 0y: Initial parameter vector
mo + 0 (Initialize the vector of the complemenarily modulated substances' density) Po (Initialize the vector of the co-consumed substances' density)
f + 0 (Initialize timestep)
while 0, not converged do t ++ gt 4 VoJ=(0(-1) (Get gradients w.r.t. stochastic objective at timestep t) density)
- (1 (1-1/Tm) m + Tm (Update the complementarily modulated substances'
f- Pe- (1-1-(ro-1)+pmm (Update the co-consumed substances' density) To Duca gel
0. x 00-10 ymppi (Update parameters/synaptic strengths) end while return 0, (Resulting Parameters)
2.2.3 BIOLOGICALLY PLAUSIBLE REALIZATION OF RMSPROP
The second type of substances p that is co-consumed by potentiation and depression stimuli is bal- anced by two factors: the intraneuron supplementation speed st, and the consumption speed sT. First, we set the supplementation speed 5+ to be proportional to the difference between p(t) and its resting value Prest:
s (1) = k [Prest - p(()),
(10)
in which ++ is a constant. Second, since p is co-consumed during both potentiation and depression, consider the rate of consumption for pto be dependent the absolute value (g(()] of the gradient. In addition, we make the consumption rate proportional to the current concentration. Consequently, the product of [g()) and p(t) sets the consumption rate:
(1) kー p(t) l9((),,
(11)
where - is also constant. The overall dynamics of is controlled by st and ST:
dp(t) =s((1) (())
Tp dt
(12)
b+ Prestt p(t)] K- p(t) [g(())
[Properties of Our Biological RMSProp] Surprisingly, we show that the equilibrium the dy- namics in (12) converges to the desired quantity of 1/(. + e) as what is used for weight update in the original RMSProp implementation (Tieleman & Hinton 2012) based on (3). To see this, first note that at a constant gradient at equilibrium, the exponentially smoothed mean square root of gradient /ut converges the absolute value of gradient lg). the RMSProp term converges to
1/(1gl + e). Furthermore, setting dp(l)/dt di. = 0 and g(() in (12) gives the equilibrium value of as:
0 k (prett p(oo)] b- -((o)) lgl
p(00) kt Prest
(13)
1-1g]+ k-
which has a form very similar to that of the equilibrium value of the original RMSProp. 1/(lg +c). We can force the equilibrium value p(xo) of the proposed dynamics be 1/(1g) + <) by properly choosing constant values &+ k and Prest of our model:
k Prest
11
(14)
1: [g
lif
lgl +
k
Presek" k
5

---

Under review as a conference paper at ICLR 2023
Bring these two constrains into (12). we have:
dp(() Tp dt Prest p(O]
Prest p(() lg(()), where Prest 1/c
(15)
In the above, the constant k+ is absorbed into the time constant Tp. and the hyperparameter Tp sets the duration of the time window for the exponential moving average of p. Based on the same reasoning used to determine the value of Tm. we set Tp = 1 1-92"
For example, we set Tp 1000
when 32 0.999 is chosen in the Adam optimizer.
The proposed dynamics can be again discretized time using the first-order forward Euler method:
PI-1 _ (Prest - Prest Pt 19:1).
Pt
(16)
which simplifies to the following final rule for updating p:
Pt + Pt-1 (Tp 1) + Preat
(17)
Tp + Prest Igel
We conclude our proposed bio-plausible Adam optimizer (Bio-Adam) in Algorithm !
3 REALIZE WEIGHT SYMMETRY VIA PREDISPOSITION
3.1 PRELIMINARIES: THE WEIGHT TRANSPORT PROBLEM
It is generally believed that learning by backpropagation (BP) is not necessarily biological plausible in the brain for many reasons [Strkk (1989): [iling et al. (2019). One of the primary reasons is that requires symmetrical weights between the forward and backward paths in order to propagate correct error information. This issue is also known as the weight transport problem Grossberg (198);; Ororbia II et al. (2017).
Using x (1+1) and (1+1) to denote the input and output vectors of layer (1+ 1) in an ANN, we have ((++ = (xx((+)), where a is a non-linear activation function (Rumelhart et al. 1986). The input vector is given by:
z (1+1) (1+ y + b0+1)
(18)
where (((1-), 6(-1)) are the forward weights and bias in layer (1+ 1). respectively, For given loss function L. define error signals as g(++1 (L+0x BP propagates 8(++1) (()) by:
(ll) 0L/Oy (w(+H))7g(+1)
(19)
The backward weight (w(++))) is the transpose of the forward weights, which necessitates that the synaptic strength of the forward and backward routes identical. This is considered biologically implausible for biological synapses that transmit signals in single direction.
3.2
PROPOSED BIO-PLAUSIBLE WEIGHT SYMMETRY
3.2. BIOLOGICAL MODELLING
Our bio-plausible setting is shown in Figure ( Following Lillicrap et al. (2016), we use a pair of synapses Wiy and Wys between neuron and neuron to carry forward and backward signals, respectively. In addition, we employ the multi-compartments neuron model, which hypothesizes neurons' basal and apical dendrites separately store two independent variables (Guerguiev al.) 2017). The basal dendrites receive the summed forward input signal æ. In ideal situations. the apical dendrites receive the sum of top-down error signal e = Ol/ which is the partial derivative of given loss function L with respect to the neuron's 's output y. Given the fact that the backward weight may not necessarily be equal to the forward weight, the top-down error signal may not be precisely equal to dLyOy, but rather provides only estimation of it. The neuron processes the two stored variables æ and e through its internal dynamics to generate the forward output y = o(x) to its next layer; and the backward error with respect to the input 5 aL/Oz co (z) and propagates it to neurons in its previous layer. We refer interested readers to (Sacramento et al.; 2018 Payeur et al, 2021) for more discussion of interneuron dynamics.

---

Under review as a conference paper at ICLR 2023
ej
Wit
ei
LTP (+)
(
Apical
Backward
Predisposition
N 10.1)
Soma
Fungtion
6j
di
a
yi
10
Forward
Iterations
Basal
Wij
-2
100
(A) Neuron model settings
(B) Predisposition function (ww, +)
(C) convergence example
Figure 2: The model and the step function we use to realize weight symmetry. (A) The solid red lines represent axons and synapses that convey information in single direction; the solid black lines represent dendrites that can transmit information bidirectionally through electrical spikes or molecular diffusion; and the dash red line represent our required variables sharing that helps build symmetrical weights. (B) An example series of stepsize function with different temperature param- eter T. The solid line works for LTP while the dash line works for LTD. (C) Initialize two weights as 1 respectively, and run gradient descent with stepsize = 0. 0 and hyperparameter T=1.
3.2.2 PROPOSED BIO-PLAUSIBLE WEIGHT SYMMETRY LEARNING RULE
Under the above bio-plausible setting, there are two synaptic weights to be updated in learning. One is the weight Wij of the forward path synapse that connects to a basal dendrite of neuron i, and the other is the weight Wij of the backward path synapse that connects an apical dendrite of neuron j. To facilitate successful learning, the adjusting goals of them are different: the forward weight wh is updated by the top-down error signal directly to minimize given loss function, while the backward weight Wii is updated to be symmetric with respect to its corresponding forward weight Wi to convey accurate error information.
Here, we discuss the proposed mechanisms that make the backward weight to be symmetric to the forward weight. The attainment of weight symmetry is accomplished in two stages. The first stage achieves the establishment of symmetry, i.e., aligning two initially different weights at the start of the learning process. The second stage deals with the maintenance of symmetry, which ensures that the values of forward and backward weights remain identical throughout the subsequent learning process. In the following, we first describe the maintenance of symmetry in the second stage assuming that the forward and reverse weights have already been aligned in the preceding first stage. Then, we elaborate on how we establish weight symmetry at the beginning of learning in the first stage.
[Maintenance of Weight Symmetry by Gradient Transport] Under the setting of biologically plausible multi-compartment based neural modeling, we may assume that the forward and backward synapses have access to the signals yy and 8, locally. Note the product of these two signals is the gradient of the forward synaptic weight Wiy. Therefore, although the backward synapse with weight Wy; cannot access Wij. it may obtain the gradient of wij based on the locally available signals yj and 6, In other words, we employ grudient transport between the two synapses, which considered biologically plausible (Akrout et al.) 2019). Gradient transport allows the same gradient to be used in the weight updates of both the forward and backward synapses:
9hy (1) (1) (1)
(20)
With gradient transport, one can immediately see that for a BP method such as SGD, the amounts of weight change for the pair of forward and backward weights wij and wys are always identical. To see the same under the proposed Bio-Adam, we note that the momentum m and the concentration p of the co-consumed substances are integrated from the same gradient g. making them identical across the two synapses. Hence, the amounts of forward and backward weight changes mediated by Bio-Adam are also identical. As result, both BP and Bio-Adam maintain the initially established weight symmetry between Wiy and Wy; since the amounts of update applied to the two weights are always identical.
[Establishment of Weight Symmetry by Predisposition] Building upon gradient transport, we achieve establishment of weight symmetry by further introducing a biologically plausible process
7

---

Under review as a conference paper at ICLR 2023
called predisposition. In this work, specifically propose predisposition functions p(w, that depend on two factors: synaptic strength w, and the direction of weight change (LTP or LTD).
Injecting p(w, =) into the weight update rule of Bio-Adam in ( results in more comprehensive model of our bio-plausible synaptic dynamics:
Wt + Wi-1 p(we 1, sign( 7 my Pt-
(21)
Here, the negative momentum term -m determines the updating type: -m > corresponds to LTP, and -m correspond LTD.
Now let us consider applying this rule to a pair of forward and backward weights wij and Wji- When LTP update is applied both Waj and wys by predisposition we would like potentiate the the weaker synapse of the two more than the other stronger one. As illustrated by the solid lines in Figure 2 (B), we achieve this predisposition effect by defining the predisposition functionn p(u, +) in the form of the sigmoid function (Han & Moraga 1995) with hyperparameter T, which usually considered as temperature:
p(w, +) 2/(1 exp(w/T)), (ITTP).
(22)
Similarly, for a LTD update applied to wiy and Whs the desired predisposition effect would depress the larger weight more than the smaller one. As shown in the dashed lines in Figure 2 (B)), achieve this by defining the predisposition function p(w,- -) according to:
p(w. -) = 2/(1 + (ep(-w/T)
(LTD).
(23)
Under practical settings like small learning rate, we can see that the two predisposition effects de- signed above diminish the initial difference between way and Wys over time and eventually establish weight symmetry,
4
EXPERIMENTAL RESULTS
4.1 BIO-ADAM
Figure 3 compares the train/test loss curves of ANNs based on the widely adopted ResNet18 archi- tectures (He et al. 2016) trained by four different optimizers on the well-known CIFARIO dataset (Krizhevsky et al. 2009 Except for RMSProp, the performance of the remaining three optimizers on the test set is comparable. SGD+momentun performs the best on the train set, and our proposed Bio-Adam performs marginally better than Adam. RMSProp performs the worst.
In addition, we evaluated the wall-clock times of all four optimizers to determine the computing overhead Bio-Adam imposes. Running 200 training epochs on 4 NVIDIA RTX 3090 GPUs in par- allel, Bio-Adam takes 55m52s, which is slightly slower than Adam's 's 54m07s; RMSProp's 's 53m15s; and MGD+ 52m56s.
2.4
RMSProp
RMSPiop
Adam
Adam
2
Bio-Adam
Bio-Adam
Loss test
1.5
Mamantmm
0.2
SGD+Mo Momentum
1.2
0.05
08
8e-3
2e-3
Epochs
Epochs
120
40
80
120
160
203
Figure 3: Cifar10 experiment. Network architecture: ResNet18 (He al. 2016;; All strengths of weight decay are set to zero; Learning rate >=0.01 for RMSProp and for Adam and Bio-Adam; Detail setting for each optimizer: RMSProp: 3g=0.99, ==10 8; Adam:
>=0.0001
(3,, (0)= 0.999), ==10- 8; Bio-Adam: (ri, (200 1000), P0e SGD+momentum: 31=0.9.
8

---

Under review as a conference paper at ICLR 2023
1.0
We ght Radio (1 (WI)))
L15
Matrix Alighment (")
12
Layer Depti
-12
10
10
Laver
1.00
085
70
120
20
100
(A) Weight Alignment with T=10
(B) Weight norm ratio with T=10
18
1.25
10
Weight Ratio (1M))))
1.20
Natri Alignment (*)
60.
12
Laver Depth
1.30
10
10
40-
Layer
100
(C) Weight Alignment with T=5
(D) Weight norm ratio with T=5
Figure 4: Cifarl0 experiment. Network architecture: ResNet20 (He et al. 2016); Weight decay are set to zero; Learning rate >=0.0001 for Bio-Adam with predisposition turned on. For (A) and (B), temperature T=10. For (C) and (D), temperature T=5
4.2 WEIGHT SYMMETRY
Throughout the training process, predisposition can establish and maintain the alignment between forward and backward paths. As demonstrated in Figure exhibit the alignment behavior of each layers individually in two forms: Sub-figures (A) and (C) evaluate the angle between the forward
weight matrix W and the backward weight matrix B as:
(B W) (180% /r)///css0// w/[[|B|l/ W (12)
and Sub-figures (B) and (D) evaluate the weight norm ratio B|||///// W/l2).
Comparing different temperatures T, we can see that the lower temperatures establish symmetry more quickly. However, a temperature that is too low may hinder training performance. When com- paring the (train/test) accuracy after the 100 epochs training, Adam reaches 199.17%/91.5444 Bio- Adam reaches (99.2 099.24%//1.53%/ Bio-Adam Predisposition (T=10) reaches (97.88%/91.5 1.55%) which still comparable Adam's. However, Bio-Adam Predisposition (T=5) only reaches (95.31 999...3%..
5 CONCLUSION
Based on synaptic dynamics, we present two ideas that, respectively. implement the bio-plausible Adam optimizer (Bio-Adam) and overcome the weight transport problem BP. Bio-Adam elimi- nates the biologically problematic RMSProp term in favor of a synaptic substances dynamic p that converges to the same equilibrium as RMSProp's. s. Meanwhile, Bio-Adam provides a clear biolog- ical explanation for the momentum term In addition, inspired by the predisposition property of synapses observed in neuroscience, we describe a novel method to build and maintain the sym- metry between forward and backward paths using only local signals and without separate training phase. The results of this investigation may impact future neuroscience research on the dynamics of synapses and how they contribute to learning.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap. and Douglas B Tweed. Deep learning without weight transport. Advances in Neural Information Processing Systems, 32:976- 984, 2019.
Timothy VP Bliss and Sam F Cooke. Long-term potentiation and long-term depression: clinical perspective. Clinics, 66:3-17, 2011.
Marina Chistiakova, Nicholas M Bannon, Maxim Bazhenov, and Maxim Volgushev. Heterosynaptic plasticity: multiple mechanisms and multiple roles. The Neuroscientist, 20(5):483-498, 2014.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. 11929, 2020.
Wulfram Gerstner, Wernr M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press, 2014.
Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cog- nitive science, 11(1):23-63 63, 1987.
Jordan Guerguiev, Timothy P Lillicrap, and Blake A Richards. Towards deep learning with segre- gated dendrites. Elife, 6:e22901, 2017.
Jun Han and Claudio Moraga. The influence of the sigmoid function parameters on the speed of backpropagation learning. In International workshop on artificial neural networks, pp. 195-201. Springer, 1995.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Bernd Illing, Wulfram Gerstner, and Johanni Brea. Biologically plausible deep learning-but but how far can we go with shallow networks? Neural Networks, 118:90-101, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412.6441 2014.
Marlen Knobloch and Isabelle M Mansuy. Dendritic spine loss and synaptic alterations in alzheimer's 's disease. Molecular neurobiology. 37(1):73-82, 2008.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications, 7(1): 1-10, 2016.
Magdalena Miranda, Juan Facundo Morici, María Belén Zanoni, and Pedro Bekinschtein. Brain- derived neurotrophic factor: a key molecule for memory the healthy and the pathological brain. Frontiens in cellular neuroscience, pp. 363, 2019.
Alexander G Ororbia II, Patrick Haffner, David Reitter, and Lee Giles. Learning adapt by minimizing discrepancy. arXiv preprint 1rXiv:111..11555 2017.
Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake Richards, and Richard Naud. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neu- roscience, 2017):1010-1019 2021.
Filip Ponulak and Andrzej Kasiñski. Supervised learning in spiking neural networks with resume: sequence learning, classification, and spike shifting. Neural computation, 22(2):467-510, 2010.
10

---

Under review as a conference paper at ICLR 2023
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back- propagating errors. nature, 323(6088):533-5233 1986.
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcir-
cuits approximate the backpropagation algorithm. Advances Neural Information Processing Systems, 31:8721-8732, 2018.
Sumit Bam Shrestha and Garrick Orchard. SLAYER: spike layer error reassignment in time. In Advances in Neural Information Processing Systems, pp. 1419-1428, 2018.
David Stork. Is backpropagation biologically plausible. In International Joint Conference on Neural Networks, volume 2, pp. 241-246. IEEE Washington, DC, 1989.
Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop, coursera: Neural networks for machine learning. University of Toronto, Technical Report, 6, 2012.
Wenhui Wang. Hangbo Bao, Li Dong. Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal,
Owais Khan Mohammed, Saksham Singhal, Subhojit Som, et al. Image as a foreign language: Beit pretraining for all vision and vision-language tasks. arXiv preprint arXiv:2208. 10442, 2022.
Grzegorz Wiera and Jerzy W Mozrzymas, Extracellular metalloproteinases in the plasticity of exci- tatory and inhibitory synapses. Cells, 10(8):2055, 2021.
Yukun Yang, Wenrui Zhang, and Peng Li. Backpropagated neighborhood aggregation for accu- rate training of spiking neural networks. In International Conference on Machine Learning, pp. 11852-11862. PMLR, 2021.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 3066):511-1141, 2018

---

Under review as a conference paper at ICLR 2023
A ADDITIONAL EXPERIMENT
A.1 TRAINING SPIKING NEURAL NETWORKS
A.1.1 CIFARIO
Here, we train spiking neural networks (SNNs) Yang et al. 2021 the CIFAR10 dataset Krizhevsky al. 2009). SNNs are known for their noisy and sparse gradients due to spiking neuron's s natural discontinuous 0/1 firing activity (Zenke & Ganguli 2018 Shrestha & Orchard 2018).
The experiment uses the recently proposed neighborhood aggregation (NA) direct training al- gorithm YYnng et al., (021) to compute and propagate gradients. We use their open-source haarrpppttttBaccopopageeeedddd
code -ewtoo-piiii-euuu--eeeeeeee Neural-Networts and only modify the optimizers to make a comparison be- Training-of-SSikiee tween these four different optimizers: SGD+momenruum, RMSProp, Adam and Bio-Adam.
fortrionnfo---curattee
Acc/%
Acc/%
4
85
RMSProp
85
20 epochs
2
Adam
Bio-Adam
80
SGD+Momentum
60 epochs
Epochs
Epochs
150
250
300
100
(A) Test Accuracy
(B) Train Accuracy
2 Loss
2 Loss
0.5
Epochs 3
0.2
Epochs
50
200
100
150
200
250
300
(C) Test loss
(D) Train loss
Figure 5: Cifar10 experiment on a spiking neural network trained by neighborhood aggregation
Yang et al. 2021). Network architecture: AlexNet 1024-1024 ciCoooovoluuton with depth i, kernel size j; Px:average pooling with kernel size x; 1024: fully connect layer with neurons number 1024); All strengths of weight decay set zero; Learn- ing rate 0=0.0 for SGD+momentum, ==0.0055 for Adam, Bio-Adam and R.MSProp: Detail setting for each optimizer: RMSProp: 3,=0.99, e=10- 8; Adam: (31, (0.9, 0.999), e=10-b -8, Bio-Adam: (TI, 72) (100, 1000), 10e SGD+momentum: 30=0.9
1-5603--2-384433-222844-2
As shown in Figure 5 Bio-Adam, Adam and RMSProp reach comparable performance, where Bio-Adam has the marginally better testing accuracy and lower testing loss. Meanwhile, SGD+Momentum performs relatively poor, which confirms conjecture about the advantages of different optimizers: Adam-liked optimizers has faster convergence when work with noisy/sparse gradient.
12

---

Under review as a conference paper at ICLR 2023
A.1.2 ROBUSTNESS EVALUATION MNIST
In Figure 6 we empirically evaluate the robustness of the hyperparameters of our proposed Bio- Adam optimizer on the MNIST dataset (in solid lines). As a comparison, we also evaluate the performance of the vanilla Adam optimizer (in dashed lines).
As in the previous section, an SNN is trained using the NA algorithm (Yang et al. 2021 The x-axis represents the learning rate, and the y-axis represents the training loss in Figure (A) (B) and represents the testing accuracy in Figure 6 (C) (D). All results are recorded after 10 epochs of training.
Bio-Adam
vanilla Adam
Tp 100
--. B2 = 0.99
1.0
T
1000
-- B2 0.999
1.0
Tp- 10000
BB = 0.9999
0.8
0.8
Training Loss
Training Loss
0.6
0.6
0.4
0.4
0.2
0.2
Learning rate
Learning rate
0.0
0.0
10
10
10-3
10 -5
10
10-3
(A)Training Loss with
(B) Training Loss with
m 1 (B1 0)
m
10 (B) 0.9)
100.0
100.0
97.5
97.5
Testing Accuracy
Testing Accuracy
95.0
92.5
92.5
90.0
90.0
87.5
87.5
85.0
85.0
82.5
Learning rate y
Learning rate y
80.0
80.0
10
10-4
10-3
10
10
10-3
(C) Testing Accuracy with
(D) Testing Accuracy with
Zm 1 (B1 0)
Tm = 10 (B, 0.9)
(iC):convolution with depth i, kernel size j; Pxcaverage pooling with kernel size x; 300: fully connect layer with neurons number 300); All strengths of weight decay set zero; Prest 10a; The red circled data point is default setting: Tm = 10, Tp 1000, and 7 0.0001. It is reported that the bias correction term in Adam boosts its robustness the start of training (Kingma & Ba 2014). However, biology learning is lifelong process. Moreover, the bias correction terms appear to lack clear biological correspon- dence. Due these two considerations, omitted them from this experiment in order to make fair comparison.
Figure 6: Network architecture:
In Figure 6 Different combinations of Tm and Tp can achieve equivalent performance with a suitable choice of The learning rate 7, demonstrating the robustness of the hyperparameters. In the meantime, the default configuration (red circled) derived from Adam's default setting achieves the lowest train- ing loss and the highest testing accuracy among all Bio-Adam's 's results ((3,, 32) = (0.9, 0.999) in Adam corresponds to Tm 10, and тp 1000 in Bio-Adam; and adopt the same default learning rate y 0.0001). When compared to vanilla Adam, Bio-Adam gains comparable performance in both accuracy and robustness.
13


Reviews:
[{"review_id": "pClD67xP6D6", "reviewer": "Reviewer_hKeF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose an interpretation of the first-order gradient model based on the dynamic evolution of synapse. Overall, it is interesting but the explanation itself seems to be not based on the ground truth of neuroscience thus appears to be a bit post-hoc given the establishment of ADAM and RMSProp. ", "summary_of_the_review": "Overall, I think this is an interesting paper with an promising view to explain the gradient descent model with biological mechanism. However, there lacks some rigorous and detailed reasoning on how they are actually connected to each other. ", "strength_and_weaknesses": "Strength: \nThe idea of explaining the gradient update from neuroscience view is interesting. \n\nWeakness:\n1. In terms of equation (6) and (7), it seems to be a very deliberate operation to split the concentration and gradient into both positive and negative parts here. What do you refer to by the $m^+$ and $m^-$ here? The high flux or low concentrations of $Ca^{2+}$? Why can you assume that it is depending on the \"gradient\" with a linear relation? For me, there lacks some evidence here to support the reasoning procedure beyond assumption. For example, ref [1] gives the dynamics equation for LTP/LTD. \n\n2. The model for RMSProp has the same issue as above. Rewrite the existing updating rule into a flow is not convincing as you can associate the parameter values with high arbitrariness. I expect that there should be some intermediate process that explains why it takes the specific parameters. \n\n\nRef:\n[1] https://www.pnas.org/doi/epdf/10.1073/pnas.132651299", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VInf9nG0iiI", "reviewer": "Reviewer_eRea", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a bio-plausible realization of the Adam algorithm in biological neural systems. They similarly propose a method to establish weight symmetry in biological neural networks based on the concept of predisposition.", "summary_of_the_review": "The bio-plausible realization of ADAM is an interesting contribution to the learning dynamics of neural circuits. My main criticism of this work is the empirical experiments conducted in this paper. The experiments are not comprehensive, more work can be done to empirically evaluate Bio-Adam.\n\nI am willing to revise my score if more empirical experiments are conducted.", "strength_and_weaknesses": "Strengths\n---------------\nThe idea of biological neural systems implementing an adaptive gradient optimization procedure is an interesting and novel contribution to our knowledge of learning in neural circuits.\n\nWeaknesses\n----------------\n\n1. There are spelling errors:\n    (A) “Variable” on first line on Page 2 \n    (B) “Mechanism” on  second line of last paragraph in Introduction (Page 2)\n2. The RMSProp bio-plausible realization is somewhat arbitrary and underdeveloped. The paper doesn’t provide the corresponding biological substance that is co-consumed by potentiation and depression. This justification was provided for momentum.\n3. The empirical experiments are inconclusive. Evaluating Bio-Adam on a single architecture (ResNet18) is not a sufficient justification for how the method compares to its contemporaries. This makes me question the robustness of the hyperparameters and the Bio-Adam model.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2OngAagUSxW", "reviewer": "Reviewer_2dmD", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work presents a biologically motivated implementation of ADAM and weight transport solution. \n", "summary_of_the_review": "Although interesting and novel, there are no computational advantages presented and the biological connections are approximate. Therefore I find this work’s contributions limited. ", "strength_and_weaknesses": "Strengths\n- Understanding how the brain achieves credit assignment is an important and very interesting research direction. This paper contributes to this field and provides a hypothesis for experimental neuroscience to explore. \n- Bio-adam is an interesting and novel algorithm. \n- The paper is well written and clear. \n\nWeaknesses\n- The main contribution of this work is to present a biologically motivated implementation of adam (there are no computational advantages proposed). However, in my view the biological connections are undeveloped, and only loosely motivated by neurobiology. As a result I find this work’s contributions limited.   \n- I do not find gradient transport and the weight symmetry learning rule particularly biologically plausible, and as such do not find this aspect of the work that convincing. First, there is little to no evidence for 1 to 1 forward backward connections in the brain being a general principle. Second, backward synapses would need to be able to communicate signed errors, which they cannot, or alternatively there should be two sets of backward synapses for each sign. Third, in my understanding axonal signaling is not equipped to signal with the time resolution that would be required. \n- There is no advantage of bio-adam shown over a more simple update such as sgd+momentum, which would most likely be simpler for biological systems to approximate. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject