Impression learning: Online representation learning with synaptic plasticity
Authors: Colin Bredenberg
Benjamin S. H. Lyo
Eero P Simoncelli
Cristina Savin
Venue: NeurIPS 2021 Poster (2021)
Keywords: computational neuroscience
probabilistic computation
synaptic plasticity
neural sampling
Wake-Sleep

Abstract: Understanding how the brain constructs statistical models of the sensory world remains a longstanding challenge for computational neuroscience. Here, we derive an unsupervised local synaptic plasticity rule that trains neural circuits to infer latent structure from sensory stimuli via a novel loss function for approximate online Bayesian inference. The learning algorithm is driven by a local error signal computed between two factors that jointly contribute to neural activity: stimulus drive and internal predictions --- the network's 'impression' of the stimulus. Physiologically, we associate these two components with the basal and apical dendrites of pyramidal neurons, respectively. We show that learning can be implemented online, is capable of capturing temporal dependencies in continuous input streams, and generalizes to hierarchical architectures. Furthermore, we demonstrate both analytically and empirically that the algorithm is more data-efficient than a three-factor plasticity alternative, enabling it to learn statistics of high-dimensional, naturalistic inputs. Overall, the model provides a bridge from mechanistic accounts of synaptic plasticity to algorithmic descriptions of unsupervised probabilistic learning and inference.

Impression learning: Online representation learning with synaptic plasticity
Colin Bredenberg Center for Neural Science New York University
Benjamin S. H. Lyo
Center for Neural Science New York University blyo@nyu. edu
cjb617@nyu. edu
Eero P. Simoncelli
Cristina Savin Center for Neural Science, Center for Data Science New York University csavinEnyu. edu
Center for Neural Science, New York University Flatiron Institute, Simons Foundation eero wwmoncellmnny.. edu
Abstract
Understanding how the brain constructs statistical models of the sensory world remains longstanding challenge for computational neuroscience. Here, derive unsupervised local synaptic plasticity rule that trains neural circuits to infer latent structure from sensory stimuli via novel loss function for approximate online Bayesian inference. The learning algorithm is driven by local error signal computed between two factors that jointly contribute to neural activity: stimulus drive and internal predictions - the network's 'impression' of the stimulus. Physio- logically, we associate these two components with the basal and apical dendrites pyramidal neurons, respectively. We show that learning be implemented online, is capable capturing temporal dependencies in continuous input streams, and generalizes hierarchical architectures. Furthermore, ically and empirically that the algorithm is more data-efficient than a three-factor plasticity alternative, enabling it to learn statistics of nigh-dimensional, naturalistic inputs. Overall, the model provides a bridge from mechanistic accounts of synaptic plasticity algorithmic descriptions of unsupervised probabilistic learning and inference.
demonstrate both analyt-
Introduction
Sensory systems are faced with a task analogous to the scientific process itself: given a steady stream of raw data, they must extract meaningful information about its underlying structure. Because the true underlying structure of the data is rarely accessible, this "representation learning" must be largely unsupervised. Framing perception in the language of Bayesian inference has proven fruitful in perceptual and cognitive science [144], but has been difficult to connect to biology, because we still lack satisfactory account of how the machinery of Bayesian inference and learning is implemented in neural circuits []
Past work includes several examples of circuits that simultancously learn top-down generative model of incoming stimuli and perform approximate inference with respect to these models. These differ in the nature of the approximation, from maximum posteriori estimation 77, efficient population codes that embed prior structure S to either parametric 10] or sampling-bart based [IT] variational inference. Learning generally takes the form of optimizing probabilistic objective, either by backpropagation
[9] 10) or through local parameter updates, which match biological learning more
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

closely [1] 12
13 While these models are mostly restricted to static stimuli, several instances
also operate over time
Developing biologically plausible learning rules that are applicable to temporally-structured data is hampered by the fact that optimizing a probabilistic objective function in such contexts requires access to non-local information across space and time. Previous research on local approximations to credit assignment in BP address spatial credit assignment by ascribing differential functions the apical and basal dendrites of pyramidal neurons in cortex, where apical dendrites are hypothesized to receive top-down learning signals, and basal dendrites receive bottom-up sensory signals [77 25 Locally implementing temporal credit assignment is a bigger challenge (26.
Our work, which we have dubbed 'impression learning' (IL), combines the tradition of probabilistic learning 11 [4] with these recent developments in local optimization, in order to learn dynamic stimuli concurrently with perception. We propose network architecture in which top-down stimulus predictions arriving at the apical dendrites of neurons influence both network dynamics and synaptic plasticity, allowing the network concurrently learn probabilistic model of the stimuli and an approximate inference computation. We provide a mathematical derivation of synaptic plasticity rules that approximate gradient descent on novel unsupervised loss function, along with detailed analyses of the biases induced by this approximation. We explore the empirical and mathematical relationships between IL and three other methods: backpropagation (BP) (28) the Wake-Sleep (WS) algorithm [29]. and specific form of neural variational inference (NVI*) [0 31
We further demonstrate that
IL scales to naturalistic stimuli and multilayer network architectures
2 Probabilistic inference and local learning in a recurrent circuit
We construct network of neurons that aims to learn a generative model of the temporal sequence of stimuli that it receives, Pm (r,s) = IT Pm (Ir, sy(rz-1), in which s represents stimuli in an input layer The latent variables r are not defined by a physical model of the stimulus environment, but are learned in an unsupervised manner to provide the best generative explanation of stimuli received. We assume that stimuli are generated by a true probability distribution p(s|z), where s corresponds to the first layer of neural activations in an early sensory layer, and vector z ~ p(z) corresponds to the environmental factors which jointly caused that activity. Because learning is unsupervised, we do not enforce explicit correspondence between the internal and true latent features, r and z, only a correspondence between model predictions and ground truth stimuli. We also assume that the network performs online inference with respect to its model, inferring the corresponding latent cause r using Bayes' rule: Pm (r)ss Pm (r, s)/pm (s). Because the network won't, tt in general, be able to explicitly calculate Bayes' rule, we will assume that the network learns an approximate inference distribution q(rls), which it attempts to bring "close' to pm (r]s). This joint process of learning and inference, known as Bayesian latent feature extraction, provides a general framework for conceptualizing early sensory processing in the brain [5]. In subsequent sections, we will write loss function for this general latent feature extraction objective, and show how local modifications at apical and basal synapses can perform approximate gradient descent this loss.
Loss function The loss function that we propose will produce a learning algorithm where neurons alternate between sampling from the model, Pm. and performing approximate inference according to q in response to real stimuli received from p(s[z). This alternation will allow the network to learn online in way that minimally perturbs the continuity of perception. First, consider two families of hybrid probability distributions, which we denote in shorthand go and po:
T
T
ÿo = II (r s, Z,, )x:)) = II (q(relss: (z:))((s))))) pm (r,, St (1-
t=0
t=0
P0
p. (re, S: Se 0)
(a(relsai (, (,)p(sa(2))) Pm 11, t=U
II
t=0
where collection of binary random variables X determines whether, given time step, sampling occurs due to (sr: (s))(((((2) or (m, sy 1, Ae; 0,),, and the full parameter space is denoted
I Codee provided at: https: //tgttgttttb..tt// com/ htt./oolinbbbbe http://oinireeeeeerrrrreeeeeeeee Learninnn-----nnneeeeeee Camera-aaaaaaaaa Ready We use the shorthand notation 's' refer to the N x T matrix of stimuli across time.

---

0 = 100pp.0 0g). We define an objective of the form:
L = Σx,2 L [[pe]]
Ex,= ʃ [log qo log po) do drds
(2)
This loss provides generalization of the widely-used evidence lower bound (ELBO), which corresponds to the case d = VL. Importantly, we can show that = and only g(re/se; 0g)p(s.(2z)
(r,, Se 1,, 0p) Vl. If this equality were achieved, it would also imply Pm (r, 8) = ((s)s) However, this absolute minimum will not achievable unless z₁ is deterministic, because Pm 8,(11,, Ay; 0p)) has no dependency on the latent variables in the environment. Thus, our goal is inherently unachievable, and different choices of p(),) and network architectures may lead to different local minima. However, each choice will incentivize learning close correspondence between these distributions, and an approximation to gradient descent with respect to any choice will lead to local synaptic plasticity rules, making this objective particularly interesting for the computational neuroscience community.
Update derivation
We begin by taking the gradient of our new loss w.r.t. 0 = 10g. Oz]
-VoL = - VOEA.
[log qo log Pol do drds
|
(Vo(log ãe bog qo drds + / [log go log Pol Vego drds
= =- Ex,2
where the second equality follows from the product rule. Both integrals are analytically intractable. but if we can write both as expectations, they can be approximated by averaging over samples of r and s. To accomplish this, we note that Vage Vge = VV log qol do. which allows us to rewrite our expression as an expectation over r and s:
/ Vo log qo Vo log pol go drds + / [log qo - log Pol (Vo log go )o drds
VoL = Ex,
We also observe that ʃ Vo log gol go drds Vo go drds Vol O, allowing the elimination of
two terms:
P0
VoL Ex,z
Ve e log po] qo drds
log
Vo log log qo Jo drds
g0
P0
2 E Ex,x
Vo log Po] To drds
Voo log go goo drds
40
Ex,z
Vo log pol qo drds
(Vo log qo go drds
qo
/
Ex,x
Vo log pol to drds
V log qo) po drds
(3)
The approximation the second line comes from a Taylor expansion of log En about 0, i.e. when to
(which introduces a bias to the parameter updates that we examine analytically in Appendix This expansion is the core of our derivation, and not all algorithms take this approach: for this reason, in Appendix B and we show how the properties of our algorithm compare to alternatives (NVI", BP, or WS).
da
At this point, we have not yet defined p(A). We'll assume that À E {0, 1}, that p(Au = 0) p(Ao = 1) 0.5, and that the values alternate deterministically with a 'phase duration' K, i.e. AR+1 A if mod (k, K) = O, and d = ê otherwise. Under these conditions, the two integrals in Eq. (3) are equivalent, and computing our parameter updates only requires sampling from If we define X =1- X, then we have p(X') p(A) and q(r, s/z, X; p(r, s/z, X'; 0), which

---

a
Wake
network
apical
hidden
Impression
generation
Tinput
sensory
O
Sleep
stimuli
hidden
basal
Pm
environment
sensory
d
f
30k
0.4-
ground Iruth output
autocorrelation
data inference generation
prediction
stimulus
loss
20k
0.6
0-
0.4
10k
0.2
-0.4
200k
200
0
20
40
60
80
time steps
time steps
ground truth
time lag
Figure 1: Network architecture and learning. a. Model schematic. A neural network receives stimulus inputs its basal dendrites, and returns lateral and top-down prediction signals via apical synapses. A gate, Å determines whether apical or basal influences dominate network activity. b. Learning schedule: the Wake-Sleep (WS) algorithm (left) trains its synapses by alternating between prolonged periods where X = 1 (Wake) or A 0 (Sleep). In contrast, our IL algorithm alternates rapidly between A and X with phase duration K 2. c. Network loss on the artificial stimulus task. Error bars indicate +11 s.e.m. averaged across 20 network realizations. d. Comparison between ground truth stimulus (green) and the network's 's prediction (blue) for a particular stimulus dimension. Same comparison across stimulus dimensions. f. The autocorrelation function of r when the network performing approximate inference (green; A = 1), or generative mode (orange; X 0) compared to the autocorrelation of the data (grey).
we make use of as follows:
V log pol Ae drds + / Vo log del Po drds p(a))
VoL EeE
= EE
Vo log pe] qo drdsp(A) +
Σ
/
Vo log Pol qo drds p(X)
x
= 2
Σ
/
Vo log po] qo drds p(A)
Using the definitions for ğo and pa and the properties of the logarithm gives the following parameter update rule:
//< =a- Vo Na-NVolloaa + (Ax) log
A 2E
qo drds
(5)
As we will show below, this parameter update equation produces updates that require only information locally available synapses, necessary condition for any biologically-plausib algorithm.
Basic model To make the above general learning procedure concrete, we need to specify how to sample from ÿo. which in turn requires an architecture for performing approximate inference at each time step, g(r, ss;; Ry))),, and joint model of stimuli and neural activations, Pm (r,, S, 11 Ae; 8p). We map these two model components onto neural circuitry, with their own local variables corresponding to and r, and segregated synaptic parameters: the 'basal' compartment dedicated to feedforward inference (g. index 'inf") and the 'apical' compartment is dedicated to generative sampling from the model (pm. index 'gen"), ); this segregation allows their influence on neural dynamics to be selectively gated by d (Fig. [T).

---

First, the internal generative model of the circuit is implicitly defined by set of currents to the apical dendritic compartment corresponding to generated samples for the next latent variable, rzt".
gen ((1 - DD 1. r + of n):
(6)
s gen f(D + t
(7)
where D, is a diagonal transition matrix (constraining generated latent- variables to be independent AR(I) processes), D, is linear decoder, I is the identity function, 7h & (O, I) are independent white noise samples, and ogen and or denote respectively the generative standard deviation for neurons at the stimulus and latent levels. We define ky (1 5(Ae )t-1)) with 6(-) the Dirac delta function; ky is 1 only if X = and A = O. We chose a piecewise model (gated by k2) for r?th because we observed that the statisties of stimuli given previous activities r are different if a transition has just occurred (Ae = 1 and A 0), which will bias the training of the generative transition parameters D,- We chose for this case, but could alternatively have a different parametric model for after transitions have occurred. As we will show, adding this condition to our model will never affect the dynamics of our network, but will cause learning for D, to occur only on time steps when a transition has not just occurred. Nothing in our derivation requires the transition matrix D, be diagonal, but we constrained it in this way to allow for learning independent latent features. As is, D, defines the leakiness of the apical dendritic compartment of the neuron; off-diagonal components of the transition matrix would correspond to recurrent synapses. These dynamics define a probability distribution: pm (r, s) = II Pm (Te, ss|r;-1, 8p).
Second, we define our inference model, factorized conditional probability distribution (r]s) II ((rg/s;: fg)) which applies feedforward nonlinear transformation to incoming stimuli:
inf /(Ws:) + inl ne:
where W denotes the feedforward weights, of is the inference standard deviation for neurons at the latent level, and the nonlinearity f()) is the tanh function. During inference mode, the stimulus layer receives latent-associated inputs from the environment, further corrupted by the same noise as the internal representation:
inf = 5(z,) + oe E..
where giu denotes the standard deviation for neurons at the stimulus level, and s(2,) input from external stimuli. During simulations, samples are determined by combination of Pm and q, given by qa:
rs = der inf + (1 Xe r gon
(10)
= Aesta inf
+ (1 X )s
(11)
We interpret these dynamics biologically as network of recurrently connected pyramidal neurons with two sources of input, one to the apical dendrites (x:² gen or ) and one to the basal dendrites (x-inf or s!mft.. The gating variable X, determines which input source controls the circuit dynamics.
Plasticity rule interpretation Inserting our particular choice of qe and Pmt into our approximate gradient descent derivation, the parameter updates can be interpreted as local synaptic plasticity rules the basal (for q:) or apical (for Pmn) compartments of our neuron model:
1
log (rr))ss5)0)))) 0g) 2 (orint)
||re - S(Ws Gg
(12)
log Pm (T,, )e;6p) =
||re ((1 ke D, + +xI)r,-112
genz
2 ||s, - f(D,r.)) + Cp:
(13)
where Gq = -N, lo lllgl///((o/uu (a( and cp N, log( (og(v/2T (o N₂ log( / (og are constants that do not depend on network parameters. We can use these equations to evaluate our weight updates, by using the general formula in Eq. [ and calculating derivatives. For online parameter
5

---

updates, we assume that weights change stochastically at each time step, based on samples from Xo.
z, r. and s (instead of explicitly calculating the expectation in Eq. 5):
X (r+
A (w
&
ff
(14)
(oinf)
AD
(F)
X
Ae ki))
(6) (Dr -D.r=-
(15)
(oton)
AD
(
(f) (T."
(16)
(o 2
Each of these updates has the form of a local synaptic plasticity rule, under the following assumptions: W(!)) a basal synapse from neuron j to neuron i, r(()) and r,²) correspond to the pre- and post- synaptic firing rates, respectively, and f(Ws)) corresponds to the local basal current injected into neuron i. Thus, assuming that basal synapse has access to both the neuron's 's firing rate and its local basal synaptic current at particular moment in time, (W))) is local; the same principle holds for the apical updates. If X O, then network activity is driven by the generative inputs, and so the parameter updates for basal synapses depend on apically-driven activity, as has been observed experimentally [2]; similarly, apical synaptic plasticity should depend on basally-driven activity. The updates for the generative transition matrix, deteemmrinn the leakiness of the apical dendritic compartments-an are gated by 1- kg. indicating that paramcter updates are delayed upon entering 'inference' mode: this could reasonably be implemented biologically by slow cascade of biochemical processes that delay changes in neural parameters, as has been proposed by previous plasticity models [33 34]
3 Numerical Results
Validation on artificial stimuli To analyze IL performance in an environment where we have access to and control over the statistics of the latent dynamics 2,, constructed artificial stimuli as follows:
2. Az: + +o otrue
(17)
5(2)) Azz,
(18)
where A a N₂ x N₂ diagonal matrix with A < 1 Vi, A as N. N₂ random matrix with A ~ (0, )) and 7, ~ (0 1) For simplicity, we fix the dimension of the latent space and the generative noise in the network to the ground truth values, N, = N₂ neurons, and o gtrue = ogen so that in principle our model f Pm (),r can match the ground truth data distribution f p(s. z z)dz exactly. This also means that we can verify that the network has learned an optimal model by comparing its second-orde statistics to those of the ground truth distribution.
We trained the network using II, verifying that the online synaptic updates minimize the loss L (Fig. [p.. We further validate that the network has learned to accurately perform inference, so that (r(s)) 2 Pm(rss) and that the network has learned a good model of the data, so that Pm (r, s)dr x p(s). as per our original goals. We show that when the network is performing approximate inference, i.c. A = 1, Vt, stimulus reconstructions based on the network's 's latent state are closely matched to the actual stimuli, i.c. Sy f(D,rt), meaning that the network is functioning as a good autoencoder across time (Fig. [d], and across all stimulus dimensions (Fig. [e.. To verify the network's generative performance, we also show that the temporal autocorrelations for the network rates in generative mode (A, = 0 V't) closely overlap with the ground truth autocorrelation structure of z, suggesting that the learned latent features correspond (modulo a rotation) the true latent features. Note that this latent variable match occurs because we have enforced a correspondence between the true generatnn distribution and our model, and would not necessarily happen if different model architecture were used.
Algorithm comparisons Having verified that is capable of training the network on simulated data, we next compared it to alternative algorithms in the literature, including neural variational
'The parameter values and initialization details for all simulations are included in the supplementary code. which an internal cluster, N.- 100 and N; 20.

---

d
12K
cosine similarity
gradient SNR
10-1
asymptolic loss
10a-
impression
8k-
impression NVI
NVI
loss
10"
BP
1
asy
10
2k-
100k
100k
100k
10
time steps
time steps
time steps
stimulus dimension
Figure Comparing learning algorithms and effects of dimensionality. a. Loss throughout time. b. Cosine similarity between gradient updates given by IL and NVT", averaged over 10 samples. c. The signal-to-noise ratio for IL. (blue), compared to NVT" (purple) across learning. Asymptotic negative ELBO loss for IL (blue), NVI" (purple), and BP (gray) as function of the stimulus dimensionality. Error bars indicate +1 s.e.m. averaged across 20 network realizations.
inference (NVI"), BP, and WS (see Appendix for detailed mathematical comparisons and deriva- tions). In particular, NVT" provides an alternative candidate model of how the brain could plausibly learn neural representations through variational inference [31]. Because NVT' performs poorly for high-dimensional stimuli and large numbers of time steps (Appendix C 65 ) we simplified the task
by reducing the dimensionality of the latent space, N₂ = 2, and stimulus space, N₂ 4. For twenty evenly-spaced time points over the course of the learning trajectory, we compared the inference pa- rameter updates given by IL. AOLL to the inference parameter updates given by NV1 AO for time-step stimulus sequence (Fig. q 2 To get good estimates of the mean and variance of these sample parameter updates, we averaged over 10t different realizations the network noise, and compared the samples using two measures, First, we considered the cosine similarity (normalized inner product) NVI
between the two empirical mean updates, 20" (Fig. ()), where c0s E 1,, and cos(0) q T Ei= SOLL and A E Ek ANNVI this case would indicate that the parameter d
updates are anticorrelated. Because the NVI" update is unbiased, ie. E = de, L, as long q
as we have averaged over a sufficient number of samples N, a positive cosine similarity across learning between the II update and the NVT' update (Fig. (2)) indicates that update is aligned in expectation to the true gradient the loss, and hence will improve performance. This is a way of empirically verifying that the bias we introduce in our derivation does not impair the learning process.
Having verified that the IL update and the true gradient are aligned on average, we next examine whether the updates given by NVI differ in terms of their signal-to-noiss ratio (SNR) from the IL updates, where we define the SNR as:
No (A))))2 Σ 1=0 S²(AA())
SNR(AA,)
(19)
No
where 5²(-) denotes the sample variance. This measure is an average across individual parameter updates A0(+), and increases with || A0 and decreases the estimator variance grows. As Fig. 2p shows, the SNR is many orders of magnitude lower for NVI* than for II over learning, likely due the high estimator variance of the NVT", which we demonstrate analytically for simple example in the Appendix C The estimator variance has direct implications for the speed of learning and asymptotic performance, so that even though NVT" and IL can have parameter updates that aligned in expectation, due to its low variance IL will greatly outperform NVI* during training.
We verified the generality of these benefits in the same task, as varied N N, and N, concurrently, so that N2 = 2N, = 2N, We optimized learning rates for NVI*, BP, and IL separately on the lowest dimensional condition by grid search across orders of magnitude (10-² 2, 10 etc.), and found that NVI' performed worse over the entire range, while II. and BP showed similar performance (using the negative ELBO loss standard). Moreover, while NVI showed worse performance as the stimulus dimension increased, this was not the case for IL or BP (Fig. 21).
Phase duration effects The previous numerical results verify that is able to effectively learn a generative model of artificial data, and to perform inference with respect that model. However, for
7

---

inference 2
1.5
duration
0.9
correlation
0.8
T 0.0
0.7
X
-1.5
duration
200
30
time steps
phase duration
b
d
1.5
inference
14k
duration =
duration
32
duration = 32
10k
loss
0.0
6k
2k
A -1.5
0
200
0
100k
duration 32
time steps
time steps
Figure 3: The effects of phase duration on dynamics and learning. Schematic of IL with phase duration of 2 Same as a, but for a switch period of 32. c. Comparison an example neuron's 's activity through time when the network is in inference mode (green, X = and when the network is alternating phase with duration 2 (blue); the random seed and stimuli are identical in both cases. d. Same as c, but for phase duration of 32 (pink). e. The correlation across time between neurons in inference mode vs. while alternating phase, for identical random seeds. f. The negative ELBO loss for a network trained with phase duration of 2 (blue, solid line) or 32 (pink, dashed line).
IL be a valid candidate for online learning the brain, the learning process should not significantly interfere with perception. To test this, we explored how the 'phase duration' K affects the correlation between network activity in a simulation where phases every K time steps (for fixed random seed and stimulus sequence). If the learning process did not interfere with perception all, this correlation would be and if it completely disrupted perception it would be O, or even negative. In Fig. Be and d, we show two example traces with K = and K = 32, respectively, comparing the network in inference mode the network during learning. While neural trajectories for the shorter phase durations are closely correlated, they deviate considerably for longer phase durations (Fig. 3--e) Despite this, the loss profile (negative ELBBO identical. Since WS can be viewed as special case of II. for very long phase durations (Appendix B.3 see Fig. 1 for an even longer phase duration), this implics that the two methods have similar performance. However, IL operating in mode of fast fluctuations between inference and generation may be more biologically relevant, as this reduces the interference with perception without impairing learning. Moreover,
1, Yt, and simulation where A alternates
found that lengthening the duration of the inference phase alone while keeping very short bursts generative activity further reduced perceptual disturbance, while only slightly increasing the time required to learn (Fig. -lp.
Spoken digits task Having verified the performance of IL on artificial stimuli, we next tested its performance on higher-dimensional and more complex naturalistic stimuli. We used the training and test sets of the Free Spoken Digits Dataset (36), which provides audio time series of humans speaking D We transformed these time series into mel spectrograms as a coarse approximation of the initial stages of the human auditory system, shifted the inputs by constant so to make them all positive, and divided the result by the ccross-channel standard deviation. The results of Fig. 4 are shown in the original log-mel spectrogram input space.
digits 0-9
To assess the hierarchical processing capabilities of IL. we added an additional feedforward layer to the network architecture (Fig. 4b; we provide the details of how this modification affects simulation and parameter updates in Appendix [ To compare IL to NVI", we again optimized learning rates via grid search across orders of magnitude, and found that IL greatly outperformed NVI* when each was evaluated at its respective optimal learning rate (Fig. 4p). Furthermore, we observed that our
'The FSDD is available at betttp::/ggihub.comm httt:r///gghttooooom/////tttttte

---

a
network
b
c
input
output
3100
impression NV!*
frequency
-20 dB
10
1800
generation input
loss
1100
-40 dB
O
stimulus
500
600 dB
2 environment
10
-80 dB
0
epoch num
20
0
10
0
10
time steps
generated
stimulus
generated
3100
dB
frequency requency
3100
iutocomelator lotoonattion
1.0
frequency
1800
-2 dB
1800
inference generated
4B
1100
-60 dB
500
500
-80 dB
0
0.0
3100
3100
time steps
20
frequency
frequency
0
time lag
Figure 4: Learning auditory sequences in multilayer network. Hierarchical network architecture. b. Test loss across epochs for IL. (blue) and NVI (purple). c. Comparison between an example data input and the corresponding network output in inference mode (Λę = 1). d. Sample network output in generative mode (A, = 0). e. Across-frequency amplitude correlations for the data (left) and for network-generatee samples (right). f. Auto-correlation function of neuron in inference and generative modes.
trained network meets the same criteria for success as for our artificial stimuli, namely its stimulus reconstructions closely match the true stimulus while in inference mode (A, = 1 vt; Fig. 4p. and sample stimuli produced while the network is in generative mode (A; = 0 Vt) qualitatively correspond to ground-truth stimuli (Fig. 4I), and quantitatively match the structure of both spatial (Fig. 4 and temporal (Fig. 4 autocorrelation of the input. These results collectively demonstrate that IL is capable of training neural representations of complex real-world stimuli. They also show that IL can function when there is mismatch between its architecture and the structure environmental latent variables, which are in this case unknown. In general, learning may fail if the chosen network architecture too restrictive.
4 Discussion
Impression learning (IL) provides potential mechanism for the brain to learn generative models of its sensory inputs through local synaptic plasticity, while concurrently performing approximate inference with respect to these models. IL. is direct generalization of the Wake-Sleep algorithm (2)] which replaces lengthy offline 'Sleep' phases with brief substitutions of network -eenrratd samples place of incoming data, between 'inference mode' and 'generative mode' are controlled by a global signal λe, which decides whether generative signals to the apical synapses or inference signals to basal synapses dominate network activity.
way that minimally perturbs natural neural trajectories. Transitions
particular instance of three-factor plasticity [37]. because its internal model provides explicit 'credit assignment" for each individual neuron, rather than implicitly calculating via correlations between neural activity and global reward signal. This leads to lower- variance gradient estimates and faster learning. Alternative learning algorithms such as backpropagation (through time) [38] are not intrinsically probabilistic, but can be used for optimizing probabilistic objectives. Like IL. BP provides explicit credit assignment, but the parameter updates it
Computationally. IL outperforms NVI* 30 31
provides are nonlocal across both network layers and time. It is worth noting that IL was developed in purely unsupervised learning setting, whereas both BP and NVI" extend to supervised and reinforcement learning (39 40 In the context of supervised learning, several thologically-plassii approximations to BP leverage the apical-basal dendritic structure of pyramidal neurons to learn

---

based primarily on sanget-proaagaee [41 or its variants 42). It would be valuable to explore the combination of such extensions with the continuous online learning capabilities of IL.
25 21]
Local computations are considered a necessary condition for learning algorithms to be biologically- plausible. In our framework, locality is enforced through the structure of the internal graphical model (Pm) and the approximate inference distribution (g): any choice of neural network architecture with independent noise will guarantee local plasticity. Our framework relatively agnostic to the details: neurons could be either rate-based with Gaussian intrinsic noise (as in the examples presented here), or generate spikes with Poisson variability, which would result in synaptic updates analogous to empirically observed spike-timing dependent plasticity, as found in generalizations of WS [14]. It would also be possible make distinctions between excitatory and inhibitory neurons, by requiring all outgoing synapses from individual neurons to be either positive or negative, or to include more complex dendritic arborizations, as have been explored in recent experimental [43] and modeling
44 efforts. Our current model enforces hard, global phase distinctions (A, E {0, for all neurons),
which could potentially correspond to alternations between activity driven by apical dendritic calcium events and basal spiking tied to theta oscillations in the hippocampus [32]. However, cortical data indicate that input to apical and basal dendrites contribute concurrently and constructively to spiking activity 45 We are currently working to extend our derivation to these circumstances, by allowing X to be non-binary and heterogenous across neurons.
Traditional predictive coding (7] requires steady-state assumptions for learning, meaning that neural dynamics must occur on a timescale much faster than that of stimuli. In contrast, IL requires a mechanism by which the relative influence of the apical and basal dendrites of pyramidal neurons can be rapidly switched, along with learning mechanisms that operate at that timescale. If such a mechanism could be experimentally identified and controlled, our model makes the specific prediction that increasing the dominance of apical dendritic input on neural activity (A₃ I) would cause the network to sample from its generative model, i.e. the manipulation will induce structured hallucinations that mimic realistic stimuli (and associated neural activity), without being tied to the sensory world. One candidate gating mechanism is rapid inhibition targeting apical dendrites specifically [46 but much work remains to explicitly relate this mechanism to learning and plasticity.
II. predicts that synapses will use an error signal based on the difference between local dendritic compartmental currents (either apical or basal) and the neuron's 's total firing rate to perform learning. There some evidence that spiking activity driven by apical inputs to pyramidal neurons can induce plasticity at basal synapses [32 [50]. and several studies have found systematic changes in synaptic plasticity between apical and basal synapses, in particular the sign changes induced by local dendritic inputs that IL predicts (51454). Hence, IL has the potential to explain the diversity of plasticity phenomena observed experimentally and inform future experiments.
Acknowledgments and Disclosure of Funding
We thank Camille Rullán Buxó, Caroline Haimerl, Owen Marschall, Pedro Herrero-Vica Siavash Golkar, David Lipshutz, Yanis Bahroun, Tiberiu Tesileanu, Eilif Muller, Blake Richards, Guillaume Lajoie, Maximilian Puelma Touzel, and Alexandre Payeur for helpful discussions and feedback on earlier versions of this manuscript. We gratefully acknowledge the Howard Hughes Medical Institute and the Simons Foundation for their support of this work. CS is supported by National Institute of Mental Health Award IROIMH12557|-4 1-01 by the National Science Foundation under NSF Award No.1922658 and Google faculty award.
10

---

References
[1] David C Knill and Whitman Richards. Perception as Bayesian inference. Cambridge University Press, 1996.
[2] Yair Weiss, Rero P Simoncelli, and Edward H Adelson. Motion illusions as optimal percepts. Nature neuroscience, 5(6):598-6044 2002.
[3] Pascal Mamassian, Michael Landy, and Laurence T Maloney. Bayesian modelling of visual perception. Probabilistic models of the brain, pages 13-36, 2002.
[4] Daniel Kersten, Pascal Mamassian, and Alan Yuille. Object perception Bayesian inference. Annu. Rev. Psychol., 15:27,-304, 2004.
[5] József Fiser, Pietro Berkes, Gergö Orbán, and Máté Lengyel. Statistically optimal perception and learning: from behavior to neural representations. Trends in cognitive sciences, 14(3):119-330, 2010.
[6] Richard D Lange, Sabyasachi Shivkumar, Ankani Chattoraj, and Ralf M Haefner. Bayesian encoding and decoding as distinct perspectives on neural coding. bioRxiv, 2020.
[7] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):99-77, 1999.
[8] Deep Ganguli and Eero P Simoncelli. Efficient sensory encoding and Bayesian inference with heterogeneous neural populations. Neural computation, 26(10):213322134 2014.
[9] Diederik P Kingma and Max Welling. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.61 14, 2013.
[10] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pages 1286. PMLR, 2014.
[11] Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel. The Helmholtz machine. Neural computation, 7(5):889 9049,99441111 1995.
[12] Stefan Habenschuss, Johannes Bill, and Bernhard Nessler. Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints. Advances in neural information processing systems, 25:773-781, 2012.
[13] Johannes Bill, Lars Buesing, Stefan Habenschuss, Bernhard Nessler, Wolfgang Maass, and Robert Legenstein. Distributed Bayesian computation and self-organized learning in sheets of spiking neurons with local lateral inhibition. PloS one, (0(8))e0133356 2015.
[14] Peter Dayan and Geoffrey E Hinton. Varieties of Helmholtz machine. Neural Networks, 185-13331, 1996.
[15] Anna Kutschireiter, Simone Carlo Surace, Henning Sprekeler, and Jean-Pascal Pfister. Nonlinear Bayesian filtering and learning: neuronal dynamics for perception. Scientific reports, 7(1):1-
13, 2017.
[16] David Kappel, Bernhard Nessler, and Wolfgang Maass. STDP installs in winner-take-all circuits an online approximation hidden markov model learning. PLoS Comput Biol, 10(3):e 1003511 2014.
[17] Konrad P Körding and Peter König. Supervised and unsupervised learning with two sites of synaptic integration. Journal of computational neuroscience, 11(3):207-215, 2001.
[18] Robert Urbanczik and Walter Senn. Learning by the dendritic prediction of somatic spiking. Neuron, 81(3):521-533 2014.
[19] Mathieu Schiess, Robert Urbanczik, and Walter Senn. Somato-dendritic synaptic plasticity and error-backpropagaatinn in active dendrites. PLoS computational biology, ((2))e1004633 2016.
[20] Joao Sacramento. Rui Ponte Costa, Yoshua Bengio. and Walter Senn. Dendritic error backprop- agation in deep cortical microcircuits. arXiv preprint
2017.
[21] Jordan Guerguiev, Timothy Lillicrap. and Blake A Richards. Towards deep learning with segregated dendrites. ELife, 6:e22901, 2017.
[22] Blake Richards and Timothy P Lillicrap. Dendritic solutions to the credit assignment problem. Current opinion in neurobiology, 54:28-36, 2019.

---

[23] Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan M Sengupta. and Dmitri B Chklovskii. biologically plausible neural network for local supervision in cortical microcircuits. arXiv preprint arXiv:201 150001 2020.
[24] Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan Sengupta, and Dmitri Chklovskii. A simple normative network approximates local non-hebbian learning in the cortex. Advances in Neural Information Processing Systems, 33, 2020.
[25] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience, pages 1-10, 2021.
[26] James M Murray. Local online learning in recurrent networks with random feedback. ELife. 8:c43299, 2019.
[27] Owen Marschall, Kyunghyun Cho, and Cristina Savin. A unified framework of online learning algorithms for training recurrent neural networks. Journal of machine learning research, 2020.
[28] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritted zip code recognition. Neural computation, 1(4):541-551) 1989.
[29] Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214): 158--161, 1995.
[30] Rajesh Ranganath, Sean Gerrish, and David Blei. Black box variational inference. In Artificial intelligence and statistics, pages 814-822. PMLR, 2014.
[31] Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief networks. In International Conference on Machine Learning, pages 1791-1799. PMLR, 2014.
[32] Katie C Bittner, Christine Grienberger, Sachin P Vaidya, Aaron D Milstein, John Macklin, Junghyup Suh, Susumu Tonegawa, and Jeffrey C Magee. Conjunctive input processing drives feature selectivity in hippocampal cal neurons. Nature neuroscience, 18(8): (00):1133-114 1142, 2015.
[33] Stefano Fusi, Patrick J Drew, and Larry F Abbott. Cascade models of synaptically stored memories. Neuron, 45(4):599 611,9996611116 2005.
[34] Claudia Clopath, Lorric Ziegler, Eleni Vasilaki, Lars Büsing, and Wulfram Gerstner. Tag- trigger-oonsolidation: model of early and late -oottttttoootniiiiionnn and depression. PLoS computational biology, 4(12); 1000248, 2008.
[35] Justin Werfel, Xiaohui Xie, and H Sebastian Seung. Learning curves for stochastic gradient descent in linear feedforward networks. In NIPS, pages 1197-1204. Citeseer, 2003.
[36] Zohar Jackson, César Souza, Jason Flaks, Yuxin Pan, Hereman Nicolas, and Adhish Thite. wwwpsssiirrreesssee--ddee digit-dataset: v1. 0.8, 2018
[37] Nicolas Frémaux and Wulfram Gerstner. Neuromodulated ppike-tmmigg-deenddee plasticity, and theory of three-factor learning rules. Frontiers in neural circuits, 9:85, 2016.
[38] Paul Werbos. Backpropagation through time: what does and how to do it. Proceedings of the IEEE, (11001150001566 1990.
[39] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529-533, 2015.
[40] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3-4):229-25 2266, 1992.
[41] Yoshua Bengio. How auto-encoders could provide credit assignment in deep networks via target propagation. arXiv preprint arXiv: 1407 79066 2014.
[42] Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propaga-
tion. In Joint European conference on machine learning and knowledge discovery in databases, pages 105-515..00 Springer, 2015.
[43] Shannon Rashid, Victor Pedrosa, Martial A Dufour, Jason J Moore, Spyridon Chavlis, Rodrigo G Delatorre, Panayiota Poirazi, Claudia Clopath, and Jayeeta Basu. The dendritic spatial code: branch-specific place tuning and its experience-dependen decoupling. bioRxiv,
2020.
12

---

[44] Eren Sezener, Agnieszka
Dimitar Kostadinov, Maxime Beau, Sanjukta Krishnagopal, David Budden, Mareus Hutter, Joel Veness, Matthew Botvinick, Claudia Clopath, et al. A rapid and efficient learning rule for biological neural circuits. bioRxiv, 2021.
[45] Matthew E Larkum, J Julius Zhu, and Bert Sakmann. A new cellular mechanism for coupling inputs arriving at different cortical layers. Nature, 398(6725) 1999.
[46] Matthew Larkum. A cellular mechanism for cortical associations: organizing principle for the cerebral cortex. Trends in neurosciences, 36()):111155, 2013.
[47] Ausra Saudargiene, Stuart Cobb, and Bruce P Graham. computational study on plasticity during theta cycles at Schaffer collateral synapses on CA1 pyramidal cells in the hippocampus. Hippocampus, 25(2):208-22 2015.
[48] Ines Guerreiro, Zhenling Gu, Jerrel Yakel, and Boris Gutkin. Disinhibitory and neuromodulatory regulation of hippocampal synaptic plasticity. bioRxiv, 2020.
[49] Richardson N Leão, Sanja Mikulovic, Katarina E Leão, Hermany Munguba, Henrik Gezelius, Anders Enjin, Kalicharan Patra, Anders Eriksson, Leslic M Loew, Adriano BL Tort, al. Olm interneurons differentially modulate ca3 and entorhinal inputs to hippocampal CA1 neurons.
Nature neuroscience, 15(11):1524 1530, 2012.
[50] Katie C Bittner, Aaron D Milstein, Christine Grienberger, Sandro Romani, and Jeffrey C Magee. Behavioral time scale synaptic plasticity underlies cal place fields. Science, 357(6355):1033- 1036, 2017.
[51] Robert C Froemke, Mu-ming Poo, and Yang Dan. depends on dendritic location. Nature, 434(7030):221 225, 2005.
synaptic plasticity
[52] Per Jesper Sjöström and Michael Häusser. A cooperative switch determines the sign of synaptic plasticity in distal dendrites of neocortical pyramidal neurons. Neuron, 51 -38,, 2006.
[53] Johannes J Letzkus, Björn M Kampa, and Greg J Stuart. Learning rules for spike timing-dependen plasticity depend dendritic synapse location. Journal of Neuroscience, 16441):10420-10429, 2006.
[54] Robert C Froemke, Johannes Letzkus, Bjorn Kampa, Giao Hang, and Greg Stuart. Dendritic synapse location and neocortical spike- takke-tiiing-eeee teee--ininnndeedeeee plasticity. Frontiers in synaptic neuroscience, 2:29, 2010.
13


Reviews:
[{"review_id": "LY1ueK7FUF6", "reviewer": "Reviewer_ZFwx", "summary": "This work combines probabilistic learning in the tradition of the Helmholtz machine with the recent hypothetical ideas about apical dendrites carrying top-down learning signals (expectation) to implement BP using local learning rule promoted earlier by numerous authors such as Larkum,  Lillicrap and Richards.  There are innovations. The impression learning mechanism proposed does sleep-wake in real-time.  It avoids the offline sleep phase in learning by using a global switching signal that can briefly substitute real incoming data with generative samples to enable learning continuously in a way that minimally perturbs natural neural trajectories.  This presents another new approach that allows the brain to learn generative models through local synaptic plasticity while concurrently performing sampling-based approximate inference wrt these models.  Incidentally, this work has been presented in COSYNE 2021 by the authors.\n\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Conceptually, the advance might be rather incremental.   It synthesizes many current existing ideas into a coherent framework.  It is well-written,  with mathematical rigor (more so than most of the earlier models). One weakness is that the approach has not been tested rigorously and compared against BP and other biologically-plausible BPs on benchmark datasets ( MINST, CIFAR or ImageNet) to evaluate its performance or to establish its competency as are customarily done nowadays in similar submissions.  Besides, as the authors admit, there is no biological evidence for this global switching signal.  While its relevance to understanding the brain is rather limited at present, it might stimulate new experimental research. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Broader impact section is missing.", "review_text": "", "extra_scores": {"ethical_concerns": "none.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours."}, "extra_text": {}}, {"review_id": "EOWuh69KcLU", "reviewer": "Reviewer_LAYW", "summary": "In this paper the authors introduce a framework for approximate online Bayesian inference in an unsupervised setting. This framework generalises the wake-sleep algorithm and enables the derivation of a local learning rule. The generative and inference components of the model are mapped onto distal and basal dendrites, respectively. A number of experiments are conducted to demonstrate the model behaviour using both artificial (toy) and naturalistic tasks.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The framework introduced by the authors is interesting and novel, and it provides a generalization of the wake sleep algorithm. The paper is well written, but some elements are missing to make the contribution clear (see below). The main contribution is the derivation of a local learning rule with elements that can be mapped onto distal and basal dendrites. This could be of high significance for computational neuroscience but its at present hard to fully assess (see below).\n\n*Major points*:\n - The link between the learning rule derived and biology is far from clear. There is only a very brief discussion on this in page 5 and the Discussion. But if this model is to be considered for understanding how basal/distal interactions lead to synaptic plasticity, this needs more discussion.\n - Importantly the basal learning rule (14) does not seem to depend on distal activity? This would be at odds with biology, as there is strong evidence that distal activity drives basal plasticity (see for example work by the labs of Anthony Holtmaat, Jeffrey Magee and Jesper Sjostrom).\n - Given that the algorithm proposed can do faster/online learning compared to the WS algorithm, why was it not used as one of the comparisons? It seems like one of the key models to contrast with. In particular, it would be important to show that this model does indeed need shorter offline phases. This seems to be one of the key contributions here that does not appear to have been clearly demonstrated.\n - In general a closer link between the model and existing/recent observations in how basal/distal interact would strength the story.\n\n\n*Minor points*:\n\n - 119: What is the biological interpretation for the gating variable, k? \n\n - 160: Does the network still learn if the latent space dimension or generative noise level do not match their ground truth values?  \n\n - 226: It would be interesting to mention if the network can learn when the phase duration is not the same for the two phases or if it can vary over the course of training. \n\n - 274: I think the distinction between excitatory and inhibitory neurons is an important point. Perhaps this could be elaborated more. \n\n - Fig 2c, 2d, 3b: choice of colours for NVI vs IL could be more distinctive \n\n- The authors use D donation for both Diagonal and non-diagonal matrices. This can cause confusion, in particular in Appendix D.1 (model structure), where D_2 is the diagonal transition matrix, and D_1 is the prediction weights. I suggest authors use different matrix notation or more informative subscripts.\n\n - In Eq.6, when the /lambda switches to 0 from 1, the value of k becomes 1, which in result the r^{gen}_t = r_{t-1} + \\sigma_{r}^{gen} \\eta_t . However, in Eq.15, there is no update when k_t = 1. So, it is not clear why this gating bias is in the training of the generative transition parameters D_r when there is no update at that time step. Can you please clarify this.\n\n - In Eq. 3, the second line, [log /hat{p} / \\hat{q}] are approximated by Taylor expansion around 1. It is not clear why it is a valid expansion about 1. The authors explain in detail how this leads to introducing bias in parameter update in appendix A, but it would be good to briefly explain what are the consequences of this approximation, in particular as this seems to be a novel element compared to previous work.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There are a number of limitations that have not been fully addressed in terms of relating the model to biology (see comments above).", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "6kS1RjW--AY", "reviewer": "Reviewer_gqEo", "summary": "The authors are interested in understanding how brain circuits are able to perform unsupervised learning. The work is an extension of the wake-sleep algorithm, where the algorithm is asked to perform two functions: i) infer latent variables from observations and ii) build a generative model of the received observations. They start by defining a loss function to be minimized for their system to perform both functions and then interpret this minimization as a plasticity rule between the weights of a network of neurons. Their learning algorithm is compared to other existing one in the literature, and its efficiency is probed on a real-world task.  ", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Although I am not an expert on this topic I have enjoyed reading the paper. The problem of designing unsupervised learning algorithm that could be mapped onto the functioning of brain circuits is of fundamental importance for both neuroscience and AI, and this work seems a solid contribution towards this goal. The paper is well written and clearly situates its contribution with respect to the existing literature. The direct comparison with other existing approaches, as well as the probing of performance of the algorithm on a real-world task make the work appears as high-quality. The mapping between the inference part of the algorithm and processing at the basal part of neurons, and the mapping between the generative part of the algorithm and processing at the apical part of neurons seem well sounded and allows to discuss experimental results in neuroscience. It would be nice to mention how current views on plasticity relates to the idea that weight updates should depend from firing rates of pre- and post- synaptic cells as well as basal current. \n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "0iHP_Ikgjpy", "reviewer": "Reviewer_YoDe", "summary": "The authors proposed an algorithm called impression learning as a mechanism for the brain to learn the generative model of sensory inputs and meanwhile performs probabilistic inference of this model. The authors proposed a local synaptic learning rule to learn the generative model, and considered sampling-based inference. It is a valuable contribution to our understanding the general principle of neural coding.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nThe main idea is based on wake-sleep algorithm, but has novel non-trivial improvements.\nStrength: \nThis work proposes a novel mechanism for the brain to perform approximate online Bayesian inference through a local learning rule, which is quite interesting.\nWeaknesses: \nThe biological plausibility of some elements need to be clarified.\t\n1) what is the biological meaning of D_r in equation 6 ?  D_r  seems to be a recurrent connection matrix in the latent layer, but it is  diagonal and does not play a role when k_t=1.\n2) the neuron should receive both top-down (apical) and bottom-up (basal) inputs concurrently，the variable λ_t induced by dopamine may control the learning phase (the infer phase and generative phase), but should not also control the apical inputs and basal inputs.\nClarity:\nIt is OK.\nSignificance:\nIt is an interesting idea to implement concurrent the generative model learning and the model inference.  \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Not applicable.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces an unsupervised local plasticity rule to perform approximate online Bayesian inference of latent structure from sensory stimuli. All reviewers agree on the novelty and significance of the paper. However, reviewers brought up concerns about biological plausibility of some elements of the proposed algorithm, which should be addressed.
Decision: Accept (Poster)