Impression learning: Online representation learning with synaptic plasticity
Year: 2021

Abstract:
Understanding how the brain constructs statistical models of the sensory world remains a longstanding challenge for computational neuroscience. Here, we derive an unsupervised local synaptic plasticity rule that trains neural circuits to infer latent structure from sensory stimuli via a novel loss function for approximate online Bayesian inference. The learning algorithm is driven by a local error signal computed between two factors that jointly contribute to neural activity: stimulus drive and internal predictions — the network’s ‘impression’ of the stimulus. Physiologically, we associate these two components with the basal and apical dendrites of pyramidal neurons, respectively. We show that learning can be implemented online, is capable of capturing temporal dependencies in continuous input streams, and generalizes to hierarchical architectures. Furthermore, we demonstrate both analytically and empirically that the algorithm is more data-efficient than a three-factor plasticity alternative, enabling it to learn statistics of high-dimensional, naturalistic inputs. Overall, the model provides a bridge from mechanistic accounts of synaptic plasticity to algorithmic descriptions of unsupervised probabilistic learning and inference.

heading: 1 Introduction
text: Sensory systems are faced with a task analogous to the scientific process itself: given a steady stream of raw data, they must extract meaningful information about its underlying structure. Because the true underlying structure of the data is rarely accessible, this “representation learning” must be largely unsupervised. Framing perception in the language of Bayesian inference has proven fruitful in perceptual and cognitive science [1–4], but has been difficult to connect to biology, because we still lack a satisfactory account of how the machinery of Bayesian inference and learning is implemented in neural circuits [5, 6].
Past work includes several examples of circuits that simultaneously learn a top-down generative model of incoming stimuli and perform approximate inference with respect to these models. These differ in the nature of the approximation, from maximum a posteriori estimation [7], to efficient population codes that embed prior structure [8] to either parametric [9, 10] or sampling-based [11] variational inference. Learning generally takes the form of optimizing a probabilistic objective, either by backpropagation [9, 10] or through local parameter updates, which match biological learning more
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
closely [11, 7, 12, 13]. While these models are mostly restricted to static stimuli, several instances also operate over time [14–16].
Developing biologically plausible learning rules that are applicable to temporally-structured data is hampered by the fact that optimizing a probabilistic objective function in such contexts requires access to non-local information across space and time. Previous research on local approximations to credit assignment in BP address spatial credit assignment by ascribing differential functions to the apical and basal dendrites of pyramidal neurons in cortex, where apical dendrites are hypothesized to receive top-down learning signals, and basal dendrites receive bottom-up sensory signals [17–25]. Locally implementing temporal credit assignment is a bigger challenge [26, 27].
Our work, which we have dubbed ‘impression learning’ (IL), combines the tradition of probabilistic learning [11, 14] with these recent developments in local optimization, in order to learn dynamic stimuli concurrently with perception. We propose a network architecture in which top-down stimulus predictions arriving at the apical dendrites of neurons influence both network dynamics and synaptic plasticity, allowing the network to concurrently learn a probabilistic model of the stimuli and an approximate inference computation. We provide a mathematical derivation of synaptic plasticity rules that approximate gradient descent on a novel unsupervised loss function, along with detailed analyses of the biases induced by this approximation. We explore the empirical and mathematical relationships between IL and three other methods: backpropagation (BP) [28], the Wake-Sleep (WS) algorithm [29], and a specific form of neural variational inference (NVI∗) [30, 31]. We further demonstrate that IL scales to naturalistic stimuli and multilayer network architectures 1.

heading: 2 Probabilistic inference and local learning in a recurrent circuit
text: We construct a network of neurons that aims to learn a generative model of the temporal sequence of stimuli that it receives, pm(r, s) = ∏T t=0 pm(rt, st|rt−1), in which s represents stimuli in an input layer.2 The latent variables r are not defined by a physical model of the stimulus environment, but are learned in an unsupervised manner to provide the best generative explanation of stimuli received. We assume that stimuli are generated by a true probability distribution p(s|z), where s corresponds to the first layer of neural activations in an early sensory layer, and vector z ∼ p(z) corresponds to the environmental factors which jointly caused that activity. Because learning is unsupervised, we do not enforce explicit correspondence between the internal and true latent features, r and z, only a correspondence between model predictions and ground truth stimuli. We also assume that the network performs online inference with respect to its model, inferring the corresponding latent cause r using Bayes’ rule: pm(r|s) = pm(r, s)/pm(s). Because the network won’t, in general, be able to explicitly calculate Bayes’ rule, we will assume that the network learns an approximate inference distribution q(r|s), which it attempts to bring ‘close’ to pm(r|s). This joint process of learning and inference, known as Bayesian latent feature extraction, provides a general framework for conceptualizing early sensory processing in the brain [5]. In subsequent sections, we will write a loss function for this general latent feature extraction objective, and show how local modifications at apical and basal synapses can perform approximate gradient descent on this loss.
Loss function The loss function that we propose will produce a learning algorithm where neurons alternate between sampling from the model, pm, and performing approximate inference according to q in response to real stimuli received from p(s|z). This alternation will allow the network to learn online in a way that minimally perturbs the continuity of perception. First, consider two families of hybrid probability distributions, which we denote in shorthand q̃θ and p̃θ:
q̃θ = T∏ t=0 q̃t(rt, st|zt, λt; θ) = T∏ t=0 (q(rt|st; θq)p(st|zt))λt pm(rt, st|rt−1, λt; θp)1−λt
p̃θ = T∏ t=0 p̃t(rt, st|zt, λt; θ) = T∏ t=0 (q(rt|st; θq)p(st|zt))1−λt pm(rt, st|rt−1, λt; θp)λt , (1)
where a collection of binary random variables λt determines whether, at a given time step, sampling occurs due to q(rt|st; θq)p(st|zt) or pm(rt, st|rt−1, λt; θp), and the full parameter space is denoted
1Code provided at: https://github.com/colinbredenberg/Impression-Learning-Camera-Ready. 2We use the shorthand notation ‘s’ to refer to the N × T matrix of stimuli across time.
θ = [θp, θq]. We define an objective of the form:
L = Eλ,z [KL[q̃θ||p̃θ]] = Eλ,z [∫ [log q̃θ − log p̃θ] q̃θ drds ] . (2)
This loss provides a generalization of the widely-used evidence lower bound (ELBO), which corresponds to the case λt = 1 ∀t. Importantly, we can show that L = 0 if and only if q(rt|st; θq)p(st|zt) = pm(rt, st|rt−1, λt; θp) ∀t. If this equality were achieved, it would also imply pm(r, s) = q(r|s)p(s|z). However, this absolute minimum will not be achievable unless zt is deterministic, because pm(rt, st|rt−1, λt; θp) has no dependency on the latent variables in the environment. Thus, our goal is inherently unachievable, and different choices of p(λt) and network architectures may lead to different local minima. However, each choice will incentivize learning a close correspondence between these distributions, and an approximation to gradient descent with respect to any choice will lead to local synaptic plasticity rules, making this objective particularly interesting for the computational neuroscience community.
Update derivation We begin by taking the gradient of our new loss w.r.t. θ = [θq, θp]:
−∇θL =−∇θEλ,z [∫ [log q̃θ − log p̃θ] q̃θ drds ]
=− Eλ,z [∫ [∇θ(log q̃θ − log p̃θ)] q̃θ drds + ∫ [log q̃θ − log p̃θ]∇θ q̃θ drds ] ,
where the second equality follows from the product rule. Both integrals are analytically intractable, but if we can write both as expectations, they can be approximated by averaging over samples of r and s. To accomplish this, we note that ∇θ q̃θ = ∇θelog q̃θ = [∇θ log q̃θ] q̃θ, which allows us to rewrite our expression as an expectation over r and s:
−∇θL =− Eλ,z [∫ [∇θ log q̃θ −∇θ log p̃θ] q̃θ drds + ∫ [log q̃θ − log p̃θ] (∇θ log q̃θ)q̃θ drds ] .
We also observe that ∫ [∇θ log q̃θ] q̃θ drds = ∇θ ∫ q̃θ drds = ∇θ1 = 0, allowing the elimination of two terms:
−∇θL = Eλ,z [∫ [∇θ log p̃θ] q̃θ drds + ∫ [
log p̃θ q̃θ
] (∇θ log q̃θ)q̃θ drds ] ≈ Eλ,z [∫ [∇θ log p̃θ] q̃θ drds + ∫ [ p̃θ q̃θ − 1 ] (∇θ log q̃θ)q̃θ drds ]
= Eλ,z [∫ [∇θ log p̃θ] q̃θ drds + ∫ [
p̃θ q̃θ
] (∇θ log q̃θ)q̃θ drds ] = Eλ,z [∫ [∇θ log p̃θ] q̃θ drds + ∫ [∇θ log q̃θ] p̃θ drds ] . (3)
The approximation in the second line comes from a Taylor expansion of log p̃θq̃θ about 0, i.e. when p̃θ q̃θ
= 1 (which introduces a bias to the parameter updates that we examine analytically in Appendix A). This expansion is the core of our derivation, and not all algorithms take this approach: for this reason, in Appendix B and C we show how the properties of our algorithm compare to alternatives (NVI∗, BP, or WS).
At this point, we have not yet defined p(λ). We’ll assume that λ0 ∈ {0, 1}, that p(λ0 = 0) = p(λ0 = 1) = 0.5, and that the λ values alternate deterministically with a ‘phase duration’ K, i.e. λk+1 = 1 − λk if mod (k,K) = 0, and λk+1 = λk otherwise. Under these conditions, the two integrals in Eq. (3) are equivalent, and computing our parameter updates only requires sampling from q̃. If we define λ′ = 1− λ, then we have p(λ′) = p(λ) and q̃(r, s|z, λ; θ) = p̃(r, s|z, λ′; θ), which
we make use of as follows:
−∇θL ≈Ez [∑ λ [∫ [∇θ log p̃θ] q̃θ drds + ∫ [∇θ log q̃θ] p̃θ drds ] p(λ) ]
=Ez [∑ λ ∫ [∇θ log p̃θ] q̃θ drdsp(λ) + ∑ λ′ ∫ [∇θ log p̃θ] q̃θ drds p(λ′) ]
=2Ez [∑ λ ∫ [∇θ log p̃θ] q̃θ drds p(λ) ] . (4)
Using the definitions for q̃θ and p̃θ and the properties of the logarithm gives us the following parameter update rule:
∆θ ∝ 2Eλ0,z [∫ [∑ t (1− λt)∇θ log qt + (λt)∇θ log pmt ] q̃θ drds ] . (5)
As we will show below, this parameter update equation produces updates that require only information locally available to synapses, a necessary condition for any biologically-plausible algorithm.
Basic model To make the above general learning procedure concrete, we need to specify how to sample from q̃θ, which in turn requires an architecture for performing approximate inference at each time step, q(rt|st; θq), and a joint model of stimuli and neural activations, pm(rt, st|rt−1, λt; θp). We map these two model components onto neural circuitry, with their own local variables corresponding to s and r, and segregated synaptic parameters: the ‘basal’ compartment is dedicated to feedforward inference (q, index ‘inf’) and the ‘apical’ compartment is dedicated to generative sampling from the model (pm, index ‘gen’); this segregation allows their influence on neural dynamics to be selectively gated by λt (Fig. 1a).
First, the internal generative model of the circuit is implicitly defined by a set of currents to the apical dendritic compartment corresponding to generated samples for the next latent variable, rgent :
rgent = ((1− kt)Dr + ktI) rt−1 + σgenr ηt (6) sgent = f(Dsrt) + σ gen s ξt, (7)
where Dr is a diagonal transition matrix (constraining generated latent-variables to be independent AR(1) processes), Ds is a linear decoder, I is the identity function, ηt, ξt ∼ N (0, I) are independent white noise samples, and σgenr and σ gen s denote respectively the generative standard deviation for neurons at the stimulus and latent levels. We define kt = (1 − δ(λt − λt−1))λt, with δ(·) the Dirac delta function; kt is 1 only if λt = 1 and λt−1 = 0. We chose a piecewise model (gated by kt) for r gen t because we observed that the statistics of stimuli st given previous activities rt−1 are different if a transition has just occurred (λt = 1 and λt−1 = 0), which will bias the training of the generative transition parameters Dr. We chose I for this case, but one could alternatively have a different parametric model for after transitions have occurred. As we will show, adding this condition to our model will never affect the dynamics of our network, but will cause learning for Dr to occur only on time steps when a transition has not just occurred. Nothing in our derivation requires the transition matrix Dr to be diagonal, but we constrained it in this way to allow for learning independent latent features. As is, Dr defines the leakiness of the apical dendritic compartment of the neuron; off-diagonal components of the transition matrix would correspond to recurrent synapses. These dynamics define a probability distribution: pm(r, s) = ∏T t=0 pm(rt, st|rt−1, λt; θp). Second, we define our inference model, a factorized conditional probability distribution q(r|s) =∏T t=0 q(rt|st; θq), which applies a feedforward nonlinear transformation to incoming stimuli:
rinft = f(Wst) + σ inf r ηt, (8)
where W denotes the feedforward weights, σinfr is the inference standard deviation for neurons at the latent level, and the nonlinearity f(·) is the tanh function. During inference mode, the stimulus layer receives latent-associated inputs from the environment, further corrupted by the same noise as the internal representation:
sinft = s̄(zt) + σ inf s ξt, (9)
where σinfs denotes the standard deviation for neurons at the stimulus level, and s̄(zt) is input from external stimuli. During simulations, samples are determined by a combination of pm and q, given by q̃θ:
rt = λtr inf t + (1− λt)r gen t (10)
st = λts inf t + (1− λt)s gen t . (11)
We interpret these dynamics biologically as network of recurrently connected pyramidal neurons with two sources of input, one to the apical dendrites (rgent or s gen t ) and one to the basal dendrites (r inf t or sinft ). The gating variable λt determines which input source controls the circuit dynamics.
Plasticity rule interpretation Inserting our particular choice of qt and pmt into our approximate gradient descent derivation, the parameter updates can be interpreted as local synaptic plasticity rules at the basal (for qt) or apical (for pmt) compartments of our neuron model:
log q(rt|st; θq) =− 1
2 (σinfr ) 2 ‖rt − f(Wst)‖ 2 2 + cq (12)
log pm(rt, st|rt−1, λt; θp) =− 1
2(σgenr )2 ‖rt − ((1− kt)Dr + ktI) rt−1‖22
− 1 2(σgens )2 ‖st − f(Dsrt)‖22 + cp, (13)
where cq = −Nr log( √ 2π(σinfr ) 2) and cp = −Nr log( √ 2π(σgenr )2 − Ns log( √ 2π(σgens )2 are constants that do not depend on network parameters. We can use these equations to evaluate our weight updates, by using the general formula in Eq. 5 and calculating derivatives. For online parameter
updates, we assume that weights change stochastically at each time step, based on samples from λ0, z, r, and s (instead of explicitly calculating the expectation in Eq. 5):
∆W(ij) ∝ 1− λt (σinfr ) 2 (r (i) t − f(Wst)(i))f ′(Wst)(i)s (j) t (14)
∆D(ii)r ∝ λt(1− kt)
(σgenr ) 2 (r
(i) t − (Drrt−1)(i))r (i) t−1 (15)
∆D(ij)s ∝ λt
(σgens ) 2 (s
(i) t − f(Dsrt)(i))f ′(Dsrt)(i)r (j) t . (16)
Each of these updates has the form of a local synaptic plasticity rule, under the following assumptions: W(ij) is a basal synapse from neuron j to neuron i, r(i)t and r (j) t correspond to the pre- and postsynaptic firing rates, respectively, and f(Wst)(i) corresponds to the local basal current injected into neuron i. Thus, assuming that a basal synapse has access to both the neuron’s firing rate and its local basal synaptic current at a particular moment in time, ∆W(ij) is local; the same principle holds for the apical updates. If λt = 0, then network activity is driven by the generative inputs, and so the parameter updates for basal synapses depend on apically-driven activity, as has been observed experimentally [32]; similarly, apical synaptic plasticity should depend on basally-driven activity. The updates for the generative transition matrix, Dr–determining the leakiness of the apical dendritic compartments–are gated by 1 − kt, indicating that parameter updates are delayed upon entering ‘inference’ mode: this could reasonably be implemented biologically by a slow cascade of biochemical processes that delay changes in neural parameters, as has been proposed by previous plasticity models [33, 34].

heading: 3 Numerical Results
text: Validation on artificial stimuli To analyze IL performance in an environment where we have access to and control over the statistics of the latent dynamics zt, we constructed artificial stimuli as follows:
zt = Λzt−1 + σ trueηt (17)
s̄(zt) = Azt, (18)
where Λ is a Nz × Nz diagonal matrix with Λii < 1 ∀i, A is a Ns × Nz random matrix with Aij ∼ N (0, 1Nz ), and ηt ∼ N (0, 1).
3 For simplicity, we fix the dimension of the latent space and the generative noise in the network to the ground truth values, Nr = Nz neurons, and σtrue = σgenr , so that in principle our model ∫ pm(r, s)dr can match the ground truth data distribution ∫ p(s, z)dz exactly. This also means that we can verify that the network has learned an optimal model by comparing its second-order statistics to those of the ground truth distribution.
We trained the network using IL, verifying that the online synaptic updates minimize the loss L (Fig. 1c). We further validate that the network has learned to accurately perform inference, so that q(r|s) ≈ pm(r|s), and that the network has learned a good model of the data, so that∫ pm(r, s)dr ≈ p(s), as per our original goals. We show that when the network is performing approximate inference, i.e. λt = 1, ∀t, stimulus reconstructions based on the network’s latent state are closely matched to the actual stimuli, i.e. st ≈ f(Dsrt), meaning that the network is functioning as a good autoencoder across time (Fig. 1d), and across all stimulus dimensions (Fig. 1e). To verify the network’s generative performance, we also show that the temporal autocorrelations for the network rates rt in generative mode (λt = 0 ∀t) closely overlap with the ground truth autocorrelation structure of z, suggesting that the learned latent features correspond (modulo a rotation) to the true latent features. Note that this latent variable match occurs because we have enforced a correspondence between the true data-generating distribution and our model, and would not necessarily happen if a different model architecture were used.
Algorithm comparisons Having verified that IL is capable of training the network on simulated data, we next compared it to alternative algorithms in the literature, including neural variational
3The parameter values and initialization details for all simulations are included in the supplementary code, which was run on an internal cluster; Ns = 100 and Nz = 20.
inference (NVI∗), BP, and WS (see Appendix B for detailed mathematical comparisons and derivations). In particular, NVI∗ provides an alternative candidate model of how the brain could plausibly learn neural representations through variational inference [31]. Because NVI∗ performs poorly for high-dimensional stimuli and large numbers of time steps (Appendix C; [35]), we simplified the task by reducing the dimensionality of the latent space, Nz = 2, and stimulus space, Ns = 4. For twenty evenly-spaced time points over the course of the learning trajectory, we compared the inference parameter updates given by IL, ∆θILq , to the inference parameter updates given by NVI
∗, ∆θNVIq , for a 4 time-step stimulus sequence (Fig. 2a). To get good estimates of the mean and variance of these sample parameter updates, we averaged over 106 different realizations of the network noise, and compared the samples using two measures. First, we considered the cosine similarity (normalized inner product) between the two empirical mean updates, ∆θ IL
q = 1 N ∑N k=0 ∆θ IL q and ∆θ NVI q = 1 N ∑N k=0 ∆θ NVI q
(Fig. 2b), where cos(θ) ∈ [−1, 1], and cos(θ) < 0 in this case would indicate that the parameter updates are anticorrelated. Because the NVI∗ update is unbiased, ie. E[∆θNVIq ] = − ddθqL, as long as we have averaged over a sufficient number of samples N , a positive cosine similarity across learning between the IL update and the NVI∗ update (Fig. 2b) indicates that our update is aligned in expectation to the true gradient of the loss, and hence will improve performance. This is a way of empirically verifying that the bias we introduce in our derivation does not impair the learning process.
Having verified that the IL update and the true gradient are aligned on average, we next examine whether the updates given by NVI∗ differ in terms of their signal-to-noise ratio (SNR) from the IL updates, where we define the SNR as:
SNR(∆θq) = 1
Nθ Nθ∑ i=0
( ∆θ (i)
q )2 S2(∆θ (i) q ) , (19)
where S2(·) denotes the sample variance. This measure is an average across individual parameter updates ∆θ(i), and it increases with ∥∥∆θq∥∥22 and decreases as the estimator variance grows. As Fig. 2c shows, the SNR is many orders of magnitude lower for NVI∗ than for IL over learning, likely due to the high estimator variance of the NVI∗, which we demonstrate analytically for a simple example in the Appendix C. The estimator variance has direct implications for the speed of learning and asymptotic performance, so that even though NVI∗ and IL can have parameter updates that are aligned in expectation, due to its low variance IL will greatly outperform NVI∗ during training.
We verified the generality of these benefits in the same task, as we variedNs, Nz andNr concurrently, so that Ns = 2Nz = 2Nr. We optimized learning rates for NVI∗, BP, and IL separately on the lowest dimensional condition by grid search across orders of magnitude (10−2, 10−3, etc.), and found that NVI∗ performed worse over the entire range, while IL and BP showed similar performance (using the negative ELBO loss as a standard). Moreover, while NVI∗ showed worse performance as the stimulus dimension increased, this was not the case for IL or BP (Fig. 2d).
Phase duration effects The previous numerical results verify that IL is able to effectively learn a generative model of artificial data, and to perform inference with respect to that model. However, for
IL to be a valid candidate for online learning in the brain, the learning process should not significantly interfere with perception. To test this, we explored how the ‘phase duration’ K affects the correlation between network activity in a simulation where λt = 1, ∀t, and a simulation where λt alternates phases every K time steps (for a fixed random seed and stimulus sequence). If the learning process did not interfere with perception at all, this correlation would be 1, and if it completely disrupted perception it would be 0, or even negative. In Fig. 3c and d, we show two example traces with K = 2 and K = 32, respectively, comparing the network in inference mode to the network during learning. While neural trajectories for the shorter phase durations are closely correlated, they deviate considerably for longer phase durations (Fig. 3c-e). Despite this, the loss profile (negative ELBO) is identical. Since WS can be viewed as a special case of IL for very long phase durations (Appendix B.3; see Fig. S1a for an even longer phase duration), this implies that the two methods have similar performance. However, IL operating in a mode of fast fluctuations between inference and generation may be more biologically relevant, as this reduces the interference with perception without impairing learning. Moreover, we found that lengthening the duration of the inference phase alone while keeping very short bursts of generative activity further reduced perceptual disturbance, while only slightly increasing the time required to learn (Fig. S1b-d).
Spoken digits task Having verified the performance of IL on artificial stimuli, we next tested its performance on higher-dimensional and more complex naturalistic stimuli. We used the training and test sets of the Free Spoken Digits Dataset [36], which provides audio time series of humans speaking digits 0-9.4 We transformed these time series into log-mel spectrograms as a coarse approximation of the initial stages of the human auditory system, shifted the inputs by a constant so as to make them all positive, and divided the result by the across-channel standard deviation. The results of Fig. 4 are shown in the original log-mel spectrogram input space.
To assess the hierarchical processing capabilities of IL, we added an additional feedforward layer to the network architecture (Fig. 4a); we provide the details of how this modification affects simulation and parameter updates in Appendix D. To compare IL to NVI∗, we again optimized learning rates via grid search across orders of magnitude, and found that IL greatly outperformed NVI∗ when each was evaluated at its respective optimal learning rate (Fig. 4b). Furthermore, we observed that our
4The FSDD is available at https://github.com/Jakobovski/free-spoken-digit-dataset.
trained network meets the same criteria for success as for our artificial stimuli, namely its stimulus reconstructions closely match the true stimulus while in inference mode (λt = 1 ∀t; Fig. 4c), and sample stimuli produced while the network is in generative mode (λt = 0 ∀t) qualitatively correspond to ground-truth stimuli (Fig. 4d), and quantitatively match the structure of both spatial (Fig. 4e) and temporal (Fig. 4f) autocorrelation of the input. These results collectively demonstrate that IL is capable of training neural representations of complex real-world stimuli. They also show that IL can function when there is a mismatch between its architecture and the structure of environmental latent variables, which are in this case unknown. In general, learning may fail if the chosen network architecture is too restrictive.

heading: 4 Discussion
text: Impression learning (IL) provides a potential mechanism for the brain to learn generative models of its sensory inputs through local synaptic plasticity, while concurrently performing approximate inference with respect to these models. IL is a direct generalization of the Wake-Sleep algorithm [29], which replaces lengthy offline ‘Sleep’ phases with brief substitutions of network-generated samples in place of incoming data, in a way that minimally perturbs natural neural trajectories. Transitions between ‘inference mode’ and ‘generative mode’ are controlled by a global signal λt, which decides whether generative signals to the apical synapses or inference signals to the basal synapses dominate network activity.
Computationally, IL outperforms NVI∗ [30, 31], a particular instance of three-factor plasticity [37], because its internal model provides explicit ‘credit assignment’ for each individual neuron, rather than implicitly calculating it via correlations between neural activity and a global reward signal. This leads to lower-variance gradient estimates and faster learning. Alternative learning algorithms such as backpropagation (through time) [38] are not intrinsically probabilistic, but can be used for optimizing probabilistic objectives. Like IL, BP provides explicit credit assignment, but the parameter updates it provides are nonlocal across both network layers and time. It is worth noting that IL was developed in a purely unsupervised learning setting, whereas both BP and NVI∗ extend to supervised and reinforcement learning [39, 40]. In the context of supervised learning, several biologically-plausible approximations to BP leverage the apical-basal dendritic structure of pyramidal neurons to learn
[25, 21], based primarily on target-propagation [41] or its variants [42]. It would be valuable to explore the combination of such extensions with the continuous online learning capabilities of IL.
Local computations are considered a necessary condition for learning algorithms to be biologicallyplausible. In our framework, locality is enforced through the structure of the internal graphical model (pm) and the approximate inference distribution (q): any choice of neural network architecture with independent noise will guarantee local plasticity. Our framework is relatively agnostic to the details: neurons could be either rate-based with Gaussian intrinsic noise (as in the examples presented here), or generate spikes with Poisson variability, which would result in synaptic updates analogous to empirically observed spike-timing-dependent plasticity, as found in generalizations of WS [14]. It would also be possible to make distinctions between excitatory and inhibitory neurons, by requiring all outgoing synapses from individual neurons to be either positive or negative, or to include more complex dendritic arborizations, as have been explored in recent experimental [43] and modeling [44] efforts. Our current model enforces hard, global phase distinctions (λt ∈ {0, 1} for all neurons), which could potentially correspond to alternations between activity driven by apical dendritic calcium events and basal spiking tied to theta oscillations in the hippocampus [32]. However, cortical data indicate that input to apical and basal dendrites contribute concurrently and constructively to spiking activity [45]. We are currently working to extend our derivation to these circumstances, by allowing λt to be non-binary and heterogenous across neurons.
Traditional predictive coding [7] requires steady-state assumptions for learning, meaning that neural dynamics must occur on a timescale much faster than that of stimuli. In contrast, IL requires a mechanism by which the relative influence of the apical and basal dendrites of pyramidal neurons can be rapidly switched, along with learning mechanisms that operate at that timescale. If such a mechanism could be experimentally identified and controlled, our model makes the specific prediction that increasing the dominance of apical dendritic input on neural activity (λt ≈ 1) would cause the network to sample from its generative model, i.e. the manipulation will induce structured hallucinations that mimic realistic stimuli (and associated neural activity), without being tied to the sensory world. One candidate gating mechanism is rapid inhibition targeting apical dendrites specifically [46–49]; but much work remains to explicitly relate this mechanism to learning and plasticity.
IL predicts that synapses will use an error signal based on the difference between local dendritic compartmental currents (either apical or basal) and the neuron’s total firing rate to perform learning. There is some evidence that spiking activity driven by apical inputs to pyramidal neurons can induce plasticity at basal synapses [32, 50], and several studies have found systematic changes in synaptic plasticity between apical and basal synapses, in particular the sign changes induced by local dendritic inputs that IL predicts [51–54]. Hence, IL has the potential to explain the diversity of plasticity phenomena observed experimentally and inform future experiments.
Acknowledgments and Disclosure of Funding
We thank Camille Rullán Buxó, Caroline Haimerl, Owen Marschall, Pedro Herrero-Vidal, Siavash Golkar, David Lipshutz, Yanis Bahroun, Tiberiu Tesileanu, Eilif Muller, Blake Richards, Guillaume Lajoie, Maximilian Puelma Touzel, and Alexandre Payeur for helpful discussions and feedback on earlier versions of this manuscript. We gratefully acknowledge the Howard Hughes Medical Institute and the Simons Foundation for their support of this work. CS is supported by National Institute of Mental Health Award 1R01MH125571-01, by the National Science Foundation under NSF Award No.1922658 and a Google faculty award.

Review summaries:
In this paper the authors introduce a framework for approximate online Bayesian inference in an unsupervised setting. This framework generalises the wake-sleep algorithm and enables the derivation of a local learning rule. The generative and inference components of the model are mapped onto distal and basal dendrites, respectively. A number of experiments are conducted to demonstrate the model behaviour using both artificial (toy) and naturalistic tasks.

This work combines probabilistic learning in the tradition of the Helmholtz machine with the recent hypothetical ideas about apical dendrites carrying top-down learning signals (expectation) to implement BP using local learning rule promoted earlier by numerous authors such as Larkum,  Lillicrap and Richards.  There are innovations. The impression learning mechanism proposed does sleep-wake in real-time.  It avoids the offline sleep phase in learning by using a global switching signal that can briefly substitute real incoming data with generative samples to enable learning continuously in a way that minimally perturbs natural neural trajectories.  This presents another new approach that allows the brain to learn generative models through local synaptic plasticity while concurrently performing sampling-based approximate inference wrt these models.  Incidentally, this work has been presented in COSYNE 2021 by the authors.  

The authors proposed an algorithm called impression learning as a mechanism for the brain to learn the generative model of sensory inputs and meanwhile performs probabilistic inference of this model. The authors proposed a local synaptic learning rule to learn the generative model, and considered sampling-based inference. It is a valuable contribution to our understanding the general principle of neural coding.

The authors are interested in understanding how brain circuits are able to perform unsupervised learning. The work is an extension of the wake-sleep algorithm, where the algorithm is asked to perform two functions: i) infer latent variables from observations and ii) build a generative model of the received observations. They start by defining a loss function to be minimized for their system to perform both functions and then interpret this minimization as a plasticity rule between the weights of a network of neurons. Their learning algorithm is compared to other existing one in the literature, and its efficiency is probed on a real-world task.