BayesPCN: A Continually Learnable Predictive Coding Associative Memory
Authors: Jinsoo Yoo
Frank Wood
Venue: NeurIPS 2022 Accept (2022)
Keywords: machine learning
associative memory
predictive coding
continual learning
bayesian inference

Abstract: Associative memory plays an important role in human intelligence and its mechanisms have been linked to attention in machine learning. While the machine learning community's interest in associative memories has recently been rekindled, most work has focused on memory recall ($read$) over memory learning ($write$). In this paper, we present BayesPCN, a hierarchical associative memory capable of performing continual one-shot memory writes without meta-learning. Moreover, BayesPCN is able to gradually forget past observations ($forget$) to free its memory. Experiments show that BayesPCN can recall corrupted i.i.d. high-dimensional data observed hundreds to a thousand ``timesteps'' ago without a large drop in recall ability compared to the state-of-the-art offline-learned parametric memory models.

BayesPCN: A Continually Learnable Predictive Coding Associative Memory
Jason Yoo Department of Computer Science University of British Columbia Vancouver, Canada j jasony978cs. ubc.
Frank Wood
Department of Computer Science University of British Columbia Vancouver, Canada fwoodêcs ubc.
Abstract
Associative memory plays an important role in human intelligence and its mech- anisms have been linked to attention in machine learning. While the machine learning community's 's interest in associative memories has recently been rekindled, most work has focused on memory recall (read) over memory learning (wrile). this paper, we present BayesPCN, a hierarchical associative memory capable of performing continual one-shot memory writes without meta-learning. Moreover, BayesPCN able to gradually forget past observations (forget get) to free its memory. Experiments show that BayesPCN can recall corrupted i..d. high- high-dimenssoonl data observed hundreds to a thousand "timesteps" ago without a large drop in recall ability compared to the state-of-the-ae offline-leamed parametric memory models.
Introduction
One of the hallmarks of biological intelligence is the ability to robustly recall learned associations. Biological agents are able to associate stimuli of various modalities and rely on their memories to perform everyday tasks from navigation to entity recognition. Yet these agents are not only good at recall but can quickly and continually form new memories from as little How do biological agents achieve this feat while mitigating catastrophic forgetting [French, 1999]? And how can we build neural networks that demonstrate these properties? This paper addresses these questions, focusing on fast, continual learning of neural artifacts that perform robust recall of i.i.d. high-dimenssinnal observations.
single experience.
Associative memory models have been of interest in machine learning for decades [Steinbuch, 1961, Hopfield, 1982]. They have been used to solve a wide range of problems from sequence processing [Schmidhuber, 1992, Graves et al., 2014, Schlag al., 2021] to pattern detection [Widrich et al., 2020, 2021, Seidl et al., 2021]. Their study has allowed us to build models that replicate aspects of biological intelligence [Kanerva, 1988, Hawkins and Blakeslee, 2004] and provided novel perspectives machine learning toolsets like dot-product attention, transformers, and linear layers of deep neural networks [Ramsauer et al., 2020, Bricken and Pehlevan, 2021, Irie et al., 2022).
Our main aim is to recall exact memories from neural network that can rapidly and continually learn from a stream data. Towards this end we introduce BayesPCN, a Bayesian take on the generative predictive coding network (GPCN) [Salvatori et al., 202 that combines predictive coding and locally conjugate Bayesian updates There are currently number of associative memory systems that are continually learnable [Hopfield, 1982, Kanerva, 1988, Hawkins and George, 2007, Ramsauer et al., 2020, Sharma et al.. 2022]. However. to our knowledge, BayesPCN is the first parametric associative memory model to continually learn hundreds of high-dimensions observations while maintaining
"Code is available
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Example queries
memory queries
read oulputs before one-shot memory write
read outputs after one-shot memory write
Example unrelated images subsequently written into memory
read outputs after 192 memory writes of unrelated images
Figure I: BayesPCN recall before and after the one-shot write of the top left image into memory.
The first row contains example memory queries, the second row contains the memory's read outputs before write, the third row contains the memory's 's read outputs immediately after write, the fourth row contains sample images written into memory after the top left image, and the last row contains the memory's ss read outputs after 192 additional datapoints have been sequentially written into memory. The model has 3 hidden layers of width 1024. The four leftmost columns illustrate auto-associative recall while the remaining columns illustrate hetero-associative recall.
good recall performance on inputs with nontrivial amounts of corruption. In addition, it is also the first hierarchical memory which after repeated forgetting recovers its original memory state.
2 Background
2.1 Auto-Associative and Hetero-Associative Memory
Before we begin let us review two useful concepts: auto-associative memory and hetero-associative memory. The former, auto-associative memory, refers to a memory system that is content addressable but for which the query may be corrupted version of the original data item. One can think about auto-associative memory systems as a kind of approximate nearest neighbor lookup mechanism. Hetero-associitivee memory systems are key-value stores, meaning that the recalled "value" is potentially of a different type to the "key." Crucially, hetero-associative memory systems can, in large part, be implemented by auto-associative memory systems by forming "joint" object memorize by combining the key and value into one object. Hetero-associative queries can then be processed simply by looking up the joint object "corrupted" by discarding the value.
2.2 Generative Predictive Coding Network
generative predictive coding network (GPCN) [Salvatori et al., 2021] is an energy- based associative memory model that has set the state-of-the-art on number of image associative recall tasks. As illustrated in Figure 2, GPCN takes the form of fully connected neural network whose bottom layer activations characterize the sensory inputs and higher layer activations characterize the internal representations of the sensory inputs. GPCN's read operation (recall) initializes the bottom layer activations to the imput query vector and iteratively denoises the query until a stored value is retrieved by minimizing its energy function w.r.t. some or all of its neuron activations. GPCN's write operation (learning) adapts the network parameters to shape the energy function such that the stored datapoints become the local minima of the energy function.
Before describing GPCN's 's read and write mechanisms in more detail, we introduce the notation we use this paper. Let M be a GPCN with L + 1 layers of neurons where 0 and denotes the bottom and the top layer index respectively. Then, let dj be the width of the I-th layer, of E IRd be the row vector denoting the 1-th layer's pre-activation function neuron values, and f' RRi + Rdi be the I-th layer's 's activation function, for instance ReLU. In addition, let W e Rd x where

---

W
W
f x²
W
#
W
0
f E
W
10 18
070
x
1:T
Figure 2: Left: A sample 3-layer GPCN/BayesPCN architecture. Circular nodes denote neurons, arrows denote synaptic weights, x denotes the sensory neuron activations, x², x² denote the upper layer hidden neurons activations, W W denote the synaptic weights between network layers, and 2 denotes the bias term that influences the top layer neuron activities. Right: graphical model for the memory model to the left with T i.idd sensory observations z zf..r- GPCN's s graphical model black while BayesPCN's graphical model adds the red components. GPCN treats the synaptic weights constants while BayesPCN treats them normally distributed random variables.
0 S I < L be the synaptic weight matrix between I+1-th and l-th layers and W E Rddr be the "bias" row vector that influences the dynamics of the top layer neuron activities x². One can intuitively think of the model parameters W as the memory content and the activations O.L. as the read state of the memory.
GPCN's read and wrile are facilitated by predictive coding (PC) [Rao and Ballard, 1999], biologically plausible learning rule that can approximate backpropagation [Millidge et al., 2020]. PC's energy function is the sum of prediction errors across all network layers. For GPCN, the choice of prediction error is the squared Euclidean distance between the actual layer activations and the prediction of the same layer activations made by their immediate upper layer activations. This leads to the following energy function.
1-1 + 2
E(x 0:L Wirk
100 12
1
Σ
1k
GPCN's read performs gradient descent the energy the initial values of n to the memory query T and fixing the values of W0 GPCN's write obtains the read output o. for each of T' i.i.d. datapoints to store 2f takes gradient step on the energy w.r.t. W0:7 while fixing 2 and repeats until convergence.
some all of x0 and 1:LL after setting
We now take probabilistic perspective on GPCN. One can view GPCN as a hierarchical generative model where all neuron activations are modeled by normally distributed random variables and the link function between the subsequent layers is layer-speeific nonlinearity followed by linear transform. Then, GPCN's read the minimization of Equation is equivalent to maximizing the log density in Equation w.r.t. x0 and 1:L.
log oz (:WDLL log Nz= WL 1..1111 I)
Σ log N(x': pl+ (
(2)
In other words, GPCN's read performs iterated conditional modes [Besag, 1986] the hierarchical generative model in the neuron activation space (refer to Algorithms and 4). In addition, GPCN's write can be viewed as approximate maximization of the lower bound log p(=f.r (W0:L w.n.t WO:L [Millidge et al., 2021]. This follows from the fact that GPCN learns via PC.
While GPCN possesses impressive associative recall capabilities, the learning rule proposed in Salvatori et al. [2021] is not amenable to continual learning. Due to its lack of safeguards against catastrophic forgetting, continually learning GPCN leads to a forgetting with older observations requires costly offline model refitting on all observations whenever new datapoint observed. BayesPCN's foremost aim is to address this limitation while preserving GPCN's read capabilities as much possible.
information associated

---

Memory Model
BayesPCN is structurally identical to GPCN, but unlike GPCN the network parameters W treated as latent random variables. This allows BayesPCN's write operation be cast as (approximate) posterior inference over W given new observations in way similar to [Wu et al., 2018a]. In other words, storing T i..d. datapoints 26.r into memory equivalent estimating p(W0:L (o..r) Figure illustrates the graphical model for BayesPCN.
We now review the probabilistic structure of BayesPCN. We assume that, prior to any observations, the random variables z.-² and W/O5 (where t is the datapoint index) are distributed as follows
p(W)) [MN(WW;; oW
#0 =fl=
(N(WL:1,0W
[Nitt ( (*+ ,o2) :N(x;; We, ,2
il
p(z/latt w²)
=
where (p, 2) and (N V) denotes normal and matrix normal distribution [Gupta and Nagar, 2018] respectively. We remind the readers that the matrix normal distribution is normal distribution for matrix-variale data that compactly specifies the matrix elements' means, row-wise covariances, and column- covariances. The joint distribution over all random variables when there are T sensory observations is, due to the conditional independence of x:: , (1,..., T) given WO:L,
T
T I
I Hrw (5) 1=0
0:L Wo:L)= p(r1:,
= II pa 0:L (WoLL)
p(WooL)= =
pla W -I
We note that BayesPCN's structure leads desirable properties that become useful when approximating p(W-0=4|2f during the write operation. First, by exploiting the linear Gaussian marginalization formula and conditional independence we can analytically compute the marginal distribution over all activation vectors 2.:1_
p(alit) /It Madlatt W')p(W")dwor 1I (xt
(6)
1=0
p(z//a4t)
[N(al fet "at 1 (o [N (o )l)
(z
)))
if = L if
(7)
We stress that neither p(=?) nor p(=: w..F)) has an analytic form due to the presence of nonlinearities. In addition, if all activation vectors ..D:L are observed, the posterior becomes normally distributed due to the linear Gaussian posterior formula and the fact that W0: conditionally independent given z.: (the analytical posterior formulae for Equation 9 are in Appendix C).
the model parameters
L
p (x00
1 II p pf w')((W)> II (WW 1=0
(8)
1=
(MN(W Mr,U (N(W": (")"")
if < L i
(9)
We use R Uhh n* E' denote the sufficient statistics of the layer-wise network weight posterior distributions given 2f a/t1
4 Memory Operations
This section contains the memory operations that BayesPCN supports: read, wrile, and forgel. We use the shorthand h = IIL to denote all hidden layer neuron activations and W 100 to denote all network parameters.

---

4.1 Read
The read operation takes the corrupted query vector T and retrieves the memory's best guess the associated original datapoint z As in GPCN, this is done by hill climbing the memory's log density log p(x") h hllr):) w.r.t. the neuron activations x., h. To clarify, x² the current memory read state and is not directly related to the past observations >9-r- BayesPCN's auto-associative recall performs iterated conditional modes on log pog,( h/ w.r.t. x², h and BayesPCN's netero-associiatvee recall ( w.r.t. 2., h while keeping the known values of z Appendix A, the recall procedures taken from Salvatori et al. [2021] and rewritten from the probabilistic perspective, describe this in more detail.
performs gradient ascent on log p(x", , h(agrr
fixed. Algorithm and Algorithm
4.2
Write
The write operations aims to take the previous timestep posterior over the model parameters p( (W[=fa.) and the current timestep observation z! to return the current timestep posterior the model parameters p( W (=f.t). Unfortunately, the analytical form of p( (W (=f..) is unavailable due to the presence of nonlinearities in BayesPCN so one must resort to approximate posterior inference.
We propose sequential importance sampling posterior inference algorithm for estimating p( [ (=)::)) where the particles are ligh-dimensional Gaussian distributions. Let N E N be the number of sequential importance sampling particles and {I,...,NN N } be the particle index. The t-th timestep posterior estimate p D(W[21.
recursively represented as weighted Gaussian mixture of the >> ( (W|= h where (())) is the 7-th particle's importance weight and p(r) (W] WWff.t--- (12 is the n-th particle's activation conditioned posterior described in Equation 8. We note that each p(*) (W)(W (W|=f.e-1> 1 efficiently characterised by the sufficient
form p[W (w]tt)
statistics je(m), g(n), I0=L-1,(m) (0:2-1,(m), Algorithm describes the write operation in detail (see
Appendix B for its justification).
Algorithm Memory Write
1: Input: Previous network weight distribution ýl W [0) 1), current timestep observation 2
2: Output: Updated network weight distribution p W ii
3: for each model ('
( [W
b and weight " in (( W Wff do
4: Fit the parameters the current timestep variational distribution:
5:
argming D (n)
(s (h.) || (10) hell9.--1)
6: Sample h(²): h,dh)
9 (n) (h:)
7: Update the model via linear Gaussian update conditioned on x! and h
8: p(") (W I=f.e. h (+)) x p("))( of.--11 (W]
(zf, h (n)
W)
9: Compute the unnormalized particle weight: a(n))
(n) p (a) h.
m
wi-1
10: end for
11: Set P(
D 2n
( W/af.. h (
While (n,) in principle can be any distribution. BayesPCN's wrile sets it to g( (hr)
8 (argmax,, log p(=. hy [x)))) as is standard in predictive coding [Millidge et al., 2021]. In
addition, we define the BayesPCN parameter prior (of an "empty" memory) as P (W) =
EN (n) (W) where (m( R0:L-1,(m) are initialized using Kaiming initialization. We chose this prior over non-mixture prior to obtain multi-particle posterior approximation despite using a Dirac variational distribution.
5

---

We now direct our attention to obtaining (og) hl2);-11 used in the read operations from p( p(Wp))u) Using Equation 6, we obtain the approximate posterior predictive density
logp(af, hls log E wn)) / plul he W (n (Wl
(10)
log
w winpln) (a h,l04/-11)
(11)
BayesPCN's write can be seen as a special case of sequential imputation [Kong et al., 1994]. an online parameter inference algorithm that represents the parameter posterior conditioned on partial observations mixture of parameter posteriors conditioned on complete observations. In addition, whereas the parameter update proposed by Salvatori et al. [2021] is doing local gradient descent for each W BayesPCN's parameter update doing local conjugate normal-norma Bayesian update for each W Hence, like GPCN, BayesPCN's synaptic weight update only utilizes local information. Lastly, while exact Bayesian updates are invariant to the order of observations, because we are using predictive coding to fit the variational distribution BayesPCN's write is technically not invariant to the order of observations.
4.3 Forget
We also propose a ceffuusion-based forgetting mechanism for BayesPCN. As more datapoints are written into BayesPCN model, the model's belief over the true parameter location R RI-Z-1, 1r shifts and the uncertainty over its belief U1:L-1, Σ decreases. This has the effect of making the model's parameters less adaptive to new datapoints and typically magnifying R2-L-11, uu' norms as the prior's contribution weakens. One can interpret this as the memory being overloaded. The forget operation counteracts this phenomenon by nudging the model's belief over R μ toward the prior (empty memory state) and increasing its uncertainty U Σ proportional to the forget strength parameter B. Algorithm 2 describes this in detail (the analytical diffusion formulae are in Appendix D).
Algorithm 2 Memory Forget
1: Input: Previous network weight distribution pl W [0 1i forget strength 3 E [0, 1]
2: Output: Updated network weight distribution p W (1a))
3: for each model p (TP: ( in p(W 2 do
4: for each layer weight distribution pl")(W' (W"[2))) (m). (1),41 in p(n) (W|z (m) h do
, Tist
5: Obtain the one-step diffused layer weight distribution p(a) (w" o [a()) Iia 1+1,
6:
p(n)) (W [i Fii
(n).d (m),(+1 p(W (wp (n), (a),4]
Fia JaWl
7: end for
Set p(") (W'l 1 1:t1 h (n) = I (()) (w") (w"1 l " ,41 (a (n))+t)
8:
9: end for
10:
Set ((W (f..) = En p ( Wo h(?))
If applied an infinite number of times, forget reverts the posterior over the parameters O W ( to the prior ş( (W) due to the property of diffusion the particle weights are uniform. In practice, one would apply forget periodically when the memory starts to become overloaded from too much data and stops being performant. Because forget erases stored information for all of datapoints somewhat evenly, the datapoints from the distant past that forgel was applied to more would be, as the name suggests, more forgotten than the recent datapoints that forget was applied to less.
5 Related Work
We briefly review the most relevant work to BayesPCN, starting with the predictive coding networks. As covered in Section 2, GPCN [Salvatori et al., 2021] is the model that BayesPCN is built upon. Self-taught associative memory [Dovrolis, 2018, Smith et al., 2021] is a hierarchical predictive coding memory that performs unsupervised continual learning using online clustering modules as

---

building blocks, unlike BayesPCN's building blocks of artificial neurons. The sequential neural coding network [Ororbia al., 2019, 2018], while not explicitly used an associative memory, is also continually learned predictive coding network similar in structure to GPCN that combats catastrophic forgetting via task-dependent activation sparsity. BayesPCN on the other hand combats catastrophic forgetting by incorporating uncertainty over the synaptic weights.
Moving on general attractor-based memory models, Hopfield networks [Hopfield, 1982] and their modern variants [Krotov and Hopfield, 2016, Ramsauer et al., 2020, Krotov and Hopfield, 2020, Krotov, 2021, Millidge et al., 2022] are biologically motivated auto-associative memories that define the neuron activation dynamics via Lagrangian functions. Out of these models only Krotov [2021]'s model is hierarchical, but theirs is not continually learned. BayesPCN generalizes the modern Hopfield network and fits into the universal Hopfield network's 's mmmilarity-spparation projection paradigm for associative memories (refer to Appendix E). Sparse Distributed Memory (SDM) [Kanerva, 1988] is a non-hierarchical memory that exploits high-dimensionality to perform auto-associative and cetero-associati George, 2007, Cui et al., 2016] is a biologically constrained hierarchical memory that continually learns time-based patterns with sparse representations. HTM does not employ predictive coding and to our knowledge has not yet been tested on high- ligh-dimmnsionall data.
recall. Hierarchical Temporal Memory (HTM) [Hawkins and
Relative generative modeling and memory research, our work has connections to the Kanerva machine's 's memory write as inference approach [Wu et al., 201 8a, and the one-shot memory write from Bartunov et al. [2019], Pham al. [2022]. These papers' models employ meta-leariin unlike aforementioned work. We note that the Kanerva machine also applies conjugate normal-normal Bayesian update to adapt its top level memory matrix once the model meta-learned via end-to-end gradient descent. On the other hand, BayesPCN applies conjugate normal-normal Bayesian update to all of its weight matrices and this as of itself constitutes learning.
Experiments
This section assesses BayesPCN's recall capabilities on image recovery tasks and examines the effect of forget on BayesPCN's behaviour.
We used CIFAR10 [Krizhevsky al., 2009] and Tiny ImageNet [Le and Yang, 2015] datasets for all experiments, whose images are of size 3 x 32 x 32 and x 64 x 64 respectively. During the write phase of the experiments, BayesPCN models were given one image to store into their memories per timestep for up to 1024 images. Then, during the read phase of the experiments, we recorded the pixel-wise mean squared error (MSE) between the write phase images and the models' read outputs given the corrupted write phase images as inputs. For reference, the pixel-wise MSE of 0.01 distinguishable to human eyes on close inspection of the images.
BayesPCN models were compared against identity models (models that return their inputs as outputs), modern Hopfield networks (MHN) [Ramsauer et al., 2020], and vanilla GPCNs [Salvatori et al., 2021] all of which were offline learned. We note that GPCNs have set the state-of of-the-art on CIFARIO and TinylmageNet image recall tasks. BayesPCN models were also compared against online GPCNs, GPCNs that naively applied their offline memory write to sequentially arriving observations and memorized one image per timestep for up 1024 images. We used our own autodiff implementation of MHNs based on Appendix F. See Appendix F for additional experiment details.
6.1 Auto-Associative and Hetero-Associative Read
For the auto-associative recall experiments, the memory inputs at the read phase were the write phase images corrupted by pixel-wise white noise with standard deviation 0.2. For the hetero-associative recall experiments, we tested two types of training image corruption schemes. The first type, dropout, randomly blacked out 25% of the image pixels. The second type, mask, blacked out 25% of the rightmost image pixels. Image recovery under these settings is vecero-associalte one can view the non-blacked out pixels as the "key" and the blacked out pixels as the "value". We experimented with 4-layer BayesPCN models that have hidden layer sizes of 256, 512, and 1024, particle counts of 1 and 4, and activation functions ReLU and GELU. We searched over the same hy perparameetr for offline and online 4-yyer GPCNs aside from the particle count. Lastly,
recall because
7

---

White Noise CIFARTO MSE
White Noise Tiay ImageNet MSE
Sequence Length
128
256
512
1024
Sequence Length
128
256
512
1024
Identity
0.1596
0.1600
0.1600
0.1600
Identity
0.1600
0.1602
0.1601
0.1600
MIIN
0,00000
0.0030
0.0000
0.0000
MIIN
0.0000
0.0000
0.0000
0.0000
GPCN (Oftline)
0.0028
0.0046
0.0073
0.0121
GPCN (Offline)
0.0005
0,0010
0.0018
0.0067
GPCN (Online)
0.0103
0.0150
0.0191
0.0210
GPCN (Online)
0.0089
0.0112
0.0138
0.0181
BayesPCN
0.00117
0.0085
0.0146
0.0337
BayesPCN
0.0011
0.0033
0.0064
0.6606
BayesPCNN (forgel)
0.00664
0.0102
0.0145
0.0188
BayesPCNN (f (forget)
0.0026
0.0059
0.0108
0.0176
Dropout CIFAR 10 MSF
Dropout Tiny ImageNet MSE
Sequence Length
128
256
512
1024
Sequence Longth
128
256
512
1024
Identity
1,1140
L,178
1,1553
1.1481
Identity
L.629
1,0889
1,1154
L.0022
MHN
0.0000
0.0000
0.0000
0.0000
MHN
0.0000
0,0000
0,0000
0.0000
GPCN (Offline)
0.0000
0.0000
0.0000
0.0001
GPCN (Offline)
0.0000
0.0000
0.0000
0.0000
GPCN (Online)
0.0022
0.0032
0.0053
0.0073
GPCN (Online)
0.0036
0,0053
0.0069
0.0099
BayesPCN
0.0000
0.0000
0.0000
0.0001
BayesPCN
0.0000
0.0000
0.0000
0.0000
BayesPCN (forget)
0.0000
0.0001
0.0005
0.0019
BayesPCN (forget)
0,0000
0,0000
0.0002
0.0008
Mask CIFARIO MSE
Mask Tiny ImageNet MSE
Sequence Length
128
256
512
1024
Sequence Length
128
256
512
1024
Identity
1,1272
1.1373
1.1619
1.1653
Identity
1,0876
1.0884
1.0982
1.1122
MHN
0.0000
0.0000
0.0011
0.0000
MHN
0.0000
0.0000
0.0000
0.0010
GPCN (OAline)
0,00000
0.0000
0.0000
GPCN (Oilline)
0.0000
0.0000
0.0000
0.0001
GPCN (Online)
0,0127
0.0255
0.0522
0.0791
GPCN (Online)
0.0081
0,0316
0,0441
0.0698
BayesPCN
0,0000
0.0000
0.0001
0.0019
BayesPCNN
0,0000
0,0000
0,0000
0.0000
BayesPCN (forget)
0,0000
0.0008
0.0096
0.0465
BayesPCIN (forget)
0.0000
0.0003
0.0031
0.0235
Table I:
Average MSE between the training images and the associative memory read outputs for the white noise, dropout, and mask tasks CIFARIO and Tiny ImageNet. The reported values are from the best models of each category. Table 2 shows the same result with standard deviations.
evaluated 4-layer BayesPCN models, which we denote as BayesPCN (Sorgel), that applied forget with strength / 0.001 to their weights once every 64 wrile operations.
Table shows the average MSE between the ground truth images and the associative memory read outputs for models trained on data sequences of different lengths, where the reported scores come from the best models of each category. The results were averaged across three random seeds. Appendix I details how different hyperparameter values affect BayesPCN's read performance.
BayesPCN's recall errors were on par with offline GPCN's recall errors and were orders of magnitude lower than online GPCN's recall errors on all BayesPCN (forgel) performed worse than BayesPCN in these settings but still outperformed online GPCN on all hetero-assoocitiive recall tasks and sequence lengths. On aaso-associative recall tasks with sequences of length < 512, BayesPCN's recall errors were marginally higher than off GPCN's recall errors but lower than online GPCN's s recall errors. However, when the sequence length approached 1024, BayesPCN performed worse than online GPCN and on the TinyImageNee auto- associative recall task became even worse than the identity model. We hypothesize that BayesPCN's sudden performance drop stems from the fact that it is not "removing" old information unlike online GPCN and BayesPCN (forget) due to the conjugate Bayesian update's 's observation order invariance property. This causes BayesPCN to approach the overloaded memory regime, where recall of all data suffers, once enough information has been stored. Because BayesPCN (forget) deliberately removes old information and thus is farther away from the overloaded memory regime, it was able to outperform online GPCN all auto-associative recall tasks and sequence lengths.
recall tasks and sequence lengths.
GPCN and BayesPCN models overall found hetero- associative recall easier than auto-associative recall, likely because the former fixes the key bits during memory read and thus discourages the algorithm from wandering to less realistic z" values. Counterintuitively, GPCN and BayesPCN models were better at Tiny ImageNet recall over CIFAR10 recall. We attribute this behaviour to the fact that for some models, high- digh-dimmnsional data recall can ironically be easier than ciw-dimensional data recall because individual datapoints are further apart from one another in higher dimensions.
surprising discovery from this experiment was how good MHN's recall can be compared to GPCN and BayesPCN's 's recall when the memory query had moderate amount of noise, a result inconsistent with the findings from Salvatori et al. [2021]. Because MHN explicitly retains past observations in data matrix and its softmax inverse temperature was set very high (8 10, 000), MHN's recall
8

---

0.05
0.015
Batch Recail MSE
0.04
0.010
0.075
0.03
0.050
0.02
0.005
0.025
0.01
0.000
0.000
256
512
768
1024
256
512
768
1024
0
256
512
768
1024
Observation Timestep
Observation Timestep
Observation Timestep
Figure 3: From left to right, a sample BayesPCN model's 's recall MSE for white noise (ar 0.2), pixel dropout (25%), ) and pixel masking (25%) tasks with different forget strength 3 after sequentially observing 1024 CIFARIO 10 images. The results were computed from three seeds and the shaded regions' upper and lower borders indicate the maximum and minimum errors observed from the runs.
effectively selected the observation that had the highest dot dot-product similarity to the memory query in our experiments. In Appendix G, we assess how MHN's recall compares BayesPCN's recall when the memory query corruption is much higher. As expected, because the observation with the highest dot-product similarity the memory query is often not the right image recall, BayesPCN significantly outperformed MHN in hetero-associative recall tasks.
6.2 Forgetting
This section investigates the forget operation's 's effect on BayesPCN's recall behaviour: Specifically, we show that forget causes BayesPCN to better recall recent observations at the expense of older observations and that this effect is more pronounced the higher the forget strength is. We trained six structurally identical BayesPCN models on 1024 CIFARIO images sequentially and applied forget with strength 3 0, 0.001, 0.002, 0.005, 0.01, 0.02 to each model once after every 64 write operations.
Figure 3 compares the models' recall MSE on 16 data batches of size 64 that were observed during different points of the training once the models have observed all 1024 datapoints. We see that BayesPCN with no forget (B 0) performs similarly on all datapoints regardless of their observation timestep, which is likely the consequence of conjugate Bayesian update's 's observation order invariance. On the other hand, BayesPCNs with forget (3 > 0) consistently perform better on newer datapoints with this effect being strongly correlated with the value of B. As previously shown, some amount of forgetting during training can improve the overall performance of the memory across all datapoints over no forgetting for long training sequences. Lastly, the higher the forget strength was the lower the recall error on the most recent data batch was for all tasks and forget strength values.
Discussion
One can intuitively think of BayesPCN read as an approximate nearest neighbour operation with a learned similarity metric and write as the continual learning of that metric. MHN the other hand uses the dot dot-product distance as its similarity metric [Millidge et al., 2022]. This explains why MHN is not good highly masked image recall because from pixelwise dro-pro distance perspective, highly masked image queries are often closer to unrelated images than to the ground truth images.
BayesPCN is closely related to the predictive coding theories of the brain's memory system. For example, Barron et al. [2020] suggests that the hippocampus may correspond to the top layer of a predictive coding network and the neo-cortical areas may correspond to the rest of the predictive coding network. Similarly, Dovrolis [2018] posits that their predictive coding network could serve as model of the cortical column albeit without experimental results. It is however unclear how exactly BayesPCN's 's activation gradient descent and locally conjugate Bayesian weight update can be realized in the brain. Lastly, there similarities between BayesPCN's ' diffusion based forget and the synaptic homeostasis hypothesis [Cirelli and Tononi. 2022], where during sleep synaptic weight strengths are decreased average to facilitate future learning.
Aside from these biological connections, BayesPCN has many potential machine learning applica- tions. One possible line of work is using it in conjunction with neural controller that learns what

---

information to store into memory [Graves et al., 2014. Schlag et al., 2021]. Using BayesPCN in this way would confer the benefit of high query noise tolerance for neural controllers. Another line of work is using it to solve small data supervised learning problems, leveraging the fact that nearest neighbour methods work well when data availability is sparse [Ramsauer al., 2020].
To conclude, we introduced BayesPCN, an associative memory model that can robustly retrieve past observations given various corrupted inputs, perform one-shot parameter update of deep neural network to store new observations as attractors, and forget old observations to better store newer ones. We demonstrated that BayesPCN's s recall performance in most cases not significantly worse than the recall performance of its offliee-laared counterpart even though it is continually learned. There are several intriguing extensions to BayesPCN that may address some of its limitations. Currently, BayesPCN read can take up to thousands of activation gradient descent iterations to obtain the best recall result when a lot of data has been stored into memory. Supplementing BayesPCN read with amortized inference may greatly reduce this computational requirement [TTschant et al., 2022]. In addition, BayesPCN read can return a nonsensical output the memory query too far away from the stored datapoints or if the memory is overloaded. Investigating BayesPCN's latent landscape may allow us design models with more robust basin of attractions that can easily be reached via activation gradient descent. Last but not least, extending BayesPCN beyond memorizing i.i.d. data to memorizing sequences like the human episodic memory would be of great interest.
Acknowledgments and Disclosure of Funding
We would like to thank Boyan Beronov for the helpful discussions. In addition, we acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canada CIFAR AI Chairs Program, and the Intel Parallel Computing Centers program. Additional support was provided by UBC's Composites Rescarch Network (CRN), and Data Science Institute (DSI). This research was enabled in part by technical support and computational resources provided by WestGrid (www.westeg www.westgrid.ccl Compute Canada www.computecanada.co and Advanced Research Computing at the University of British Columbia (arc.ubc.ca).
References
Helen C. Barron, Ryszard Auksztulewicz, and Karl Friston. Prediction and memory: A predictive coding account. Progress in Neurobiology, 192:101821, 2020. ISSN 0301-0082. doi: https://doi.
http:///ojpmeurrooo0000000111110 101822 URL https: http://wwww//// www.sccencedirect..... wwteeeettcccceeeeeeeeeeeeeeeeeee com/science)
articalepp1//0331102200077
Sergey Bartunov, Jack W Rae, Simon Osindero, and Timothy P Lillicrap. Meta-learning deep
energy-based memory models. arXiv preprini 11X.v:191. 2011.
Julian Besag. On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society: Series B (Methodological), (8(3):259-277) 1986.
Trenton Bricken and Cengiz Pehlevan. Attention approximates sparse distributed memory. Advances in Neural Information Processing Systems, 34, 2021.
Chiara Cirelli and Giulio Tononi. The why and how of sleep-dependent synaptic down-selection. Seminars in Cell & Developmental Biology, 125:91-100, 2022. ISSN 1084-9521. doi: https: 1odoi.org/10.1000 101 wwwp:///.eemm...m./mm 2ad.dd...222....... 2021.02.007. URL https: //wtps://ww..r sciencedirect com/sc stt//ciiecc///e///////// ence/ Special Issue: Interplay between non-canonical and canon- ical Wnt signalling by Terry Van Raay/ Special Issue: The making of memories by Patricio Opazo
arti article/ hall//ppi//eee aetticle/ppi/S1044995555 /p11//11114922210000111111
and Victor Anggono.
Yuwei Cui, Subutai Ahmad, and Jeff Hawkins. Continuous online sequence learning with an unsupervised neural network model. Neural Computatioon, 0008933. 10.1162/neco_s a 00893. URL https: //doi. /ttt::::/doi..or/11..11111 wttp///.....22//co.oooaa.oo.
nov 2016. doi:
Constantine Dovrolis. A neuro-inspired architecture for unsupervised continual learning based on online clustering and hierarchical predictive coding, 2018. URI. https: http///xxvvvo/////e//e////// /bbbs/ 1810. 1810.09391.
10

---

Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, (4):128-135, 1999.
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines, 2014. URL https: //arxiv.ors orgg abs/1410. 5401.
A.K. Gupta and D.K. Nagar. Matrix Variate Distributions. Monographs and Surveys Pure and Applied Mathematics. CRC Press, 2018. ISBN 9781351433006. URL aattps://boon google. ca/ bebbooks? 1==
Jeff Hawkins and Sandra Blakeslee. On Intelligence. Times Books, New York, 2004.
Jeffrey Hawkins and Dileep George. Directed behavior using a hierarchical temporal memory based system, August 16 2007. US Patent App. 11/622,448.
John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8):2554-2556 1982.
Kazuki Irie, Róbert Csordás, and Jürgen Schmidhuber. The dual form of neural networks revisited: Connecting test time predictions to training patterns via spotlights of attention. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 9639-9659. PMLR, 17-23 Jul 2022. URL htwp:::tttpp/eeeeee httttp:://proceedinggs
mlr. mlr.press/ pwwps/vve2e htt.///////ere..../e// html.
Pentti Kanerva. Sparse distributed memory. MIT press, 1988.
Augustine Kong, Jun S Liu, and Wing Hung Wong. Sequential imputations and bayesian missing data problems. Journal of the American statistical association, 89(425):278-288 1994.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Dmitry Krotov. Hierarchical associative memory. arXiv preprint arXiv:2107.06446, 2021.
Dmitry Krotov and John Hopfield. Large associative memory problem in neurobiology and machine learning. arXiv preprint wwXiv:2008.0699 2020.
Dmitry Krotov and John J Hopfield. Dense associative memory for pattern recognition. 2016. doi:
00.48550/ARXIV..O 1606.01 164. URL https://arxiis http://aaaxvv...gaabaaa/aaa
01164.
Ya and Xuan Yang. Tiny imagenet visual recognition challenge. CS 23/N, 7(7):3, 2015.
Beren Millidge, Alexander Tschantz, and Christopher Buckley. Predictive coding approximates backprop along arbitrary computation graphs, 2020. URL https://arxiv.or org/ 2006. 04182.
Beren Millidge, Anil Seth, and Christopher Buckley. Predictive coding: a theoretical and experi- mental review, 2021. URL http://///xxxv.ooo/b/bbaeee /ttp........tttt///raia...oo..g
12979.
Beren Millidge, Tommaso Salvatori, Yuhang Song, Thomas lukasiewicz, and Rafal Bogacz. Univer- sal hopfield networks: A general framework for single-shot associative memory models. arXiv preprint arXiv:2202.04555 2022.
Alexander Ororbia and Daniel Kifer. The neural coding framework for learning generative models. Nature Communicationns, 13(1), April 2022. doi: 10.1033/<44477.0222000000 296-2---- URL https: //doi //dpi.org///..oo.////oo/// 1011110///4/4022222222 -22-2262227
Alexander Ororbia, Ankur Mali, Daniel Kifer, and C. Lee Giles. Lifelong neural predictive coding: Learning cumulatively online without forgetting, 2019. URL https://axxiv.org/ wrt../////..b....... 10696.
Alexander G. Ororbia, Ankur Mali, Daniel Kifer, and C. Lee Giles. Conducting credit assignment by aligning local representations, 2018. URL https: //axxiv./// arg/abs/1803.00
Kha Pham, Hung Le, Man Ngo, Truyen Tran, Bao Ho, and Svetha Venkatesh. Generative pseudo- inverse memory. In International Conference on Learning Representations, 2022. URL https: // //operevvie peet/forum?? :dr-nan4
EZB#. EBBB..

---

Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02221 2020.
Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-fiele effects. Nature neuroscience, 2(1):79 077799--7111 1999.
Tommaso Salvatori, Yuhang Song. Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bogacz, and Thomas Lukasiewiez. Associative memories via predictive coding. Advances in Neural Information Processing Systems, 34, 2021.
Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber. Linear transformers are secretly fast weight programmers. In International Conference on Machine Learning, pages 9355-9366. PMLR, 2021.
Jürgen Schmidhuber. Learning to Control Fast- Weight Memories: An Alternative to Dynamic Recurrent Networks. Neural Computation, 4(1):131-139 01 1992. ISSN 0899-76677. doi: 10.1162/neco. 1992.4.1.133
URL httphttts:///o..rrg wwtp://///....///..oooooooo www.//eeo.oe..eee..o 1992. 4.992.4.1.00 131.
Philipp Seidl, Philipp Renz, Natalia Dyubankova, Paulo Neves, Jonas Verhoeven, Marwin Segler, Jörg K. Wegner, Sepp Hochreiter, and Günter Klambauer. Modern hopfield networks for few- and zero-shot reaction template prediction, 2021. URL https://axxiy,.. org/ http://bbbbb/.//3//////
Sugandha Sharma, Sarthak Chandra, and lla R. Fiete. Content addressable memory without catas- trophic forgetting by heteroassociation with fixed scaffold, 2022.
James Smith, Cameron Taylor, Seth Baer, and Constantine Dovrolis. Unsupervised progressive learning and the stam architecture. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth Interna- tional Joint Conference on Artificial Intelligence, IJCAI-21, pages 2979-2987 International Joint Conferences on Artificial Intelligence Organization, 8 2021. doi: 10.24963/jccai.2221/110 URL https://doi. org/10. .00.024933/// 14963/ijcat 2021/410. Main Track.
K. Steinbuch. Die lermmatrix. Kybernetik, 1:36-45, 1961.
Alexander Tschantz, Beren Millidge, Anil K Seth, and Christopher Buckley. Hybrid predictive coding: Inferring, fast and slow, 2022. URL httph: https: //axiiv/////ivva//a http///aww..rrrbbs////aa// 02169.
Michael Widrich, Bernhard Schäfl, Milena Pavlović, Hubert Ramsauer, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter, Geir Kjetil Sandve, Victor Greiff, Sepp Hochreiter, et al. Modern hopfield networks and attention for immune repertoire classification. Advances Neural Information Processing Systems, 33: 33:18832-18845, 2020.
Michael Widrich, Markus Hofmarcher, Vihang Prakash Patil, Angela Bitto-Nemling, and Sepp Hochreiter. Modern hopfield networks for return decomposition for delayed rewards. In Deep RI. Workshop NeuriPS 2021, 2021. URL httpt:://penreeviieeee
net/forum?id=cOPQSI
Yan Wu, Greg Wayne, Alex Graves, and Timothy Lillicrap. The kanerva machine: A generative distributed memory. arXiv preprint arXiv:1804.01756 2018a.
Yan Wu, Gregory Wayne, Karol Gregor, and Timothy Lillicrap. Learning attractor dynamics for generative memory. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31, Cur- Associates, Inc., 2018b. URL https: http///preeennn...// newris...urps.... wwwuurps..cppppeeeee /ccpappe//018///ilee Ge22555611166e0000099366666-86 Paper pdf.
12


Reviews:
[{"review_id": "_Dv9YHwGJzx", "reviewer": "Reviewer_QHLK", "summary": "This paper proposes the BayesPCN, an associative memory model that is trained with predictive coding (in contrast to Hebbian learning employed by Hopfield networks). It seems to be an extension of a similar recent work, the generative predictive coding network (GPCN). Rather than using the GPCN's offline write mechanism, this work alters the weights W by Bayesian inference, enabling continual learning in the network. In tandem they propose a forgetting mechanism that incorporates the Bayesian prior. They then compare recall errors for CIFAR10 and Tiny ImageNet across the GPCN, BayesPCN, and the modern Hopfield network (MHN).", "questions": "* What are the details of the MHN models used in the experiments for Table 1? What is the size of the MHN? What hyperparameters were searched over?\n* Which hyperparameters were best for each model in Table 1? For example, was the forgetting \\beta set to 0.0 in these experiments? These hyperparameters could be included in a Supplementary document.\n* Lines 242-245 claim that BayesPCN recall quality deterioriated relatively evenly across time. Where is the evidence for this?\n* In Table 1 and in Figure 4, is the MSE the average across all datapoints thus far in the seqeuence (e.g. sequence length 512 averages 512 datapoints, whereas sequence length 128 averages 128 datapoints)? Or is it just showing the average across the most recent datapoints?\n* What is the standard deviation of MSE across seeds? i.e. can you add these to Table 1 & Figure 4? While the checklist answer does not include these because \"we are not necessarily claiming that BayesPCN obtains a better score than the baselines\", the discussion does assert that \"BayesPCN's recall performance is comparable to its offline-learned counterpart even though it is continually learned.\" Error bars would be useful to evaluate how comparable they are.\n\nSuggestions\n* The PDF appears to be rendered as an image, and it is not possible to search the document or highlight text. Please fix.\n* In Section 4.2 line 137, the text refers to the particle model described in Section 3. There is no explicit reference to a particle model in Section 3.\n* Line 45: is 'combing' supposed to be 'combining'?", "limitations": "Limitations of the model are discussed briefly. e.g. that the model can return nonsensical output for a memory query that is too far away, and that there is not a good biologically plausible implementation of the locally conjugate Bayesian weight update.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The introduction and explanation of the model details, such as the write algorithm, are clear enough. Recasting the GPCN as a Bayesian network is interesting, and to my knowledge the proposed forget mechanism is a novel idea. However, the experiments are the weak point of this paper. Many details on experiment parameters are missing, which I list in the questions below. This makes it challenging to interpret some of the findings. Finally, the claims of novelty in the introduction of the paper are not well-supported by the experiments.\n\nIn the first claim, the authors emphasize that the model is continually learnable unlike the GPCN. However, they do not show any results for any common continual learning evaluations, such as class-incremental continual learning. Second, they say that \"BayesPCN is the first parametric associative memory model to continually learn hundreds of very high-dimensional observations (>10,000 dimensions) while maintaining good recall performance.\" While my area of expertise is not in associative memory models, it is my understanding that the capacity of modern Hopfield networks is quite large. The authors do not show any experimental evaluations which result in high error for MHNs and relatively low error for the BayesPCN. They only show an example in Figure 3 of the MHN failing with high dropout and masking. By contrast, this is something more thoroughly evaluated by the GPCN paper (see their Figure 9 and Table 1), though even their work does not provide a good summary comparison between the two.\n\nThe authors then state that BayesPCN is \"the first hierarchical memory which after repeated forgetting recovers its original memory state.\" While this third claim may be technically true, it does not seem a particularly significant or notable addition to the literature. If I understand correctly, the authors are saying that a dedicated forget mechanism (other than overwriting old memories) does not exist in models like the GPCN or hierarchical temporal memory (HTM) networks. And the authors show in Figure 4 that BayesPCN recall error is actually smaller later in a sequence, suggesting that it recovers the original images quite well. However, why is it notable or useful to have a *hierarchical* memory that can do this? I assume if it were the first associative memory model to do this, the authors would have claimed that instead.\n\nThis is not my area of study, so perhaps I am missing some context that would make this paper's significance more apparent. As is, I am rating it a borderline reject.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_6JVQG-QAop", "reviewer": "Reviewer_HskX", "summary": "This paper proposes an interesting extension of PC-based memory models, that tackles an existing problem of the original generative network: the inability to remember stored memories when presented with new ones, i.e., to write new memories without resetting the model. Generative predictive coding networks were, in fact, trained only once using full batch training on the whole dataset. This work tackles this problem by introducing a new method to write memories in the database. Additionally, it allows to use a similar method to implement a 'forget' operation, that cancels memories from the database. ", "questions": "In Table 1, you show the average MSE of every data point, under specific kinds of corruption. However, what is the MSE of the first stored datapoints when all the patterns are stored? What is the MSE of the last ones? \nParticularly, I have the feeling that the reported MSE only show the weakness of the model when confronted with the other baselines, and no experiment shows the advantages. You could perform the same experiments using the standard generative network trained via running inference, and make it memorize one datapoint at the time, exactly how you did for BayesPCNs. I would guess that your model would perform much better on the retrieval of past memories. Why is something like that not included?\n\n\n\nLine 237: \"A surprising discovery from this experiment was how good MHN’s recall is compared to GPCN and BayesPCN when the memory read input has a moderate amount of noise.\"\nThis follows as the induced Gaussian noise has mean zero, and hence the expected normalized dot product between a memory + Gaussian noise and the original memory is 1. As MHNs use dot products to perform similarity scores, they actually perform quite well under this very specific task. This, however, does not happen if the vector of the pixels is not normalised before performing the MHNs operation: in this case, the model will often return the 'whitest' image of the dataset, as the colour white is the one with the highest norm in the rgb collar system. \n\n\nFig.1 is slightly confusing: the description reads nicely and well explains the point, but the image is just a bunch of pictures. I would improve it by adding some pointers directly in the Figure, as well as removing the forth line. ", "limitations": "Yes they have.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1) The model works well, and is indeed able to store memories sequentially without the need of re-training the model from scratch, or without a loss in performance on memories stored multiple 'write' operations before;\n\n2) This method could be potentially interesting in context that are more general than a generative model to perform associative memories, such as continual learning;\n\n3) The literature and related work is well covered.\n\nWeaknesses:\n\n1) I personally found the description of the 'write' operation to be unclear, and it took me a couple of readings to fully understand it: The role of the n-th particle weight $\\omega^{n}$ is not well explained, and the same applies to the n-th model $p^{n}$. Hence, a better and clearer explanation seems needed. It also says that these concepts are introduced in Sec.3, where exactly? \n\n2) The experiments seem to not really show the potential and limitations of the model on the task that concerns this paper the most: continual 'writing' of memories. I will list below a couple of questions I have about the performance of the model that are both a personal curiosity, and a tip on some experiments that would make the paper more (in my opinion) interesting. \n\nThese are my main concern, and a rebuttal that addresses the above two points would make me increase the final score. \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["Legal Compliance (e.g., GDPR, copyright, terms of use)"]}, "extra_text": {}}, {"review_id": "T9RWMxebl34", "reviewer": "Reviewer_yM5G", "summary": "This paper extends upon predictive coding neural networks by making the weights Bayesian and using this relationship to make connections to associative memory models within a deep learning framework more broadly. The model is also able to forget over time. ", "questions": "See above. ", "limitations": "No I have requested that they add a clear limitations section. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Fig. 1 add labels to the rows and columns rather than putting everything in the caption. \n\nI think you should cite the paper http://proceedings.mlr.press/v80/marino18a/marino18a.pdf because of its relations to predictive coding. Discussing relations between your model and VAEs could be interesting in general. \n\nThe Kanerva Machine should be added as a benchmark? Also a standard VAE as per the results of: https://www.pnas.org/doi/abs/10.1073/pnas.2005013117 . I would also cite the follow up to the Kanerva Machine: https://proceedings.neurips.cc/paper/2018/file/6e4243f5511fd6ef0f03e9f386d54403-Paper.pdf \n\nLine 119 difference between hetero and auto associative is confusing. Line 188 talking about SDM as non hierarchical -- this descriptor should apply to the other models mentioned too like MCHNs. Another sentence on the differences between your model and the Kanerva Machine could be helpful. \n\nThe images in App. H generated from the BayesPCN are not impressive. Some commentary on this as a limitation in the main text should be present. A limitations section more broadly would be good. \n\nCan you control for the continual learning abilities? E.g. can you learn offline with BayesPCN and see if it outperforms the GPCN?\n\nI'd like to see more experimental results and examples where BayesPCN performs better than existing approaches. You also note that MHN does worse for more heteroassociative tasks but dont show this in Table 1. Also what heteroassociative approach are you using here? The correct one is probably to concatenate the key and value into one large vector as is done in https://arxiv.org/abs/1606.01164 \n\nRegarding forgetting it would be useful to know the memory capacity and performance and potentially benchmarking in general against a method like: https://proceedings.mlr.press/v162/sharma22b/sharma22b.pdf \n\nWhere does eq. 33 come from? ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "H7SYnJX10tB", "reviewer": "Reviewer_HvfQ", "summary": "In this work, the authors have proposed Bayes predictive coding network (PCN), a model inspired by predicting coding literature from Rao and Ballard. They offer the inclusion of hierarchical associative memory, capable of performing continual one-short memory writes without the need for meta-learning heuristics. This mechanism also helps the model gradually forget partial least important information to free memory. Experiments are performed on cifar and tiny-imagenet to show the efficiency of the proposed method. ", "questions": "* Can authors comment on how the current approach compares/differs against, the neural coding network [1-3] which is proposed to be a stable predicting coding network that works on a wide variety of networks. \n* Ablation study with varied value of Sigma_w and Sigma_x is required, current approach uses Sigma_w = 1 and Sigma_x = 0.01\n* Can authors report results with deeper BayesSPCN\n* How are baseline models trained and did the authors conduct hyperparameter optimizations of those? Can authors report all these ranges?\n* Hyper-parameter optimization- No detail about hyper-parameter optimization, authors should provide reasoning how these parameters were chosen. \n* Statistical significance: - Experiments should be conducted for K trials and average performance and the standard error should be reported.\n* Can authors comment on the robustness and stability of the given model?\n* Can authors provide computational overhead using temporal weights both during train and inference?\n\n* [1] https://ojs.aaai.org/index.php/AAAI/article/view/4389\n* [2] https://www.nature.com/articles/s41467-022-29632-7\n* [3] https://arxiv.org/pdf/1905.10696.pdf", "limitations": "Authors have failed to include Broader Impact and Societal or negative impact sections. Please refer to the NeurIPS deadline, before submitting your manuscripts.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n* Well-written paper. However, writing can be improved. Since this paper can be easily understood researchers working on predicting coding and bio-inspired approaches. But for people outside the field, it's difficult to understand.\n* Incremental approach \n* Model section is well defined.\n\n# Weakness\n* Experimental section needs more details\n* Related work is missing some citations\n* Ablation study is missing", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["Discrimination / Bias / Fairness Concerns", "Inadequate Data and Algorithm Evaluation", "Inappropriate Potential Applications & Impact  (e.g., human rights concerns)"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept