Biological learning in key-value memory networks
Authors: Danil Tyulmankov
Ching Fang
Annapurna Vadaparty
Guangyu Robert Yang
Venue: NeurIPS 2021 Poster (2021)
Keywords: neuroscience
computational neuroscience
memory
recall
biological learning
learning rule
synaptic plasticity
key-value network

Abstract: In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in machine learning commonly use a key-value mechanism to store and read out memories in a single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet their biological relevance is unclear. We propose an implementation of basic key-value memory that stores inputs using a combination of biologically plausible three-factor plasticity rules. The same rules are recovered when network parameters are meta-learned. Our network performs on par with classical Hopfield networks on autoassociative memory tasks and can be naturally extended to continual recall, heteroassociative memory, and sequence learning. Our results suggest a compelling alternative to the classical Hopfield network as a model of biological long-term memory.

Biological learning in key-value memory networks
Danil Tyulmankov'
Ching Fang" Columbia University ching. fawg@coolmmiia..ee.m udu
Columbia University dt:256@cclumbia. edu
Annapurna Vadaparty
Guangyu Robert Yang
Columbia University
Columbia University
Stanford University
Massachusetts Institute of Technology yanggremit. edu
Abstract
In neuroscience, classical Hopfield networks the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, evemory-augmeee learning commonly use key-value mechanism to store and read out memories in single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet their biological relevance is unclear. We propose an implementation of basic key-value memory that stores inputs using a combination of biologically plausible three-factor plasticity rules. The same rules are recovered when network parameters are meta-learned. Our network performs par with classical Hopfield networks on autoassociative memory tasks and can naturally extended continual recall, heteroassociative memory, and sequence learning. Our results suggest compelling alternative to the classical Hopfield network as a model of biological long-term memory.
neural networks in machine
Introduction
Long-term memory an essential aspect of everyday lives. It is the ability rapidly memorize an experience or item, and to retain that memory in retrievable form over prolonged duration (days to years in humans). Neural networks capable of long-term memory have been studied in both neuroscience and machine learning, yet wide gap remains between the mechanisms and interpretations of the traditions.
In neuroscience, long-term associative memory is typically modeled by variants of Hopfield networks [Hopheld 1982 Amit 1992 Willshaw et al. 1969.. Rooted in statistical physics, they of the earliest and best known class of neural network models. A classical Hopfield network stores an activation pattern & by strengthening the recurrent connections W between co-active neurons using bhologically-plausibe Hebbian plasticity rule,
W + W + ET
and allows retrieval of memory from corrupted version through recurrent attractor dynamics,
#t+1 sign ( x::)
thereby providing a content-addressabte and pattern-completing autoassociative memory.
In more recent parallel thread machine learning, various memory networks have been devised to augment traditional neural networks Grrves al. 2014 Sukhbaatar et al. 2015 Munkhdalat et al.
Equal contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

neural networks utilize a more stable external memory system analogous to computer memory. in contrast to more volatile storage mechanisms such as recurrent neural networks [Rodriguez et al. 2019]. Many memory networks from this tradition can be viewed as consisting of memory slots where each slot can be addressed with key and returns a memory value, although this storage scheme commonly lacks mechanistic interpretation in terms of biological processes.
2019 Le et al. 2019 Bartunov et al., 2019].
Key-value networks date back to at least the 1980s with Sparse Distributed Memory (SDM) as a
model of human long-term memory [Kanerva 1988 1992]. Inspired by random- acccess m
in computers, it is at the core of many memory networks recently developed in machine learning
[Graves et al., 2014 2016 [Sukhbaatar al., 2015 Banino al.] [2020] Le et al. 2019). basic
key-value network contains key matrix K and value matrix V. Given a query vector æ, memory
read operation will retrieve an output y as
h f(K))
(3)
y Vh.
where f is an activation function that sparsifies the hidden response h.
Variations exist in the reading mechanisms of key-value memory networks. For example, f may be the softmax function [Sukhbaatar et al. 2015). making memory retrieval equivalent to the "key-value attention" " [Bahdanau et al. 2013| used in recent natural language processing models [Vaswani et al. 2017]. It has also been set as the step function [Kanerva 1988 and hard-max function [Weston et al. 2.15]. There is an even greater variation across writing mechanisms of memory networks. Some works rely on highly flexible mechanisms where external controller learns which slots to write and overwrite [Graves et al. 2016 2014), although appropriate memory write strategies can be difficult learn. Other works have used simpler mechanisms where new memories
be appended
sequentially to the existing set of memories (Sukhbaatar et al. [015 Hung et al. 2019 Ramsauer
al.
2020
or written through gradient descent [Bartunov et al. 2019 Munkhdala ai et al. 2019 Le
et al.
2019 Krotov and Hopfield (016). Kanerva| [1992] updates the value matrix through Hebbian
plasticity, but fixes the key matrix. Munkhdalai al. [2019] turns memory write into key-target value regression problem, and updates an arbitrary feedforward memory network using metalearned
local regression targets.
clear advantage of key-value memory over classical Hopfield networks is the decoupling of memory capacity from input dimension (Kanerva 1992 Krotov and Hopfield, 2020). In classical Hopfield networks, the input dimension determines the number of recurrent connections, and thus upper bounds the capacity. In key-value memory networks, by increasing the size of the hidden layer, the capacity can be much larger when measured against the input dimension (although the capacity per connection is similar).
There exists a gap between these two lines of research on neural networks for long-term memory- classical Hopfield networks in the tradition of computational neuroscience and key-value memory in machine learning. In our work, we study whether kcy-value memory networks used in machine learning can provide an alternative to classical Hopfield networks as biological models for long-term memory.
Modern Hopfield Networks (MHN) Krotov and Hopfilld 2016 2020 Ramsauer et al. 2020) Krotov
202T] begin to address this issue, suggesting neural network architecture for readout of key-value pairs from a synaptic weight matrix, but lacking a biological learning mechanism. MHNs implement an autoassociative memory (key is equal to the value, so V KT) [Ramsauer et al., 2020 Krotov and Hoptield 2020). but they can be used for heteroassociative recall by concatenating the key/value vectors and storing the concatenated versions instead. To query the network, keys can be clamped, and only the hidden and value neurons updated [Krotov and Hopfield 2016). With particular choice of activation function and one-step dynamics, this architecture is mathematically equivalent to a fully connected feedforward network [Krotov and Hopfield 2016), and thus analogous to the memory architecture proposed by Kanerva
1992].
It remains unclear whether biological mechanism can implement the memory write. We will show that such biologically-plausbbe plasticity rules exist. We consider a special case of the MHN architecture and introduce biologically plausible learning rule, using local three-factor synaptic plasticity [Gerstner al. 2018), as well as respecting the topological constraints of biological neurons with spatially separated dendrites and axons. We first suggest simplified version of the learning

---

rule which uses both Hebbian and non- non-Hebbian plasticity rules to store memories, and evaluate its performance in comparison to classical Hopfield networks. Adding increasing biological realism to our model, we find that meta-learning of the plasticity recovers rules similar to the simplified model. We finally show how the feedforward, slot-based structure of our network allows it be naturally applied to more biologically-ootvaaee key-value memory compatible with applications of the classical Hopfield network, particularly as a biologically plausible mechanistic model of long-term memory in neuroscience.
memory tasks. Thus, the addition of our learning rules makes
2 Simplified learning mechanism
We consider a neuronal implementation of slot-based key-value memory, and endow it with biologically plausible plasticity rule for memorizing inputs. We first describe a simplified learning rule for storing key-value pairs, which sets an upper bound on the network memory capacity and serves as benchmark against existing memory networks. In later sections, we modify this rule to further increase biological realism.
(a) Read
(b) Write key
(c) Write value
Ke
Vi
K₁
Ket1
V
Kr41
Ve+1
Sppdln weigght local third factor
1/+
I1
8/
71
h+
Figure I: Network architecture and read/write mechanism. (a) Memory reading. The network is given query z (input layer activation with corrupted version of stored key œ)), selects the most similar stored key through upproximately-one
hidden layer activity h. and returns the corresponding value ÿ ~ y. (b) Writing keys. The input æe is written into the 1th slot of the input-to-hidden weight matrix K2 by a "pre-only" plasticity rule, selecting the corresponding hidden neuron via local third factor [ty]] 1. (c) Writing values. The same 7th hidden neuron is selected through an intermediate hidden unit activation hi and the target ye is written to the hidden-tor to-eutttt weight matrix V. by a Hebbian update.
The network operates sequentially and consists of three fully-connected layers of neurons (Fiuree T): a dddimensioonl input with activity at time given by the vector æt, an N-dimensional hidden layer ht and an mi-imeensinnal output layer yt- The 7th key memory slot corresponds to the synaptic weights from the input layer to single hidden neuron (2th row of the key matrix K,, storing key m.)). The corresponding value stored in the weights from that hidden neuron to the output (ith column of the value matrix V,, storing value y:).
2.1 Reading
The network stores a set of key-value pairs {(x,, y: such that if query ă. e.g. corrupted version of a stored key æ. is presented to the network, it returns the corresponding value y. Given a query æ (Figure 1b) the hidden layer computes its similarity h; to each stored key æ; (ith row of K,) normalized dot product:
h softmax K₂
(4)
where the softmax function normalizes the hidden unit activations, and can be approximated biolog- ically with inhibitory recurrent connections. The output layer then uses these similarity scores to compute the estimated value as the weighted sum of the stored values through a simple linear readout:
N
ÿ = Vih =
hays
(5)
3

---

Assuming uncorrelated keys, the dot product of the query ă will be maximal with its corresponding stored key æ, and near-zero for all other stored keys, so h will be approximately one-hot. Thus, Equation 5) reduces to y ~ y as desired. If target values are binary, y: E {+1, -1)m (neurons are either active or silent), we use (yg when evaluating performance.
Mathematically, this architecture is equivalent to an MHN constrained to the heteroassociative setting with a fixed choice of activation functions [Krotov and Hopfield 20201, and recurrent dynamics updated for exactly one step [Krotov and Hoptield 2016 Alternatively, it can be thought of as an differentiable version of an SDM [Kanerval 1988 with a softmax rather than step function activation function in the hidden layer [Kanerva 1992]. Unlike these networks, however, we introduce a novel biologically plausible writing mechanism to one-shot memorize key-value pairs by updating both the key and value matrices.
2.2 Writing keys
Key-value pairs are learned sequentially. Given input æ, time t. write it into slot of the key matrix using a non-Hebbian plasticity rule, where the presynaptic neuronal activity alone dictates the synaptic update, rather than pre- and postsynaptic co-activation as in traditional Hebbian rule. A local third factor [Tr] E {0, 1 Figure Tp, red circle) gates the plasticity of all input connections to hidden unit i, enabling selection of a single neuron for writing. Biologically, this may correspond to a dentritic spike [Stuart and Spruston 2015 Gambino et al., 2014]. which occur largely independently of the somatic activity h2 performing feedforward computation This plasticity rule resembles behavioral time scale plasticity (BTSP) [Bittner et al., 2017]. recently discovered in hippocampus,
brain structure critical for formation of long-term memory.
In this simplified version we approximate local third factors as occurring in the deast-recently-usee
neuron by cycling through the hidden units sequentially:
(1 if t i mod N
[rc]
(6)
0 otherwise
This can be biologically justified in several ways. Each neuron may have internal timing mechanism that deploys a local third factor every N timesteps; the neurons may be wired such that a dendritic spike in neuron i primes neuron i for dendritic spike the next time step: or the local third factors may be controlled by an external circuit that coordinates their firing. Alternatively, we consider simpler mechanism, not requiring any coordination among the hidden layer neurons: each neuron independently has some probability p of generating a dendritic spike:
[rr] Bernoullipp)
To gate whether a stimulus should be stored at all, we include a scalar global third factor ge E (0. 1] Biologically, this may correspond to neuromodulator such acetylcholine [Rasmusson 2000 that affects the entire population of neurons, controlled by novelty, attention, or other global signals. Although it can also be computed by an external circuit, in experiments this value is provided as part of the input. Thus, the learning rate of the synapse between input unit j and hidden unit i is the product of the local and global third factors:
In = celccl
(8)
Finally, to allow reuse of memory slots, we introduce a forgetting mechanism, corresponding to rapid synaptic decay mediated by the local third factor. Whenever a synapse gets updated we have by 1, so we can zero out its value by multiplying it by 1 - [m]]s]- If there is no update (either third factor is zero), the weight is not affected. The synaptic update is therefore:
K = (1 (1-18 nt O K₁ + nh @ [1x]]
where c indicates the Hadamard (element product, and 1 (1, 11,1,..,1) 1).
"The same synaptic update could be accomplished without third factors, using Hebbian rule, if the hidden layer were ooe-hot. However, since the hidden layer activity is determined by its weight matrix and input, there is no simple neuronal mechanism to independently select a hidden neuron. More complicated versions may involve an external circuit which controls the activity of the hidden layer during writing, or strong feedforward weights which do not undergo plasticity (Tyulmankov al. 2021

---

We compute the accuracy as function of the number of stored stimuli (Fiuuee 2). By design, the plasticity rule with sequential local third factors performs identically simple non-biological key-value memory (TVT, (Hung et al. 2019]. Appendix B}. It has perfect accuracy for T S N since every pattern stored in a unique slot. For > N, previously stored patterns are overwritten one-by-one and accuracy smoothly degrades. With random local third factors, accuracy degrades sooner due to random overwriting. Importantly, this holds for arbitrary network sizes, unlike the classical Hopfield network, which experiences a "blackout catastrophe" where all stored patterns are erased if the network size is large and the number of inputs exceeds its capacity [McCloskey and (we do not see this here due to relatively small network 2.3 Writing values
Having stored the key 24, the hidden layer activity this intermediate stage is given by:
h softmax(K(+11))
(10)
Note we are using the updated key matrix with æt stored in the 7th slot. In the idealized case of random uncorrelated binary random memories æ; E [+1,-1]+ the 2th entry of 1t will be equal to 2,,- normalization via the softmax, h will be approximately one-hot, with [ha]e == 1.
d. All other entries will be near 0, since they are uncorrelated with Et. Thus, after
To store the value, the output layer activity is clamped to the target yr Figure Te), Biologically, this can be achieved either by strong one-to-one residual connections from the input the output layer, or by having common circuit which drives activity in both the input and output layers. Since the only hidden unit active is the one indexing the ith column of the value matrix, we can update the synapse between hidden unit and output unit & by Hebbian rule with learning rate [n, AA = q (7%) ) and rapid decay rate analogous to the key matrix
V = (1 n V n. O yye (k)))
(11)
Results
3.1 Benchmark:
autoassociative recall
As simple evaluation of our algorithm's performance and comparison to other memory networks, we consider the classical autoassociative memory recall task where the key is equal to the value, (d m. Figure T). The network stores a set of T stimuli {æ,] and the query x is a corrupted version of stored key (60% of the entries are randomly set zero). The network returns the corresponding uncorrupted version (Appendix A Figure Al.
Cohen 1989 Robins and McCallum 1998
size).
Next, by measuring the number of patterns that the network can store before its recall accuracy drops below threshold an imperfect measure because some networks may have accuracy that stays above threshold longer but drops sharply after that. Nevertheless, this metric allows us to investigate empirically how the network's performance scales with its size (Figure 2p). The empirical scaling of the classical Hopfield network C' ~ 0. 141,, consistent with theoretical calculations [mit et al. 1985 ), and the sequential algorithm's 's capacity scales approximately as C' ~ 1.0.N, as expected analytically. Importantly, the random algorithm's capacity also scales linearly, C ~ 0.16N, with slope similar to the Hopfield network. Note, however, that with the same number of hidden neurons our network has twice as many connections as the Hopfield network. We also note that although the theoretical capacity of the Hopfield network can scale C ~ 2N [Cover 1965 Gardner 1988] to our knowledge there is not a learning algorithm that achieves this bound. Other work on autoassociative memory shows exponential scaling, but lacks a biologically plausible readout [Demircigil et al. 2017
0.98), we can estimate this network's memory capacity C. This is
3.2 Meta-learning of plasticity rules
We now introduce parameters that can be meta-learned optimize performance on a particular dataset without sacrificing biological plausibility. First, we enable varying the scale of the synaptic
'If local third factors are dendritic spikes in the dendrites of hidden layer neurons, it may not be realistic for the axons of the same neurons to be affected by these dendritic spikes. We consider this case for simplicity, as an upper-bound bound performance. We will later lift this assumption without significantly hurting performance.
5

---

(a) 1.0
(b)
Sequential
Sequential C-1 .00ON
Random
Capacity (38% acc)
Random C-0.162N
0.9
Hopfield
Hopfllld C-0.147N
Accuracy
TVT
0.8
50
0.7
0
25
50
125
20
140
160
Number stimuli
Network size
Figure 2: Our network (d = N 40) with sequential and random (p 0.1) local third factors, Hopfield network, and TVT [Hung et al. 2019) performance on the autoassociative memory task. (a) Accuracy function of stored stimuli. (b) Capacity (maximum number of stored stimuli at Z 98% accuracy) as a function of network size.
plasticity rates each one is multiplied by learnable parameter 17" 77": In/ = 9: [elkk(">), and [m]les = 4: [7
(12)
Next, further improvement biological plausibility, remove the assumption that the hidden- to-output synapses (hidden layer axons) have access to the local third factors (hidden layer dendritic spikes). Note that n, above no longer contains Y term. Instead, forgetting mechanism, the hidden-to-output synapses decay by a fixed factor À each time a stimulus stored:
A = (1 9c) + 9
(13)
where X, is the decay at time and qe {0. 1}. Although in general ,"" matrix which sets learning/decay rate for each synapse or neuron, we consider the simpler case where each is scalar, shared across all synapses.
.7 and À can each be
Most importantly, we parameterize the update rule itself. Each of the pre- and postsynaptic firing rates is linearly transformed before the synaptic weight is updated according to the prototypical "pre-times-post rule. The jth input neuron's transformed firing rate is given by
(**))), = 7f xj
(14)
and others f".g*. l are analogous. Although these functions can take arbitrary forms in general, this simple parameterization enables interpolating between traditional Hebbian, anti-Hebbian, and non-Hebbian (pre- post-only) rules. The update rules can be summarized as follows:
K (1 n K. n @ 10 (a.))) V41 A o + gen O (g"()))
(15)
(16)
We begin by training this network on the benchmark task from subsection 3. parameters using stochastic gradient descent (Adam, (Kingma and Ba (014 Figure 34 shows the performance of the meta-learned algorithms in comparison to the corresponding simplified versions (section 2) for either sequential and random local third factors. Importantly, removing the unrealistic rapid synaptic decay in the hidden-to-output synapses and replacing with a passive decay does not significantly impact performance, long the decay rate appropriately set. Indeed, this even slightly improves performance for longer sequences.
optimizing these
haal--hirdd-accto---eeddd
In the case of random local third factor, we also compare values of (not trained). Empirically, we find that x 4/N produces desirable performance: ensures that the probability no local third factor occurring (and therefore storage) is small (< 2%) while minimizing overwriting. Computing the capacity (Figure 3p), we see that there an advantage for the probability to scale with the network size (p- 4/N) rather being fixed value (p = 0.1).
We next investigate the training trajectories. Figuree 3p shows the loss and accuracy curves over the course of training for networks with sequential and random local third factors. Training data

---

(a)
(b)
Sequential
150
Sequenbal C-1 0100
Trained
Capacity (98% acc)
Random 4/N C-0.183N
Random 0.025
Random 0.1 C-0.095N
Acouracy
Trained
100
Random
Trained
50
0.7
0
120
Number stimuli
Network size
(c)
40
Sequential
(d)
Inpul-to-hidden, pre-synaplic
Hidden-lo-oulpul, pre-synaplic
35
Rendom
3000
wmmm
30
25
20
15
MMW
Iterations
Inpul-to-hidden, post-synaptic
Hidden-to-oulput, post-synaptic
0.9
0.8
Accurapy
0.4
0.7
60
0.2
0.6
0.0
0.5
0.0
0.5
1000
2000
3000
lterations
Figure 3: (a) Performance comparison of simple (dashed curves) and meta- (solid) network (d N 40, p varied, shown in legend.) (b) Capacity of trained network with sequential or random local third factor. In the random case, p is either fixed or scales with N. (c) Training the sequential and random (p 0.1) network from (a.b). Although accuracy is not 1.0 during training, the network successfully generalizes to unseen sequence lengths. (d) Learning trajectory of the plasticity parameters in the sequential network converges to qualitatively similar solutions as the original simplified network (see also Figure C4). Note that the input-to-hidden plasticity is independent of the post-synaptic firing rate (bottom left plot, 79* R 0)
consists of sequence lengths between T = N/2 and T = 2N (above capacity for both versions of the network), so accuracy does not reach 1.0. Nevertheless, the network successfully generalizes to lengths outside of this range, indicating robust mechanism for memory storage.
Most importantly,
examine the plasticity rule discovered by optimization, Figure 3 shows the slope 7 and offset b parameters for the firing rate transfer functions g² ,", I' in the sequential algorithm over the course of meta-leaming. The plasticity rule for the input-te -ot-ttto-hdddnn connections Figure 3/1, left) becomes pre-dependent (ā ~ 0.5, T 0) but not post-dependent (Ø9
30 0.5), analogous to the idealized plasticity rule described in section 2 Similarly, the plasticity rule for the hidden-to-outpur connections becomes Hebbian (both pre-dependent and post-dependent, 30" T5" z 0, but " = a 1). The random version shows qualitatively similar trained behavior
Figuur C4).
Since the performance and parameterization of the meta-learned algorithm is almost identical to the simple case when using sequential local third factors, we only consider the random version for meta-learning in subsequent sections.

---

3.3 Continual, flashbulb, and correlated memory tasks
We next test our plasticity rules on more ecologically relevant memory tasks. These tasks reflect the complexity of biological stimuli and the functionality needed for versatile memory storage.
(a)
Continual recall
(c)
Correlation=0.6
1.0
Sequential
1.00
Sequential
0.9
Random
0.95
Random
Accuracy
Trained
Accuracy
Trained
0.8
Hopfield
0.90
Hopfield
0.7
0.85
0.6
0.80
0
25
50
75
100
125
150
175
50
75
100
125
150
175
Delay interval
Number of stimuli
(b)
Flashbulb: Sequential
Random
Hopfield
1.0
Acouracy
Regular
Flashbulb
100
100
150
Delay interval
Delay interval
Delay interval
Figure 4: (a) Performance on continual recall task, accuracy measured as a function the number of timesteps between the storage and recall of memory. "Trained" corresponds to the meta-learned algorithm with random local third factors. (b) Same as (a), but including five "flasbull" memories. Performance of simplified algorithm with meta-plasticity, using sequential (left) or random (middle) local third factors. For Hopfield network (right), increasing darkness of each line corresponds to higher write strength for flashbulb memories, with values 10, 50, 10², and 10 (c) Same as Figure 2h. but with memories having correlation of 0.6.
In realistic scenarios, memories are stored and recalled in a continual manner the subject sees an ongoing stream of inputs and is required recall stimulus that shown some time ago (Appendix A) rather than being presented a full dataset to memorize before testing, as in the benchmark autoassociative task. Figure A2h shows such sample dataset with delay interval of 2. Our learning algorithm naturally suited for this task due to its decay mechanisms recent stimuli stored, but older ones forgotten. Its accuracy decreases in steps of width N due to the sequential nature of the local third factor (Appendix A). Performance with random third factors decays smoothly since writing is stochastic, but both networks show better overall performance than the Hopfield network FFguure 4
Next, we consider "flashbulb" memories, phenomenon where highly salient experiences are remem- bered with extreme efficacy, often for life [Brown and Kulik 1977). Functionally, these memories may be used to avoid adverse experiences seek out rewarding ones. We modify the continual recall task such that a small number stimuli are deemed salient and accompanied by a stronger global third factor (q: 10) akin to a boosted neuromodulatory influence learning (Figure A2p) We add a simple meta meta--tasticyty mechanism to our plasticity rule, a stability parameter [S.lk for each synapse, initially set to O. If the learning rate for that synapse crosses threshold [m.ky > 1, then [S,= = for all & >> 0 and suppresses subsequent plasticity events. Thus, the learning rate for the key matrix is as follows (learning rate n). is analogous):
In. by = (1 - [S )4 (rel:
(17)
*For fair comparison, we introduce an empirically chosen synaptic decay parameter 0.95 the Hopfield network. We also considered version of the Hopfield network with bounded weights (Parisi] 1986). designed for continual learning. but found that synaptic weight decay has better overall performance. Furthermore, we include global third factor which controls overall plasticity identically model.

---

With this meta- plaatticcit,ty the network retains flashbulb memories with minimal effect on its recall performance on regular memory items (Figure 4p, left, middle). To perform this task in the Hopfield network, introducing synaptic stability is significantly detrimental to performance, so we simply store flashbulb memories as large-magnitude updates to the weight matrix. As a result, it exhibits a tradeoff between storage of regular items and efficacy of flashbulb memories (Figure 4p, right). Thus, an important benefit of the slot-based storage scheme is that flexible treatment of individual memories can be naturally implemented.
Finally, real-world stimuli are often spatially and temporally correlated - for instance, two adjacent video frames are almost identical. Storing such pattemss in a Hopfield network causes interference between attractors in the energy function, decreasing its capacity. In contrast, by storing stimuli in distinct slots of the key and value matrices, key-value memory can more easily differentiate between correlated but distinct memories. To verify this, we use a correlated dataset by starting with a template vector and generating stimuli by randomly flipping some fraction of its entries (Figure A2c). The performance of the plasticity rule is similar to that for uncorrelated data (Figure 20), with minor degradation due to spurious recall of similar stored memories (Figure 4c). Figure shows similar results for varying correlation strengths.
3.4 Heteroassociative and sequence memory
The network and learning mechanism is agnostic the relationship between the input and target patterns, and so naturally generalizes to heteroassociative memory. To draw comparisons with Hopfield-type networks, we consider the Bidirectional Associative Memory (BAM) (Kosko| 1988], a generalization of the Hopfield network designed for heteroassociative recall. We evaluate the networks modified version of the recall task from kubsection 3.1 where values are distinct from keys, y/: E (11, - for d * m (Figure 5p, top: Figure A3p. Compared to the autoassociative task (Figure? 2). the Hopfield-type network's performance on the heteroassociative task deteriorates Figuure 5 bottom). On the other hand. the performance of our network remains unaffected in the heteroassociative task, as its factorized key-value structure allows more flexibility in the types of memory associations learned.
A more biologically relevant version of heteroassociative memory is sequence learning. Experimental and theoretical evidence suggests that hippocampal memory can serve as a substrate for planning and decision making through the replay of sequential experiences [Foster and Wilson 2006 Pfeiffer
and Foster 2013 Mattar and Daw 2018). As a simple probe for a similar functionality. we use a sequence recall task where the network is presented with a sequence of patterns to memorize, using the value at time f as the key at time + 1. Afterwards, it is prompted with a random pattern from the sequence and tasked with recalling the rest. By adding recurrent loop, using the output Yr at time as the input xt+1 at the next timestep (Figure 5p. top), our network with sequential local third factors can perform sequence learning without error until its capacity limit (Figure 5b, bottom). BAM does not perform as well, likely due to propagating errors from incorrect recall of earlier patterns in the sequence.
Finally, we consider a more complex task that requires our memory module to be integrated within larger system, demonstrating the modular nature of our memory network. In the "copy- paste" task [Graves et al., 2014). the system must store a variable-length sequence of patterns ((11, 82.......... ST+1) ) and output this sequence when the end-of-sequence marker is seen (Figure A3). We train an external network as a "controller" to generate 2t, yt. and qe (Figure 5b. top) (for BAM., qi scales the magnitude of the Hebbian update). With this controller, our network with sequential local third factors successfully learns the task and generalizes outside the sequence lengths seen (Figure 3p. bottom). The random and trained networks do not perform as well, likely due to lower memory capacity. BAM successfully learns the task, but is not able to generalize outside the sequence lengths seen in training.
4 Discussion
We proposed models of biological memory by taking inspiration from key-value memory networks used in machine learning. These biologically plausible models use combination of Hebbian and non-Hebbian three-factor plasticity rules to approximate key-value memory networks. Due to the flexibility of their structure, they can naturally be adapted for biologically relevant tasks. Our

---

(a)
(b)
(c)
1/
3/x
FF
FF
Heteroassociative Recall
Sequence Recall
Copy-Paste
1.0
Sequential
1.0
1.0
0.9
Random
Accuracy
Trained
Accuracy
Accuracy
0.8
0.8
0.8
BAM
0.7
0.6
20
40
20
40
Number of stimuli
Number patterns
Number of patterns
Figure 5: (a) Architecture (d = N 40, 20) and performance heteroassociative version
of the benchmark task (Figure 2). (b) Recurrent architecture modification for sequence recall (d - N sequence presented. (c) Network is embedded in a larger system with a feedforward network to perform the copy-paste task = N 40, 40). Dashed line is the maximum number of patterns shown during training.
40, 40), Fz41 y:- Accuracy plotted as a function of the length of the entire
results suggest an alternative framework of biological long-term memory that focuses on feedforward computation, opposed to recurrent attractor dynamics. Importantly, both our hand-designed and meta-learned results propose a role for recently discovered non-classical plasticity rules [Bittner et al.) propose
2017) that are advantageous for this type of feedforward computation. Furthermore, an architecture where individual memories are stored more independently of each other than in Hopfield networks. Such factorized design creates opportunities for more versatile use and control of memory.
Several questions remain. First, although long-term memory describes many categories of memory supported by various brain regions, we have not disambiguated these differences and their implications on testing and interpreting as a true model of the brain, it is critical to make direct comparisons between our algorithm and experimental findings in memory-related behavior and neural activity. Furthermore, our aim is not to be competitive with state-of-the-as memory networks but rather to provide a biologically realistic learning algorithm for MHNs that is par with classical Hopfield networks. To this end, focus on artificial stimuli with simple statistical structures; remains unclear how our models will perform with more complex and naturalistic data, or how they compare to their non- non-biological counterparts.
model. To validate our network not merely plausible model but
Taken together, our results take neuroscience-minder approach to connect two lines of work in memory networks that have been largely disparate.
5 Acknowledgements
We are particularly grateful for the mentorship of Larry Abbott. We also thank Stefano Fusi, James Whittington, Emily Mackevicius, and Dmitriy Aronov for helpful discussions. Thanks to David Clark for bringing Bidirectional Associative Memory to our attention. Research supported by NSF NeuroNex Award DBI-1707398, the Gatsby Charitable Foundation, and the Simons Collaboration for the Global Brain. G.R. Y. was additionally supported as a Simons Foundation Junior Fellow. C.F. was additionally supported by the NSF Graduate Research Fellowship.
10

---

References
D. J. Amit. Modeling brain function: The world of attractor neural networks. Cambridge university press, 1992.
D. Amit, H. Gutfreund, and H. Sompolinsky. Storing infinite numbers of patterns in spin-glass model of neural networks. Physical Review Letters, 55(14):1530, 1985.
D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations, 2015.
A. Banino, P. Badilea, Köster, M. Chadwick, Zambaldi, D. Hassabis, C. Barry, M. Botvinick, D. Kumaran, and C. Blundell. Memo: A deep network for flexible combina- tion of episodic memories. arXiv preprint urXiv:2001.1091: 2020.
S. Bartunov, J. W. Rae, S. Osindero, and T. P. Lillicrap. Meta-learning deep energy-based memory models. arXiv preprint arXiv:1910.02720, 2019.
K. C. Bittner, A. D. Milstein, C. Grienberger, S. Romani, and J. C. Magee. Behavioral time scale synaptic plasticity underlies cal place fields. Science, 357(6355):1033-1033 2017.
R. Brown and J. Kulik. Flashbulb memories. Cognition, (11):73-99, 1977.
T. M. Cover. Geometrical and Statistical Properties of Systems of Linear Inequalities with Applica- tions in Pattern Recognition, IEEE Trunsactions on Electronic Computers, EC-14(3):326-3344 June 1965. ISSN 0367-7508. Conference Name: IEEE Transactions on Electronic Computers.
M. Demircigil, J. Heusel, M. Löwe, S. Upgang, and F. Vermet. On Model of Associative Memory with Huge Storage Capacity. Journal of Statistical Physics, 168(2):288822)) 2999, July 2017. ISSN 1572-9613.
D. J. Foster and M. A. Wilson. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature, 440:680-683, 2006.
F. Gambino, S. Pagès, V. Kehayas, D. Baptista, R. Tatti. A. Carleton, and A. Holtmaat. Sensory- evoked ltp driven by dendritic plateau potentials in vivo. Nature, 515(7525):116-11) 2014.
E. Gardner. The space of interactions in neural network models. Journal of physics A: Mathematical and generul, 21)::25, 1988.
W. Gerstner, M. Lehmann, V. Liakoni, D. Corneil, and J. Brea. Eligibility traces and plasticity on behavioral time scales: Experimental support of neohebbian three-factor learning rules. Frontiers in Neural Circuits, 12:53, 2018. ISSN 1662-5110.
A. Graves, G. Wayne, and I. Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5400 2014.
A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwe Barwiiikk,, S. G. Col- menarejo, E. Grefenstette, Ramalho, Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471-444 2016.
J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(3):2554-2558, 1982.
C.-C. Hung, T. Lillicrap. J. Abramson, Y. Wu, M. Mirza, F. Carnevale, A. Ahuja, and G. Wayne. Optimizing agent behavior over long time scales by transporting value. Nature communications, 10(1):1-12, 2019.
P. Kanerva. Sparse distributed memory. MIT press, 1988.
P. Kanerva. Sparse distributed memory and related models, volume 92. NASA Ames Research Center, Research Institute for Advanced Computer Science, 1992.
D. Kingma and J. Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.

---

B. Kosko. Bidirectional associative memories. IEEE Transactions on Systems, man, and Cybernetics, 18(1):99-60, 1988.
D. Krotov. Hierarchical Associative Memory, arXiv:2107.06-44 [cs], July 2021. arXiv: 2107.06446.
D. Krotov and J. Hopfield. Large associative memory problem in neurobiology and machine learning. arXiv preprint NXXv:2008.06999, 2020.
D. Krotov and J. Hopfield. Dense associative memory for pattern recognition. arXiv preprini orXiv:1606..00 164, 2016.
H. Le, T. Tran, and S. Venkatesh. Neural stored-progran memory. arXiv preprint arXiv:1906.08862, 2019.
M. G. Mattar and N. D. Daw. Prioritized memory access explains planning and hippocampal replay. Nature Neuroscience, 21:1609-1617, 2018. doi: 10.
M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
T. Munkhdalai, A. Sordoni, T. Wang, and A. Trischler. Metaleamed neural memory. arXiv preprini arXiv:190..09720, 2019.
G. Parisi. A memory which forgets. Journal of Physics A: Mathematical and General, 19(10)L611,, 1986.
B. Pfeiffer and J. Foster. Hippocampal place-cell sequences depict future paths to remembered goals. Nature, 497:74-79, 2013.
H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandve, Greiff, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217, 2020.
D. D. Rasmusson. The role of acetylcholine in cortical synaptic plasticity. Behav Brain Res., 115(2):
205-218, 2000. doi: 10. 00.1066//0166--42880000000000000
A. Robins and S. McCallum. Catastrophic forgetting and the pseudorehearsal solution in hopfield- type networks. Connection Science, 1012): 1998. doi: 10. 00.10800095540098111655 URI.
https://doi.onn orw.ioorro11.. 090g/000.1000//44409911111100000
N. Rodriguez, E. Izquierdo, and Y.-Y. Ahn. Optimal modularity and memory capacity of neural reservoirs. Network Neuroscience, 3(2):551-566, Jan. 2019. ISSN 2472-1751.
G. J. Stuart and N. Spruston. Dendritic integration: 60 years of progress. Nature neuroscience, 18 (12):1713-1721, 2015.
Sukhbaatar, A. Szlam, J. Weston, and R. Fergus. End-to-end memory networks. arXiv preprint arXiv:1503.08895, 2015.
D. Tyulmankov, G. R. Yang, and L. Abbott. Meta-learning local synaptic plasticity for continual familiarity detection. bioRxiv, 2021. doi: 10. 00..0. .436277.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, F.. Kaiser, and Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998 -008-6008, 2017.
J. Weston, A. Bordes, S. Chopra, A. M. Rush, B. van Merriënboer, A. Joulin, and Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv: 1502.05698. 2015.
D. J. Willshaw, O. P. Buneman, and H. C. Honguet-Higgins Non-holographic associative memory. Nature, 222(5197):960 962, 1969.
12


Reviews:
[{"review_id": "xSBnifyLMwt", "reviewer": "Reviewer_dV7b", "summary": "The paper proposes a new ML-inspired model for biological associative memory. Auto-associative memory networks are a popular model for biological memory, with one-shot learning implemented by hebbian learning and attractor dynamics / gradient descent for retrieval. In contrast, a traditional key-value memory system has fast feedforward retrieval (linear-nonlinear-linear map), with more complex, often gradient based, parameter optimization. Here the authors develop local learning rules for the parameters of a standard key-value network, linking its properties to some recent experimental observations in rodent hippocampus. They also generalize the basic setup to an online metalearning scenario where the gating of plasticity and the functional form of the synaptic updates are themselves parametrized, with hyperparameters adapted to the learning task. The learning procedures are applied to a range of memory tasks and compared to traditional Hopfield networks.\n\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall: The idea is extremely obvious: copy one-to-one the item to be stored into one unique set of synapses, which results into an essentially one hot encoding of the item; learn the reverse map by correlations (essentially copying the target output into the corresponsing readout weights). The mechanistic interpretation seemed unnecessarily convoluted i would think that if you have a random ultra sparse activation in the hidden layer, this activity can simultaneously gate the learning of both the key(in) and value (out) weights at the same time, without additional gating. Presumably the extra steps are there for biological plausibility, not the core computation. Some of the numerical experiments are set in a nonstandard manner and some of the numerical experiments could be improved (see detailed comments). I appreciate the ambition of exploring such a range of tasks and variants but naturally this comes at the cost of getting less insight from the individual pieces. Sometimes less (done extra thoroughly) is more. \n\nDetailed comments:\n- i don't quite understand why the two learning processes need to be different, as long as the softmax is more like a WTA, so the activity in the hidden layer is very sparse h should be able to safely gate\n- i have some conceptual issues with benchmark 1: in the traditional hopfield net the cue is a valid pattern - in a spin (+/-1) representation the corrupted bits would be flipped rather than zeroed out. In this way they add to the variance of the current in the hidden layer neurons. by setting them to zero there is no added variance due to the input noise in the h currents, which artificially aids performance. the larger the hidden layer, the more likely for spurious items to be retrieved, restricting capacity analysis. Was the retrieval for your implementation of the hopfield dynamics done consistently? same for tvt.\n- what were the pattern statistics of the stored items?\n- it seems that for large stimulus values hopfield degrades the most gracefully among the items compared... why is that?\n- what is the loss function being optimized for the hyperparameter learning ?\n- 'improves performance for longer sequences' on line 196: by sequence you mean number of stored items? i did not see any reference to temporal dependence in the input-output pairs in 4.2.\n- presumably the hidden layer sparsity scaling with number of units is orthogonal to the meta-learning and changes in gating in 4.2 \n- with the random rewrite version, would you see ever catastrophic forgetting if the number of stored items is larger than N? (in the cycle through neurons version you obviously just overwrite the oldest so forget T-N items)\n-missing a bit the biological and/or computational relevance of the metalearning scenario beyond the point that it need not be quite so simple\n- biological circuits are almost invariably recurrent with attractor dynamics as a not uncommon dynamical feature; do you see any role for recurrence in this framework (maybe in the context of sequential inputs?)\n-for preferrential encoding of salient information not encoding the non-salient information should be the best (because it effectively reduces the number of stored inputs), at least intuitively; not sure how the 'synaptic stability' was implemented exactly, but there are for sure ways to make that work.\n- i am somewhat unsure if the comparison to hopfield is really fair in the sense that with key-value one has the extra degree of freedom of the size of the hidden layer. Maybe one way to equate neural resources would be to set N such that the total number of synapses of the two systems is the same (?)\n- minor, with the changes in variable naming for hopfield the capacity is o(d) but for KV its o(N) which somehow gets confused in the description.\n- since the hopfield net is not designed to do anything other than autoassociative memory its use for some of the more complex tasks seems dubious (and it's unclear how it was set up at all)\n- would have liked a more in depth discussion of the relevance of these findings and with biological relevance in mind neural signatures that can differentiate between the different proposals.\n\nSignificance: Technically very simple, potentially relevant for interpreting recent data on hippocampal plasticity during behavior.\n\nOriginality: Interesting point of intersection between ML and neuroscience.\n\nClarity: Overall, the text is very clear. In the key learning part the gradual forgetting and link to eq7 needs clarification.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No obvious negative impact beyond that of standard uses of regular key-value memory systems.", "review_text": "", "extra_scores": {"ethical_concerns": "No concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "uKv93is8mAo", "reviewer": "Reviewer_vmVW", "summary": "This paper proposes a biologically plausible neural circuit KEVIN that approximates key-value memory (slot) networks. The neural circuit is a key-value memory network with a key matrix and a value matrix.\n“Pre-only” rules are used to store inputs in the key matrix slots based on presynaptic activity, gated by a “local third factor”: Cycling through slots sequentially or via independent slot-probabilities. This local third factor is also used to allow for overwriting/forgetting of key matrix slots. A scalar “global third factor” is used to (de)activate storage of an input independently of individual slots.\nIn the simplified version, this local third factor is also used for writing to the value matrix. These update rules are then refined and parameterized to interpolate between “pre-only” and “post-only” rules.\nKEVIN is analyzed on and evaluated against classic Hopfield networks in three simple long-term memory tasks (continual storage, storage of single event over long time, storage of correlated inputs) and a heteroassociative memory task (sequence memorization). Optimization of the parameterized rules in meta-learning experiments shows that the rule for key matrix slots becomes “pre-only” and the rule for value matrix becomes both pre- and post-dependent.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n## Major:\nThe paper proposes and motivates an interesting biologically more plausible key-value memory network. However, the proper placement of the new method and its discussion w.r.t. other state-of-the-art methods are insufficient. The experimental setup, in this form, is inadequate (no error bars, no natural/more complex tasks).\n\n### Originality\n1.  The proposed method with its biologically plausible aspect, to my knowledge, is new.\n2.  The state-of-the-art w.r.t. possible storage capacity is not reflected appropriately in the paper. While classic Hopfield networks, as pointed out and compared to in the paper (e.g. Section 4.1), indeed exhibit linear storage capacity, modern Hopfield networks achieve exponential storage capacity [1][2][3][4]. One could argue about the biological plausibility of modern Hopfield networks but regardless, it should be mentioned that KEVIN with its linear storage capacity would scale exponentially worse than modern Hopfield networks. As a result, KEVIN would likely be outperformed by such modern Hopfield networks in the experiments conducted in the paper. W.r.t. the correlation task, the learned mapping of the patterns into the associative space would furthermore allow to separate patterns that are correlated in the input space well in the association space. ([1] and [3] are cited in the Introduction but the relation of the KEVIN storage capacity to this state-of-the-art is missing. This would be relevant for the assessment of KEVIN’s performance in the experimental section and its potential for realistic applications. It would also emphasize the impact of such networks given their recent successes in i.a. [3][4][5].)\n\n### Quality\n3.  See point 2 - comparison of KEVIN storage capacity only to classic Hopfield networks is misleading.\n4.  The paper keeps pointing out a clear gap between Hopfield networks and key-value memory networks. With the connection of classic Hopfield networks to modern Hopfield networks [3] and thereby key-value memory networks, such as the Transformer [5], this statement is too general and should be refined and made more specific.\n5.  The experimental section and plots in the paper lack information about repetition of runs with different random seeds and have no error bars.\n6.  Experiments on more realistic/complex datasets (e.g. RNN benchmarking datasets) would have been important to judge the performance of KEVIN in larger systems (see point 7).\n\n### Clarity\n6.  More information on the experimental setup in the main paper would be useful (see minor comments).\n\n### Significance:\n7.  The biologically more plausible aspect is interesting but the significance is hard to judge without experiments on more realistic datasets, especially with the poor performance on the one more complex copy-paste task.\n\n## Minor:\n1.  The experimental setup should be described in more detail in the main paper (e.g. how large the FF Controller and KEVIN network is in Section 4.4.).\n2.  Figure 1: Please also mention the variable letters with their names in the caption.\n3.  Figure 1: The positioning/illustration of the local third factor for gating is a bit confusing, maybe this could be refined (otherwise it looks like the local third factor is the hidden state w.r.t. Fig1.a and Fig.1.b).\n4.  Figure 2: Please specify what “Network size” refers to (e.g. number of hidden neurons).\n5.  Figure 2: It would be interesting to also have a plot with the number of synapses instead of the number of neurons on the x-axis.\n6.  Figures: Maybe the colors could be modified so that they are distinguishable in grayscale for print and color vision deficiency.\n7.  W.r.t. memory capacity of RNNs, [6] would also be relevant in the Introduction.\n8.  Since $\\mathbf{q}$ is used for the query vector in Eq1, it might be beneficial to rename the global third factor to something other than $q_t$.\n9.  Typos: Eq.3: is it not $\\mathbf{V}^T_t$?; 104: “slots”;\n\n## References\n\n[1] Krotov, D., & Hopfield, J. J. (2016). Dense associative memory for pattern recognition. Advances in neural information processing systems, 29, 1172-1180.\n\n[2] Demircigil, M., Heusel, J., Löwe, M., Upgang, S., & Vermet, F. (2017). On a model of associative memory with huge storage capacity. Journal of Statistical Physics, 168(2), 288-299.\n\n[3] Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Gruber, L., ... & Hochreiter, S. (2020). Hopfield Networks is All You Need. In International Conference on Learning Representations.\n\n[4] Widrich, M., Schäfl, B., Pavlovic, M., Ramsauer, H., Gruber, L., Holzleitner, M., ... & Klambauer, G. (2020). Modern Hopfield Networks and Attention for Immune Repertoire Classification. In Advances in Neural Information Processing Systems (pp. 18832-18845).\n\n[5] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).\n\n[6] Leibold, C., & Kempter, R. (2006). Memory capacity for sequences in a recurrent network with biological constraints. Neural computation, 18(4), 904-941.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors did not address the societal impact of their work", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "20"}, "extra_text": {}}, {"review_id": "HsHRheqOUmo", "reviewer": "Reviewer_dTBY", "summary": "The paper proposes KEVIN--a biologically plausible key-value memory using a 2-layer feed-forward neural network wherein the key and value matrices are the weights of each layer. Here, each row of the key matrix is updated toward an input vector (non-Hebbian update). The value matrix is updated using  Hebbian learning. To control the synaptic plasticity rate, the authors use a global and two alternative local (sequential or random) third factors that take inspiration from the brain. In addition to the main design, there are modifications for specific tasks such as learning to scale synaptic plasticity rates with gradient descent and disabling subsequent weight updates conditioned on the current learning rate. The authors compare the proposed model with Hopfield networks in various synthetic tasks, showing competitive results. ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The method is incremental as it combines different known techniques (slot-based and Hebbian writing), although there are interesting connections to biological mechanisms in the brain. One interesting finding is that the simplified local third factor and the trained one show similar behaviors. The model is simple and claimed to be implementable with biological neurons. I am curious about how the Sequential KEVIN is biologically plausible, especially how to implement Eq. 4 with real neurons. Also, the proposed architecture seems related to a classic model ADAM [1] (also 2-layer feed-forward net). The idea is also to map the input to some pattern (index) and use it to retrieve the value. It is good to have ADAM as one baseline in the experiment. \n\nOverall, the paper is easy to read. However, there remain unclear details of implementations. For example, how is $q_t$ (in Eq. 6) computed?  What is the exact loss to train f, g (in Eq. 12, 13)?  Some part of writing is confusing: where are Fig. A5 (Line 213) and A4 (line 256)? Fig. 5 caption, why $y_t = x_{t-1}$?\n\nMy main concern is the experiment results. Not to mention the toy tasks, the overall performance of KEVIN is not impressive. The best performer is Sequential KEVIN, yet it is obvious thanks to the sequential slot-based writing and it is unclear this mechanism is biologically plausible. Hopfield network still shows competitive and better performance in Auto-associative Recall task. In other tasks, KEVIN shows improvement. However, this is attributed to additional modifications and specific designs for the task, which makes the model no longer simple and generic. Besides, the authors should consider comparisons with gradient-based baselines such as RNN to validate the usefulness of KEVIN. \n\n[1] J. Austin. ADAM: A distributed associative memory for scene analysis. In M Caudhill and C Butler, editors, First International Conference on Neural Networks, volume IV, page 285, San Diego, 1987.                         ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4h"}, "extra_text": {}}, {"review_id": "87l1pAflhML", "reviewer": "Reviewer_Dwry", "summary": "The authors present \"Biological Key-value Memory Networks\", a biologically plausible method of storing patterns of data in memory for later recall. The method is a simple key-value implementation with each neuron in the hidden layer encoding one key for one memory. The authors provide evidence that this algorithm performs on par with Temporal Value Transport (TVT) and better than Hopfield nets. \n\n_____\n\nPost rebuttal update:\nThanks to the added clarity from the author responses I have increased my score from 4 to 6.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The contribution is interesting and the paper is generally well written. The authors explore a number of relevant experiments and highlight the strengths of their algorithm. However, because of a number of issues I cannot recommend the work for publication in its current state. The following is a number of strengths and weaknesses of the paper in no particular order.\n\n## Strengths\n\n* The algorithm is intuitive and easy to understand. Furthermore, the biological implementation and learning rules are more or less reasonable.\n* The experiments are well varied and thorough in scope. It is also interesting that this simple algorithm tackles tasks that are alternatively solved by more complicated methods (e.g. sequence memorization in bird songs is thought to be handled via RL). Furthermore, the authors show the modularity of their algorithm by building it into other frameworks to perform more complicated tasks.\n\n## Weaknesses\n\n* There is no theoretical analysis of the capacity of the algorithm.\n* There are no error bars or confidence intervals in the experiments. Especially for a fast algorithm like this, it won't take much time to run each experiment multiple times and and add confidence intervals. This is particularly necessary for example in Fig. 5, where the accuracy of the network at 60 patterns is higher than at 50, indicating a possibly high error range for the plot. Another example is Fig. 4a, is the plateaud drop in the sequential learning task a real feature or is it just a quirk of that single run of the expriment?\n* Comparison to some other models is lacking, e.g. the authors do not mention reservoir networks. Some references to prior work is lacking, e.g. Fixing the weights connected to specific neurons to reduce catastrophic forgetting was previously considered in _Continual Learning via Neural Pruning_ circa 2019.\n* Some confusion regarding Hebbian learning rule. In my understanding, Hebbian 'fire together, wire together' implies that the pre and post part of the plasticity are the activities of the neurons. If, as in eq. 9, the learning rule is dependent on a third factor, this is no longer Hebbian.\n* In Eq. 9, why not just use the activity $h'_t$ instead of this local third factor? This is what the meta learning rule learns and is motivated already in lines 135-136.\n* The simplicity of the algorithm is a little deceiving since it requires a mechanism to indicate when to store and retrieve information. This should be highlighted since competing methods can store and retrieve without queuing (albeit it comes at a cost of performance). In other words, whether or not to store an incoming pattern (information regarding whether it is new or not), needs to be provided to the network, which is why it performs well in the correlated recall task. In a Hopfield net, the dynamics of the evolving energy landscape itself decides if a new energy minimum should be created or if a preexisting minimum should be updated (that is the network itself decides if the information is new and should be stored).  _This needs to be properly addressed in the paper, so the reader understands where the performance improvements are coming from._\n\n## Clarity\nEven though the paper is in general readable, there are a number of issues that need to be addressed.\n* There are many references to the appendix which are mislabeled (e.g. line 273 A4->A3, line 248 A3->C5.\n* Eq. 13 what is $\\tilde\\eta^y$?\n* (optional) I would suggest moving the learning trajectory plots (Fig 3d) to the appendix, and instead using the freed up space for perhaps parts of Fig A1 or A2, which explain the workings of the experiments. While the trajectory is interesting, the main takeaway is that it started from zero and ended at the reported values in lines 209-211.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The significance of the proposal was not satisfactorily established for acceptance. While the committee accepts that the authors' burden is to prove their proposal is an interesting and valid model for biological memory (rather than being competitive with state-of-the-art non-biological memory networks), the experiments were still deemed inadequate in a few different dimensions (see reviewer comments). The metrics for a proposal like this are a bit unclear, but some on the committee would have liked to see scientific merit as evidenced by the proposal being more consistent with actual biological experimental data than past proposals. That was not made clear, and the issue was convoluted by the incomplete analytical comparisons to prior work. For example, it would have helped to place this in context better compared to the sparse distributed memory models of Kanerva et al. from twenty years ago, which seemed very similar, though this proposal uses a more biologically complex soft-max than a step function.  Further, the sequential KEVIN method was not compelling biologically.  Overall, this proposal seemed to fall in the middle of biological plausibility and computationally efficiency, making it not clear how to judge its overall merits and significance, especially with the lack of standard experimental evidence. 
Decision: Accept (Poster)