Biological learning in key-value memory networks
Authors: Danil Tyulmankov
Ching Fang
Annapurna Vadaparty
Guangyu Robert Yang
Venue: NeurIPS 2021 Poster (2021)
Keywords: neuroscience
computational neuroscience
memory
recall
biological learning
learning rule
synaptic plasticity
key-value network

Abstract: In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in machine learning commonly use a key-value mechanism to store and read out memories in a single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet their biological relevance is unclear. We propose an implementation of basic key-value memory that stores inputs using a combination of biologically plausible three-factor plasticity rules. The same rules are recovered when network parameters are meta-learned. Our network performs on par with classical Hopfield networks on autoassociative memory tasks and can be naturally extended to continual recall, heteroassociative memory, and sequence learning. Our results suggest a compelling alternative to the classical Hopfield network as a model of biological long-term memory.

Biological learning in key-value memory networks
Danil Tyulmankov'
Ching Fang" Columbia University ching. fawg@coolmmiia..ee.m udu
Columbia University dt:256@cclumbia. edu
Annapurna Vadaparty
Guangyu Robert Yang
Columbia University
Columbia University
Stanford University
Massachusetts Institute of Technology yanggremit. edu
Abstract
In neuroscience, classical Hopfield networks the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, evemory-augmeee learning commonly use key-value mechanism to store and read out memories in single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet their biological relevance is unclear. We propose an implementation of basic key-value memory that stores inputs using a combination of biologically plausible three-factor plasticity rules. The same rules are recovered when network parameters are meta-learned. Our network performs par with classical Hopfield networks on autoassociative memory tasks and can naturally extended continual recall, heteroassociative memory, and sequence learning. Our results suggest compelling alternative to the classical Hopfield network as a model of biological long-term memory.
neural networks in machine
Introduction
Long-term memory an essential aspect of everyday lives. It is the ability rapidly memorize an experience or item, and to retain that memory in retrievable form over prolonged duration (days to years in humans). Neural networks capable of long-term memory have been studied in both neuroscience and machine learning, yet wide gap remains between the mechanisms and interpretations of the traditions.
In neuroscience, long-term associative memory is typically modeled by variants of Hopfield networks [Hopheld 1982 Amit 1992 Willshaw et al. 1969.. Rooted in statistical physics, they of the earliest and best known class of neural network models. A classical Hopfield network stores an activation pattern & by strengthening the recurrent connections W between co-active neurons using bhologically-plausibe Hebbian plasticity rule,
W + W + ET
and allows retrieval of memory from corrupted version through recurrent attractor dynamics,
#t+1 sign ( x::)
thereby providing a content-addressabte and pattern-completing autoassociative memory.
In more recent parallel thread machine learning, various memory networks have been devised to augment traditional neural networks Grrves al. 2014 Sukhbaatar et al. 2015 Munkhdalat et al.
Equal contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

neural networks utilize a more stable external memory system analogous to computer memory. in contrast to more volatile storage mechanisms such as recurrent neural networks [Rodriguez et al. 2019]. Many memory networks from this tradition can be viewed as consisting of memory slots where each slot can be addressed with key and returns a memory value, although this storage scheme commonly lacks mechanistic interpretation in terms of biological processes.
2019 Le et al. 2019 Bartunov et al., 2019].
Key-value networks date back to at least the 1980s with Sparse Distributed Memory (SDM) as a
model of human long-term memory [Kanerva 1988 1992]. Inspired by random- acccess m
in computers, it is at the core of many memory networks recently developed in machine learning
[Graves et al., 2014 2016 [Sukhbaatar al., 2015 Banino al.] [2020] Le et al. 2019). basic
key-value network contains key matrix K and value matrix V. Given a query vector æ, memory
read operation will retrieve an output y as
h f(K))
(3)
y Vh.
where f is an activation function that sparsifies the hidden response h.
Variations exist in the reading mechanisms of key-value memory networks. For example, f may be the softmax function [Sukhbaatar et al. 2015). making memory retrieval equivalent to the "key-value attention" " [Bahdanau et al. 2013| used in recent natural language processing models [Vaswani et al. 2017]. It has also been set as the step function [Kanerva 1988 and hard-max function [Weston et al. 2.15]. There is an even greater variation across writing mechanisms of memory networks. Some works rely on highly flexible mechanisms where external controller learns which slots to write and overwrite [Graves et al. 2016 2014), although appropriate memory write strategies can be difficult learn. Other works have used simpler mechanisms where new memories
be appended
sequentially to the existing set of memories (Sukhbaatar et al. [015 Hung et al. 2019 Ramsauer
al.
2020
or written through gradient descent [Bartunov et al. 2019 Munkhdala ai et al. 2019 Le
et al.
2019 Krotov and Hopfield (016). Kanerva| [1992] updates the value matrix through Hebbian
plasticity, but fixes the key matrix. Munkhdalai al. [2019] turns memory write into key-target value regression problem, and updates an arbitrary feedforward memory network using metalearned
local regression targets.
clear advantage of key-value memory over classical Hopfield networks is the decoupling of memory capacity from input dimension (Kanerva 1992 Krotov and Hopfield, 2020). In classical Hopfield networks, the input dimension determines the number of recurrent connections, and thus upper bounds the capacity. In key-value memory networks, by increasing the size of the hidden layer, the capacity can be much larger when measured against the input dimension (although the capacity per connection is similar).
There exists a gap between these two lines of research on neural networks for long-term memory- classical Hopfield networks in the tradition of computational neuroscience and key-value memory in machine learning. In our work, we study whether kcy-value memory networks used in machine learning can provide an alternative to classical Hopfield networks as biological models for long-term memory.
Modern Hopfield Networks (MHN) Krotov and Hopfilld 2016 2020 Ramsauer et al. 2020) Krotov
202T] begin to address this issue, suggesting neural network architecture for readout of key-value pairs from a synaptic weight matrix, but lacking a biological learning mechanism. MHNs implement an autoassociative memory (key is equal to the value, so V KT) [Ramsauer et al., 2020 Krotov and Hoptield 2020). but they can be used for heteroassociative recall by concatenating the key/value vectors and storing the concatenated versions instead. To query the network, keys can be clamped, and only the hidden and value neurons updated [Krotov and Hopfield 2016). With particular choice of activation function and one-step dynamics, this architecture is mathematically equivalent to a fully connected feedforward network [Krotov and Hopfield 2016), and thus analogous to the memory architecture proposed by Kanerva
1992].
It remains unclear whether biological mechanism can implement the memory write. We will show that such biologically-plausbbe plasticity rules exist. We consider a special case of the MHN architecture and introduce biologically plausible learning rule, using local three-factor synaptic plasticity [Gerstner al. 2018), as well as respecting the topological constraints of biological neurons with spatially separated dendrites and axons. We first suggest simplified version of the learning

---

rule which uses both Hebbian and non- non-Hebbian plasticity rules to store memories, and evaluate its performance in comparison to classical Hopfield networks. Adding increasing biological realism to our model, we find that meta-learning of the plasticity recovers rules similar to the simplified model. We finally show how the feedforward, slot-based structure of our network allows it be naturally applied to more biologically-ootvaaee key-value memory compatible with applications of the classical Hopfield network, particularly as a biologically plausible mechanistic model of long-term memory in neuroscience.
memory tasks. Thus, the addition of our learning rules makes
2 Simplified learning mechanism
We consider a neuronal implementation of slot-based key-value memory, and endow it with biologically plausible plasticity rule for memorizing inputs. We first describe a simplified learning rule for storing key-value pairs, which sets an upper bound on the network memory capacity and serves as benchmark against existing memory networks. In later sections, we modify this rule to further increase biological realism.
(a) Read
(b) Write key
(c) Write value
Ke
Vi
K₁
Ket1
V
Kr41
Ve+1
Spodln weiight local third factor
1/+
I1
8/
71
h+
Figure I: Network architecture and read/write mechanism. (a) Memory reading. The network is given query z (input layer activation with corrupted version of stored key œ)), selects the most similar stored key through upproximately-one
hidden layer activity h. and returns the corresponding value ÿ ~ y. (b) Writing keys. The input æe is written into the 1th slot of the input-to-hidden weight matrix K2 by a "pre-only" plasticity rule, selecting the corresponding hidden neuron via local third factor [ty]] 1. (c) Writing values. The same 7th hidden neuron is selected through an intermediate hidden unit activation hi and the target ye is written to the hidden-tor to-eutttt weight matrix V. by a Hebbian update.
The network operates sequentially and consists of three fully-connected layers of neurons (Fiuree T): a dddimensiooal input with activity at time given by the vector æt, an N-dimensional hidden layer ht and an mi-imeensinnal output layer yt- The 7th key memory slot corresponds to the synaptic weights from the input layer to single hidden neuron (2th row of the key matrix K,, storing key m.)). The corresponding value stored in the weights from that hidden neuron to the output (ith column of the value matrix V,, storing value y:).
2.1 Reading
The network stores a set of key-value pairs {(x,, y: such that if query ă. e.g. corrupted version of a stored key æ. is presented to the network, it returns the corresponding value y. Given a query æ (Figure 1b) the hidden layer computes its similarity h; to each stored key æ; (ith row of K,) normalized dot product:
h softmax K₂
(4)
where the softmax function normalizes the hidden unit activations, and can be approximated biolog- ically with inhibitory recurrent connections. The output layer then uses these similarity scores to compute the estimated value as the weighted sum of the stored values through a simple linear readout:
N
ÿ = Vih =
hays
(5)
3

---

Assuming uncorrelated keys, the dot product of the query ă will be maximal with its corresponding stored key æ, and near-zero for all other stored keys, so h will be approximately one-hot. Thus, Equation 5) reduces to y ~ y as desired. If target values are binary, y: E {+1, -1) (neurons are either active or silent), we use (yg when evaluating performance.
Mathematically, this architecture is equivalent to an MHN constrained to the heteroassociative setting with a fixed choice of activation functions [Krotov and Hopfield 20201, and recurrent dynamics updated for exactly one step [Krotov and Hoptield 2016 Alternatively, it can be thought of as an differentiable version of an SDM [Kanerval 1988 with a softmax rather than step function activation function in the hidden layer [Kanerva 1992]. Unlike these networks, however, we introduce a novel biologically plausible writing mechanism to one-shot memorize key-value pairs by updating both the key and value matrices.
2.2 Writing keys
Key-value pairs are learned sequentially. Given input æ, time t. write it into slot of the key matrix using a non-Hebbian plasticity rule, where the presynaptic neuronal activity alone dictates the synaptic update, rather than pre- and postsynaptic co-activation as in traditional Hebbian rule. A local third factor [Tr] E {0, 1 Figure Tp, red circle) gates the plasticity of all input connections to hidden unit i, enabling selection of a single neuron for writing. Biologically, this may correspond to a dentritic spike [Stuart and Spruston 2015 Gambino et al., 2014]. which occur largely independently of the somatic activity h2 performing feedforward computation This plasticity rule resembles behavioral time scale plasticity (BTSP) [Bittner et al., 2017]. recently discovered in hippocampus,
brain structure critical for formation of long-term memory.
In this simplified version we approximate local third factors as occurring in the deast-recently-usee
neuron by cycling through the hidden units sequentially:
(1 if t i mod N
[rc]
(6)
0 otherwise
This can be biologically justified in several ways. Each neuron may have internal timing mechanism that deploys a local third factor every N timesteps; the neurons may be wired such that a dendritic spike in neuron i primes neuron i for dendritic spike the next time step: or the local third factors may be controlled by an external circuit that coordinates their firing. Alternatively, we consider simpler mechanism, not requiring any coordination among the hidden layer neurons: each neuron independently has some probability p of generating a dendritic spike:
[rr] Bernoullipp)
To gate whether a stimulus should be stored at all, we include a scalar global third factor ge E (0. 1] Biologically, this may correspond to neuromodulator such acetylcholine [Rasmusson 2000 that affects the entire population of neurons, controlled by novelty, attention, or other global signals. Although it can also be computed by an external circuit, in experiments this value is provided as part of the input. Thus, the learning rate of the synapse between input unit j and hidden unit i is the product of the local and global third factors:
In = celccl
(8)
Finally, to allow reuse of memory slots, we introduce a forgetting mechanism, corresponding to rapid synaptic decay mediated by the local third factor. Whenever a synapse gets updated we have by 1, so we can zero out its value by multiplying it by 1 - [m]]s]- If there is no update (either third factor is zero), the weight is not affected. The synaptic update is therefore:
K = (1 (1-18 nt O K₁ + nh @ [1x]]
where c indicates the Hadamard (element product, and 1 (1, 11,1,..,1) 1).
"The same synaptic update could be accomplished without third factors, using Hebbian rule, if the hidden layer were ooe-hot. However, since the hidden layer activity is determined by its weight matrix and input, there is no simple neuronal mechanism to independently select a hidden neuron. More complicated versions may involve an external circuit which controls the activity of the hidden layer during writing, or strong feedforward weights which do not undergo plasticity (Tyulmankov al. 2021

---

We compute the accuracy as function of the number of stored stimuli (Fiuuee 2). By design, the plasticity rule with sequential local third factors performs identically simple non-biological key-value memory (TVT, (Hung et al. 2019]. Appendix B}. It has perfect accuracy for T S N since every pattern stored in a unique slot. For > N, previously stored patterns are overwritten one-by-one and accuracy smoothly degrades. With random local third factors, accuracy degrades sooner due to random overwriting. Importantly, this holds for arbitrary network sizes, unlike the classical Hopfield network, which experiences a "blackout catastrophe" where all stored patterns are erased if the network size is large and the number of inputs exceeds its capacity [McCloskey and (we do not see this here due to relatively small network 2.3 Writing values
Having stored the key 24, the hidden layer activity this intermediate stage is given by:
h softmax(K(+11))
(10)
Note we are using the updated key matrix with æt stored in the 7th slot. In the idealized case of random uncorrelated binary random memories æ; E [+1,-1]+ the 2th entry of 1t will be equal to 2,,- normalization via the softmax, h will be approximately one-hot, with [ha]e == 1.
d. All other entries will be near 0, since they are uncorrelated with Et. Thus, after
To store the value, the output layer activity is clamped to the target yr Figure Te), Biologically, this can be achieved either by strong one-to-one residual connections from the input the output layer, or by having common circuit which drives activity in both the input and output layers. Since the only hidden unit active is the one indexing the ith column of the value matrix, we can update the synapse between hidden unit and output unit & by Hebbian rule with learning rate [n, AA = q (7%) ) and rapid decay rate analogous to the key matrix
V = (1 n V n. O yye (k)))
(11)
Results
3.1 Benchmark:
autoassociative recall
As simple evaluation of our algorithm's performance and comparison to other memory networks, we consider the classical autoassociative memory recall task where the key is equal to the value, (d m. Figure T). The network stores a set of T stimuli {æ,] and the query x is a corrupted version of stored key (60% of the entries are randomly set zero). The network returns the corresponding uncorrupted version (Appendix A Figure Al.
Cohen 1989 Robins and McCallum 1998
size).
Next, by measuring the number of patterns that the network can store before its recall accuracy drops below threshold an imperfect measure because some networks may have accuracy that stays above threshold longer but drops sharply after that. Nevertheless, this metric allows us to investigate empirically how the network's performance scales with its size (Figure 2p). The empirical scaling of the classical Hopfield network C' ~ 0. 141,, consistent with theoretical calculations [mit et al. 1985 ), and the sequential algorithm's 's capacity scales approximately as C' ~ 1.0.N, as expected analytically. Importantly, the random algorithm's capacity also scales linearly, C ~ 0.16N, with slope similar to the Hopfield network. Note, however, that with the same number of hidden neurons our network has twice as many connections as the Hopfield network. We also note that although the theoretical capacity of the Hopfield network can scale C ~ 2N [Cover 1965 Gardner 1988] to our knowledge there is not a learning algorithm that achieves this bound. Other work on autoassociative memory shows exponential scaling, but lacks a biologically plausible readout [Demircigil et al. 2017
0.98), we can estimate this network's memory capacity C. This is
3.2 Meta-learning of plasticity rules
We now introduce parameters that can be meta-learned optimize performance on a particular dataset without sacrificing biological plausibility. First, we enable varying the scale of the synaptic
'If local third factors are dendritic spikes in the dendrites of hidden layer neurons, it may not be realistic for the axons of the same neurons to be affected by these dendritic spikes. We consider this case for simplicity, as an upper-bound bound performance. We will later lift this assumption without significantly hurting performance.
5

---

(a) 1.0
(b)
Sequential
Sequential C-1 .00ON
Random
Capacity (38% acc)
Random C-0.162N
0.9
Hopfield
Hopfllld C-0.147N
Accuracy
TVT
0.8
50
0.7
0
25
50
125
20
140
160
Number stimuli
Network size
Figure 2: Our network (d = N 40) with sequential and random (p 0.1) local third factors, Hopfield network, and TVT [Hung et al. 2019) performance on the autoassociative memory task. (a) Accuracy function of stored stimuli. (b) Capacity (maximum number of stored stimuli at Z 98% accuracy) as a function of network size.
plasticity rates each one is multiplied by learnable parameter 17" 77": In/ = 9: [elkk(">), and [m]les = 4: [7
(12)
Next, further improvement biological plausibility, remove the assumption that the hidden- to-output synapses (hidden layer axons) have access to the local third factors (hidden layer dendritic spikes). Note that n, above no longer contains Y term. Instead, forgetting mechanism, the hidden-to-output synapses decay by a fixed factor À each time a stimulus stored:
A = (1 9c) + 9
(13)
where X, is the decay at time and qe {0. 1}. Although in general ,"" matrix which sets learning/decay rate for each synapse or neuron, we consider the simpler case where each is scalar, shared across all synapses.
.7 and À can each be
Most importantly, we parameterize the update rule itself. Each of the pre- and postsynaptic firing rates is linearly transformed before the synaptic weight is updated according to the prototypical "pre-times-post rule. The jth input neuron's transformed firing rate is given by
(**))), = 7f xj
(14)
and others f".g*. l are analogous. Although these functions can take arbitrary forms in general, this simple parameterization enables interpolating between traditional Hebbian, anti-Hebbian, and non-Hebbian (pre- post-only) rules. The update rules can be summarized as follows:
K (1 n K. n @ 10 (a.)))
(15)
V41 A o + gen O (g"()))
(16)
We begin by training this network on the benchmark task from subsection 3. parameters using stochastic gradient descent (Adam, (Kingma and Ba (014 Figure 34 shows the performance of the meta-learned algorithms in comparison to the corresponding simplified versions (section 2) for either sequential and random local third factors. Importantly, removing the unrealistic rapid synaptic decay in the hidden-to-output synapses and replacing with a passive decay does not significantly impact performance, long the decay rate appropriately set. Indeed, this even slightly improves performance for longer sequences.
optimizing these
haal--hirdd-accto---eeddd
In the case of random local third factor, we also compare values of (not trained). Empirically, we find that x 4/N produces desirable performance: ensures that the probability no local third factor occurring (and therefore storage) is small (< 2%) while minimizing overwriting. Computing the capacity (Figure 3p), we see that there an advantage for the probability to scale with the network size (p- 4/N) rather being fixed value (p = 0.1).
We next investigate the training trajectories. Figuree 3p shows the loss and accuracy curves over the course of training for networks with sequential and random local third factors. Training data

---

(a)
(b)
Sequential
150
Sequenbal C-1 0100
Trained
Capacity (98% acc)
Random 4/N C-0.183N
Random 0.025
Random 0.1 C-0.095N
Acouracy
Trained
100
Random
Trained
50
0.7
0
120
Number stimuli
Network size
(c)
40
Sequential
(d)
Inpul-to-hidden, pre-synaplic
Hidden-lo-oulpul, pre-synaplic
35
Rendom
3000
wmmm
30
25
20
15
MMW
Iterations
Inpul-to-hidden, post-synaptic
Hidden-to-oulput, post-synaptic
0.9
0.8
Accurapy
0.4
0.7
60
0.2
0.6
0.0
0.5
0.0
0.5
1000
2000
3000
lterations
Figure 3: (a) Performance comparison of simple (dashed curves) and meta- (solid) network (d N 40, p varied, shown in legend.) (b) Capacity of trained network with sequential or random local third factor. In the random case, p is either fixed or scales with N. (c) Training the sequential and random (p 0.1) network from (a.b). Although accuracy is not 1.0 during training, the network successfully generalizes to unseen sequence lengths. (d) Learning trajectory of the plasticity parameters in the sequential network converges to qualitatively similar solutions as the original simplified network (see also Figure C4). Note that the input-to-hidden plasticity is independent of the post-synaptic firing rate (bottom left plot, 79* R 0)
consists of sequence lengths between T = N/2 and T = 2N (above capacity for both versions of the network), so accuracy does not reach 1.0. Nevertheless, the network successfully generalizes to lengths outside of this range, indicating robust mechanism for memory storage.
Most importantly,
examine the plasticity rule discovered by optimization, Figure 3 shows the slope 7 and offset b parameters for the firing rate transfer functions g² ,", I' in the sequential algorithm over the course of meta-leaming. The plasticity rule for the input-te -ot-ttto-hddenn connections Figure 3/1, left) becomes pre-dependent (a ~ 0.5, T 0) but not post-dependent (Ø9
30 0.5), analogous to the idealized plasticity rule described in section 2 Similarly, the plasticity rule for the hidden-to-outpur connections becomes Hebbian (both pre-dependent and post-dependent, 30" T5" z 0, but " = a 1). The random version shows qualitatively similar trained behavior
Figuur C4).
Since the performance and parameterization of the meta-learned algorithm is almost identical to the simple case when using sequential local third factors, we only consider the random version for meta-learning in subsequent sections.

---

3.3 Continual, flashbulb, and correlated memory tasks
We next test our plasticity rules on more ecologically relevant memory tasks. These tasks reflect the complexity of biological stimuli and the functionality needed for versatile memory storage.
(a)
Continual recall
(c)
Correlation=0.6
1.0
Sequential
1.00
Sequential
0.9
Random
0.95
Random
Accuracy
Trained
Accuracy
Trained
0.8
Hopfield
0.90
Hopfield
0.7
0.85
0.6
0.80
0
25
50
75
100
125
150
175
50
75
100
125
150
175
Delay interval
Number of stimuli
(b)
Flashbulb: Sequential
Random
Hopfield
1.0
Acouracy
Regular
Flashbulb
100
100
150
Delay interval
Delay interval
Delay interval
Figure 4: (a) Performance on continual recall task, accuracy measured as a function the number of timesteps between the storage and recall of memory. "Trained" corresponds to the meta-learned algorithm with random local third factors. (b) Same as (a), but including five "flasbull" memories. Performance of simplified algorithm with meta-plasticity, using sequential (left) or random (middle) local third factors. For Hopfield network (right), increasing darkness of each line corresponds to higher write strength for flashbulb memories, with values 10, 50, 10², and 10 (c) Same as Figure 2h. but with memories having correlation of 0.6.
In realistic scenarios, memories are stored and recalled in a continual manner the subject sees an ongoing stream of inputs and is required recall stimulus that shown some time ago (Appendix A) rather than being presented a full dataset to memorize before testing, as in the benchmark autoassociative task. Figure A2h shows such sample dataset with delay interval of 2. Our learning algorithm naturally suited for this task due to its decay mechanisms recent stimuli stored, but older ones forgotten. Its accuracy decreases in steps of width N due to the sequential nature of the local third factor (Appendix A). Performance with random third factors decays smoothly since writing is stochastic, but both networks show better overall performance than the Hopfield network FFguure 4
Next, we consider "flashbulb" memories, phenomenon where highly salient experiences are remem- bered with extreme efficacy, often for life [Brown and Kulik 1977). Functionally, these memories may be used to avoid adverse experiences seek out rewarding ones. We modify the continual recall task such that a small number stimuli are deemed salient and accompanied by a stronger global third factor (q: 10) akin to a boosted neuromodulatory influence learning (Figure A2p) We add a simple meta meta--tasticyty mechanism to our plasticity rule, a stability parameter [S.lk for each synapse, initially set to O. If the learning rate for that synapse crosses threshold [m.ky > 1, then [S,= = for all & >> 0 and suppresses subsequent plasticity events. Thus, the learning rate for the key matrix is as follows (learning rate n). is analogous):
In. by = (1 - [S )4 (rel:
(17)
*For fair comparison, we introduce an empirically chosen synaptic decay parameter 0.95 the Hopfield network. We also considered version of the Hopfield network with bounded weights (Parisi] 1986). designed for continual learning. but found that synaptic weight decay has better overall performance. Furthermore, we include global third factor which controls overall plasticity identically model.

---

With this meta- plaatticcit,ty the network retains flashbulb memories with minimal effect on its recall performance on regular memory items (Figure 4p, left, middle). To perform this task in the Hopfield network, introducing synaptic stability is significantly detrimental to performance, so we simply store flashbulb memories as large-magnitude updates to the weight matrix. As a result, it exhibits a tradeoff between storage of regular items and efficacy of flashbulb memories (Figure 4p, right). Thus, an important benefit of the slot-based storage scheme is that flexible treatment of individual memories can be naturally implemented.
Finally, real-world stimuli are often spatially and temporally correlated - for instance, two adjacent video frames are almost identical. Storing such pattemss in a Hopfield network causes interference between attractors in the energy function, decreasing its capacity. In contrast, by storing stimuli in distinct slots of the key and value matrices, key-value memory can more easily differentiate between correlated but distinct memories. To verify this, we use a correlated dataset by starting with a template vector and generating stimuli by randomly flipping some fraction of its entries (Figure A2c). The performance of the plasticity rule is similar to that for uncorrelated data (Figure 20), with minor degradation due to spurious recall of similar stored memories (Figure 4c). Figure shows similar results for varying correlation strengths.
3.4 Heteroassociative and sequence memory
The network and learning mechanism is agnostic the relationship between the input and target patterns, and so naturally generalizes to heteroassociative memory. To draw comparisons with Hopfield-type networks, we consider the Bidirectional Associative Memory (BAM) (Kosko| 1988], a generalization of the Hopfield network designed for heteroassociative recall. We evaluate the networks modified version of the recall task from kubsection 3.1 where values are distinct from keys, y/: E (11, - for d * m (Figure 5p, top: Figure A3p. Compared to the autoassociative task (Figure? 2). the Hopfield-type network's performance on the heteroassociative task deteriorates Figuure 5 bottom). On the other hand. the performance of our network remains unaffected in the heteroassociative task, as its factorized key-value structure allows more flexibility in the types of memory associations learned.
A more biologically relevant version of heteroassociative memory is sequence learning. Experimental and theoretical evidence suggests that hippocampal memory can serve as a substrate for planning and decision making through the replay of sequential experiences [Foster and Wilson 2006 Pfeiffer
and Foster 2013 Mattar and Daw 2018). As a simple probe for a similar functionality. we use a sequence recall task where the network is presented with a sequence of patterns to memorize, using the value at time f as the key at time + 1. Afterwards, it is prompted with a random pattern from the sequence and tasked with recalling the rest. By adding recurrent loop, using the output Yr at time as the input xt+1 at the next timestep (Figure 5p. top), our network with sequential local third factors can perform sequence learning without error until its capacity limit (Figure 5b, bottom). BAM does not perform as well, likely due to propagating errors from incorrect recall of earlier patterns in the sequence.
Finally, we consider a more complex task that requires our memory module to be integrated within larger system, demonstrating the modular nature of our memory network. In the "copy- paste" task [Graves et al., 2014). the system must store a variable-length sequence of patterns ((11, 82.......... ST+1) ) and output this sequence when the end-of-sequence marker is seen (Figure A3). We train an external network as a "controller" to generate 2t, yt. and qe (Figure 5b. top) (for BAM., qi scales the magnitude of the Hebbian update). With this controller, our network with sequential local third factors successfully learns the task and generalizes outside the sequence lengths seen (Figure 3p. bottom). The random and trained networks do not perform as well, likely due to lower memory capacity. BAM successfully learns the task, but is not able to generalize outside the sequence lengths seen in training.
4 Discussion
We proposed models of biological memory by taking inspiration from key-value memory networks used in machine learning. These biologically plausible models use combination of Hebbian and non-Hebbian three-factor plasticity rules to approximate key-value memory networks. Due to the flexibility of their structure, they can naturally be adapted for biologically relevant tasks. Our

---

(a)
(b)
(c)
1/
3/x
FF
FF
Heteroassociative Recall
Sequence Recall
Copy-Paste
1.0
Sequential
1.0
1.0
0.9
Random
Accuracy
Trained
Accuracy
Accuracy
0.8
0.8
0.8
BAM
0.7
0.6
20
40
20
40
Number of stimuli
Number patterns
Number of patterns
Figure 5: (a) Architecture (d = N 40, 20) and performance heteroassociative version
of the benchmark task (Figure 2). (b) Recurrent architecture modification for sequence recall (d - N sequence presented. (c) Network is embedded in a larger system with a feedforward network to perform the copy-paste task = N 40, 40). Dashed line is the maximum number of patterns shown during training.
40, 40), Fz41 y:- Accuracy plotted as a function of the length of the entire
results suggest an alternative framework of biological long-term memory that focuses on feedforward computation, opposed to recurrent attractor dynamics. Importantly, both our hand-designed and meta-learned results propose a role for recently discovered non-classical plasticity rules [Bittner et al.) propose
2017) that are advantageous for this type of feedforward computation. Furthermore, an architecture where individual memories are stored more independently of each other than in Hopfield networks. Such factorized design creates opportunities for more versatile use and control of memory.
Several questions remain. First, although long-term memory describes many categories of memory supported by various brain regions, we have not disambiguated these differences and their implications on testing and interpreting as a true model of the brain, it is critical to make direct comparisons between our algorithm and experimental findings in memory-related behavior and neural activity. Furthermore, our aim is not to be competitive with state-of-the-as memory networks but rather to provide a biologically realistic learning algorithm for MHNs that is par with classical Hopfield networks. To this end, focus on artificial stimuli with simple statistical structures; remains unclear how our models will perform with more complex and naturalistic data, or how they compare to their non- non-biological counterparts.
model. To validate our network not merely plausible model but
Taken together, our results take neuroscience-minder approach to connect two lines of work in memory networks that have been largely disparate.
5 Acknowledgements
We are particularly grateful for the mentorship of Larry Abbott. We also thank Stefano Fusi, James Whittington, Emily Mackevicius, and Dmitriy Aronov for helpful discussions. Thanks to David Clark for bringing Bidirectional Associative Memory to our attention. Research supported by NSF NeuroNex Award DBI-1707398, the Gatsby Charitable Foundation, and the Simons Collaboration for the Global Brain. G.R. Y. was additionally supported as a Simons Foundation Junior Fellow. C.F. was additionally supported by the NSF Graduate Research Fellowship.
10

---

References
D. J. Amit. Modeling brain function: The world of attractor neural networks. Cambridge university press, 1992.
D. Amit, H. Gutfreund, and H. Sompolinsky. Storing infinite numbers of patterns in spin-glass model of neural networks. Physical Review Letters, 55(14):1530, 1985.
D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations, 2015.
A. Banino, P. Badilea, Köster, M. Chadwick, Zambaldi, D. Hassabis, C. Barry, M. Botvinick, D. Kumaran, and C. Blundell. Memo: A deep network for flexible combina- tion of episodic memories. arXiv preprint urXiv:2001.1091: 2020.
S. Bartunov, J. W. Rae, S. Osindero, and T. P. Lillicrap. Meta-learning deep energy-based memory models. arXiv preprint arXiv:1910.02720, 2019.
K. C. Bittner, A. D. Milstein, C. Grienberger, S. Romani, and J. C. Magee. Behavioral time scale synaptic plasticity underlies cal place fields. Science, 357(6355):1033-1033 2017.
R. Brown and J. Kulik. Flashbulb memories. Cognition, (11):73-99, 1977.
T. M. Cover. Geometrical and Statistical Properties of Systems of Linear Inequalities with Applica- tions in Pattern Recognition, IEEE Trunsactions on Electronic Computers, EC-14(3):326-3344 June 1965. ISSN 0367-7508. Conference Name: IEEE Transactions on Electronic Computers.
M. Demircigil, J. Heusel, M. Löwe, S. Upgang, and F. Vermet. On Model of Associative Memory with Huge Storage Capacity. Journal of Statistical Physics, 168(2):288822)) 2999, July 2017. ISSN 1572-9613.
D. J. Foster and M. A. Wilson. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature, 440:680-683, 2006.
F. Gambino, S. Pagès, V. Kehayas, D. Baptista, R. Tatti. A. Carleton, and A. Holtmaat. Sensory- evoked ltp driven by dendritic plateau potentials in vivo. Nature, 515(7525):116-11) 2014.
E. Gardner. The space of interactions in neural network models. Journal of physics A: Mathematical and generul, 21)::25, 1988.
W. Gerstner, M. Lehmann, V. Liakoni, D. Corneil, and J. Brea. Eligibility traces and plasticity on behavioral time scales: Experimental support of neohebbian three-factor learning rules. Frontiers in Neural Circuits, 12:53, 2018. ISSN 1662-5110.
A. Graves, G. Wayne, and I. Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5400 2014.
A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwee Barwiiikk,, S. G. Col- menarejo, E. Grefenstette, Ramalho, Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471-444 2016.
J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(3):2554-2558, 1982.
C.-C. Hung, T. Lillicrap. J. Abramson, Y. Wu, M. Mirza, F. Carnevale, A. Ahuja, and G. Wayne. Optimizing agent behavior over long time scales by transporting value. Nature communications, 10(1):1-12, 2019.
P. Kanerva. Sparse distributed memory. MIT press, 1988.
P. Kanerva. Sparse distributed memory and related models, volume 92. NASA Ames Research Center, Research Institute for Advanced Computer Science, 1992.
D. Kingma and J. Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.

---

B. Kosko. Bidirectional associative memories. IEEE Transactions on Systems, man, and Cybernetics, 18(1):99-60, 1988.
D. Krotov. Hierarchical Associative Memory, arXiv:2107.06-44 [cs], July 2021. arXiv: 2107.06446.
D. Krotov and J. Hopfield. Large associative memory problem in neurobiology and machine learning. arXiv preprint NXXv:2008.06999, 2020.
D. Krotov and J. Hopfield. Dense associative memory for pattern recognition. arXiv preprini orXiv:1606..00 164, 2016.
H. Le, T. Tran, and S. Venkatesh. Neural stored-progran memory. arXiv preprint arXiv:1906.08862, 2019.
M. G. Mattar and N. D. Daw. Prioritized memory access explains planning and hippocampal replay. Nature Neuroscience, 21:1609-1617, 2018. doi: 10.
M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
T. Munkhdalai, A. Sordoni, T. Wang, and A. Trischler. Metaleamed neural memory. arXiv preprini arXiv:190..09720, 2019.
G. Parisi. A memory which forgets. Journal of Physics A: Mathematical and General, 19(10)L611,, 1986.
B. Pfeiffer and J. Foster. Hippocampal place-cell sequences depict future paths to remembered goals. Nature, 497:74-79, 2013.
H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandve, Greiff, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217, 2020.
D. D. Rasmusson. The role of acetylcholine in cortical synaptic plasticity. Behav Brain Res., 115(2):
205-218, 2000. doi: 10. 00.1066//0166--42880000000000000
A. Robins and S. McCallum. Catastrophic forgetting and the pseudorehearsal solution in hopfield- type networks. Connection Science, 1012): 1998. doi: 10. 00.10800095540098111655 URI.
https://doi.onn orw.iiorroo1.. 090g//00.1000//44409991111100000
N. Rodriguez, E. Izquierdo, and Y.-Y. Ahn. Optimal modularity and memory capacity of neural reservoirs. Network Neuroscience, 3(2):551-566, Jan. 2019. ISSN 2472-1751.
G. J. Stuart and N. Spruston. Dendritic integration: 60 years of progress. Nature neuroscience, 18 (12):1713-1721, 2015.
Sukhbaatar, A. Szlam, J. Weston, and R. Fergus. End-to-end memory networks. arXiv preprint arXiv:1503.08895, 2015.
D. Tyulmankov, G. R. Yang, and L. Abbott. Meta-learning local synaptic plasticity for continual familiarity detection. bioRxiv, 2021. doi: 10. 00..0. .436277.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, F.. Kaiser, and Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998 -008-6008, 2017.
Weston, A. Bordes, S. Chopra, A. M. Rush, B. van Merriënboer, A. Joulin, and Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv: 1502.05698. 2015.
D. J. Willshaw, O. P. Buneman, and H. C. Honguet-Higgins Non-holographic associative memory. Nature, 222(5197):960 962, 1969.
12


Reviews:
[{"review_id": "a370599GoC", "reviewer": "Reviewer_3w4j", "summary": "The authors study a set of synaptic plasticity rules for Key-Value Networks (KVN), which is also a variant of the Modern Hopfield Network (MHN).  The authors propose a biologically-plausible mechanism for writing keys and values into synaptic weights using Hebbian and non-Hebbian plasticity rules. Similar plasticity rules arise as a result of a meta learning algorithm. The authors also claim that this submission implements KVNs with neurons. However, as I explain below, this has already been done in a prior work. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper studies a set of plasticity rules for the key and value synaptic weights and demonstrates the advantage of the Key-Value Network (a limit of the Modern Hopfield Network) over the conventional Hopfield network from 1982.  The authors introduce local third factors that make it possible to write a key into synaptic weights of a single hidden unit and discuss possible biological realizations of this mechanism. All these results (sections 3.1, 3.2, 4) look novel, relevant to the NeurIPS community, and, from my perspective, justify the acceptance of this submission. \n\nHowever, I think the submission has some issues with the overall metamessage, and also misses some of the results presented in prior work. \n\n1. The model that the authors describe in section 3 (before section 3.1) is mathematically identical to the model studied in [KH2016], [KH2020] if that model is run for only one step in time. \n\n Consider figure 3 and equation (10) in [KH2016]. This is literally the the network shown in Fig1 of the present submission defined by equations (2) and (3). $K_t=\\xi^\\mu_i$, $V_t= \\xi^\\mu_\\alpha$, $f=$softmax, $g=$linear or sign, $v_i = \\mathbf{x_t}$, $c_\\alpha=\\mathbf{y_t}$. That network has also been described in terms of  hidden neurons in [KH2020]. I think the relationship between equations (2,3) and that prior work should be acknowledged in section 3.\n\n Also, heteroassociative setting (section 4.4) can be implemented by concatenating keys and values in the feature layer of the Modern Hopfield Network, as it was done in [KH2016] (keys = images, values = labels). The keys can be clamped, and the values and the hidden neurons can be updated for one step. \n\n\n2.  Key-Value Networks (KVNs) are a limiting case of the Modern Hopfield Networks (MHNs) that are run for one step update in time, and have an almost one-hot encoded output of the softmax, as pointed out by the authors.  For this reason the dichotomy between Hopfield Networks and the Key-Value Networks, which the authors are emphasizing throughout the paper, is in fact a dichotomy between two kinds of Hopfield networks (conventional Hopfield network from 1982) and a limit of the Modern Hopfield Network. For this reason it is confusing to say just Hopfield network when referring to the model from [H1982], because then a casual reader may get an incorrect impression that the authors are referring to any kind of Hopfield Network (modern or classical). See e.g. line 12, or 288 for the specific instances where this terminology leads to an incorrect understanding. \n\n This seems to be an easy to fix issue. For instance, the authors could add “classical” or “conventional” in front of the Hopfield network every time they refer to the model from [H1982]. And explain somewhere in the Introduction (or Related Work) the relationship between KVNs and MHNs, which they have already done in lines 57-67. \n\n But, most importantly, I don’t see this model as an alternative to Hopfield Networks (as the authors claim in lines 12-13). Moreover, this model is a limiting case of the Modern Hopfield Network with the specifically chosen write operation.  In other words, when the synaptic weights are fixed the model studied in this submission is identical to a one step update of the MHN. The novelty of this submission is in the plasticity rules, and the third factor, not the architecture.  At the same time, the beginning of section 3 is written as if the authors were introducing a new architecture. \n\n\n3. [Feedforward computation vs. recurrent attractor dynamics for long-term memory in lines 283-284] In the limit considered by the authors MHNs are converging to the fixed point attractor in one step, see section 5 of [KH2016] or theorem 4 in [RA2020]. For this reason there is no mathematical difference between recurrent convergence to an attractor and feedforward computation in this limit.  \n\nDespite these issues I think that the novel plasticity rules and the introduction of the third factors are interesting and valuable results. So is the meta learning of those plasticity rules. If the authors can address issues 1, 2, 3 above, I would vote accept for this submission. \n\n\n[H1982] Hopfield, PNAS 1982, Neural Networks and Physical Systems with Emergent Collective Computational Abilities\n\n[KH2016] Krotov and Hopfield, NeurIPS 2016, Dense Associative Memory for Pattern Recognition, https://arxiv.org/abs/1606.01164 \n\n[RA2020] Ramsauer et al., ICLR 2021, Hopfield networks is All You Need, https://arxiv.org/abs/2008.02217\n\n[KH2020] Krotov and Hopfield, ICLR 2021, Large Associative Memory Problem in Neurobiology and Machine Learning, https://arxiv.org/abs/2008.06996\n\n----------------------------------------------------------------------------------------------------------------------------------------------------\n\nPost-rebuttal comments. The authors have addressed my concerns. Overall, the paper presents interesting three-factor learning algorithms for the KVNs or MHNs networks. These results look novel and relevant to the NeurIPS community. I vote accept on this submission. I have updated my scores. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This is mostly an exploratory work, which is unlikely to lead to negative societal impacts in the near future. ", "review_text": "", "extra_scores": {"ethical_concerns": "no concerns", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "QVKXIY1U-ck", "reviewer": "Reviewer_Y8iA", "summary": "This paper proposes a key-value memory network based on Hebbian and non-Hebbian plasticity rules. It utilizes the key-value form for efficient memory read and write, and used Hebbian rules for biological plausible computation. Experiments demonstrate that the proposed approach outperforms Hopfield network with simulations.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The idea of using plasticity rules in a key-value memory network is interesting. The proposed approach also works with different tasks such as autoassociative recall, continual, flashbulb, and correlated memory tasks, which extends the ability of traditional key-value memory networks. \n\n2. There are also existing biological-inspired works for memory network, such as the Neural Turing Machines (NTM). What is the relationship between NTM and the proposed approach? comparison and discussions should be added.\n\n3. In the experiment, several tasks are involved and compared, while the comparison are mostly with traditional Hopfield networks. Comparisons should be carried out with more recent studies such as NTM and other key-value based approaches.\n\n4. In Figure 2 (a), the performance of Hopfield is higher when the number of stimuli increases up to 100, please explain.\n\n5. Some symbols are confusion. \n1) In equation (4) and (6), what do i and j refer to respectively? \n2) How to select the qt in equation (6)? \n3) In equation (12)-(13), what is the meaning of g?\n\n============================================\n\n\nPost-rebuttal:\nThe authors have addressed most of my concerns. I have increased my score from 5 to 6.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "0JLhJUyYA-8", "reviewer": "Reviewer_Fi9m", "summary": "In this article the authors propose a key-value memory model for long-term memory that incorporates Hebbian and non-Hebbian learning. The model is tested on auto- and hetero-associative memory tasks as well as sequence learning, where it performs similar or better than Hopfield networks and other memory networks. The model is also evaluated on a copy-paste task, by embedding the memory model in a larger neural network that acts as a controller.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: I did not find enough of novelty in using Hebbian plasticity for fast weight memory modules. However, the proposed synaptic update rule is quite interesting, sufficiently simple, and it does the job well. The proposed model per se is quite simple and performed well, despite using only feedforward computation, as opposed to recurrent attractor dynamics as in e.g Hopfield networks. The authors also nicely describe the advantage of their model over Hopfield networks. The authors reference prior work throughout the paper, while also emphasizing how the current implementation is different from those approaches. One recent paper that the authors may want to address is Limbacher T, Legenstein R, H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks, NeurIPS 2020, where a similar key-value memory approach is taken.\n\nQuality: Some details are missing (see additional feedback). Other than that the empirical methodology appears standard, and is reported in sufficient detail to recreate the results. The resulting claims are justified by the performance of the model. Strengths and limitations are sufficiently discussed.\n\nClarity: The writing is clear and succinct. Few, minor grammatical errors/typos. One more careful proof read would be good.\n\nSignificance: While the results are interesting to those focused on biologically inspired models, the current framing may be too narrow for the NeurIPS community. To clearly demonstrate that their model is of more general interest it would be necessary to try the model on a wider range of more naturalistic tasks and to compare it to memory augmented networks from the machine learning literature.\n\nAdditional feedback: The learning rate $\\eta_t^v$ in Equation 9 seems to be undefined. Is it equal to $\\eta_t^k$ (i.e., does it contain the local third factor)? If yes, then the statement \"..it may not be realistic for the axons of the same neurons to be affected by these plateau potentials [local third factor]\" would make sense. However, in Equation 10, where the learnable parameter $\\tilde{\\eta}^v$ is defined, $\\eta_t^v$ is not dependent on the local third factor. This is somehow confusing. For the forgetting mechanism (Eq. 11), since the state is almost one-hot, couldn't it be used in the forgetting mechanism instead of using a fixed factor? The authors do not state how the weights matrices $K$ and $V$ are initialized. I assume that they are zero-initialized, no? For the copy-paste task, the authors say that they embedded the memory model in a feedforward neural network that acts as a controller. In Figure 5c, the feedforward network seems to have feedback connections. To my understanding, a feedforward system can't have feedback connections.\n\n---\n\nUPDATE AFTER REBUTTAL: The authors raised convincing points, intend to address the various concerns of the reviewers, and I have raised my score.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a biologically plausible key-value-type memory model using Hebbian and non-Hebbian plasticity, related to Hopfield networks and modern Hopfield networks.

The reviewers agree that the proposed model is interesting. Key-value-type memory is important for many artificial neural network models (Memory-augmented neural networks). The question whether such memory could be implemented through biological mechanisms is thus well-justified. The paper is well-written, and presents convincing simulation results.

In the first reviews, there were some questions regarding the relation to Hopfield/ modern Hopfield networks. These issues could be resolved with the author's responses.

On the negative side, it was noted that the model could be tested on a wider range of more naturalistic tasks. Nevertheless, the manuscript is suitable for publication at NeurIPS.
Decision: Accept (Poster)