Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The authors study a set of synaptic plasticity rules for Key-Value Networks (KVN), which is also a variant of the Modern Hopfield Network (MHN).  The authors propose a biologically-plausible mechanism for writing keys and values into synaptic weights using Hebbian and non-Hebbian plasticity rules. Similar plasticity rules arise as a result of a meta learning algorithm. The authors also claim that this submission implements KVNs with neurons. However, as I explain below, this has already been done in a prior work. 
main_review: The paper studies a set of plasticity rules for the key and value synaptic weights and demonstrates the advantage of the Key-Value Network (a limit of the Modern Hopfield Network) over the conventional Hopfield network from 1982.  The authors introduce local third factors that make it possible to write a key into synaptic weights of a single hidden unit and discuss possible biological realizations of this mechanism. All these results (sections 3.1, 3.2, 4) look novel, relevant to the NeurIPS community, and, from my perspective, justify the acceptance of this submission. 

However, I think the submission has some issues with the overall metamessage, and also misses some of the results presented in prior work. 

1. The model that the authors describe in section 3 (before section 3.1) is mathematically identical to the model studied in [KH2016], [KH2020] if that model is run for only one step in time. 

 Consider figure 3 and equation (10) in [KH2016]. This is literally the the network shown in Fig1 of the present submission defined by equations (2) and (3). $K_t=\xi^\mu_i$, $V_t= \xi^\mu_\alpha$, $f=$softmax, $g=$linear or sign, $v_i = \mathbf{x_t}$, $c_\alpha=\mathbf{y_t}$. That network has also been described in terms of  hidden neurons in [KH2020]. I think the relationship between equations (2,3) and that prior work should be acknowledged in section 3.

 Also, heteroassociative setting (section 4.4) can be implemented by concatenating keys and values in the feature layer of the Modern Hopfield Network, as it was done in [KH2016] (keys = images, values = labels). The keys can be clamped, and the values and the hidden neurons can be updated for one step. 


2.  Key-Value Networks (KVNs) are a limiting case of the Modern Hopfield Networks (MHNs) that are run for one step update in time, and have an almost one-hot encoded output of the softmax, as pointed out by the authors.  For this reason the dichotomy between Hopfield Networks and the Key-Value Networks, which the authors are emphasizing throughout the paper, is in fact a dichotomy between two kinds of Hopfield networks (conventional Hopfield network from 1982) and a limit of the Modern Hopfield Network. For this reason it is confusing to say just Hopfield network when referring to the model from [H1982], because then a casual reader may get an incorrect impression that the authors are referring to any kind of Hopfield Network (modern or classical). See e.g. line 12, or 288 for the specific instances where this terminology leads to an incorrect understanding. 

 This seems to be an easy to fix issue. For instance, the authors could add “classical” or “conventional” in front of the Hopfield network every time they refer to the model from [H1982]. And explain somewhere in the Introduction (or Related Work) the relationship between KVNs and MHNs, which they have already done in lines 57-67. 

 But, most importantly, I don’t see this model as an alternative to Hopfield Networks (as the authors claim in lines 12-13). Moreover, this model is a limiting case of the Modern Hopfield Network with the specifically chosen write operation.  In other words, when the synaptic weights are fixed the model studied in this submission is identical to a one step update of the MHN. The novelty of this submission is in the plasticity rules, and the third factor, not the architecture.  At the same time, the beginning of section 3 is written as if the authors were introducing a new architecture. 


3. [Feedforward computation vs. recurrent attractor dynamics for long-term memory in lines 283-284] In the limit considered by the authors MHNs are converging to the fixed point attractor in one step, see section 5 of [KH2016] or theorem 4 in [RA2020]. For this reason there is no mathematical difference between recurrent convergence to an attractor and feedforward computation in this limit.  

Despite these issues I think that the novel plasticity rules and the introduction of the third factors are interesting and valuable results. So is the meta learning of those plasticity rules. If the authors can address issues 1, 2, 3 above, I would vote accept for this submission. 


[H1982] Hopfield, PNAS 1982, Neural Networks and Physical Systems with Emergent Collective Computational Abilities

[KH2016] Krotov and Hopfield, NeurIPS 2016, Dense Associative Memory for Pattern Recognition, https://arxiv.org/abs/1606.01164 

[RA2020] Ramsauer et al., ICLR 2021, Hopfield networks is All You Need, https://arxiv.org/abs/2008.02217

[KH2020] Krotov and Hopfield, ICLR 2021, Large Associative Memory Problem in Neurobiology and Machine Learning, https://arxiv.org/abs/2008.06996

----------------------------------------------------------------------------------------------------------------------------------------------------

Post-rebuttal comments. The authors have addressed my concerns. Overall, the paper presents interesting three-factor learning algorithms for the KVNs or MHNs networks. These results look novel and relevant to the NeurIPS community. I vote accept on this submission. I have updated my scores. 

limitations_and_societal_impact: This is mostly an exploratory work, which is unlikely to lead to negative societal impacts in the near future.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this article the authors propose a key-value memory model for long-term memory that incorporates Hebbian and non-Hebbian learning. The model is tested on auto- and hetero-associative memory tasks as well as sequence learning, where it performs similar or better than Hopfield networks and other memory networks. The model is also evaluated on a copy-paste task, by embedding the memory model in a larger neural network that acts as a controller.
main_review: Originality: I did not find enough of novelty in using Hebbian plasticity for fast weight memory modules. However, the proposed synaptic update rule is quite interesting, sufficiently simple, and it does the job well. The proposed model per se is quite simple and performed well, despite using only feedforward computation, as opposed to recurrent attractor dynamics as in e.g Hopfield networks. The authors also nicely describe the advantage of their model over Hopfield networks. The authors reference prior work throughout the paper, while also emphasizing how the current implementation is different from those approaches. One recent paper that the authors may want to address is Limbacher T, Legenstein R, H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks, NeurIPS 2020, where a similar key-value memory approach is taken.

Quality: Some details are missing (see additional feedback). Other than that the empirical methodology appears standard, and is reported in sufficient detail to recreate the results. The resulting claims are justified by the performance of the model. Strengths and limitations are sufficiently discussed.

Clarity: The writing is clear and succinct. Few, minor grammatical errors/typos. One more careful proof read would be good.

Significance: While the results are interesting to those focused on biologically inspired models, the current framing may be too narrow for the NeurIPS community. To clearly demonstrate that their model is of more general interest it would be necessary to try the model on a wider range of more naturalistic tasks and to compare it to memory augmented networks from the machine learning literature.

Additional feedback: The learning rate $\eta_t^v$ in Equation 9 seems to be undefined. Is it equal to $\eta_t^k$ (i.e., does it contain the local third factor)? If yes, then the statement "..it may not be realistic for the axons of the same neurons to be affected by these plateau potentials [local third factor]" would make sense. However, in Equation 10, where the learnable parameter $\tilde{\eta}^v$ is defined, $\eta_t^v$ is not dependent on the local third factor. This is somehow confusing. For the forgetting mechanism (Eq. 11), since the state is almost one-hot, couldn't it be used in the forgetting mechanism instead of using a fixed factor? The authors do not state how the weights matrices $K$ and $V$ are initialized. I assume that they are zero-initialized, no? For the copy-paste task, the authors say that they embedded the memory model in a feedforward neural network that acts as a controller. In Figure 5c, the feedforward network seems to have feedback connections. To my understanding, a feedforward system can't have feedback connections.

---

UPDATE AFTER REBUTTAL: The authors raised convincing points, intend to address the various concerns of the reviewers, and I have raised my score.

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a key-value memory network based on Hebbian and non-Hebbian plasticity rules. It utilizes the key-value form for efficient memory read and write, and used Hebbian rules for biological plausible computation. Experiments demonstrate that the proposed approach outperforms Hopfield network with simulations.
main_review: 1. The idea of using plasticity rules in a key-value memory network is interesting. The proposed approach also works with different tasks such as autoassociative recall, continual, flashbulb, and correlated memory tasks, which extends the ability of traditional key-value memory networks. 

2. There are also existing biological-inspired works for memory network, such as the Neural Turing Machines (NTM). What is the relationship between NTM and the proposed approach? comparison and discussions should be added.

3. In the experiment, several tasks are involved and compared, while the comparison are mostly with traditional Hopfield networks. Comparisons should be carried out with more recent studies such as NTM and other key-value based approaches.

4. In Figure 2 (a), the performance of Hopfield is higher when the number of stimuli increases up to 100, please explain.

5. Some symbols are confusion. 
1) In equation (4) and (6), what do i and j refer to respectively? 
2) How to select the qt in equation (6)? 
3) In equation (12)-(13), what is the meaning of g?

============================================


Post-rebuttal:
The authors have addressed most of my concerns. I have increased my score from 5 to 6.

limitations_and_societal_impact: Yes.
review_title: null

Meta-review: The authors propose a biologically plausible key-value-type memory model using Hebbian and non-Hebbian plasticity, related to Hopfield networks and modern Hopfield networks.

The reviewers agree that the proposed model is interesting. Key-value-type memory is important for many artificial neural network models (Memory-augmented neural networks). The question whether such memory could be implemented through biological mechanisms is thus well-justified. The paper is well-written, and presents convincing simulation results.

In the first reviews, there were some questions regarding the relation to Hopfield/ modern Hopfield networks. These issues could be resolved with the author's responses.

On the negative side, it was noted that the model could be tested on a wider range of more naturalistic tasks. Nevertheless, the manuscript is suitable for publication at NeurIPS.
Decision: Accept (Poster)