Paper: Correlated dense associative memories (ICLR-2024)

Reviewer comment (Writing):
In the Abstract, the authors claim that "[their] results have implications for both machine learning and neurobiology." However, after reading the paper I was left wondering what precisely those implications were.

Author response:
Reviewer oKZA found that the implications for machine learning and neurobiology were unclear. In conjunction with additional experiments, we have added to the Introduction and Discussion to highlight our contributions.

Reviewer comment (Novelty):
The overall motivation of the paper is, to me, rather unclear. Yes, the paper extends past work on sequence modeling using heteroassociative memories, but it is not made clear why this extension is of interest. This lack of clarity is accentuated by the fact that the simulations are not woven together into a coherent story. Rather, they read as independent vignettes, and it is not made clear why each is scientifically interesting.

Author response:
Thank you for the feedback. We have now overhauled major parts of the paper to more clearly communicate our results and tell a coherent, overarching story.

Reviewer comment (Novelty):
In the list of contributions, the authors make an interesting claim regarding links between their model and studies of inhibitory plasticity in neuroscience. However, this link is never made concrete later in the paper. If there is a deeper link to neuroscience experiment, expanding on this point could help resolve my concerns regarding overall motivation, framing, and significance.

Author response:
To make this more concrete, we discuss our motivations from neuroscience in the introduction and replicate data from a neuroscience experiment.

Reviewer comment (Theory):
The weighted combination of autoassociative and heteroassociative terms in the update rule (2) resembles that used in previous works on discrete-state sequence models, c.f. the cited work of Kleinfeld & Sompolinsky (1988), work by Kanter & Sompolinsky (1986, Temporal Association in Asymmetric Neural Networks, not cited in the submitted manuscript), or eq. 13 of the cited work by Chaudhry et al. (2023). In those sequence models, tuning the relative weighting of the associative and heteroassociative terms changes the dwell time of the system in a given state. Here, no clear motivation for the inclusion of the autoassociative term is given. The authors' empirical results (see for instance Figure 2) are consistent with the intuition that networks with small $b$ dwell near the initial state for longer, but they do not probe this carefully. Indeed, I think the authors' results on how varying $b$ determines which scales are "selected" are more precisely explained in terms of an increase in dwell time. This would be easy to probe by examining intermediate timesteps. In particular, they should examine how their model's dynamics relate to diffusion on the graph. This is also relevant to the question of what the overall objective of the proposed heteroassociative learning rule is. Finally, the parameterized weighting in (2), with $b$ and $1-b$, is a bit puzzling given that the authors proceed to consider $b = -100$; at this scale they are effectively considering $b$ and $-b$ as the weights. In short, more specific motivation for the combined update rule should be provided (including for why the authors are interested in continuous memories in this particular setting), and the resulting behavior probed more carefully.

Author response:
We have expanded on our motivations from both neuroscience and machine learning in the introduction.

Our initial inclination was to construct our model such that we could rigorously test it while varying a single parameter, $b$. However, as the reviewers noted, this became less natural outside the range of $b \in [0,1]$. So, while parsimony of this kind has its advantages, we completely agree with the reviewers that it offers little advantage in this case. We therefore separated the auto- and hetero-associative strengths into two parameters – $a$ for the auto-associative weighting and $h$ for the hetero-associative weighting. This reparameterization further aided us in the theoretical analysis of the model.

Reviewer comment (Experiments):
In a similar vein to the above, the simulations do not probe some of the most basic properties of associative memories. How does the capacity of their model scale with the number of neurons? Related to point (3) above, how quickly does the model converge to each target state?

Author response:
When $a, h \neq 0$, regular notions of ‘capacity’ no longer seem fitting. This is because ‘capacity’ is normally measured in the pure auto-associative case by giving a noise-corrupted or partial memory pattern, and observing whether and how closely the model's dynamics converge to the uncorrupted or complete memory pattern (e.g., see Amit et al. 1985 for the classical model and Demircigil et al. 2017 for the dense model). In the pure hetero-associative case, ‘capacity’ has (to our knowledge) only ever been studied in the linear sequences case (e.g., see Löwe 1998 for the classical model and Chaudhry et al. 2023 for the dense model). However, in our model we study general mixtures of both auto- and hetero-association, as well as arbitrary memory graphs (not just linear cycles). It is therefore unclear whether there exists an appropriate notion of ‘capacity’ for our model.

One can, however, study the model in a similar spirit of analysis. To this end, we have completed theoretical and numerical work to demonstrate the dynamics of our model in the thermodynamic limit. By analysing the energy function, we show how (under certain assumptions of the memory graph) the network can exhibit one of four dynamical modes – auto-association, narrow hetero-association, wide hetero-association, and neutral quiescence – as controlled by choices of $a$ and $h$. Additionally, we provide some propositions on how parameter choices and the memory graph structure influences dynamics.

Reviewer comment (Presentation):
The introduction barely discusses the contributions of this work; the contributions are relegated to single-sentences at the end with few details. It’s hard to interpret the results or impact of this work from the intro-- much of it reads like a background.

Author response:
We appreciate Reviewer TfVu’s praise of the introduction (under ‘Strengths’) insofar as it provided a `good overview` and `crisp summary` of the field, its intuitions, progress, and open problems. Nevertheless, we recognise it is helpful if the introduction also incorporates details about the results and impact of the current work. We have therefore revised the introduction section to integrate further details of this paper’s contributions, including two new sub-sections (one for neuroscience, one for machine learning).

Reviewer comment (Presentation):
The results section is hard to parse; it’s unclear to me what research questions the authors are trying to answer with their results section. I suggest adding an overview section and explicitly stating the goals of your experiments.

Author response:
We thank Reviewer TfVu for the suggestion; we have added an overview subsection to the beginning of the results section, where we explicitly state the goals of each experiment.

Reviewer comment (Experiments):
Experimental design is poorly discussed. It's difficult to find basic information about the experiments, such as what dataset is used, or what refutable hypotheses each experiment is testing. The authors mention that they used both random and partially-random memory patterns; in the partially-random case they "use some real-world data." Where does this real-world data come from and what are its structural properties?

Author response:
In our initial submission, no real datasets were used – all data were simulated*. In our original submission (in the final paragraph of section 3.3, titled ‘Oriented recall in a symmetric regime’) we said that if a modeller wishes to dedicate some of the memory pattern vector contents (e.g., half) to intentionally create correlations between some patterns, this would reduce the amount of usable memory (a fact reflected in the reduced capacity of classical associative memory networks, e.g., see Theorem 2.2 of Löwe, Annals of Applied Probability 1998). In the case of random patterns (as we used in that experiment), this matters little since we are only interested in the dynamics of the network, not the actual contents of the random memories. It was in this context that we stated `The downside of this technique is that it reduces the usable content of memory vectors, e.g., where instead of using random memory patterns we use some real-world data.` However, the wording `we use` is confusing, so we have edited this sentence to clarify that we used random memory patterns in that experiment, not real-world data.

*Based on suggestions from reviewers, in our revision we include new experiments with real data.

Reviewer comment (Evaluation):
There is insufficient discussion of the potential applications or usefulness of CDAM. The authors state that “our results have implications for both machine learning and neurobiology” but do not convincingly back up this claim. They discuss some connections to transformers in the future work section, but it is purely speculative.

Author response:
Reviewer TfVu comments that we did not adequately discuss potential applications or uses of CDAM. Therefore, we have added further discussion to address this, as well as added new theoretical and numerical analyses to demonstrate potential applications vis-à-vis including graph clustering and replicating a neuroscience experiment’s results. We have also further added to the Discussion section regarding the link to Transformers, as well as a subsection addressing neuroscience implications.

Reviewer comment (Novelty):
The novelty claim of this work is not immediately clear to me. Can you explain how CDAM qualitatively differs from the method and capabilities of hetero-associative MCHNs discussed by Millidge et al. (2022) which you cite?

Author response:
`The novelty claim of this work is not immediately clear to me. Can you explain how CDAM qualitatively differs from the method and capabilities of hetero-associative MCHNs discussed by Millidge et al. (2022) which you cite?`

In Millidge et al. (2022), section 3.5 describes how we may generally consider the relationships between auto- and hetero-associative models, and notes how Transformers’ attention mechanisms take the hetero-associative form. Appendix B of Millidge et al. (2022) goes on to use the simple example of splitting CIFAR10 and MNIST images into upper and lower halves, treating the upper-half as the query and the lower-half as the hetero-associated memory (or ‘value’, in the Transformer language). So, given the upper-half of a CIFAR10 or MNIST image, the network can recall the lower-half. This is where the work of Millidge et al. (2022) stops.

In the CDAM framework, we can consider the hetero-associative experiments shown in Millidge et al. (2022)’s Appendix B as one which sets the memory graph very specifically as a union of all memorised pairs of upper- and lower-half memories (assigned to two separate vertices) of CIFAR10 or MNIST images (where the the upper-half vertex has a directed edge to the lower-half vertex). In the current paper, we work with arbitrary memory graphs and demonstrate a much more general framework, with applications and connections to, e.g., graph clustering and replicating data from a neuroscience experiment.

Reviewer comment (Experiments):
The experiments are mostly run on randomly generated data. This makes its impact on real-world problems hard to judge. How does CDAM perform on real-world data, or how do you expect it to perform?

Author response:
2. & 3. Reviewer TfVu asks about performance on real-world data and tasks. 

We have now included new experiments which use real-world data. We hasten to add that applying associative memory models directly to real-world problems is normally inadvisable where domain-specific models exist. It is common, however, for researchers to marry existing, e.g., gradient-based, models with associative memory models, e.g., Saha et al. ‘End-to-end Differentiable Clustering with Associative Memories’ ICML 2023 and Auer et al. ‘Conformal Prediction for Time Series with Modern Hopfield Networks’ NeurIPS 2023. Such integrations constitute independent works in their own right, and are beyond the scope of this paper, which aims to investigate associative memory networks at a more fundamental level and from a more theoretical perspective, e.g., similar to Chaudhry et al. ‘Long Sequence Hopfield Memory’ NeurIPS 2023 or Burns & Fukai ‘Simplicial Hopfield networks’ ICLR 2023. Nevertheless, we acknowledge that it is helpful for applied researchers to see examples of potential applications, thus our additions.

More broadly, we wish to emphasise one of the theoretical aims of this paper is to provide evidence of the inherent computational capabilities of dense hetero-associative networks. This is scientifically interesting from the perspective of AI interpretability due to the connection between CDAM and the attention mechanism of Transformers. Our results suggest that single attention heads in Transformers could be performing any or all of the demonstrated computations, including simulating computations using combinations of auto- and hetero-association. This constitutes a notable addition to the recent literature on Transformers’ capabilities in this area, e.g., see Liu et al. ‘Transformers Learn Shortcuts to Automata’ ICLR 2023.

Reviewer comment (Evaluation):
On what types of real-world problems should I use CDAM, and why? You claim that your results show that CDAM can be used to model multi-scale representations of community structures in graphs, oriented recall in a symmetric connection regime, and temporal sequences with distractors. In which domains do these problems appear, and how does CDAM compare to current approaches?

Author response:
2. & 3. Reviewer TfVu asks about performance on real-world data and tasks. 

We have now included new experiments which use real-world data. We hasten to add that applying associative memory models directly to real-world problems is normally inadvisable where domain-specific models exist. It is common, however, for researchers to marry existing, e.g., gradient-based, models with associative memory models, e.g., Saha et al. ‘End-to-end Differentiable Clustering with Associative Memories’ ICML 2023 and Auer et al. ‘Conformal Prediction for Time Series with Modern Hopfield Networks’ NeurIPS 2023. Such integrations constitute independent works in their own right, and are beyond the scope of this paper, which aims to investigate associative memory networks at a more fundamental level and from a more theoretical perspective, e.g., similar to Chaudhry et al. ‘Long Sequence Hopfield Memory’ NeurIPS 2023 or Burns & Fukai ‘Simplicial Hopfield networks’ ICLR 2023. Nevertheless, we acknowledge that it is helpful for applied researchers to see examples of potential applications, thus our additions.

More broadly, we wish to emphasise one of the theoretical aims of this paper is to provide evidence of the inherent computational capabilities of dense hetero-associative networks. This is scientifically interesting from the perspective of AI interpretability due to the connection between CDAM and the attention mechanism of Transformers. Our results suggest that single attention heads in Transformers could be performing any or all of the demonstrated computations, including simulating computations using combinations of auto- and hetero-association. This constitutes a notable addition to the recent literature on Transformers’ capabilities in this area, e.g., see Liu et al. ‘Transformers Learn Shortcuts to Automata’ ICLR 2023.

Reviewer comment (Reproducibility):
Could you explain what anti-Hebbian learning rule you applied and why?

Author response:
`Could you explain what anti-Hebbian learning rule you applied and why?`

We used a symmetric rate-based rule, since this is a rate-based model without a notion of spike timing (for background, see Roberts & Leen, 2010 and Shulz & Feldman, 2013). In our initial parametrization of the model with $b$, when $b<0$ the model entered an anti-Hebbian auto-associative and Hebbian hetero-associative regime. Our initial inclination was to construct our model such that we could rigorously test it while varying this single parameter. However, as reviewers indicated (and we agreed), this is slightly unnatural given the structure of the model’s dynamics (Eq 2 in the initial submission, Eq 1 in the revision). We therefore separated the auto- and hetero-associative strengths into two parameters – $a$ for the auto-associative weighting and $h$ for the hetero-associative weighting. This reparametrization aids in interpretation of the model and avoids the unnaturalness present in the prior use of highly negative values of $b$. It also allows for any combination of anti-Hebbian and Hebbian rules applied to the auto- and hetero-association (with anti-Hebbian corresponding to negative values of $a$ or $h$, whereas Hebbian corresponds to positive values).

Reviewer comment (Experiments):
In the present form the paper demonstrates the possibility to write pre-defined patterns with a given graph connectivity H to CDAM and read the patterns from it. While this is of course a non-trivial capability of the model, and a first step in studying this model, it would be great to illustrate the utility of the proposed framework for some useful downstream applications that a graph community might be interested in.

Author response:
Reviewer K2Ug said, `it would be great to illustrate the utility of the proposed framework for some useful downstream applications that a graph community might be interested in.`

Generally speaking, applying associative memory models directly to real-world problems is inadvisable where domain-specific models exist. It is common, however, for researchers to marry existing, e.g., gradient-based, models with associative memory models, e.g., Saha et al. ‘End-to-end Differentiable Clustering with Associative Memories’ ICML 2023 and Auer et al. ‘Conformal Prediction for Time Series with Modern Hopfield Networks’ NeurIPS 2023. Such integrations constitute independent works in their own right, and are beyond the scope of our paper, which aims to investigate associative memory networks at a more fundamental level and from a more theoretical perspective, e.g., similar to Chaudhry et al. ‘Long Sequence Hopfield Memory’ NeurIPS 2023. Nevertheless, we have added theoretical and numerical analysis to show connections between our model and graph theory concepts and techniques, including a graph clustering experiment.

Reviewer comment (Novelty):
Could the authors imagine this network being helpful for some graph-related tasks that ML or graph community might be interested in?

Author response:
1. `Could the authors imagine this network being helpful for some graph-related tasks that ML or graph community might be interested in?`

Yes, we could. As mentioned above, we believe exploring this fully is outside of this first step’s scope. However, we have included theoretical and numerical analyses to help deepen the connection to graph-related tasks, such as graph clustering experiment and temporal sequence recall.

Reviewer comment (Theory):
I am not sure I understand how the model uses the "transition" term (with the matrix H in equation 2) in situations when one node on the graph is connected to 2 other nodes. How does the model decide which of the two connected nodes it should give more weight and transition to?

Author response:
2.  `How does the model decide which of the two connected nodes it should give more weight and transition to?`

This is decided by the graph’s (weighted) adjacency matrix. In the case of an unweighted graph, all connections have an equal, arbitrary value (conventionally $1$), and so from an active node the weighted transition is equally spread to all connected nodes. However, our method can also accommodate weighted graphs (by letting the adjacency matrix elements take values from $\mathbb{R}$), in which case the transitions can be unequal. As an example, we demonstrate our method applied to a weighted graph.

Reviewer comment (Theory):
I am a little skeptical about using the model with negative $\beta$. In that case the softmax in equation 2 will pick the pattern with the smallest overlap with the current state $S$. Why would this setting be useful?

Author response:
3. `Why would [setting $\beta<0$] be useful?`

We view this as useful only to aid theoretical understanding of the components and dynamics of dense associative memory networks and related models. When $\beta<0$, we can interpret this from the neuroscience perspective as an inversion of contributions to the excitatory-inhibitory balance, which is highly studied and important for a range of neuropsychiatric disorders, such as autism, epilepsy, and schizophrenia.

Reviewer comment (Experiments):
Do the authors see any interesting phenomena for $b$ in the range [0,1]? When $b$ is less than 0, as in most case studies presented, the model needs not only jump to the next pattern (on the graph), but also unlearn the current pattern. Technically, there is nothing wrong with this, but keeping $b$ in the range [0,1] would be more natural.

Author response:
4. Reviewer K2Ug asked some questions relating to the parametrization of the model with $b$. Our initial inclination was to construct our model such that we could rigorously test it while varying this single parameter. However, as the reviewers noted, this becomes less natural outside the range of $b \in [0,1]$. So, while parsimony can offer advantages, in this case we agree it does not. We therefore separated the auto- and hetero-associative strengths into two parameters – $a$ for the auto-associative weighting and $h$ for the hetero-associative weighting. This reparametrization aids in interpretation of the model and avoids the unnaturalness present in the prior use of $b$.

Reviewer comment (Experiments):
The paper does not include experiments on real temporal data. It is thus unclear how such a system could be applied to real problems.

Author response:
Indeed, our original submission only used simulated data, which was a limitation. In our revision, we include new experiments with real data.

Reviewer comment (Theory):
The paper is missing analysis of the hetero-association dynamics as \( t \rightarrow \infty \) at different values of \( b \) (i.e., why did you choose to display results only at \( t=10 \)?). Will the state blow up or converge to meaningful values?

Author response:
For practical applications, it is helpful if the network does not require many updates, which is why our initial work focussed on analysing the network states after $t=10$ updates. Nevertheless, it is still helpful to know about the long-term dynamics. We therefore added analysis of long-term dynamics both through numerical simulations and theoretical analysis.

Reviewer comment (Evaluation):
Experiments only show correlations, not the actual state itself. I feel that the hidden state \( S^{(t)} \) is blowing up in value and cannot be run at low values of \( b \) for too long.

Author response:
For practical applications, it is helpful if the network does not require many updates, which is why our initial work focussed on analysing the network states after $t=10$ updates. Nevertheless, it is still helpful to know about the long-term dynamics. We therefore added analysis of long-term dynamics both through numerical simulations and theoretical analysis.

Reviewer comment (Experiments):
Missing experiments that would consider alternative formulations of anti-hebbian modulation. Why should anti-hebbian modulation also amplify the signal from hetero-association? See Question 1.

Author response:
Our initial inclination was to construct our model such that we could rigorously test it while varying a single parameter, $b$. However, as the reviewers noted, this became less natural outside the range of $b \in [0,1]$. So, while parsimony of this kind has its advantages, we completely agree with the reviewers that it offers little advantage in this case. We therefore separated the auto- and hetero-associative strengths into two parameters – $a$ for the auto-associative weighting and $h$ for the hetero-associative weighting. This reparameterization further aided us in the theoretical analysis of the model.