Adaptive Hopfield Network: Rethinking Similarities in Associative Memory
Authors: Shurong Wang
Yuqi Pan
Zhuoyang Shen
Meng Zhang
Hongwei Wang
Guoqi Li
Venue: ICLR 2026 Poster (2026)
Keywords: similarity measure
associative memory
Hopfield network
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: Propose an adaptive similarity that captures context subtleness and strives for correct retrieval defined by variant distribution.

Abstract: Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability.
However, existing models evaluate the quality of retrieval based on proximity, which cannot guarantee that the retrieved pattern has the strongest association with the query, failing correctness.
We reframe this problem by proposing that a query is a generative variant of a stored memory pattern, and define a variant distribution to model this subtle context-dependent generative process.
Consequently, correct retrieval should return the memory pattern with the maximum a posteriori probability of being the query's origin.
This perspective reveals that an ideal similarity measure should approximate the likelihood of each stored pattern generating the query in accordance with variant distribution, which is impossible for fixed and pre-defined similarities used by existing associative memories.
To this end, we develop adaptive similarity, a novel mechanism that learns to approximate this insightful but unknown likelihood from samples drawn from context, aiming for correct retrieval.
We theoretically prove that our proposed adaptive similarity achieves optimal correct retrieval under three canonical and widely applicable types of variants: noisy, masked, and biased.
We integrate this mechanism into a novel adaptive Hopfield network (`A-Hop`), and empirical results show that it achieves state-of-the-art performance across diverse tasks, including memory retrieval, tabular classification, image classification, and multiple instance learning.
Our code is publicly available at https://github.com/shurongwang/Adaptive-Hopfield-Network.

Published as a conference paper at ICLR 2026
ADAPTIVE HOPFIELD NETWORK: RETHINKING SIMI- LARITIES IN ASSOCIATIVE MEMORY
Shurong Wang Yugi Pan", Zhuoyang Shen', , Meng Zhang', Hongwei Wang'* & Guoqi Li "Zhejiang University IIntittte of Automation, Chinese Academy of Sciences shurong. 220int1. z jju. edu. cn, gueqi 1i0ia.
ABSTRACT
Associative memory models are content-addressablle memory systems fundamen- tal to biological intelligence and are notable for their high interpretability. How- ever, existing models evaluate the quality of retrieval based on proximity, which cannot guarantee that the retrieved pattern has the strongest association with the query, failing correctness. We reframe this problem by proposing that a query is generative variant of stored memory pattern, and define variant distri- bution to model this subtle context-dependent generative process. Consequently, correct retrieval should return the memory pattern with the maximum posteri- ori probability of being the query's 's origin. This perspective reveals that an ideal similarity measure should upproximate the likelihood of each stored pattern gen- erating the query in accordance with variant distribution, which is impossible for fixed and pre-defined similarities used by existing associative memories. To this end, we develop adaptive similarity, novel mechanism that learns to approximate this insightful but unknown likelihood from samples drawn from context, aiming for correct retrieval. We theoretically prove that our proposed adaptive similar- ity achieves optimal correct retrieval under three canonical and widely applicable types of variants: noisy, masked, and biased. We integrate this mechanism into a novel adaptive Hopfield network (A-HH (A-Hop), and empirical results show that it achieves state-of-the-arr performance across diverse tasks, including memory re- trieval and various classification tasks. Our code is publicly available here.
INTRODUCTION
Associative memory represents a fundamental paradigm in information storage and retrieval, func- tioning as a content-addressable memory system that serves cornerstone of biological intelli- gence (Miyashita, 1988; Pearce & Bouton, 2001), particularly in the hippocampus and neocortex (Wang et al., 2014). Unlike conventional computer memory, which retrieves data based on spe- cific address, associative memory retrieves stored patterns by using a partial or noisy variant of the pattern itself as a cue. This memory paradigm enables robust pattern completion, error correction, and fault-tolcrant information processing, making it a compelling model for both understanding biological cognition and developing artificial intelligence systems.
The computational modeling of associative memory has evolved dramatically since its inception. Hopfield (1982) pioneered this field by introducing a recurrent neural network, dubbed Hopfield network, capable of storing and retrieving patterns through energy minimization. Subsequent work (Krotov Hopfield, 2016; Demircigil et al., 2017) extended memory capacity using steeper energy function. A pivotal breakthrough came with the establishment of a profound connection between the modern Hopfield network and the attention mechanism (Vaswani et al., 2017), achieved by using the softmax(-) function to further separate memories (Ramsauer al., 2021). This insight not only unified two previously disparate fields but also inspired further refinements that strengthened associative memory's 's performance from different perspectives (Millidge al., 2022; Hu et al., 2023; Wu et al., 2024a), and broadened applications to tasks like clustering (Saha et al., 2023), time series prediction (Wu et al., 2024b), and more (Krotov et al., 2025).
Despite these significant advances, a critical and unaddressed limitation pervades the literature: the absence of rigorous framework for assessing retrieval accuracy. Current evaluations typically rely

---

Published as a conference paper at ICLR 2026
on proximity- based criteria, such as e-retrieval (Ramsauer al., 2021; Hu et al., 2023; Wu al., 2024a; Hu et al., 2024; 2025), which deem retrieval successful if the retrieved pattern is sufficiently close to certain stored pattern. However, proximity does not establish correctness; ensuring the retrieval is a valid memory provides no guarantee that is the correct one, that is, the one that has the strongest association with the query. This oversight leads to universal reliance fixed, pre-defined similarity measures (c.g., inner product or Euclidean distance between two memory patterns). Such one-size-fits-al metrics fail to capture the nuanced, context-dependent association, or similarity, between the query and the stored memory patterns. For instance, the word click is semantically similar to tap, phonetically similar to clique, and orthographically to clock illustrating that an appropriate notion of similarity is context and task dependent while fixed metrics cannot adapt to such context, nor can they certify correctness.
Our central premise is that correctness is inherently generative: a query x emerges as a variant of an unknown stored pattern &a- So, to properly define and achieve correct retrieval, we should model the generative process that transforms stored pattern & into query x. To this end, we encapsulate the context-dependet and application-related subtleness into a probabilistic framework centered on the concept of variant distribution V({I...),, joint distribution over stored patterns Ei--N and memory variants x, where the likelihood pv captures how probable that we observe & and coincidentally generates for (Ex, x) V.E....N) Under this view, correct retrieval returns the memory pattern ÉA: maximizing the posterior pv that is, the likelihood of x originates from &x when observed x as query. With further decomposition, maximizing pv(Ex|x)
equivalent to maximizing (v Ex), i.c., given &x. how probable would it generates x as its variant. The correct retrieval is therefore finding the pattern & that is most likely produce x by varianting itself. This perspective yields an insight that optimal correct retrieval can be achieved by forcing the similarity measure to mimic the behavior of py (x(Ex).
However, it is not possible to derive the variant distribution V(XI..N) and the likelihood (y(x({x) on most occasions. Thus, we need to reconstruct the unknown by mining deeply from what is observable: the query stored patterns EI.-N, and samples matching the context that vaguely describe V. Building on these motivations, we introduce an adaptive similarity framework that learns to approximate py (x(Ex) from samples observed from the variant distribution, without assuming the variant type is known a priori. Integrating this novel similarity measure into the Hopfield energy yields adaptive Hopfield network that strives for correct retrieval by capturing the underlying variant distribution. Our key contributions are as follows:
We introduce the variant distribution to model how queries emerge from stored patterns, and formalize correct retrieval as a robust and meaningful criterion for evaluating the theoretical accuracy of associative memories.
We propose adaptive similarity derived from this framework and prove its optimality for three canonical and widely applicable types of memory variants: noisy, masked, and biased.
We build a novel adaptive Hopfield network (A- (A-Hop) that incorporates learnable adaptive similarity, achieving state-of-the-art performance among computational associative memo- ries on tasks including memory retrieval, tabular classification, image classification, etc.
2 BACKGROUND
We consider associative memory that stores N memory patterns denoted by the memory matrix E [E1: Ez; : EN] E RdxN where each column vector & E IRd represents a memory pattern. Given a memory variant (query) x E Rd, the goal is to retrieve the stored memory that is most associated with it. For simplicity, we denote [n] 1 { Z < k < n1,, and & < E means & is one of the column vectors of the memory matrix =. Appendix A. contains collection of notations.
2.1 HOPFIELD NETWORKS
Hopfield network is line of associative memory that retrieves the most relevant stored memory through a similarity- based matching process. The original Hopfield network (Hopfield, 1982) uses d binary neurons a (-1,+1)d represent the states of the memory system that is limited to storage of binary values. For retrieving, the model sets the query as the initial state (i.e., ( x),
2

---

Published as a conference paper at ICLR 2026
Table 1: Summary of all Hopfield network by components; Hop (Hopfield, 1982), H-Hop (Krotov & Hopfield, 2016), H-Hop (Demircigil et al., 2017), M-Hop (Ramsauer et al., 2021), N-Hop (Mil- lidge et al., 2022), SoHop (Hu et al., 2023), U-Hop (Wu et al., 2024a), and A-Hop (Ours).
Model
sim(ç, x)
sep(s)
mod(E)
E(x)
Нop (Original)
1 ax>=E's
D-Hop (Dense)
{
st
-(1 x) x)++1
E-Hop (Exponential)
{ ['x
exp(s)
exxp1'==xx
M-Hop (Modern)
[xx
softmax(s)
x - Ise (x
N-Hop (Universal)
- x||1
arg max(s)
S-Hop (Sparse)
8 x
sparsemax(s)
x V1(8E'x)
H-Hop (Uniform)
E E x
a entmmax(s)
+ &
1 Φx ( + (xx
A-Hop (Adaptive)
w ftpt(E,
multiple
-Ise x))
and updates more neuron(s) iteratively through the following dynamics until convergence:
N Sa. Ek.j-
(++1)
sgn
where Túj
R=1
vectorized retrieval dynamics exists when updating all neurons simultancously in one iteration:
a (++1) sgn = (
Years later, Krotov & Hopfield (2016) improves the memory capacity of Hopfield network from (dd) ((24/2) when storing random samples. They adopted higher-order polynomial or expo- nential function (Demircigil et al., 2017) to distinguish each stored memory to alleviate the fuzzy memory problem: a (1+1) sgg(E( a (9)))) or (1+1) = sgn (g exp(= ( (0))).
This concept has evolved significantly and was extended to memories with continuous value. Mod- em Hopfield networks abstract retrieval as a one-iteration update (Ramsauer et al., 2021), and their retrieval dynamics T(x) can be unified under three-step procedure (Millidge et al., 2022):
(I) Similarity [s = sim sim x)]: The query x is compared against stored patterns E using the similarity function sim( -), obtaining vector of similarity scores R.V. where $ sim x).
(2) Separation [p = sep(s)]: The similarity scores (logits) s are transformed by a separation function sep(-) into a probability distribution p. The separation function sharpens the scores and emphasizes patterns with high similarity.
(3) Readout ly pp: The final retrieved pattern is computed as weighted combination of stored memory patterns, using the weights p provided by the separation function.
Combining each step gives the unified retrieval dynamics:
Txx) = E sepp(sim(Exx)
Concretely, Ramsauer et al. (2021) proposed using softmax(-) function as the separation function, which further enlarges the memory capacity and draws tight connection between associative mem- ory and attention mechanism, with the retrieval dynamics being T(x) = E softmax (= x). Later, Hu al. (2023) proposed sparse Hopfield network substituting sof(ma with sparsemax()). for inducing sparse selection while retaining differentiabilityy More recently, Wu et al. (2024a) at- tempted to store memory patterns in a kernel space with greater separation among patterns, giving modulation step to the existing three-step unified framework. For clarity, use the modulation function mod(.) to describes how memory patterns are stored or pre-trained for better retrieval and larger capacity. So, it broadens the unified framework to:
rise adding
= E scp(sim (c)))) x)).
The kernelized Hopfield network (Wu et al., 2024a) adopted mod(3) = $ T T for a learnable matrix & IR P xd that projects memory patterns into kernel space with the retrieval dynamics T Tx). The kernel & is trained to minimize separation loss defined on E, that the expected Fuclidean distance between any memory
being T(x) = E sepp(($ E) ( x)) = E sep(( + Φ
3

---

Published as a conference paper at ICLR 2026
patterns is maximized. A succeeding work (Hu et al., 2024) uses spherical codes to find the optimal kernel that maximizes the capacity of the kernelized Hopfield network.
Furthermore, the energy-based view is defining feature of Hopfield networks: memory retrieval can be viewed as descending on a energy landscape (a Lyapunov function) E(-) whose minima coincide with stored patterns (or their modulated version). Formally, the retrieval dynamics T(x) and the corresponding energy function Exx) are jointly and carefully designed such that each update monotonically decreases the energy (i.e., ET (x)( < E(x)), and successful retrieval occurs when being sufficiently close to a generalized fixed point near a specific memory pattern &a E E (i.e.. [x] E& ll2 < e). This principled linkage between dynamics and energy ensures convergence and provides a powerful interpretable model of memory retrieving for Hopfield networks. Connecting to the previous unified framework, the separation function decides the direction of the retrieval dynamics T()) and the modulation function reshapes the geometry of the energy landscape E((), and we organize all existing Hopfield networks' components and energy function in Table
However, across existing formulations, the energy and retrieval dynamics are anchored to fixed, aasn-ago similarity measure (typically the dot product). Apart from that, the energy and dy- namics are solely determined by stored memories 3\r that overlook the nature of the subtle, nuanced, context-specific association between queries and memories required for "correct" retrieval. This fundamental gap motivates our work: to refine the energy and retrieval dynamics around learnable, adaptive similarity measure while preserving the precious interpretability of Hopfield networks.
METHODS
In this section, we first establish a rigorous probabilistic framework define correct retrieval, clim- inating limitations of conventional proximity-based metrics (Section 3.1). To make this concept practical, we develop the similarity footprint (Section 3.2), a multi-scale descriptor measuring asso- ciation between queries and memory patterns, and use it to learn an adaptive similarity integrated into Adaptive Hopfield Network (A (a-Hop) that achieves optimal correct retrieval for noisy, masked, and biased types of variants, and has a decreasing, convergent, and bounded energy (Section 3.3).
3.1 VARIANT DISTRIBUTION AND CORRECT RETRIEVAL
Conventional analyses of associative memory (Ramsauer et al., 2021; Hu et al., 2023; Wu et al., 2024a; Hu et al., 2024) mostly focus on e-retrieval:
Definition 1: e-retrieval (Hu et al., 2023; Wu et al., 2024a; Hu et al., 2025)
Given a query x E I and the retrieval result y E IRd given by the memory system, a memory pattern & E 3 is said to be c-retrieved if ||y E11 <
While e-retrieval ensures the retrieval result y lies near a certain stored memory pattern &a E E, it provides guarantee that & is the most appropriate match for query x. The query may have stronger associations with a different pattern &, (j # k), denoting that & could be the more appropriate match for x. This identifies that proximity alone an insufficient proxy for correctness.
To address this limitation, we use a probability distribution to model the generative process of the query x. We posit that query x not an arbitrary vector but variant of a specific stored mem- ory pattern & E E generated by a context-depended process. We formalize this via the variant distribution, which models the relation of memory patterns & E E and queries x Bd variants:
Definition 2: Variant distribution
variant distribution V(E) is a joint distribution over pair (5. x) E E x Rd where & E E is one of the stored memory patterns and E IRd is an arbitrary query. For (g. x) ~ V(B), the probability density function Pv( (E, x) (or po x) when unambiguous) measures the likelihood of observing and the same time.
Additionally, the posterior py (E x) represents the likelihood that query x originates from memory pattern x, and the likelihood py models how probable that & generates x. This leads to a rigorous definition of the context-dependent correct retrieval:
4

---

Published as a conference paper at ICLR 2026
Definition 3: Correct retrieval
A query x is said to be correctly retrieved under V(E), if the retrieval result satisfies that:
arg min (lly - -11 = arg max (pv(5" x)).
e'ea
g'ea
In Eq. 1, the left-hand side identifies the closest memory pattern to the retrieval result y (given by the memory system), while the right-hand side is ground truth (the most probable origin of query given by variant distribution V( Thus, intuitively, correct retrieval requires that the closest memory pattern coincides with ground truth. With further derivation,
arg max (po(x()) Pv(5') pv(x) arg (xgx (V())) (2) E'E
arg max E'EE
E'E
This reformulation is necessary modeling the likelihood pv (x(E) more tractable than directly estimating the posterior Py(E x). The likelihood py(x(E) is conditioned on a single, finite, known memory &. while the posterior pv(E(x) requires estimating complex function that maps the entire query space Rd to discrete distribution over E. Given that the prior Pv(E) is typically uniform or be easily estimated from samples, the central challenge of achieving correct retrieval reduces to accurately modeling py(x (x|$) { in other words, how probable does x generate & under V(E)? With this in hand, it is possible model three canonical and common variant types rigorously:
Definition 4: Noisy variant
generated by adding Gaussian noise to a certain memory pattern & E. Formally, (x E) ~ NO0, diag(or) holds for (g, x) ~ Viuisy (3), where diag(v) transform vector bu to a diagonal matrix. The likelihood of noisy variant is:
query x a noisy variant
PVaay (x(E) = (2 diag(or)) exp (-2(x - 5) T) didag(o)
Noisy variants have been widely studied (Krotov & Hopfield, 2016; Hu et al., 2023; Wu et al., 2024a), and it occurs in scenarios such as noise. Specially, under isotropy a1, the respective likelihood reduces to: PV (x($) (2xa) exp(- E E 20).
Definition 5: Masked variant
A masked variant of memory pattern & E = is obtained by changing values in each dimension with probability Pmasked to numbers generated by G. The likelihood of masked variant is:
PViestos (x(E) = exp In Prankked
Σn 8-x8(x; E:)]
II
po (x: [1- [1-8(x; E::]
Here, 5(-) denotes the Dirac's delta function, and the value of 0(x) is 1 iff and otherwise. Masked variants arise in real-world scenarios such as information loss during transmission, the same object appearing in different background, and more.
Definition 6: Biased variant
Adding global bias to memory patterns gives the biased variant. Formally, E d holds for (E, x) ~ Volased (E) and a constant vector d E Rdd The likelihood biased variant:
Pirsod (x(E) =
Σ 6(x: - E: - di))
Biased variants occur as systematic difference, such as changes in light conditions or use of filters.
We visualize the conditional probability density function pv (x(E) in Fig. 1, providing intuition akin to an electron cloud, with memory pattern & as the atom nucleus and its variants orbiting electrons. A direct observation that (x[E) varies significantly across contexts, and may be an- alytically intractable. For instance, even though visualizing the noisy + masked variant (Fig. I (d)) is possible by composing these two operations, deriving its likelihood (v is analytically cum- bersome. Consequently, although V(E) is principled tool to link queries with memory patterns, it poses two challenges for correct retrieval: (1) the underlying variant type is generally unknown a priori; and (2) the resulting variant distribution ((=) can be too complex model explicitly.
5

---

Published as a conference paper at ICLR 2026
a
noisy
bb)
masked
(c)
biased
d
noisy + masked
Figure 1: Visualization of probability density function (x(()() for noisy, masked, biased, and noisy masked variants, Darker regions indicate larger (x(x(E). and the central dark point represents &.
3.2 SIMILARITY FOOTPRINT
In the previous section, we established correct retrieval as selecting the pattern that maximizes py(g))). While (x({ constitutes the ideal similarity metric (guiding the memory system to return arg (
per Eq. 2), it is often analytically intractable or unknown in real- world scenarios. To address this, we propose mining richer evidence from observable quantities to construct a tractable surrogate that mimics the selectivity of the true likelihood p>(x(g). We introduce similarity footprint, a multi-scale descriptor capturing the relation between query x and memory patterns E\t in multiple subspaces. By replacing proximity saasee scalar similarity measure (e.g.. E x) with a structured descriptor (the footprint, a d-dimensional vector), the model could form accurate and robust decision based on the comprehensive subspatial evidence.
We begin with a base similarity measure (e.g.. dot product _xx x or negative squared Euclidean distance - E x x12). and define the k-optimal similarity between & and x:
sim (K) (E, DC(d).DD=
max {sim(En,x))),
where vD A [VD,, VDa "","""" !Dipil"
Here, Vn is a sub-vector of v containing only the elements corresponding to indices in D. In- tuitively, sim, (E, quantifies the largest alignment between x and within their most consistent k-dimensional subspace. This formulation inherently filters out corruption pure noise by focusing on the most informative dimensions while ignoring outliers. Consequently, we define the similarity footprint as the vector aggregating all k-optimal similarity (im(*) across all dimensionalities:
ftptsim (
x)
[sim (1) (E.x), sim (2) x),
sim (4) (E.x)]
This vector serves the rich multi-scale descriptor of the subspatial relation between and x, offering more evidence for measuring similarity. While nai'ee computation the footprint requires evaluating all 2d subspaces (impractical), efficient computation is possible for decomposable similarity measures (Bq. 6). For such measures, whose result is the aggregation of similaritiy in each dimension, the footprint computation reduces to Old log d) via sorting, and the procedure is visualized in Fig. 2 bottom part. Let q be the dimension- -iiie similarity vector (Fig. 2 (1)), where qi sim(E.. E., x;) for e [d]. and let be the vector q sorted decreasing order (Fig. (2)). Then, the similarity footprint can be calculated as (Fig.2 (3)):
ftpt (E, = UQ.
where is the lower-left triangular matrix of 1dxds (i.c., U = 1f l d. and U,, otherwise). Such simplication is valid because (*) x) = Σ
Di=1 qi holds for decomposable
similarity measures, and rigorous proof is provided Appendix A.2 Theorem 3.
3.3 ADAPTIVE SIMILARITY AND ADAPTIVE HOPFIELD NETWORK
The similarity footprint provides a structured, multi-scale descriptor of the association between query x and a memory patter &. To exploit these subspatial evidences and construct a similarity measure that adapts to the underlying variant distribution V(3), we introduce adaptive similarity as
a learnable linear combination of the footprint elements: Sain (g. x) = w ftpt.im (g.x) x) w Uq
(Fig. 2 (4)), for some weight vector IR This formulation enables the model to learn the relative

---

Published as a conference paper at ICLR 2026
Adaptive Hopfield Network
Adaptive Similarity Training
Adaptive Similaritt
Similarity Score <t
Output
Loss
Separation softmax(()
Pbo
Way
Separation softmax(
Footprint fipeu.
Footprint 1o (6
Adaptive Similarity .(.))
Sort wan (6)
Sort scri(e)
Adaptive Similarity -(:--)
Pattern &x
Input
Pointwise Similarity xil
Pointwise Similarity
9: - x
Stored patterns
Input
E - Ex]
Generator (E,, ~ V(E)
Puttern
Input
Proximity-Basse Based Similarity
(c) Adaptive Stmilarity
E x
ftptax (E..
scin (5:.. x)
9.94
E
Waite
Input/Query
Vector Form
Atpta. ($2 x)
E x 119.6
8ain (5 no 333
Nakara pastree
Similarity Score (Conventiona))
Stored Pattern
Valuo altagati
Stored Pattern
Pointwise Similarity 0, IC x.."
Sort (decreasing)
Foutprint Botaa(6 Uiq
Similarity Score (Ours)
The architecture of adaptive Hopfield network (a), the training procedure for adap- tive similarity (b) and its design choice (c). Bottom: An illustrative example of memory retrieval procedure involving two 16 x 16 image patterns and one query. The conventional proximitiy similarity (d) fails retrieve the correct pattern while the adaptive similarity (e) succeeds. The com- putation of proposed adaptive similarity consists of four steps: (1) the pointwise similarity E TRd computed as q, = -E. x2 (2) elements in q are sorted in decreasing order to obtain q. (3) the footprint is calculated as ftptais (E, x) Uq. and (4) the adaptive similarity score is computed as (E, x) wiigffpt (E,
Figure 2: Top:
importance of subspaces with varing sizes and pay extra attention on more informative subspaces. As an example, for masked variants, the model could assign larger weights to the first m terms in the footprint, effectively ignoring the "tail" of the footprint containing corrupted dimensions, whereas for noisy variants, it might distribute weights to larger subspaces for global view.
To further enhance the model's expressiveness, we combine footprints of multiple base similarities (Fig. 2 (c)) and derive the final similarity function s(g, x) and its vectorized form s()
x(, Σ Bk wk ftpttime (E,x) and k=1
[ x) [s(Ei,x), s(E2,
(EN, x)]
where B1...B are learnable scalars weighting B different base similarities. In this work, we adopt two simple and common base similarities: dis(E, (is(E,x) = - and dot x) = Tx. Integrating this adaptive similarity into the unified associative memory framework (Table 1) using the softmax()) as the separation function yields Adaptive Hopfield Network (A-Hop, complete achitecure illustrated in Fig. (a)()), and its retrieval dynamics can be formulated
= T(x) = E scp(ssE, = E =ootmax(33mn Sais (E. + Baot Saot (E,x)).
(4)
The parameters w's and Is's are optimized align the model's 's behavior with the underlying variant distribution. We construct training set by drawing samples from the variant distribution V(E and minimize the discrepancy loss between the model's predicted likelihood (x & scp(s(E, and the underlying ground-truth likelihood pv (x({x) (see Fig. 2 (b) for training procedure):
( V) [E [- log (x)gx((((()))
(5)
7

---

Published as a conference paper at ICLR 2026
For intuitive understanding of adaptive similarity, we demonstrates comprehensive procedure of calculating the adaptive similarity score between &1, &2 (pixel art of two chicken) and the query (a noisy masked variant of {1) in the botton part of Fig. 2. While rigorously, Adaptive Hopfield Network satisfies the following theoretical properties.
Definition 7: Optimal correct retrieval
We say a retrieval dynamics T T(x) achieves optimal correct retrieval under V(=), if for any xE, ~ V it achieves correct retrieval (Def. 3) for query x.
Theorem 1: A Hop's retrieval dynamics for optimal correct retrieval
The following retrieval dynamics adopted by A-Hop achieves optimal correct retrieval for noisy, masked, and biased variants, with careful design of x):
Tx) = E sep(s x))
First, A-Hop is theoretically capable of achieving optimal correct retrieval (its retrieval always correct, see Def. 7) for noisy, masked, and biased variants when weights w's and the adaptive simi- larity s(=, x) are ideally configured. A complete walk-through and proof to Theorem is presented in Appendix A.2.1. While optimal correct retrieval is not always guaranteed for continuous and parameter-efficient adaptive similarity (c.g., s(E x) = w ftpt(f.x)), we discuss the tradeoff be- tween correctness and adaptivity in Appendix A.3.1, and show that learnable adaptive similarity can attain high retrieval accuracy with large probability. Together, this theoretical analysis and the empirical results in ablation study (Appendix 6.4.6) validate the design choice illustrated in Fig 2.
Theorem 2: A-Hop's 's decreasing, convergent, and bounded energy function
Energy E(x) will be monotonically decreasing, convergent, bounded from below, for isotropic noisy, and biased variants, if the following energy is used:
Exx) Isc (s(3, x))
Second, consist with existing associative memories, A- Hooop guarantees monotonically decreasing, convergent energy that can be bounded from below by - In n ||b]1// / during the iterative retrieval process. Notably, among the models compared in Table 1, this is the only energy function that be bounded from below even as ||ll| -> oc, which guarantees robust retrieval behavior for unbounded patterns, A detailed proof and explanation Theorem 2 is provided in Appendix A.2.2.
EXPERIMENTS
We evaluate the effectiveness of A-Hop tasks including memory retrieval, tabular classification, image classification, and multiple instance learning, demonstrating that A-Hop achieves state-of- the-art performance on these tasks. A further ablation study validates our design choice of adap- tive similarity (Appendix A.4.6). Due to space constraints, full descriptions of baselines, metrics, datasets, and implementation details are moved Appendix A.4.
4.1
MEMORY RETRIEVAL
(a) 2048 patterns
(b) 4096 patterns mask
plot settings
mask
(any a.ln 6AD. L6 3)
(8.0.0) (6.3
O A-Fop
X-toop
M-Hop
00000
U-Hop
O MLS
bias
noise
bias
hoise
60.4 (05.5)
Figure 3: Retrieval accuracy (1) of 64-dimensional synthetic memory patterns.
8

---

Published as a conference paper at ICLR 2026
Prior work on Hopfield networks primarily assesses retrieval accuracy (Def. 9) under two settings: (1) masking half of the dimensions and (2) adding Gaussian noise. However, real-world data often exhibits compounded corruptions. To rigorously stress-test retrieval correctness, introduce mixed variant parameterized by triplet (dmaak, Anaise, dbias) E [0, 1| that controls the intensity (difficulty) of masking, noise, and bias, respectively (see Appendix A.4.3 for formal definitions).
We evaluate A- against existing baselines on 12 mixed variant settings (listed in Fig. 3 (c)) with 64-dimensional (d 64) random memory patterns at scales of N 2048 (Fig. 3 (a)) and N 4096 (Fig. 3 (b)). Furthermore, Table 2 details performance under high-intensity corruption using either N 2048, 64 synthetic vectors or N 2048, 784 MNIST digits as memory patterns.
As shown in Fig. 3, existing associtive memory baselines perform poorly when masking or bias is involved as their retrieval accuracy is less than 20% on the mask axis and less than 60% on the bias axis while А- achieve near- perfect accuracy on these settings. It is because fixed proximity- based similarity cannot distinguish between irrelevance and missing data (masking), and have no chance to learn the biased term added to memory pattern. In Table 2, baselines exhibit sharp per- formance collapse as difficulty increases, while A-Hop maintains robust retrieval gaining a 6% to 20% accuracy increment. A- A-Hop's 's adaptive similarity creates noticable empirical gap by achiev- ing higher retrieval accuracy and lower retrieval error in all settings. This highlights its robustness and impressive adaptability to align similarity to the underlying variant distribution through learning.
Table 2: Retrieval accuracy (f) and error (L) between models. Each cell contains the mean accuracy or error with standard deviation in a smaller font. Results of the best-performing model are bolded. Difficulty stands for mixed variant setting with (dmasks dnoise, doias) = (d, d, d).
Dataset
Synthetic
MNIST
Difficulty
0.4
0.5
0.6
0.7
Metrics
Accuracy
Error
Accuracy
Error
Accuracy
Error
Accuracy
Error
M-Hop
.5202.00
.1762.01
.1952
300 o1
.875..01
.013
.661
.068
N-Hop
.260
.417
.059
+5
.50#..s
.143=.m
.176+.m
.37++00
U-Hop
.487 +.0
.295 +.02
.177+.02
.764
.764
.064
.526+.2
.164+.01
U2- U2-Hop'
.521
.1762:01
.195 oz
.298
.878
.013
.660
.068
A- A-Hop
.724
.106
.360
.227
.939
.005... .m
.849
.015
4.2
TABULAR CLASSIFICATION
Table Predictive performance (f) between models on tabular data, Each cell contains mean aceu- racy AUC-ROC score with standard deviation in smaller font. Results of the best rest-performing associative memory are bolded, and the best other model is underlined.
Model
Adult
Bank
Vaccine
Purchase
Heart
M-Hop
.085_
.7975 not
.8822.00
.6325 002
Ug-Hop
.81722.03
.0022_0o2
.7971 003
.825
.64732.002
A-Hop
.8634:.00
.9139 .002
.8042
90007.001
.7315::002
Extra Trees
.8595
.9098=1m
.7932+
.8166+.00
.717.5 003
Random Forest
885921.00
.1132_003
.7918:100 003
.9002..001
.7254.1002
AdaBoost
.8597..000
.9094_000 _011
.801 002
.8865
.7294.:.00
XGBoost
.8600
.9152=003
.8034 002
.9032+t.uu
.7370# 003
We investigate the utility of HoH as memory-based classifier for tabular data (setup details in Appendix A.4.4). Unlike image data, tabular data is often heterogeneous, containing a mix of continuous and categorical features with varying scales and sparsity.
A-Hop consistently outperforms all competing associative memories (M-Hop and U2 Uo-Hop), demonstrating that adaptive similarity crucial for handling the mixed feature types inherent in
Ug Hop U-Hop whose kernel optmized by Eq. 5, rather than the original separation loss.

---

Published as a conference paper at ICLR 2026
tabular data. Furthermore, A-Hop surpasses standard deep learning baselines (Extra Trees, Ran- dom Forest, AdaBoost) in all datasets. While the gradient-boosting method XGBoost (Chen & Guestrin, 2016) retains slight edge on 4 out of 5 datasets, A-Hop significantly narrows the gap compared to prior approaches, offering a differentiable alternative to tree ensembles. Significantly, the performance gap between A-Hop and other memory models is widest the Adult (about 5%) and Heart (about 10%) datasets. We hypothesize that these datasets contain more subtle and com- plicated variant distribution where the "informativeness" of dimensions varies per query, but the results suggest that A- A-Hop can find the most informative subspace more accurately and can retrieve relevant prototypes even when the distance in the original feature space is misleading.
4.3 IMAGE CLASSIHCATION AND MULTIPLE INSTANCE LEARNING
Table 4:
Classification accuracy ( each model on images, and AUC-ROC score (t) of each model in multiple instance learning task. Each cell contains accuracy or AUC-ROC score with standard deviation in a smaller font. Results of the best-performing ussociative memory are bolded.
Image Classification
Multiple Instance Learning
Dataset
CIFARIO
CIFAR100
Tiny ImageNet
Dataset
Tiger
Fox
Elephant
UCSB
HoH
+5123+000
.2464 005
40055
M- X-Hop
.8924+t.05
+3277
.9344+.00
.8815 =8855
U U-Hop
.589 002
.2877 1077-.02
.1144+.0 o.0
S S-Hop
48923+
.6433
.9365 ou2
.8794 024
A A-Hop
.5637 oo
.2904=002 002
.02
A-Hop
.9030
.6753 013
.9451
.8935_00 022
Following established protocols, we evaluate A-Hop on image classification (Wu al., 2024a) and multiple instance learning (Ramsauer et al., 2021; Hu et al., 2023) by integrating as a component within larger and more complicated deep neural network architectures (i.e., Hop dLayer, Hopfie PeffieldPooll. dPoolic ing (Ramsauer et al., 2021)).
As shown in Table 4, A A-Hop consistently achieves the highest accuracy and AUC-ROC scores among all Hopfield variants. This demonstrates that the benefits of adaptive similarity extend to complex, nigh-dimensional data and can enhance the performance of sophisticated models like the Vision Transformer. While the absolute gains in these complex architectures are naturally smaller than in isolated retrieval tasks (as the associative memory is not the core mechanism). the consis- tent superiority of Hop validates its role as general-purpose, robust associative memory layer. Nevertheless, the consistent improvement confirms that optimizing the similarity measure remains valuable factor for enhancing performance in complex deep learning systems.
4.4 ABLATION STUDY
Due to page limit, ablation study are moved to Appendix A.4.6.
CONCLUSION
We reframe associative memory retrieval as problem of correct retrieval under a task- and context- dependent variant distribution, motivating similarity measure that approximates the likelihood that stored pattern generated the query. Building on this principle, we propose adaptive similarity, prove its optimality for three canonical variant families (noisy, masked, biased), and instantiate new adaptive Hopfield network, A-Hop. This perspective clarifies why fixed, pre-defined similarities are inherently limited: they cannot align to the prevailing variant distribution and thus struggle to guarantee correctness, whereas adaptivity enables the model to capture the underlying variant distribution through samples, shifting towards correctness.
Empirically, A- Hop establishes state-of-the-art performance among Hopfield networks across mem- ory retrieval, tabular classification, image classification, and multiple instance learning. The gains are most pronounced under mixed variant settings where adaptive similarity maintains impressively high retrieval accuracy and low error. In downstream tasks, A-Hop consistently improves over prior Hopfield variants. Ultimately, adaptive similarity is a key principle for advancing associative memories, paving the way for more powerful and resilient memory systems.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work was partially supported by the CAS Project for Young Scientists in Basic Research (YSBR-116), National Natural Science Foundation of China (625B1029, 62325603, 62236009, U22A20103), Beijing Science and Technology Plan (2241100004224011), and Shanghai NeuHe- lium Neuromorphic Technology Co., Ltd.
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. this study, no human subjects or animal ex- perimentation was involved. All datasets used were sourced in compliance with relevant usage guidelines, ensuring no violation of privacy. We have taken care to avoid any biases or discrimi- natory outcomes in research process. No personally identifiable information experiments were conducted that could raise privacy or security concerns, We are committed to maintaining transparency and integrity throughout the research process.
used, and no
REPRODUCIBILITY STATEMENT
Code We provide code help understand this work, and is publicly available at: https:// // anonymous 4 4open. www.sccieeeneeeeeeeeeeeeeeeeeeee CaileneerrAAaptti -tddaptie---oooieellltt
Datasets All datasets are either included the repo, or a description for how to download and preprocess the dataset is provided. All datasets are public and raise ethical concerns.
Hyperparameters All parameters of our proposed framework in Appendix A.4.
Environment Details of experimental setups are provided in Appendix A.4.
Random Seed we do not set random seed specifically for all random behavior, with the random seed determined PyTorch.
LARGE LANGUAGE MODELS USAGE
Large Language Models (LLMs) were used to aid polishing of the manuscript. Specifically, we used an LLM assist refining the language, improving readability, and ensuring clarity various sections of the paper. The model helped with tasks such as sentence rephrasing, grammar checking. and enhancing the overall flow of the text.
It is important to note that the LLM was not involved in the ideation, research methodology. or experimental design. All research concepts, ideas, and analyses were developed and conducted by the authors. The contributions of the LLM were solely focused on improving the linguistic quality of the paper, with no involvement the scientific content or data analysis.
REFERENCES
Barry Becker and Ronny Kohavi. Adult. UCI Machine Learning Repository, 1996.
Leo Breiman. Random Forests. Machine Learning, 45(1):5-32 October 2001. ISSN 1573-0565.
doi: 10. 00.10033/4111003334444444 URL https: htpp:://iin /link,
Company: Springer Distributor: Springer Institution: Springer Label: Springer Number: 1 Publisher: Kluwer Academic Publishers.
1023/A:
P. Bull, I. Slavitt, and G. Lipstein. Harnessing the power of the crowd to increase capacity for data science in the social sector. In ICML #Dala4Good Workshop, 2016.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Min- ing, KDD 16. pp. 785-794, New York, NY, USA. 2016. Association for Computing Machin- ery. ISBN 9781450342322. doi: 10.1145/993967722999785. URL https: :toss///doi /doi. //tpiioogi.....////////////o..o.
1145 29397785

---

Published as a conference paper at ICLR 2026
Mete Demircigil, Judith Heusel, Matthias Löwe, Sven Upgang, and Franck Vermet. On a Model of Associative Memory with Huge Storage Capacity. Journal of Statistical Physics, 168(2):288- 299, July 2017. ISSN 1572-9613. org/ 10. 007 / 1005550----1 017 017-1806-y
doi: 10.11077/119555-07--88866 URL https://ddo
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255, leee, 2009.
Yoav Freund and Robert Schapire. A Decision-Theoretii Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55(1):119-139 1997. ISSN 0022-0000. doi: 10. 10.1006/jsss.1997
hwwp://wwww.....c www.neciieeeddre.... URL https::/ww...iiieeeeteee
Elisa Drelie Gelasca, Jiyun Byun, Boguslaw Obara, and B.S. Manjunath. Evaluation and benchmark for biological image segmentation. IEEE International Conference on Image Processing, Oct 2008. URL http://t///.
bttt:://visioe /ttp:/vviiiooneoo ece. ucsb http:ppbbiiaatiionseeeeee.. -CIPP8 pdf.
Pierre Geurts, Damien Ernst, and Louis Wehenkel. Extremely randomized trees. Machine Learning, 63(1):3-42, April 2006. ISSN 1573-0565, ///oii.or/// 100
doi: 10. 10.1007/110944:06-6666-11 URL https:
100./0009000199990 /t/do/.org/1..100/11199999999999 006 6226-1.
J. J. Hopfield. Neural networks and physical systems with emergent collective computational abil- ities. Proceedings of the National Academy of Sciences, 79(8):2554-22 2558, April 1982. doi: 10. 10.1073/pn 77.3//m9..99...5544 79. 8 2554.
URL https: /tt:://wwwwww /www. pww.s.pas..... org / doi /abs/ habs/10 11.11......111.1 pnas. Publisher: Proceedings of the National Academy of Sciences.
Jerry Yao-Chieh Hu, Donglin Yang, Dennis Wu, Chenwei Xu, Bo-Yu Chen, and Han Liu. On Sparse Modern Hopfield Model. Advances in Neural Information Processing Systems, 36:27594 December 2023. URL URL.tttpss//rroceedigg..nneennne
-onsfrect-Cof..eecce..
html.
Jerry Yao-Chieh Hu, Dennis Wu, and Han Liu. Provably Optimal Memory Capacity for
Modern Hopfield Models: Transformer-Compatie Dense Associative Memories as Spher- ical Codes. Advances Neural Information Processing Systems, 37:70693-70729, Decem- ber 2024. URL https:: http::::///coceednig..o wtw.eeeddiinnnnnenineeeeeeeeeeee neurips cc/ ccppaper__ppeeeeeeeeeeeeeeeeeeee fareer_ffllee files/paper/2// hash/822 828 4 2es.saeeeeedeeefccccaaaaaea..22. -astaaee-eeeeeeeaaaaaaattt
Abstract Conference html.
Jerry Yao-Chieh Hu, Bo-Yu Chen, Dennis Wu, Feng Ruan, and Han Liu. Nonparametric modern hopfield models. In Forty-second International Conference Machine Learning, 2025, URL net http://eeee..ett/rrooummo//////
Kaggle. Cardiovascular disease dataset. https:/ /ww. http:/wwwwgwel.ee//////////../tt kaggle. datasets -aw.rrrriiooaaclaaaaaaeeeeeeeeee dataset, 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Dmitry Krotov and John J. Hopfield. Dense Associative Memory for Pattern Recognition. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https: /ttp:///ppppsssee nips wttpeeerfiieeepppeeeeeeee (aa333cc199r11022d000oooooo
Abstract html.
Dmitry Krotov, Benjamin Hoover, Parikshit Ram, and Bao Pham. Modem Methods in Associative Memory, July 2025. URL / wttp://v...rr//////.....o. (abbs/2207..11111 ar1iv:25002. [cs].
/abs////////
Gert Lanckriet and Bharath K. Sriperumbudur. On the Convergence of the Concave-Conve Procedure. In Advances in Neural Information Processing Systems, volume 22. Curran As- sociates, Inc., 2009. URL https: https::/ nips dc wwt.pppeccppppp.pppcpep hash/8b50 4 wow.aeeeeeaaeeeeeeee33322222222 tttt
12

---

Published as a conference paper at ICLR 2026
Beren Millidge, Tommaso Salvatori, Yuhang Song, Thomas Lukasiewicz, and Rafal Bogacz. Uni- versal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models. In Proceedings of the 39th International Conference on Machine Learning, pp. 15561-15583. PMLR, June 2022. URL https:: http///doooceeee/e///ee////....e html. ISSN: 2640-3498.
/ .al.ggssll......... mlr www.....e.//e..mmmmm..m 62/mi dge
Yasushi Miyashita. Neuronal correlate of visual associative long-term memory in the primate tempo- ral cortex. Nature, 811-6193) October 1988. ISSN 1476-4687. doi: (0.103/333817200
URL httpt::////..... http://wwwwoo. nature.uureeee. com/ www.articees//335e888888 Group.
Publisher: Nature Publishing
Sérgio Moro, Paulo Cortez, and Paulo Rita. A data-driven approach to predict the success of bank telemarketing. Decision Support Systems, 62:22-31, 2014. ISSN 0167-9236. doi: 10.1016/j.ds. dss. 2014.03.001. URL https://www. http://wwwwcccceeeeeeee.o.. com/ cwwm/science/artcccleeeooo
6016792361400061X,
John M. Pearce and Mark E. Bouton. Theories of Associative Learning in Animals. Anmual Review
of Psychology, Volume 52, 2001):111-139, February 2001. ISSN 0066-4308, 1545-2085. doi: 10. 10.11466aamuureexsyyc......11111
URL https: www. annua revi ews org content// Dormmalss/0.1114///areeeesp..ppp
111. Publisher: Annual Reviews.
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gru- ber, Markus Holzleitner, Thomas Adler, David Kreil, Michael K Kopp, Günter Klambauer, Jo- hannes Brandstetter, and Sepp Hochreiter. Hopfield networks all you need. In International Conference on Learning Representations, 2021. URL https: : opwnrevieeeeeeeeeeeeeeeeeeeeeeee net/ forum? (.=tt189Rnz. icd.
Bishwajit Saha, Dmitry Krotov, Mohammed J Zaki, and Parikshit Ram. End-to-end differentiable clustering with associative memories. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th Inter- national Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 29649-29670. PMLR, 23-29 Jul 2023. URL press/v202 sahaa3a html.
mmrr.
C. Sakar and Yomi Kastro. Online Shoppers Purchasing Intention Dataset. UCI Machine Learning Repository, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL. https: http:///ooooeeeeeeeoeeoee htt.://rooeddd...............e.e www.inuu.pp.... cc cc/ccc//////eeee (appprrrilles httlllee/pppp/211177 hash/3f5ee24354ddee
Jane X. Wang, Lynn M. Rogers, Evan Z. Gross, Anthny J. Ryals, Mehmet E. Dokucu, Kelly L. Brandstatt, Molly Hermiller, and Joel Voss. Targeted enhancement of cortical-hippocampaa brain networks and associative memory. Science, 345(6200;:005-10551 August 2014. doi: 10.1126/science. 0111266/ccennce.1222000. 1252900. URL sttps://wwit http://wwww.ieenc.ccee...cc.e.ce org//oi/ /tf/fuulll////////// 1126/
science 1252900. Publisher: American Association for the Advancement of Science.
Shurong Wang, Zhuoyang Shen, Xinhao Qiao, Tongning Zhang, and Meng Zhang. Dynfrs: An efficient framework for machine unlearning random forest. In The Thirteenth International Conference on Learning Representations, 2025, URL https:/ peatts:://opereviieet net / forum? id=nsCOeCLR8e.
Dennis Wu, Jerry Yao-Chieh Hu, Teng- Yun Hsiao, and Han Liu. Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models. In Proceedings of the 41st International
Conference Machine Learning, pp. 53471-53514. PMLR, July 2024a. URL https:
//proceedings... biigs.ml. ml www.ssprrsss.ee...
22.35/wu444 html. ISSN: 2640-3498.
Yu-Hsuan Wu, Jerry Yao-Chieh Hu, Weijian Li, Bo-Yu Chen, and Han Liu. STan- Hop: Sparse Tandem Hopfield Model for Memory- Enhanced Time Series Pre- diction. International Conference on Representation Learning, 2024:30886-30225, 2024b. URL htt. http://eeeeeepeee..eeaa/////////
htmi.
html.
13

---

Published as a conference paper at ICLR 2026
A
APPENDIX
APPENDIX CONTENTS
A.I Notations
15
A.2 Theorems
16
A.2.1 Optimal Correct Retrieval
17
A,2.2 Unified Adaptive Similarity and Energy Function
20
A.3 Discussion.
23
A.3.1
On Good Design Choice of Adaptive Similarity
23
A.3.2 On More Complicated Variant Distribution
27
A.4 Experiments
28
A.4.1
Baselines and Metries
28
................................
A.4.2
Datasets
................................
28
A.4.3
Memory Retrieval
29
................................
A.4.4
Tabular Classification
29
A.4.5
Image Classification and Multiple Instance Learning
30
A.4.6
Ablation Study
30
A.4.7
Multimodal Settings
31
A.4.8
Time Complexity
33
14

---

Published as a conference paper at ICLR 2026
A.1
NOTATIONS
Table 5: Notations and symbolds used in this work.
Symbol
Description
&. E1. E
A specific memory pattern (d x 1) (or memory, stored memory pattern, stored pattern..
The d N memory matrix, with each memory pattern being its column vector. The dimensionality memory patterns.
N
The number of stored memory patterns.
The similarity function that measures how strong the association are between the inputs similarity, similarity measure, measure, association).
(im((, x)
sep(s)
The separation function, turning the output of sim(- -) (logits) to probability distribution.
mod(() [
The modulation function that governs how memory patterns are stored and learned.
(x)
The energy landscape, defined the vector space memory patterns.
T T(x)
The retrieval dynamics, defined on the same vector space as memory patterns. The probability distribution
p
produced sep(-).
The similarity score vector produced by sim(-.).
The query vector. Also, the input to the associative memory The retrieval result vector. Also, the output of associative memory. The variant distribution memory matrix E, governs how queries generated. Each query sampled from this distribution logether with its origin memory pattern The joint probability density function that measures the likelihood that & and x are observed together.
y
V(E)
xv(E,x
The conditional probability density function (posterior) that measures the likelihood that originates from x when observed x.
pv(E)x)
The joint probability density function (likelihood) that measures the likelihood that & generates when observed g.
pv(x(E)
The dimension-wist similarity vector whose value of the i-th index measures the similarity between the value of i-th index in and &.
q
q
The sorted version q (sorted in ascending order). The upper right triangle matrix of ones.
U
dis(E, x)
The (negative and squared) Euclidean distance similarity - -||x - -11
dot(f.
The dot product similarity x &.
The k-optimal similarity function that finds a d-dimensional subspace that maximizes the similarity sim(--) -) of the inputs within that subspace.
sim (E,x)
The similarity footprint function that generates the rich descriptor between & with sim(:; being the base similarity (or footprint).
ftptamm (E.x)
Satm (E.
The adaptive similarity function adopting ftpt.am(::) with sim(-,)) as the base similarity. The vectorized form the adaptive similarity function 8sim (-, +), and returns vector that measures the adaptive similarity between &. and for i E [N].
Ssim (E, x)
The final adaptive similarity function that aggregate multiple Saima, (v+)) for different base similarity sim(:s) -) footprint.
(5,
The vectorized form of the final adaptive similarity function s(,, .), and returns vector that measures adaptive similarity between & and for E [N].
s x)
The weight vector that turns the footprint into scalar, which is designed to extract information from the rich descriptor.
w
8
Scalar used to aggregate different adaptive similarities Saim (sim())) (.)). The loss function used to optimize w's and B's
C(g, V)
[n]
The set integers less than or equal to n. Return the sign (-1 or + +1) of the input.
sgn(#)
8(x)
The Dirac delta that returns when input and returns 0 otherwise.
Transpose of vector / matrix.
diag(v)
Transform vector v to a diagonal matrix.
Vo
A sub-vector of containing only the elements corresponding to indices in D.
|[v]|p
The E, norm.
Ise(v)
The log-sum-exp function.
15

---

Published as a conference paper at ICLR 2026
A.2 THEOREMS
We define the retrieval accuracy that estimates the retrieval performance of an associative memory under certain variant distribution.
Definition 8: Retrieval accuracy
Retrieval accuracy for an associative memory with retrieval dynamics T(-) is the probability that correct retrieval is met:
( arg arg (ln(())()))) margmin(( - arg max (r (pV
B(gx)~V((E)
E'E
E'E3
Pr
arg (i E'1l2) } = arg (ax
(Ex))~((E)
E'E=
e'ea
However, Def. is usually intractable p> is unknown and complicated. Therefore, we define empirical retrieval accuracy based on samples drawn from V(=), which is computable, and used in experiments (Section 4).
Definition 9: Empirical retrieval accuracy
Empirical retrieval accuracy for an associative memory with retrieval dynamics T(-) can be esti- mated by performing abundant retrieval tests:
arg minn -E'112 $)
B(gx)~~V())
E'EZ
Pr ({x))~((E)
arg (l - - E
e'E
Theorem 3: Equivalence form for decomposable base similarity
For decomposable similarity measure sim(-.) satisfying:
sim x) = Σ
sim(E, x.)
(6)
1=1
Let q: sim(E;, x,) (1 S d), and qí (1 (1 S d) be the j-th largest element in q. We have:
sim (x)
q
Proof. Let (N) A arg p (sim(ED xD)). and define Q(A) {q: i E D(*)), then
we know:
sim (k) (E. x) =
Σ q =
Σ (EQ(A)
k)D())
Then, we try to prove by induction that (k) Q 1k- (qx} for S d. and we let Q
For, 1, we can see that Q(1) arg max { q. satisfying the induction hypothesis.
Next, for S d, we assume that the hypothesis holds for all 1 < k < k, we can see that
k-1
1(=-1)= = Q(K-2) (qx-1} ((k-3) U (Âp- - = U (
j=1
Now, suppose the followings:
(1) We can find qi-.d s.t. a 41-1, Q(k) Q(k-1) U (a). This not possible because
(qr,... 2,- a} would be better choice for Q(k-1), and this contradicts with the assumption.
(2) We can find a E qu q qr. (()) ((k-1) U {a). This is not possible because
16

---

Published as a conference paper at ICLR 2026
there would be & elements in q1.-d that is larger than qu. that qy is the (K + laagestt element, which contradicts with the definition of ql- Q(A) ((=-1) U {a}. This is not possible because
(3) We can find E qu-..d s.t. q {Âr,... 9 9 is better choice for Q(k) compared to (â),". a because q
Therefore, by contradiction, we can see that D(R) ((k U ( and the hypothesis holds due to induction. Consequently,
A:
Q
U ==1
q
sim (E,
Σ (EQuK)
q
q
j
A.2.1 OPTIMAL CORRECT RETRIEVAL
We now start to prove Theorem
Let us begin with simple variant - the isotropic noisy variant.
Lemma 1: Optimal correct retrieval for isotropic noisy variant
A Hop achieves optimal correct retrieval (Def. 7) for isotrophic noisy variant Visisy (=) (Def. 4,
(x (x-E) E (O,00 II) for (E, x) DoOiS some E R.) for arbitrary memory matrix Rdx
Proof. We claim that the optimal correct retrieval is achieved when using sep(-) arg max(-), and ft (E, only (i.e., 31 and 32 = 0 in Eq. 4). That is, the retrieval dynamics should be:
T(x) = arg max { (w') fiptais (5'xx)
e'EE
arg max { ||5' -x(
E'E
This step be satisfied by setting W 1, and w, for 2 S i I d. Then, optimal retrieval is achieved only when Eq. is met. We first estimate the right-hand side of Eq. 1:
arg max {Pv.,, (E\x) E'EZ
arg max { PVr.. (x(E) É'EZ
d
arg max
In 2 - ||x E'11
e'
2
20
arg max
--llE' x112]
É'ez
The first step comes from Eq. 2, and assuming that the prior p(E) is uniform (which often the case for memory retrieval) or can be easily obtained from samples. And the second step comes from Def. 4. We can see that the derived results coincide with the retrieval dynamics derived before. Therefore, plugging the retrieval dynamics to the left-hand side of Eq. gives:
arg migmin (I](()) E'1l2] arg mig e'EE
arg max {-11e" x112] -5112
E'ez
N
arg max -115"" -x x11²} -Eall2 Er
8"
k=1
max -115" - x112] |- |1Ex x x(2] Ex 8"
Rm
arg max
1|E x||2]
E:
The second step holds as there always exists a g E E that let arg max." {-1Es" x 0, since the resulting vector of the arg max(-) E E, and g' iterates every column vector of E, thus must have coincided with resulting vector, and the thrid step holds for similar reason.
x113 E'1l2 =
17

---

Published as a conference paper at ICLR 2026
Therefore, show that the left-hand side and right-hand side of Eq. the same (arg max - -x12)) Thus, the requirement for correct retrieval is met for all (f,x) v(=), yielding optimal correct retrieval.
If we adopt footprint that does not sort the dimension-wind wise similarity vector by substituting q in Eq. 3 to q and gives ft (5.x) x) = Uq, we can prove the optimality for the standard noisy variant defined in Def. 4, which is more general than Lemma 1. However, the footprint ftptais (E, x) Uq
achieves high empirical retrieval accuracy, but harder to estimate analytically.
Lemma 2: Optimal retrieval for noisy variant
-oH achieves optimal correct retrieval (Def. for noisy variant Vnoisy memory matrix E RdxN
[ (Def. for arbitrary
Proof. Following the same spirit in the proof of Lemma One can see that the right-hand side (RHS) of Eq. is (similar to Lemma 1):
1
arg max (Pyine (E'xx)} arg max e'cE
In |(dii(())))))
2 (x -x- -x) diag(o)) E')
e'c
2
d
( x)²
arg max e'
While the left-hand side (LHS) of Eq. is (step follows how RHS is resolved in Lemma l):
arg
(x)
<'1l2}
arg max { {w
U (Ex - =x)
ELEE
arg max { [ (Ex x)
EL
d
Σ
u - x2
arg max E
Here, v² E R&d denotes dimension-wise square operation over E Rdd so that (Ex x)}
(Ex.i - x;) Also, we let uu = w U T for simplicity. One can see that LHS equals RHS when:
Vi, i [d], u.
Since U is full-rank matrix, so that w = =u holds. When we set w in the following way:
T 0
i=1
W,
2sisd
LHS and RHS of Eq. 1 are the same, satisfying the requirement for correct retrieval. Furthermore, can tell that Lemma is special case of this lemma.
Lemma 3:
Optimal retrieval for masked variant
A Hop achieves optimal correct retrieval (Def. 7) for masked variant (mmasked()) (Def. 5 for arbitrary memory matrix = dx and for uniform generator (pg(-) is constant).
Proof. As in Lemma 1, we first reformulate the RHS of Eq. 1, and find the suitable choice for w make LHS of Eq. equal RHS. We let q be the fimension-wise similarity vector with qi
18

---

Published as a conference paper at ICLR 2026
-(E: xx)), and the vector that sort q in ascending order. The RHS can be expanded as:
arg max {Pransed (5)1x)}
E'Ea
arg f'ea
In Pmnasked
Σn 8(x; [ Σ1
8(x; E:) In (x(
1=1
arg max E'EZ
(ln Pmaased + (l1pp))
[1 8(x: - -Ei
1=1
d - d
arg max E'Ea
8(x; - Ei)
arg
((q))
E'E
Here, the second step is valid as Pç a constant, and we term this constant pç. As pa is a constant, it must be less than or equal to to make p.(() valid probability density function. Additionally, we know 0 S Pmasked < 1 from definition, and therefore, In Pmasked + In pg < 0, and this explains why sign change occurs in step three. The derived RHS suggests designing discrete adaptive similarity:
8dis (E. x) w (T)))()))))))))))))()))))) = Σ w.)(g:)
Then, the LHS would be (first step following that of Lemma and recall we use arg max(.) as separation function):
arg min (x)) - E' = arg max {w 8(q.)) = arg max E'E3
Σ w. -(ā.))
ELEE
& EB
Setting 1 concludes that LHS equals RHS for all (f.x) V(E), and thus, the optimal correct retrieval is achieved for this concrete adaptive similarity Sais x).
It can be shown that it is impossible to find a continuous (E x) for masked variant's ' optimal correct retrieval, unless more constraints and are made. Typically, such continuous function is possible if || - x||2 2 (can be bounded from below) for s > 0.
Lemma 4: Optimal retrieval for biased variant
A-Hop Hop achieves optimal correct retrieval (Def. 7) for biased variant Viissd [3) (Def. 5 for arbi- trary memory matrix = dx N and an arbitrary difference vector d Rd
Proof. Following the proof to Lemma 1, the LHS of Eq. 1 is:
d
arg max {Prino (E')x)} = arg max
d
8(x; - gl -d)d)
E'EE
f'EE
1=1
arg max
-d +
Σ
5(x; - di)
-di)
e'c
(=1
arg max E'EE
{-|[x g
2
The last step follows that the maximum score is both 0 before and after the transform, and the goal assign high score (0) when x - ==d. =
Here, we use similar continuous adaptive similarity
defined in the main text:
sain (E, w Uq 9 T
19

---

Published as a conference paper at ICLR 2026
with q E, and set a =w with 2d". Then, the RHS of Eq. is:
arg min (1] El1l2) = arg max {u 1--- q'ap
f'cE
ELC
arg max (2d q d d'd)
arg max - (q d) d) (q (q- d)}
arg max
{ ( || d
E.6
arg max
{- E [2]
ELCE
This follows immediately that LHS equals RHS, and the optimal correct retrieval achieved when:
[2d] 2dd,
==1
W;
-2d,-1
2sisd
It finally comes down Theorem 1:
Theorem 1: A-Hop's retrieval dynamics for optimal correct retrieval
ret-dyn The following retrieval dynamics adopted by A H-H achieves optimal correct retrieval for noisy, masked, and biased variants, with a careful design of s(E, x):
T(x) = E sep(s(E, x))
Proof. First all, sep(-) arg max(-) :-)
is crucial for achieving optimal correct retrieval, as transforms the left-hand side of Eq. (see Lemma 1):
arg arg min (x] - E'lla} = arg max (Ex.
E'
E.EE
In Lemma 2, we see that using the following adaptive similarity achieves optimal correct retrieval:
Sais (5, x) = w Uqq with w, = {(-2--
-
i=1 2
In Lemma 3, we see that using the following adaptive similarity achieves optimal correct retrieval: (Eia x) = w 5()) with w, = 1 for E [d]
In Lemma 4, that using the following adaptive similarity achieves optimal correct retrieval:
2dd with w; 22d, 2d,-1
i i=
Sais (E, U(x 2) -xx (x-E)) Tx E
2 2 d
One can see that achieving optimal correct retrieval is not easy, and it requires the sacrifice of continunity. However, we can build continuous adaptive similarity inspired from the proof of Theorem that achieve high retrieval accuracy (at least, empirically). For more discussion on this topic, please read Appendix A.3.1.
A.2.2 UNIFIED ADAPTIVE SIMILARITY AND ENERGY FUNCTION
We find that the adaptive similarity in Lemma has the form:
s(E. x) -(x Q) (x E
while that Lemma 2 has the form:
s(E, -(x E) diag(a)(x E
20

---

Published as a conference paper at ICLR 2026
for some diagonal matrix diag(a), and a, > for all i E [d]. Meanwhile, for Lemma 4 has the form:
x(, (x 5) (x { b bx (x-E) for some real vector b. That is being said that we can unifies these three adaptive similarity by:
Sunify (E. x) -x 5) diag(a)(x
<) + b (x E)
However, this similarity is too tough, we can analysis simpler one:
Sunity Sunity (5.x) = (x 5) (x - {) + b (x E)
If we use softmax(-) function as the separation function and Sunify (E, x) as the similarity function, and construct an energy function (with Sunity [E,x) being the vectorized form of Sunity(E,X]]
E(x) Iso(Suniry (E)
whose gradient is:
Ex) = soll x)) V Sunify
Letting pxxx) & (aatmax (Ex:))::
vxE(x) Σ Px(x)V Sunify (En)fy(E
N
Σ
px(x) (-2x + 2E, + b)
i=1
N
2x b -
px(x) &:
i=1
Retrieval on the gradient flow gives:
dx -=-VxE(x)
= -2x +b > (x(x) &.
dt
Then, consider using gradient descent with step 77. where 77 > for discrete-time retrioval:
(1+1) = x(1 Vxx(((()))
N
= (1 (1-2n) 2n) (() + n/b + 2n Σ ()
By setting = 2 that would cancels the x term on the RHS and remove the coefficient before the summation, which is wonderful:
(++1) 1 b E\t softmax(sumit, (E,
2
From Lemma 4, we know that the setting = 2d is optimal for noisy variant, plugging this in gives:
(++1) d Txx (t) = softmax(suniy (E, x))
(8)
suggesting that we add new bias term - d for biased variants, which coincidentally. remove the bias vector d. However, when there is no bias, Eq. 8 reduce the simple retrieval dynamics we are familiar with.
We then further analysis the behavior of the energy (Eq. 7) retrieval dynamies (Eq. 8):
Lemma 5: Rewriting the energy
Let the energy function be F(x) -Ise(s(E, x)), where s(E,,) -(x - T (T)
Ex)
(x E.). Then, the energy can be written as
E(x)
|x|
Ise Ax +xx c
for some matrix A E Nxx and vector c E R
21

---

Published as a conference paper at ICLR 2026
Proof.
E(x)
Ise x))
exp | 1xx | + + b) Tx x + b.
In
exp(- [|X||||| ||x + exp [22 b) [ x -
n + ||x|| lse (Ax+c)
for A, = 26; + and c; = E.- || 113 Also, we can omit the term In n as it is a constant.
Therefore, have decomposed the energy function Exx) into convex function g(x) ||x| and a concave function -h(x) - Ise( Ax c).
Lemma 6: Decreasing energy
Energy function Exx would be monotonically decreasing using the retrieval dynamics:
(1++1) = TTx
E softmax(s(3 x)) for s(E,, x) = -x T E,) +b)+bb E,)
Proof. Using the concave convex procedure (Lanckriet & Sriperumbudur, 2009), we construct convex surrogate function (x( for each iteration by linearizing the concave function -h((x around the current x(();
U.(x) = g(x) [xx(()) vxh(x(0) - (xxx(())
|x12 - Vxh(x(0) x (Vah(x(t)) x(1)) h(x(()))
The next (1+1) is the minimizer of (x)),, i.e., x(++1) arg min (Ux())} and we can find it by setting gradient to zero:
VxU((x) 2x Vx((x(())= =0
((+1) [x(((() = A softmax + c)
We can add term - =|x back to softmax(-) as independent index Thus, by denoting (x(x) = softmax (1 c); = softmax (s,,xx(s(x)x)))))))))))))))) x)); (follows Lemma 5), we have:
(++ 1
A; pi(x!()
2
N
1 N
Σ
c (x())))
- Σb> (x
i=1
i=1
1
S softmax(s(E,x (())
b
and this agrees with what we have derived before (Eq. 8).
Then, by convexity of h(x) (recall -xx) is concave), have the following inequality:
h h(x) > (x) + (x x(())
g(x) l((x) < (x) - (1) Vxh(x (x- (00)))=0+++) -U(()
=
with the equality holds iff (4), and recall that (1+1) the minimum value of Ux)),, we have:
(++2) < (++1) S (x) = (xx(())
with equality holds when (1) ((+1), Thus, E(x((+1)) & E(x(() completes the proof.
22

---

Published as a conference paper at ICLR 2026
We can see that Exx) | [ that E(x) + t00 as |||| to0, E(x) is coer- cive, meaning its level sets are compact. Additionally, (x( is 2-strongly convex as V2Ui(x) = 21, therefore,
Uz(x!0) - U,(x((+1)) 2 ||x(0) - x1++1)1
Along with Inequality 9:
E(x(0)- - E(x(++1)) 2 Ux((x())
Ut(x((+1)) 2 ||x xx
This yields that
F(x F(x
Σ - -x
This mean that F(xx()) bounded from above. We can see that the sequence of x: (x(*) must remain within the level set fx E(x) I E(x(o))) as the sequence E(x(O) is non-increasing. Therefore, fx(*)) must remains in the compact {x E(x) S {((x()) and be bounded.
Theorem 2: A-Hop's decreasing, convergent, and bounded energy function
energy Energy E(x) will be monotonically decreasing, convergent, bounded from below, for isotropic noisy, and biased variants, if the following energy is used:
(x) Ise (s (3)
Proof. In Lemma 6, we have proven that the energy function will be monotonically decreasing.
Also, from above analysis we can see that Exx((t) could be bounded by:
E(x()) < ((x Σ [x(++)) -x(11
On the other hand, we can try to bound E(x(t)) from below:
F(x(1)) - In exp (-
x Ex112 + +bT E:))
N
In
Σ
exp
(-||x
x
E 112 + |lb||2 ||x E:lla)
N
|bll
|blll
2
In
exp
+
|bbl2
2
2
1=1
|bb
In N
As ||b|2 2 bounded, we can see that (x) bounded from below. Since F(x) is monotonically decreasing and bounded from below, we can simply put that it is convergent.
We try to find the property for very different energy landscape and different retrieval dynamics, also these retrieval dynamics can guaratee optimal correct retrieval, for iostropic noisy, and biased variant, when the separation function is arg max(-) and has weight b choosen ideally. Theorem tries to connect the noval correct retrieval and the traditional energy analysis.
A.3 DISCUSSION
A.3.1 ON GOOD DESIGN CHOICE OF ADAPTIVE SIMILARITY
Achieving optimal correct retrieval is costly, as it requires designing "weird" similarity function for corner cases discrete function that makes the model unlearnable. For instance, it is impossible
23

---

Published as a conference paper at ICLR 2026
for continuous similarity function to achieve optimal correct retrieval for masked variants, as they have a hard time distinguishing (5, Xx,| = 0 and E - E. = c for some arbitrary small c. Also, using softmax(() !) as the separation function make it hard to prove whether optimal retrieval is achieved or not, as we cannot exclude the effect of other memory patherns from the one receive largest similarity score, while using softmax(-) is crucial for learnable adpative similarity.
Therefore, achieving optimal correct retrieval for a certain variant distribution is not the only golden criteria for similarity measure. More generally, we think good (adaptive) simliarty measures should be assessed from the following aspects:
Adaptivity Can the similarity measure adapt to wide range of variant distribution? This criteria measures the breadth of the similarity.
Correctness How accurate can the similarity measure predicts the variant distribution? How often does the associative memory achieves correct retrieval? This criteria measures the depth of the similarity.
Learnability Can this similarity measure be learned easily when some samples from the variant dis- tribution is provided? How many learnable parameters it requires? This criteria measures the scale of the similarity.
Efficiency What is the time complexity of calculating the similarity score? This criteria measures the speed of the similarity.
It is ideal but not likely that we can find a perfect adaptive similarity that meets all above mentioned criteria, because there are tradeoff's between these aspects. For example, for the adaptive similarity introduced in the proofs to Lemma 2, 3 and 4, they only achieves optimality (optimal correctness) in noisy, masked and biased variants but would have poor performance in other variants (i.e., poor adaptivity). This is because adaptivity is sacrified when every corner case (cases happen in a very low probability but need extra care) is handled carefully. On the other side, optimality would be lim- ited for similarity with strong adaptivity because the outcome for some corner cases could be con- tradicting for different scenarios. Moreover, learnable adaptivity similarity might not often achieve optimality as the learning alogithm cannot always guarantee convergence of weights to the optimum state (e.g., minimum loss). Also, large-scale similarity measure with plenty of learnable parameters are not likely to have good efficiency, as their computation procedure could be complicated.
The design of the adaptive similarity x)) = fttt (E, x) + ftpt (E.x) in Eq. 4 (il- lustrated in Fig. 2) a balanced result of considering four aspects - adaptivity, correctness, learaailiity, and efficiency. We will explain the design choice of this adaptive similarity from the perspectives of these four criteria.
The similarity footprint The footprint is the core mechanism of adaptive similarity (Eq. 4 and Fig. 2). Sorting and cumulative sum is not the essential part of it, while the idea of exploiting subspaces is. The idea is that considering subspatial information will give extra benefits compared to considering the whole Rdd itself. The footprint picks d most useful subspaces from all 2d 1 subspaces for much better efficiency (from computationally impossible possible). However, sorting and cumulative sum happen to be the right algorithm for computing the similarity score efficiently for decomposable similarity measure (Eq. 6), it doesn't mean that the footprint is about sorting dimensions. Regard the footprint as mining the most valuable subspatial information from all 2d subspaces, and it provides useful evidence for guiding the model to make decision.
Recall that ftptzim (E, x)x sim (*) (E,x), x), and sim (k) (E, x) is the k-optimal similarity between and x, which finds the optimal k-dimensional subspace such that & and x is most associated (closest) on this subspace. That to say, the footprint finds a representative for each dimensionality & E [d] and regards the one that minimize the distance between & and x the most useful k-dimensional subspace. But the positional information of each subspaces is not used (i.e., which dimensions contributes to sim (*) (5. x) is unkonwn) this would harm the correctness compared to a similarity measure that exploits all 2d 1 subspaces. Although finding the positional information of each subspaces is possible (which would benefits correctness), it would need (dd) time for computation (which is not worthy).
However, compared to fixed proximity- based similarity (c.g., {Tx), the footprint offers a multi- scale descriptor to the relation between & and (the evidence for deciding whether & generates
24

---

Published as a conference paper at ICLR 2026
x). The k-optimal similarity sim (x.)) ignores d k non non-informative dimensions, which make it inherently suitable for handling masked dimensions and heavily noisy dimensions (by simply ignore them). Therefore, the footprint enables better correctness against fixed proximity-basee similarity as it offer richer evidence (d v.s. 1) for models to make the right decision.
Linear combination of footprint elements The proposed adaptive similarity calculate the sim- ilarity score the dot product of (t. (a d-dimensional vector) and learnable weight w (also a d-dimensional vector). This allows the model to behave differently (by choosing differ- ent weight) when facing different variant distribution. Also, one can see that when Wa and Wx 0 for 1 I h < d. the adaptive similarity degenerates to its base similarity as w ftpt.im (E. x) sim(g. x). That is to say, the adaptive similarity has better expressiveness (better adaptivity) and is at least as good as the fixed proximity-basee similarity.
However, one can use a more complicated method (e.g., train neural network to let a({, x) MLP((E fttt (5. x)) to extract information from footprints and gains better correctness and adaptivity. But this also brings more computation cost and may make the model harder to train. Using a linear map is the simplest and fastest way to exploit the footprint, and empirical results demonstrates that it can largely improve the performance of existing associative memory models in the task of memory retrieval.
Aggregating among different similarities The proposed adaptive similarity aggregate the sim- ilarity footprint with base similarity dis({.x) = -Il - x1| and dot(g. = = x. Aggregating adaptive similarities with different base similarity would improve model's adaptivity and correct- ness but reduce its efficiency (scale up the runtime by a constant). As an example, when we find similarity 8 (::-)) that achieves optimal correct retrieval under variant distribution V1, and another similarity -2(-,)) that achieves optimal correct retrieval under another variant distribution V2. Ide- ally, the aggregated model can set B1 and 32 under V and B1 and 32 = I under V22 to achieve optimality under both variant distribution.
Think of base similarity as base vectors in a vector space, and the aggregated model resembles the span of these basis. The learnable parameter $1...B controls how each base similarity is used in different scenarios (the weight of each basis). It is clear that span(Vi, V2, Vn) covers more points in the vector space than span(vi) for B > 1, and covering more points means achieving better correctness in more occasions.
We can further improve the proposed adaptive similarity by letting it aware of dimension-specific information by setting q, = a, siim(: x,) + b, (for < i S d) with learnable weight a, b E IRd. This modification introduces (dd) extra parameter to the model. but its complexity does not change as we already have (d) learnable parameter (for B = 2 O())). From different perspective, is now the dimension-wei wisee similarity vector after a affine transformation.
Next, we show that the proposed adaptive similarity s(, x) w Uq is suitable (has high correct- ness) for noisy + masked variant. Here, we assume that, with large probability, the affection brought by masking is much larger than that of noise. That is to say, for a certain dimension
i S d and |x; Ex.il when the i-th dimension is masked and A mossy Ex,al
(Ex. x) (E), let A masked
when the i-th dimension is not masked, we will have A masked >>
A. moisy
with large probability.
The intuition is that sorting (i.e., q (q), Fig. 2 (2)) can separates masked and un-masked dimensions. Since A musked >> A moisy with large probability, q: (the dimension-wise similarity) would preserve this ordering for masked and un-masked dimensions. Suppose that x; is masked (i.e. x; = 0) while x, is not masked (i.e., X, = ENj + Ej with e, ~ ~N(00, o))), then q; x mmAminked and q < jaiiy), which results in q < q with large probability. When there are exactly m dimensions are masked, and by sorting Qi... in decreasing order (to obtain q), the values of qi-.d-m would be significantly larger than that of qu-m++1: d.--d- It is because qm are dimensions that are not masked, while 9d-m+1---m are dimensions that are masked. By setting Wd-m = 1 and w, 0 for i # d m. the associative memory excludes all the masked dimensions and treats the remaining subspace as noisy variants. Let s(E. x m = k) denotes the adaptive similarity when assuming there are exactly k masked dimensions. Then, we define the resulting adaptive similarity
25

---

Published as a conference paper at ICLR 2026
as the expected similarity score
x(,
Pr [exacctly is dimensions are masked]
s(E, x m =
d
(1 Pmmasked ) A
o(d - o(d-k- Σ
Pinaseed
5(d - k -1) (1)
(10)
(
Σ
Pimusked
(1
Pmasked )a
k=1
P Puusseed
Pmmasked)
(Uā);
i=1
w UU
This suggests that the proper choice of w is to let w, - (i) P Additionally, we can let qi = J. (E: - x.)² (1 S i S d) to turn noisy variants into isotropic noisy variants as suggested in the proof Lemma which is to say, the optimal choice for and b is a, -0," and b, = 0 for 1 < i < d. Furthermore, one can examine that the adaptive similarities that achieve optimal correct retrieval in noisy variants (Lemma 2) masked variants (Lemma 3) will not achieve optimality, or even cannot sustain a high retrieval accuracy. This is because the similarity used in Lemma 2 will confuse masked and un-masked dimensions, and similarity used in Lemma will always returns 0 as adding noise make & x; 0 not possible.
Then, we will show that the w T Uq adaptive similarity achieve optimal correct retrieval for noisy and biased variants, and achieves correct retrieval with high probability for masked variants.
guarantees optimal correct retrieval. With Pmasked O, w turns into w, 0(d i) (1 S i S d) (i.c., wa and w, 0 for 1 S i < d - 1). This degenerates the model to the one defined in the proof to Lemma 2 (with a, -0, and enable the model to have global view of the entire Ed space, and electron cloud of said adaptive similarity would look like a shpere illustrated in Fig. (a).
For noisy variants, using the w derived in Eq. 10 with Pronske
Next, for biased variants, similar to how we deal with the noisy variants, we let Pmasked = 0 in the weight derived in Eq. 10, which is w, = 5(d-i) (1 < < d). Then, we let a; = 1 and b, = d3 (1 < i S d) so that for (E, x) Vniased(S) we will always have qi = -E, x,) b,
d3 + b, 0. As q O, then q = 0 as well, and this would results in s(E x) w Uq 0 achieving the largest value of such adaptive similarity. In other words, the adaptive similarity s(E, x) is maximized when (E.x) Vonsea (E) (i.e., + d).
Finally, for biased variant, achieving optimal correct retrieval is not possible for continuous adaptive similarity function, unless the difference between query and any stored pattern can be bounded. Formally, when there exists 0 < < 1, s.t. for all (E. x) ~ V(E) and i E [d]. we always have
c << (Ei-xx - xx,)² = -q < Then, we set (w UU; = c 3(d-i) and this leads to:
s(Exx) T TUp
Σ
3(--1) q
Let s'(E,x) - 8(ā:) (the adaptive similarity used in Lemma 3), we have:
3(d=''(( <
E 3(d-1)- < s((Ex) < - ((d-)((()xx))
(11)
1=1
3d s(E,x) E ( 2_ -2
As result of Eq. 11, s' x() s' (E2,x) iff x(E1,x) s(2,,x) for E1, 3 (i.e., s(E.x) preserves the ordering of s' x)) And since (E, x) achieves optimal correct retrieval (Lemma 3), s(E, x) also achieve optimal correct retrieval in this case.
26

---

Published as a conference paper at ICLR 2026
However, when qi cannot be bounded by such O, try find the probability that s(E. x) does not achieve correct retrieval. Let Vm--aased( [)) denotes the variant distribution when exactly m dimensions will be masked (i.c., s'(E, x) d - for (E.x) V masked ())). Let Per denotes the probability of s(S, x) = w T "q not achieving correct retrieval under masked variants. Formally, when E\t and & generated randomly,
Perr
Pr (E:x))Vussaaaf
[s x) 8('', C'EB,E''
Pr ((x))~Vuaal13 E'E
(uq)s <
w, (Uğ);
j=s' (1,))+
Σ m=1
m) Prmaseed
(1 Pmasked m
Pr (5.x) E'EE,E''E
(
w: (Uq')
Lind +1
This is diffieult to estimate because it involves sortings and summation over random variables of different distribution. We computed the numerical error Per for &, x E [] through Monte Carlo estimation. With dimensionality d = 64, Pei 6.8 x 10-11 1.6 x 10-9 1.2 x 10-6 and 0.001 for Pmasked = 0.1, 0.25, 0.5 and 0.75, respectively, when using the w derived from Eq. 10. This result verify that s x) w 'ğ achieves correct retrieval with high probability.
The main point of this section is that achieving optimal correct retrieval is too strict so that it forces us abandon other properties. However, optimal correct retrieval itself is a good property when the underlying variant distribution is known priori, but this is not always the case. Therefore, good design choice of adaptive similarity should consider not only optimality, but also adaptivity, learnability, efficiency.
A.3.2 ON MORE COMPLICATED VARIANT DISTRIBUTION
Variant distributions discussed so far are simple. In previous analysis, we assumes that all memory variants follows a similar distribution. For example, (E, x) ~ (=) ensures that all memory patterns generates a noisy variants, sharing a similar (x E). We say a variant distribution general knowing Py (x) is equivalent to knowing Ex(,) for arbitrary i.j E [N] and i * j. In other words, we can obtain (x({)); by substituting x, with x,. However, there could be cases where each memory patterns generates variants quite differently. Intuitively, we say each pattern generates on their own, meaning that (x({) has completely diffeerntt form for different memory patterns, and we call such memory variant isolated.
By looking closely to the adaptive similarity function:
s(E,x) = Up
TU
One can see that it uses a universal weight w for all pairs of (E.x) ~ V(B), assuming that the variant distribution it is trying to model is general. However, such limiltaion be easily broken by introducing more weights. For instance, we use separate weights for different memory patterns, i.e., for N memory patterns, we spare weight We to memory pattern &x. Therefore, the adaptive similarity that can suit isolated variant distribution look like:
s(Ex, x) w Ua
This can fit each isolated likelihood (x(E)) as the weights are no longer shared. But this rises more problem: (1) it requires more samples, and it might be impossible in real-world scenarios. (2) it requires samples generated by each memory pattern &R. as each individual weight wi: is optimized by samples involving Ex.
Additionally, the adaptive similarities are a family of similarity measures that can fit to the variant distribution through sampling, it is not solely Eq. 4. When proving the optimal correct retrieval for noisy, masked, and biased variants, we propose more adaptive similarity as theoretical tools. We wrote Eq. 4 in the main text simply because it is the most effective ones for retrieving under noisy, masked, biased, and mixed settings, and it requires minimum trainable weight, as examined in ablation study (Appendix A.4.6).
27

---

Published as a conference paper at ICLR 2026
One more thing is that the priori py(g(x) are often ignored in this work. Well, it should not be ignored in all cases. However, we can use a bias term b E IN and use by capture the occurrance of pattern Er. By assuming similarity is a logits of the posteriori pv (v(E)x) (e.g.. log (E(x), then we can see that arg m (log (logrv({) (5])) = arg maxg'ez (log (x(("()) + log (og))(()) } and we set pass s(E, + b to the separation function so that b handles the term log pF
A.4 EXPERIMENTS
A.4.1 BASELINES AND METRICS
For memory retrieval test, we compare Adaptive Hopfield network A A-Hop against: Modern Hopfield network M- Hop (Ramsauer et al., 2021); Universal Hopfield network N- N-Hop (Millidge al., 2022) with sim(E, = - x and scp(.) arg max(.) as they report leading perfor- mance for such configuration when masking out half of the dimensions in memory patterns (similar to masked variant, Def. 5) Kernelized Hopfield network U-Hop (Wu et al., 2024a) with the kernel optimized by separation loss proposed by Wu et al. (2024a); Kernelized Hopfield network U2 Hop with the kernel optimized by loss defined by variant distribution (Eq. 5) rather than the separation loss; and Multi-Layer Perceptrons MLP that has 4 layers and input dimensionality d, output dimen- sionality N, trying fit (E|x but unsatisfactory. We estimate the empirical retrieval accuracy of each model (Def. 9), and the mean squared error (({x)))V(E)) [(T(x) <) (T(x) - }) We wrote a generator that can generate noisy, masked, biased, and mixed variants.
For tabular classification test, we compare the A-Hop with a memory- classifier (Ap- pendix A.4.4) with: M-Hop uses the same classifier framework but the unlearnable similarity func- tion; HopHop that uses the same classifier framework but a different similarity function, and its kernel function is optimized by the classification loss; Extremely Randomized Trees (Geurts et al., 2006), or Extra Trees, a tree-based classifier; Random Forest (Breiman, 2001), yes, the famous Random Forest classifier; AdaBoost (Freund & Schapire, 1997). a classic boosting classifier; and XGBoost (Chen Guestrin, 2016), an enhanced Gradient Boosting Decision Tree. All models are tuned with 5-fold cross-validation
the training set. We measure the test accuracy for the dataset with a positive sample rate 0.2 < % pos < 0.8, and the ROC-AUC score otherwise, following the settings in Wang et al. (2025).
For image classification task, we follow the settings in Wu et al. (2024a), where they replaced the attention component with a HopfieldLayer ldLayer (Ramsauer et al., 2021). We experiment by inte- grating AoHop, M-Hop, and U-Hop into the hopfieldLayer Similarly, we follow the settings in Ramsauer et al. (2021); Hu al. (2023), where they HopfieldP colloonig ing for multiple in- stance learning. We integrate A A-Hop, M o-Hoo, 5-fold cross-validation report the mean ROC-AUC of all folds the result.
and U-Hop Hop to Hopfie SerfieldPooo dPoo l ling, and run a
For all experiments, we report the results with the mean and standard deviation of five runs.
A.4.2 DATASETS
We used a total of 12 datasets to assess the performance of A-Hop on tasks including tabular clas- sification (Adult, Bank, Vaceine, Purchase, and Heart), and image classification (CIFAR 10, CIFAR 100, and Tiny ImageNet), and multiple instance learning (Tiger, Fox, Elephant, UCSB).
Adult (Becker & Kohavi, 1996) The prediction task for this dataset is classify individuals' in- come levels as either above or below $50,000 annually. The data was extracted by Barry Becker from the 1994 Census database.
Bank (Moro et al., 2014) To predict the success of a term deposit subseription, this dataset records the outcomes of telemarketing campaigns from a banking institution in Portugal.
Vaccine (Bull et al., 2016) We use this dataset from DrivenData competition to predict if a person received seasonal flu vaccine. The data consists of 26,707 survey responses detailing 36 behavioral and personal attributes.
Purchase (Sakar& & Kastro, 2018) The objective with this dataset is to forecast the online shopping intentions of visitors to an e-commerce website, determining whether a user will proceed with purchase.
28

---

Published as a conference paper at ICLR 2026
Heart (Kaggle, 2018) This dataset facilitates the prediction of cardiovascular disease presence. It contains health-related data from 70,000 patients, as provided by Ulianova.
CIFAR 10 (Krizhevsky & Hinton, 2009) is a classic image recognition dataset consisting of 60,000 32 x 32 color images in 10 classes, with 6,000 images per class.
CIFAR 100 (Krizhevsky & Hinton, 2009) is a more challenging version of CIFAR 10, containing the same number of images but split into 100 fine-grained classes.
Tiny ImageNet (Deng et al., 2009) is a subset of the ImageNet dataset designed for educational purposes. It contains 100,000 images from 200 classes, downsized to 64 x 64 pixels.
Tiger, Fox, Elephant (Deng et al., 2009) These specific class subsets extracted from the large- scale ImageNet database, often used for fine-gr teeegrained image classification tasks.
UCSB (Gelasca et al., 2008) This dataset is composed of 58 breast cancer histopathology images stained with Hematoxylin and Eosin (H&E). The primary challenge it presents the ac- curate segmentation of individual cells from the complex tissue background, which is a critical precursor to classifying cells as benign or malignant.
A.4.3 MEMORY RETRIEVAL
We first introduce the intensity of variant setting. A triplet (dmask; duoiiee dhias) E [0. 1] is used to describe a variant setting. This mean that we will first add Gaussian noise n ~ [O, dnoisc to a certain memory pattern & E = obtaining x & + n. Then, will choose d dmask indices, and set these indices of x to random numbers choosen uniformly from [-1, 1]. Finally, we will add a bias d to + d with d; s; doius: where s; is random sign sampled uniformly from {-1, + +-) Then, the generator return (E, x) as a sample of (()). Therefore, different triplet describle different variant settings, and thus, corresponds different variant distribution ((
In the memory retrieval task, we use the retrieval dynamics written in Eq. 4. To learn the weight w's optimizer Adam for 200 epoches, and use learning rate 0.1 in all settings. However,
and fi's we argue that number of epoches (N-epoch) and learning rate (1r) should be tuned when applying A A-Hop to other memory retrieval settings.
For U- HoHop and U2 Hoop we tuned them carefully. For the original U- HoHp, it optimize a separation loss, and we try make it as small as possible. However, the retrieval accuracy of U-Hop is not satisfactory, and it is reasonable since it is not optimized for correct retrieval, but for e-retrieval. We found that the domain of the memory pattern in their experiments is [0, 1]d and it is harmful to dot product-base based methods (think of using only 2-4 of the space) as dot product highly relies signs.
Therefore, for fair comparision, we sample the random vectors uniformly from [-1, 1] and change the domain of pixels in MNIST images to [-1,
A.4.4 TABULAR CLASSIHICATION
We develop memory-based model for tabular classification that takes advantage of the excellent memory retrieval effectiveness of associative memories. The insight is that classification is hierar- chical, and we divide instances into cases, and further classify cases into the final class. For instance, there are type diabetes and type II diabetes, where each type here resembles the idea of cases. An- other samples is that cats has plenty of breeds, and associative memory can capture specific idea of orange cat, blue cat, while they have a hard time figuring out the generalize idea of cats. So, let associative memory match instances into cases by choosing some instances in the training set as the representatives of cases, or use K-means cluster to produce such representatives, and pass the retrieval probability to a multi-layer perceptron (MLP) for classifying cases. That is, the model can be represented as y MLP (LayerNo rm(sim x))). and this can be trained on a conventional machine learning fashion by minimizing classification loss:
L(D) = Exxxy)>Dly )
(12)
We no longer tune weights in adaptive memories using loss defined in Eq. 5, as they can be tuned using the classification loss when participanting into going forward in the network.
We tuned the hyperparameters by grid searching on the training data:
29

---

Published as a conference paper at ICLR 2026
Table Hyperparameters tuned in tabular classification
Namc
Domain
Neppoch batch-size
(50, 100, 150, 200, 250}
{100, 1000)
1r
{2. 10 10 8 5 10- 10-110---0000 ,,10"
init
Cluster, No cluster
A.4.5 IMAGE CLASSIFICATION AND MULTIPLE INSTANCE LEARNING
For image classification, we follow the settings in Wu et al. (2024a), and place the adaptive similarity the Hopfield layer inside image Transformer. For M-Hop and U-Hop, we use the hyperparame- ter suggested in Wu et al. (2024a), and find the optimal number of epoch and learning rate for A Hop via grid search. We run five iterations of separate loss optimization for A- A-Hop before testing.
Table 7: Hyperparameters tuned in image classification
Name
Domain
epoch
{25, 42,, 40, 50) {10-3, 10-1)
1r
For multiple instance learning. we follow the settings Hu et al. (2023), and use Hopfiel dPool ing the backbone. Similarly, we place the adaptive similarity in the core Hop- field component replacing the fixed dot product. All experiments are run on a 5-fold validation, and the ROC-AUC scored taken the mean of all folds.
Table 8: Hyperparameters tuned multiple instance learning
Name
Domain
Neepoch
{10, 20, 40} (10-3, 10 10
1r 1r.decay
(0.9, 0.75)
A.4.6 ABLATION STUDY
We conduct four different ablation studies to see the effective of different components.
In the first experiment (Table 9), we tested if sorting the dimension-wist similarity vector q and the upper-right triangle matrix U is needed. The results shows that both of them are necessay for high retrieval accuracy.
Table Retrieval accuracy (t) and error (L) between unsorted and sorted q. Each cell contains the mean accuracy or error with standard deviation in a smaller font. Results of the best-performing model are bolded.
Conditions
Synthetic (d = 0.4)
Synthetic (d 0.5)
sorted? use U?
Accuracy
Error
Accuracy
Error
5172+.004
1900
2094
.2658+..05
.5444+.0
o0
.1888+.15
.2738=
6928
.1133
.3374+
.23244..00
.7280..00
.1033:.01
.36341.000
.2207-t.01
Next, we look for a better matrix U (Table 10), which is the core of similarity measure. Our results show that the upper-right triangle structure of U is optimal, while making U learnable and initialize
30

---

Published as a conference paper at ICLR 2026
it with the upper-right triangle yields the best result, but does not adopt learnable U in other experiments to keep minimum learnable parameters. Another finding is that a randomized matrix U is better than not having U (when U = D). Therefore, along with Table 9, we find that the footprint structure essential to adaptive similarities.
Table 10: Retrieval accuracy (t) error (L) between different configuration of U. Each cell contains the mean accuracy error with standard deviation in smaller font. Results of the best- performing model are bolded, and the second are underlined.
Conditions
Synthetic (d = 0.4)
Synthetic (d = 0.5)
U learnable?
initialization?
Accuracy
Error
Accuracy
Error
X
Random
.6928
+3140+0019
.2055 4.00s
16802+.013
.. +.04
.3458 4-025
.2298 +.00
X
U
.7242=
.1066:
.3600)
.2272+.00
Random
.7176
-34144-023
:2292
.7114::0199
.1096.1.00
13528..00
.2270
V
U
.7280::034
.1033::011
.3634..0100
.2207::011
We also tested the effect of different footprint with different base similarity (see Table 11). Results shows that two footprints (dis and dot) is always better than one alone, and we found that ftptais performs better than ftpt in this setting. However, even though retrieval accuracy degrades by re- moving one of the footprint, using fip foplaot only is still better than other Hopfield networks.
Table 11: Retrieval accuracy (1) and error (1) between different usage of footprint. Each cell con-
tains the mean accuracy or error with standard deviation smaller font. Results of the best- performing model are bolded.
Conditions
Synthetic (d 0.4)
Synthetic (d 0.5)
use ftptain? use ftptaot
Accuracy
Error
Accuracy
Error
X
.59262.016
.1517..008
.25201.00 007
22515
X
.64582002
.13171.01
.32861 013
.23262..07
.7242::016
.1056:1007
.3600+004
.2272:2.005
Finally, we tested the number of samples needed for adaptive similarity to mimic the variant distri- bution. It looks like training on only 512 samples provides a good enough adaptive similarity for 2048 64-dimension memory patterns.
Table 12:
Retrieval accuracy (1) and error (L) between different number of samples provided for learning. Each cell contains the mean accuracy or error with standard deviation in a smaller font. Results of the best-performing model are bolded.
Conditions
Synthetic (d 0.4)
Syuthetic (d 0.5)
number samples?
Accuracy
Error
Accuracy
512
.7204+.02
1077=
.3541 4.028
.23 03
.7242
.1056-mm
.6000.024
.2272+.00
A.4.7 MULTIMODAL SETTINGS
We designed a multimodal memory retrieval task to further test the memory retrieval capacity of different associative memories. this setting, we have M different modalities, and N different concepts. For each concept < i < N. it has M different patterns &. E.2., ... E. its M different representations different modalities. Furthermore, each modality S < M has a unique encoder Aj, which encodes pattern f Rd in the j-th modality into hidden state
31

---

Published as a conference paper at ICLR 2026
Table 13: Retrieval accuracy (1) between models in the multimodal memory retrieval task. Each cell contains the mean accuracy with standard deviation in a smaller font. Results of the best persoreeroomnn model are bolded.
Model
Number of concepts
Phrase
Phrase 2
128
256
512
1024
M- M-Hop
M-Hop
.173
.092:
.044
.021=.0
U2-Hop
U2-Hop
.574..0
.349:.02
.1462.02
.081 =.02
Uz-Hop
A-Hop
.6361.03
.465
.2762:03
.183=
A-Hop
U2-Eop
+.04
.709+.o
.492+.04
.328 .02
A-Hop
A-Hop
+.77
-998+.00
.993+.00
.988
bas E Rdd,, by setting h. A Here, d is the dimensionality of each pattern, and d, is the dimensionality of each hidden state, and we can see that Aj E Rddx for all S S M.
The objective of this task that: given a hidden state variant of unknown concept and unknown modality, find its corresponding patterns in all modalities. Formally, let h' = h.j + e for some unknown concept 1 i N unknown modality S j < M, and a noise vector e E Rd drawn from the normal distribution (O, oI) (the isotorpic noisy variant, see Def. 4). Then, h' will be the query of the retrieval task, and M patterns yi...M should be retruned for each modalities. Finally, the retrieval correct is the following holds (i.e., correct retrieval is obtained in each modality):
Vj. 1.15j S M, s.t. arg min flly E'1l2] arg max (pv(E' Ei.j e'ea,
E'Ea,
Here, =, [Ei.ji Ez.ji 1..i--.--0 EN. collects all concepts' pattern in the j-th modality.
To address this challenge, the retrieval process requires two distinct associative memories. The first associative memory stores N patterns in form of concat(h,;, h.,,, ... .,. (i.c., the concatena- tion of M d, -dimensional vectors into one M d,, dimensional vector), remembering all the hidden states of the same concepts. When query h' comes in, a vector of form concat(h', h'...... h') given to the associative memory, it will find the corresponding concept i, such that one of the b;,1, hi,2, ... b.. is close to h', The second associative memory stores N x M patterns in form of concat(h,,, Ei.)) for all 1 < S N and S S M, serves as the decoder that finds the original pattern E.,, from hidden state h,,- When a query of form concat(h, v) is given to (v E IRd, the mask, can be any arbitrary d-dimensional vector), it will find the corresponding hidden state- satee-ptterrn pair concat(h,, Eig) so that h close to ha, and the original pattern is restored as E. Therefore, an additional M queries the second associative memory will retrieve all M patterns of the i-th concept all modalities, respectively.
For intuitive example, let h' stands for the textual embedding (hidden state) of inhaler, which (ideally) is a variant of the textual embedding of inhalator (represented by hg,1). So, when passing concat(hi h', hh) to the first associative memory, and it will retrieve the corresponding hidden states concat(y1. Y3) of inhalators in different modalities, which should satisfy that concat(y1, y2, concat (1,,1, h422, hg.3). Here, h is the textual embedding of inhalator, h is the visual embedding of inhalator, and h is the acoustic embedding. Then, we can retrieve the textual, visual, and acoustic pattern (i.e., Ex, Êx.2 and (x.3) of inhalator by querying the second associative memory (the decoder) with concat(y,, v) three times for all 1 < < 3.
In our experiment, we let M 3 (i.e.. three modalities), and N E {128, 256, 512, 1024) (varying number of patterns), d = 32 (each pattern is 32-dimensional vector), and d, = 16 (each hidden state a 16-dimensional vector). Furthermore, each pattern É., is generated randomly and inde- pendently (i.c., each dimension of E., follows the uniform distribution on [-1, 1]), and the encoder matrix A, is generated similarly. For each query, certain hidden state h; is randomly picked from all N x M hidden states, and noise (0, with 0.2 is added it (i.e., h' h + c).
We use different associative memories as the first (Phrase 1) and second (Phrase 2) associative memory mentioned above. In Table 13, we observe that A-Hop greatly outperforms all other combinations in all settings. For all other combinations other than A-Hop + A-Hop (last row in Table 13), sharp degeneration in retrieval accuracy can be observed the number of concepts (N)
32

---

Published as a conference paper at ICLR 2026
doubles. However, A-Hop + A-Hop retains 98.8% accuracy even for N 1024 concepts, while the runner-up (A-Hop + U2-Hop) only scores 32.8%. By observing the last four rows Table 13, one tell that the retrieval accuracy increases the involvement of A-Hop increases, which indicates that A- A-Hop is better surrogate for the underlying variant distribution both phrases.
A.4.8 TIME COMPLEXITY
Training time (L) and inference time (4) of Hop with and without sorting. Ratio denotes the quotient of the runtime of sorted version divided by the unsorted version.
Table 14:
sort?
Scaling dimensionality
Scaling number of patterns N
128
256
N
2048
2048
2048
2048
4096
8192
Training time (ms,
V
840
1573
3609
840
1531
2949
X
2093
3059
8224
2093
4011
7983
ratio
2.49
1.95x
2.27x
2.49X
2.62x
2.71x
Inference time (ms, 4
V
.355
.368
.414
.355
.446
.570
X
.287
.318
.326
.287
.364
.467
ratio
1.23x
1.15x
1.27
1.23x
1.23x
1.22x
When conducting memory retrieval for a query x, we need to find the adaptive similarity between and all N stored memory patterns f1--N- Computation of the adaptive similarity requires sorting the dimemsion-win wise similarity vector q1-d, which needs Od d) time for comparison-basee sort- ing algorithms, and is the time complexity bottleneck for the proposed adaptive similarity (Eq. 4). Therefore, the overall time complexity of memory retrieval is ONN (oNolog d) for calling the adaptive similarity for times, which is higher than that of the Hopfield network with fixed proximity- basedd similarity (O(Nd).
We conducted a runtime comparison experiment sorted and unsorted adaptive similarity, and the results are listed in Table 14. In the experiment, each model (sorted and unsorted) is trained for 200 epoches with the same batch size 256, and the same learning rate 0.01, and optimizer AdamW. For the inference phrase, each model performs 2000 memory retrieval tasks simultane- ously (in one batch) with the same configuration (but different sorting operation). The experiment run NVIDIA GeForce GTX 3090 GPU, and the time elapsed each task is measured by torch. cuda Event elapsed.t ime (end_event..
As listed in Table 14, training sorted A-Hop is merely times slower than unsorted one, and it needs at most 30% more time during the inference phrase. This is smaller than what is expected (3x v.s. 8x, as log, 256 = 8), suggesting that A-Hop light- As written in the official documentation, torch sort () is implemented via radix sort, which might be the reason why the adaptive similarity is faster than expected.
33


Reviews:
[{"review_id": "Web5qrlNc6", "reviewer": "Reviewer_67xP", "summary": "This paper challenges the fixed, proximity-based similarity measures in traditional Hopfield networks, arguing they fail to capture context-dependent associations and cannot guarantee \"correct retrieval\" of a query's true origin. To solve this, the authors introduce the Adaptive Hopfield Network (A-Hop), which reframes retrieval as a probabilistic problem governed by a \"variant distribution\". A-Hop replaces the fixed metric with a learnable \"adaptive similarity\" that is built from a similarity footprint of sorted, dimension-wise similarities. The key advantage is that A-Hop's similarity measure is not fixed. It learns from samples to align with the task's specific context, such as noise or masking.", "questions": "1. Does the author aim to release their code? The current link provided in the paper leads to a folder containing only a readme file.\n\n2. Want to clarify the complexity of the proposed method compared to the modern Hopfield network. The proposed method requires a sort operation ($\\mathcal{O}(d \\log d)$) for each of the $N$ memory patterns during the similarity calculation. A modern Hopfield network (M-Hop) performs this in $\\mathcal{O}(Nd)$ ($d$ for dot product and need to repeat for N memory patterns), but A-Hop increases this to at least $\\mathcal{O}(Nd \\log d)$, is my understanding correct?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper's first strength lies in its novel reframing of the retrieval problem in associative memories through a probabilistic framework, and go beyond traditional epsilon-retrieval to a new concept term as \"correct retrieval\". The goal becomes finding the memory pattern that is the most probable origin of the query, based on a clear mathematical definition: maximizing the a posteriori probability\n\n2. The authors demonstrate that A-Hop achieves state-of-the-art performance across a diverse set of four distinct tasks.\n\n3. The paper is in general well-written.", "weaknesses": "1. My main concern is whether the proposed similarity footprint and adaptive similarity, $s(\\xi,x) = w^{\\top}U\\tilde{q}$, can approximate the posterior distribution, about which the author spends a whole subsection (3.1) discussing the novelty of framing the retrieval problem in this probabilistic way. Maybe I miss some part of the content, but I feel like it lacks a strong theoretical justification for why this specific basis (a linear combination of cumulative sums of sorted dimension-wise similarities) is a universal approximator for an arbitrary likelihood function $p_{\\nu}(x|\\xi)$.\n\n2. The authors write a detailed and insightful discussion section in the appendix (A.3), but this crucial context is never linked to or integrated into the main text. This discussion addresses fundamental questions (e.g., the trade-off between the provably optimal and the learnable formulation) that should be visible to the reader. It would be great if the authors could move the most critical parts of the discussion section into the main text, likely condensing and replacing some of the dense definition-heavy material in section 3.1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oAci16WKUR", "reviewer": "Reviewer_L7xN", "summary": "This paper proposes an adaptive similarity metric to augment Hopfield Network-based storage and retrieval. It is a common observation that since the task is memory retrieval, if the query deviates from the stored patterns, then the retrieved pattern may not necessarily be 'semantically' close to the query. The authors analyze three particular query variants, namely, noisy variants of patterns, masked variants of patterns, and biased variants of patterns. The basic problem is to predict the most likely stored pattern given a query. The approach taken is to consider the query as a generative variant of the stored patterns and ask the question,  what is the maximum a posteriori probability that a given patttern is likely given the query. To estimate this, they propose an approach in which the similarity between query and stored pattern vectors are computed at the individual dimension level and the vectors are sorted. A combination of Euclidean and cosine distance are used and the softmax operator is applied to select the final stored pattern.", "questions": "It would also be useful to address this problem in the context of cross-modal Hopfield retrieval, particularly using textual queries where it will be easier to show performance against truly conceptual variants. For example, if the stored model is a visual description is that of inhalator, queries such as inhaler should still be able to find them", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The beginning of the paper was on the right track in terms of addressing a problem since the current limitation of Hopfield networks was their ability to handle conceptual variants of the stored patterns. Modeling the query as a generative variant of the store patterns is also reasonable. Experimental results are indicating that the technique works under the modeled transformations of stored patterns. The proofs of the theorems are provided in the appendix which adds to some of the clarifications.", "weaknesses": "While the approach was well-motivated, the method presented was a bit under-whelming. Why would sorting along each dimensions help in finding similarity. It is also known that such operations can bring dis-similar vectors close together as well and create distractions. The tabular results are too abstract to interpret and not enough time is devoted in the paper. Having an illustration of the method through visual examples in case of image retrieval would strengthen the understanding of the  method. \n\nThe definitions and theorems are stated but not proved in the text which is fine but no reference to the appendix section is made.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q78ujCMiev", "reviewer": "Reviewer_ohC5", "summary": "This paper introduces an Adaptive Hopfield network (A-Hop) that learns a context-dependent similarity measure for content-addressable memory. \nBy reframing retrieval as a maximum a posteriori problem under a variant distribution, the authors derive an adaptive similarity function that approximates the likelihood of a stored pattern generating the query. \nThey prove that, with appropriate learned weights, \nA-Hop achieves optimal correct retrieval for three common variant types (noisy, masked, biased). \nEmpirically, integrating this adaptive similarity into Hopfield networks yields state-of-the-art performance on synthetic memory retrieval tasks and on downstream benchmarks (tabular data, image classification, multiple-instance learning), outperforming prior Hopfield variants.", "questions": "1. where did you validate the design choices in the adaptive similarity? if there is no, please add\n\n2. any runtime comparison (or complexity analysis)? please Report the computational overhead of A-Hop. should be at least provided to show how the $O(d \\log d)$ footprint computation scales with the dimensionality and number of memories. \n\nLLM disclaimer: I used LLM to polish my language.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Tackles a fundamental issue (retrieval correctness vs. proximity) with a clear probabilistic framework.\n\n- Offers theoretical guarantees for optimal retrieval under well-defined generative scenarios (noise, masking, bias).  I skimmed through the proofs. Looks reasonable to me, but I didn't check them line-by-line.\n\n- Extensive experiments across diverse tasks, showing consistent improvements over strong baselines. Code provided. I skimmed through them. Looks reasonable to me. I didn't check line-by-line nor run them at my end.", "weaknesses": "1. [major] Incremental novelty: Core idea is essentially learning a weighted distance (metric learning) for Hopfield retrieval similar to Wu & Hu et al 2024. It is a relatively straightforward extension of known concepts.\n\n2. [minor] Theory vs. implementation gap: Proofs assume per-dimension weighting (unsorted footprint) for optimal retrieval, but the actual method uses sorted feature footprints without a formal optimality guarantee for that sorting. Please correct me if I am wrong.\n\n3. [minor] Sorting overhead: Computing the similarity footprint requires sorting features ($O(d \\log d)$ or $O(n \\log n)$). This adds computational cost. The paper does not discuss runtime impact or scalability to very high-dimensional data.\n\n4. [very minor] Reliance on supervision: The adaptive similarity weights are learned using ground-truth associations (either known memory-query pairs or class labels). The method may not apply directly in fully unsupervised retrieval settings without a way to obtain such training signals.\n\n5. [very very minor, almost personal opinions] I kind of like this paper. It shows some solid efforts on theory and numerical validations. This might already be a good paper, but I really hope it is better, even great. But the clarity is not good enough in general. I feel it can be more precise and concise in many places. It will be great if the authors can polish more.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)