Bound by semanticity: universal laws governing the generalization-identification tradeoff
Authors: Marco Nurisso
Jesseba Fernando
Raj Deshpande
Alan Perotti
Raja Marjieh
Steven M Frankland
Richard Lewis
Taylor Whittington Webb
Declan Iain Campbell
Francesco Vaccarino
Jonathan D. Cohen
Giovanni Petri
Venue: ICLR 2026 Poster (2026)
Keywords: generalization
multi-object reasoning
cognitive science
Primary area: applications to neuroscience & cognitive science

Abstract: Intelligent systems must form internal representations that support both broad generalization and precise identification. 
Here, we show that these two goals are fundamentally in tension with one another. 
We derive closed-form expressions proving that any model whose representations have a finite semantic resolution, impairing long-range similarity computations, must lie on a universal Pareto front linking its probability of correct generalization $p_S$ and identification $p_I$.
We extend this analysis to general input spaces and to parallel processing scenarios, predicting a sharp $1/n$ collapse in the capacity of processing multiple inputs at the same time.
A minimal ReLU network reproduces these laws: a resolution boundary emerges during learning, and empirical $(p_S,p_I)$ trajectories closely match the theory for linearly decaying similarity.
Finally, we show that the same limits appear in far more complex systems, including a convolutional neural network and state-of-the-art vision–language models, indicating that learned finite-resolution similarity are broad and foundational informational constraints rather than toy-model artifacts.
Together, these results provide a precise theory of the generalization–identification tradeoff and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.

Published as a conference paper at ICLR 2026
BOUND BY SEMANTICITY: UNIVERSAL LAWS GOVERN- ING THE DENERALIZATION-IDEETICCCCTT TRADEOFF
Marco Nurisso', Jesseba Fernando Raj Deshpande', Alan Perotti Raja Marjieh', Steven M. Frankland', Richard Lewis", Taylor Whittington Webb', Declan Iain Campbell Francesco Vaccarino', Jonathan D. Cohen Giovanni Petri*.11
'Dipartimento di Scienze Matematiche, Politecnico di Torino
Network Science Institute, Northeastern University
"Institute for Experiential AI, Northeastern University
4 NP Lab, Network Science Institute, Northeastern University London
5 CENTAI Institute
'Department of Psychology, Princeton University
7 Program in Cognitive Science, Dartmouth College
$ "Department of Psychology. University of Michigan
Microsoff Research
Prrineeton Neuroscience Institute
Department of Physics, Northeastern University
12 Intesa Sanpaolo AI Research
ABSTRACT
Intelligent systems must form internal representations that support both broad generalization and precise identification. Here, we show that these two goals are fundamentally in tension with one another. We derive closed-fomm expressions proving that any model whose representations have finite semantic resolution, impairing long-range similarity computations, must lie on a universal Pareto front linking its probability of correct generalization ps and identification pı. We extend this analysis to general input spaces and to parallel processing scenarios, predicting sharp 1/n collapse in the capacity of processing multiple inputs at the same time. A minimal ReLU network reproduces these laws: resolution boundary emerges during learning. and empirical (ps. trajectories closely match the theory for linearly decaying similarity. Finally, we show that the same limits appear in far more complex systems, including convolutional neural network and state-of-the- art vision-language models, indicating that learned finiterresolution similarity are broad and foundational informational constraints rather than toy-model artifacts. Together, these results provide a precise theory of the generalizztion-iidniffcaa tradeoff and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.
INTRODUCTION
Background. Modern neural networks are surprisingly good performing variety of tasks, rivaling and often surpassing human performance. However, they still exhibit striking limitations in their capabilities to process information, often when they need to process multiple objects the same
time (Campbell et al. 2024, Gong and Zhang. 2024 Rahmanzadehgervii et al. 2024 Rane et al.
2024 Zhang and Wang 2024 Lewis et al. 2022). Similar limitations are also commonly observed in
humans when performing working (short-term) memory tasks Miller 1956);
Luck and Vogel (199)]
Cowan (2001).
Neural networks employ distributed representations (Hinton et al. 1986 Hinton 1986 Smolensky
1990) to process inputs. They enable efficient generalization in unseen situations through, for instance, compositionality. but at the same time suffer from the binding problem -the inability to maintain associations between features when processing multiple inputs simultaneously (Roskies) 1999 Greff et al. 2020 Treisman and Gelade 1980).

---

Published as a conference paper at ICLR 2026
Cognitive science offers rich literature about the ways in which internal representations can help to generalize. The celebrated Shepard's 's Universal Law of Generalization (Shepard 1958a 1987)
states that representations should be arranged in the "psychological space" in structured way, which echoes the real structure of the entities that are represented. This law has received through the years numerous empirical validations and theoretical support (Shepard. 1958b [Sims] 2018 (Tenenbaum and Griffiths| 2001] CChater and Vitányi 2003). This fundamental idea resonates with recent works neural network interpretability. showing that feature vectors in the latent spaces of large neural networks are often organized in rich geometric structures (Arora et al. 20188 Engels et al. 2024
al. 2022 Zhong et al. 2023 Shai et al., 2024 Modell al., 2025).
Frankland et al. (2021) proposed that these two facts the striking information processing limitations, and the generalization through structured representations- are strongly related, and are at the heart of a fundamental trade-off which puts in tension generalization versus identification of representations.
Our contribution. We investigate the fundamental tradcoff between representational fidelity and distinctness under finite semantic resolution. More precisely, we provide:
1. A framework that quantifies the exact Pareto front between identification and similarity perfor- mances, demonstrating how finite resolution creates an inescapable tradeoff;
2. Closed-form expressions for this tradeoff across multiple inputs, noise levels, and varying resolu- tions, revealing a sharp 1/n collapse in multi-item (n) processing capacity;
3. Empirical validation showing how this resolution boundary self-organizes during neural network training, with empirical trajectories closely following our theoretical predictions;
4. Confirmation that these limits persist across architectures from simple ReLU networks, to CNNs, to vision- languaee models, establishing emergent finite resolution as universal constraint rather than a model-specific artifact.
SETUP
Stimulus space and similarity functions. Assume A to be a model processing stimuli coming from a set S the structure of which is encoded by a distance function ds. For example, can be the space of color hues or days of the week, naturally arranged in circle, the set of positions of item in physical space, or more complex topological spaces, such as a torus, or the Klein bottle of natural image patches (Carlsson et al. 2008).
The model processes the stimuli coming from and builds representations by mapping them into a latent (or psychological) space M with map +b S > M. which we assume to be a bijection: this induces naturally a distance on M via d(x,, y) == ds( (ax), $ ()))) In M, the representations processed and compared through a non-negative similarity function g: M x M - JR... For example, if M is a vector space, we can choose g(x;, y) = h(((r) (Ty)) with h(x) 2 O. If
f(x=)= = (xxp(=z), this encompasses. but is more general than, the standard self-attention mechanism
of transformer (Vaswani et al. 2017)
The specific form of g is not uniquely specified by the distance d, allowing for different degrees of "semanticity" (how the metrical structure d is represented by g) with significant impacts on model capabilities. Localized functions IN := 9(x.-) reduce interference between representations, permitting more reliable distinction between them and thus accurate simultaneous processing of multiple representations. Conversely, more distributed g can reflect long-range relations of S, thus enhancing generalization capabilities, at the cost of potential interference among distinct but nearby stimuli. In the following, corroborated by seminal works in the cognitive psychology literature (Shepard 1987), we assume for simplicity that g depends only on the distance between the stimuli: g(z, y) (d(2.y) y)).
'Our similarity function includes common ML metrics: cosine similarity in embedding models, dot-product attention in transformers, and implicit similarity in contrastive learning (InfoNCE, triplet loss). While these mechanisms differ in implementation, they all measure semantic relatedness hetween representations and are subject the resolution limits we identify in this work.
2

---

Published as a conference paper at ICLR 2026
a
No resolution
Resolution p(n)
Constant
b
#2))))))
1.0
1.0-
10
0
22
0.5
0.5-
0.5
1
2
21
9 210
9
91.
Sizg
22
T2
22
Figure 1: a. On the left, exponential similarity functions centered on two stimuli æ1, E M, with the black line indicating the decision function ())()))) .) + g(æ2.p) with no resolution (see Section for explanation). On the center and right, the same quantities are shown in the case of the presence of finite resolution. Notice that the model becomes uncertain for probes far away from stimuli 11, 22. b. Visualization of the constant similarity functions of Definition
Measures of identification and generalization accuracy. Following Frankland al. 20221 , we introduce models of two simple tasks that have previously been used to measure identification and generalization accuracy, and that we use in our theoretical analyses below.
We measure the generalization capabilities of A using similarity task in which the model is asked to perform similarity judgments that respect the metric structure of the stimulus space. The model is shown stimuli T1,.... ................................ ................................ In S and additional one, called the probe, p S. It is then asked to decide which of the n stimuli is the closest to p according to the distance d. Let (Z1,... ....... (.n.....,,,,)), In), be sampled independently from M according to probability measure v. We call X the random variable encoding the index of the closest item to the probe, i.e. X argmin d(z,, p). Intuitively, the i=l......
decision function represents how the model assesses the evidence when determining which input is most similar to the probe. It formalizes the idea that the model's choice depends on relative similarity strengths rather than absolute values. We call Y the random variable indicating the model's 's decision, that we model as follows (Luce 1959):
g
Di( Tai p) = PYY
En. p))
Σ k= g g(L,,P)
We quantify the overall generalization capability as the probability of the model making the correct decision, i.e. ps == P(Y =X).
The identification task used measure how accurately stimuli can distinguished from one another. The task is the same as the similarity task, but with the exception that the probe is always one of the input stimuli p E [nn] This will result in the decision function Equation (1) always being of the form
( x.y)
Di Ini Unj = P(Y = (](21 In, x;)) Ek=1 9 Iz),,
(2)
now X In, xn we write pr == P-Y = X) to indicate the probability of the model succeeding the identification task. Equations (I) and (2) can be interpreted, independent of probabilities, in terms of relative similarity, where ps is taken to represent the average relative similarity of stimuli that are close compared stimuli that are further apart. In the same way, P1 is the average relative similarity of equal stimuli compared to different stimuli.
sxp(-pd(xas; 2z))), and the decay rate for the exponential is taken to infinity (μ + co). both ps and py approach (perfect performance); that is, identification and generalization accuracy both benefit by maximizing decay rate. Critically, however, it has been observed empirically that virtually any loss of precision (i.e., resolution) in computing the similarity function introduces a fundamental tension -referred to as "Miller's Law" (Frankland et al.) [021 between Ps (generalization) and pr (identification accuracy) with respect to decay rate, wherein generalization benefits by decreases in decay rate that dramatically degrade identification accuracy (Figure Here, provide a formal analysis of this effect, showing that it generalizes to learning in neural networks, where it imposes a fundamental constraint on the interaction between representations and efficiency of processing.
Importantly, when g(z;,
3

---

Published as a conference paper at ICLR 2026
The effect of resolution. To show this, we formally consider how a limit in the precision with which the model can compute a similarity function impacts both identification and generalization accuracy. Such limit might arise from any number of factors: computational noise, finite precision, ReI.U activations clamping negative correlations to zero (see Section 4), or imprecisely coded distant relationships. These can all be formalized where A is a noise parameter. As shown in Figure D the resolution drastically affects decision boundaries (the black line): for probes sufficiently far from both stimuli, the decision function approaches 1/2 indicating maximal uncertainty. Resolution thus represents the model's 's inherent limitation in gauging low similarities between distant stimuli.
resolution
O such that g(r,, y) as A if d(x, y) > e,
Tradeoff (Miller's Law). To analyze this, we use simplified similarity function. If LA is the indicator function over the set A, and (,(x) is the closed ball of center x and radius r over M, (x) {y E M d(z, v) r). the similarity function can be defined as follows:
Ceneralization-ldentffccell
Definition 1. The constant similarity function with resolution e and noise A is (x,1) v) (., (v) + (L)))
According to this function, the model will judge two things to be similar (ge:2 (z: y) = 1) if and only if they are closer than certain threshold e O. Outside of this "resolution region" the similarity value is fixed to a noise value A > 0.
This simplified model aligns with Shepard's Universal Law of Generalization (Shepard 1987). where similarity decays exponentially with distance: g(x, y) = exp(- y)). In Shepard's 's formulations, the parameter I controls the sensitivity to distance, with larger μ creating sharper similarity boundaries. This is conceptually similar to controlling the temperature parameter in a softmax function, in which lower temperatures induce sharper probability distributions, while higher temperatures make them more uniform. In our framework, & serves an analogous role, controlling the distance of the similarity functions or the spatial range of entanglement (or semanticity) of the representations. In standard kernel terminology, & plays role akin to kernel bandwidth, determining how decays with distance. Below, we use this to quantify the iitiiiattiii-iennifiattoooo tradcoff as function e.
3 THEORETICAL RESULTS
We use the constant similarity function defined above to derive closed form solutions for the values of ps and pr over a broad class of stimulus spaces and probability distributions over them.
Accordingly, we denote (E) as the probability measure of the closed ball of radius e centered in p. be( := =(B((p)) Furthermore, let (b(=)) = Ep- (s) be the average measure of a ball of radius in M, and Var(b(e)) its variance. The variance term Var(b(s)) captures how the probability mass of e-balls varies across space. Intuitively, this measures the heterogeneity of the stimulus space, that is how differently 'crowded' regions are, which, in turn, compromises similarity judgments. Additional assumptions and notations are described in Appendix A.22
Theorem 1 (2-item tests). Let (M, d, V, be a separable metric probability space. If. for every E M. bp is absolutely continuous on every closed sub-interval of [0, 20), then, for the noise-free constant similarity function 9e:0 holds that
v)
(=) 2 + (b(E)) - (b(E)) Var(b(=)),
(3)
(r) = 11- (b(=))
The proof's can be found in Appendix
A.3
These results have implications for neural architecture design and quantify how much identification performance must be sacrificed to gain generalization ability. These results, being independent of
NNote terminology: "resolution" (5) in this paper strictly refers to the parameter controlling the distance threshold beyond which similarities collapse to noise level A. Higher e values mean the model preserves similarity information across greater distances.
4

---

Published as a conference paper at ICLR 2026
model choices, provide multiple insights how ps, P1 depend on the resolution and on their relation.
First, note that the variance of the ball volume appears in Equation as a term responsible for decreasing the probability of success in the similarity test. This happens when the probability distri- bution is non-uniform the space is heterogeneous (as for manifold with boundary). Spaces which are homogeneous (in Haar measure) with uniform probability distributions will have Var(b(e)) = O, hence performing similarity tests on them will be easier. Therefore, models will perform better on uniform data manifolds (such as rotations), than on manifolds with varying density (such as natural images).
The specific values of (r() and Ps(e) can vary depending on the space chosen. However, assuming Var(b(e)) they are both parametrized by (b(=)), which always a non-decreasing function of e from 0 to 1. This means that, in the (ps, Pr) plane, there is "universal"" Pareto curve relating identification to generalization accuracy that independent of M and v (Figure 2 Indeed, as we will show in Section 4 the distance of empirical performances from the Pareto front directly quantifies the additional "difficulty" introduced by the heterogeneity of the stimuli space (Figure D
This curve exhibits three regimes as function of the ball's resolution e.
a
Low E regime. For small resolutions, the similarity func- tions act like Dirac deltas, meaning that representations do not interfere with one another and thus are perfectly distin- guishable (p1 x 1). However, small resolutions mean that the model is able recognize two objects similar only they are very close, limiting generalization (ps Rs 0.5, chance level).
1..
(6(=)
0.9- A-0.1
Identification test
0.8-
(5(e))
0.3.3
Medium & regime. Increasing elicits the similarity-
AA 0.6
identification tradeoff:: As increases, the similarity mea- sure for more distant stimuli becomes more robust, and thus the structure of the space can be more accurately rep- resented. However, this comes at the cost of nearby stimuli becoming more similar, thereby producing interference that decreases pr- Importantly, ps reaches maximum at (b(e)) 2 space.
a.88
0.2
0.5-
0.0
0.5
0.6
0.7
0.8
0.9
Similarity test
b
I i.e. when the average ball covers half of the
1.0
High e regime. Once increases beyond (b(=)) > 3. the cases in which stimuli interfere d(z1,p) < E, d(22. p) < outweigh the ones in which the probe too far away d(z1, p) > d(22, >> e, resulting in a decrease in both Ps and pr-
0.9
Var(b(e))
Identificttion test
The effect of noise. The result of Theorem can be read- ily extended to take into account the presence of nonzero noise outside the resolution region.
0.6
Theorem 2 (Noise). Under the same assumptions of The- orem 7 for the two-item similarity and identification tests with constant similarity functions g = Ie:A with noise level A 2 0 it holds that
0.5
0.9
Similarity test ps
(5) Figure a. The region in (ps)Pr) plane where the model's performances lie (Theorem [ The black line pa- (6) rameterized by the resolution and rep- resents the behaviour homogeneous spaces. b. Effect of het- erogeneity Var(b(=)) on the similarity test performance.
ps(c, A) = 1
2 1 (((e)) (M{)))),
2- (1-11A4 (b()()))
()(=,A)
2 2A
the model in
Proof. The proof can be found in Appendix A.4
A.4
The effect of noise can be appreciated in Figure as a monotonous decrease in both ps and pr.
5

---

Published as a conference paper at ICLR 2026
Multiple n All resolutions
Multiple Fixed resolutions
a
---3 n-2
b
n=44--
0.05
(8(=))
0.05
6.5 Chanse
Identification Pi
0.8
Identification test P
0.8
0.75
Identification test
0.5
0.6-
0.6
0.25
0.4
0.75
Similarisy aaitt Ps
0.5-
0.2
0.25
0-
0
Similarity test Ps
Similarity test Ps
Number stimuli
Figure 3: a. Similarity-idenificatiin curves for different values of n and parameterized by bp(e) [0, 1], as described by Equations and b. The colored curves correspond to similarity- identification values as the number of inputs n varies, for some fixed values of (s). c. Similarity (top) and identification (bottom) dependence on for different resolutions.
Processing of multiple stimuli. The foregoing analyses may provide formal account of why humans and large neural networks alike exhibit dramatic processing constraints simple tasks (e.g. visual working memory tasks and numerosity judgments), that demand simultaneous processing of multiple stimuli (Campbell et al. 2024). On the one hand, these tasks typically demand generalization (e.g., the processing of stimuli that involve arbitrary combinations of features, such color, shape and position). On the other hand, performance is typically evaluated based on identification accuracy by identifying individual stimuli. The results above thus suggest that these competing demands run up against the fundamental tension between identification and generalization accuracy, irrespectively of scale architecture (i.e., even in systems with billions of parameters, such VIMMs or the human brain). When such systems intrinsically value and/or are trained explicitly for generalization, then they will position themselves into the low-medium resolution/semantiiiyy regime FFiuure Indeed, we can show this is the case by explicitly deriving probabilities of success for n-item similarity and identification tasks.
Theorem (n-item tests). Under the same assumptions of Theorem 1 for the constant noise-free (A = 0) similarity function g = 9::0 we have that
bp(e)) -(1 bp()))m
()((e) Eppn
n
(E))
(e) = Eppoxr
(8)
nbp(E)
Proof. The proof can be found in Appendix A.5
First, note that, despite their apparently complicated formulations, Equations and 8 are poly- nomials in (e()) for any fixed and, given their non-linearity, the expected value over the probes cannot be simplified in general. Thus, for simplicity, we focus on the homogeneous case where by)s) b(e) Vp E M and E disappears.
Under this assumption, both similarity and identification performances are once again parameterized by M()), yielding universal pareto curves independent of M. Figure 3h shows the shape of the Pareto front for different values of n. As sanity check, note that, the resolution goes to b(s) 0, performance approaches perfect identification for any number of simultancous inputs with no capacity to generalize (0) 1/n (chance level).
As shown in Figure 3 b,c), the mapping of one curve into the next is not "uniform". For any fixed 0, increasing the number of inputs quickly degrades both identification and generalization performances. Furthermore, Equation (8) shows that for large n, p (E) zo (b(=)n)-1 identification performance decrease 1/n with rate given by b(g). For model tasked with learning structured

---

Published as a conference paper at ICLR 2026
representations of the input space, and thus optimizing for generalization (say, b(E) = 1/2 for n 2), our analyses predict that the capacity to accurately process multiple representations at the same time will be strongly constrained (Figure Bp.
Interestingly, the bottom panel of Figure 3e shows that the probability of success in the similarity test is non-monotonic in n when b(e) is small. Thus, when the model has to deal with high number of items, is convenient for it to pick low resolutions. The cost, however, is paid by the significant increase in error for low numbers of items.
These observations provide an elegant explanation for why even large neural network models struggle with multi-object reasoning Camphell al. (2024): they likely have developed representations that support generalization, but this brings 1/n decrease in identification probability as the number n of objects increase, thus generating the striking capacity limits observed in both humans and large dision-languan models. In the next section, we provide empirical evidence that neural networks obey these constraints, first in simple toy model, and then in multiple large scale networks.
4 TOY NEURAL NETWORK IMPLEMENTATION
Figure 4: Emergent resolution and tradeoff in toy architecture. (ps, pr) results for the toy model (a) of Section 4 with 50 inputs. b. The orange curve shows the average training trajectory for purely reconstruction loss. The orange insets show the learned (average) similarity function at two epochs. The gray and dashed lines show the curves of Theorem with noise levels A 0, 0.1, respec- tively. The red curve shows the average training trajectory when the loss is based on the similarity test on a circle while the purple one is trained on stimuli coming from segment. The black line shows the theoretical performances obtained with linearly decaying similarity functions, as in Propo- sition
a
Liwer viullecit
S
Consland sauileity
W
Identiiication test
Pure meonstmation
Circle
Segment
(
0.5
0.6
0.7
0.8
6.9
1.
Similarity test
We start from the toy architecture of Ellaaee et al. (2022), that permits a direct comparison with the analyses above. The input vector E R.,, whose entries we identify with features, linearly encoded by W E mx decoded by WT and then the ReLU activation function o is applied elementwise f(x) o(W Wx) (Figure 4p). When trained with reconstruction MSE loss and sparse inputs, this model displays the phenomenon known as superposition: features associated represented as orthogonally (or dissimilarly) as possible to minimize their interference in reconstruction (Elhage et al. (022).. This, in turn, means striving for good identification performance and thus the capability of processing a large number of features simultaneously.
with input dimensions
We contrast this with the effect of inducing the model to learn representations with simple forms of metric (semantic) structure. To do so, we consider two spaces of stimuli made of / points {.)..... 2z equally spaced in the interval [0, 1]: a (flat) circle, with distance d(x, y) = min([x-y], 1-1x-y1), and a segment, with distance d(x, 3) | - y/- The model was trained to perform 3-items similarity tests (as explained in Section 2 on the metric space by encoding its points, the stimuli, as i-dimensional one-hot vectors. Given this last assumption, the i-th column of W, Wi, can be interpreted as the latent embedding of x., and the model's output / (zx);
a [w] w:) == g ar)) as the non-negative similarity between x; and Zi.- The model was trained to convergence 10 times and, for each epoch, we recorded the average similarity and identification ratios ps, Pı of Equations I and ( using the learned g.
4p shows the resulting training trajectories for three different runs in the similarity- identification plane: the orange run corresponds to trainings with pure reconstruction loss, in red the run with pure similarity task loss the circle and in purple the segment. In all cases, used / = 50 stimuli, a hidden dimension of m = 10 and repeated the experiment 10 times. See Appendix A. for additional details.
Figure
7

---

Published as a conference paper at ICLR 2026
As expected, when the network is trained only on reconstruction loss, there is no improvement in Ps but a steady increase in PI. Features are arranged as orthogonally as possible but, due to the low number of hidden dimensions, some interference between them remains. If features line, visualizing the learned similarity function g(x, .) for a fixed x at the last training step shows that it is close to being a Dirac delta x, with smaller-scale random-like noise other features. Estimating this noise scale A and using that in the equations given by Theorem [ shows that the corresponding dashed curve accurately predicts the value of pr at which the training stops.
arranged
In contrast, when the network is trained on the semantic task, Figure 4 shows that (starting from the bottom left corner) both ps and Pj increasing up until the "boundary" reached, after which similarity begins to decrease. Note that the learned similarity functions g(x,)) for a fixed æ = 0.5 (the red insets) exhibit a transition from noise to a semantic function that respects the structure of the circle. Furthermore, this structure also exhibits sensitivity to resolution: the model arranges features associated with points further than a certain threshold have a negative inner product, which is then mapped to zero by the ReLU activation. Moreover, we see that this resolution decreases as training progresses, resulting in an increase of Pı and a decrease in ps-
Not surprisingly, the neural network does not learn constant similarity functions (Section 2). and thus the predictions given by Theorem [ (in gray) only provide a qualitative prediction. However, the learned similarity function (zz, -) appears to be approximately linearly decaying with distance the circle. Based on this observation, we can analytically derive the values of ps and pt for linearly decaying similarities in a circle, finding formulae that approximate Theorem
Proposition (Linear decay). On the flat circle [0, 1] with d( y) = min( - yl. 1 - yl) sampled with the uniform measure, for the two-item similarity and identification tests with linearly
decaying similarity g(z, y) max (, 1 d(a.v)
3 - -log(2) b(E)3, pr(c) 1 (1 log ())
ps(E) + b(e)
(9)
with b(e) 22, E [0, 1 1/2..
The proof can be found in Appendix A.6 Figure shows how the resulting curve (in black) provides good fit to the empirical result. Finally, when the metric space is a segment instead of circle (purple), the heterogeneity given by the presence of the two endpoints results in an overall reduced ps. as qualitatively predicted by Theorem T
5 EVIDENCE OF TRADEOFF IN REALISTIC NEURAL NETWORKS
Finally, we summarize experiments and results showing that the effects described above are also observed in networks at scale. We report details on implementations and additional results in Appendix A..8
CNNs and evolutionary distance We fine-tuned a ResNet-50 model (He et al. 2016) to analyze the generalization--deenffaiiioo tradeoff bird species images (Wah al. (011] using weighted loss function L = (1 (1-x) ax) Lit a Lxim- where a controls the bias between identification and generalization. Both tasks employed a triplet design (I1, X2, and p): for generalization, the model judged which reference is evolutionarily closer to the probe, using phylogenetic distances as ground truth (Kumar 2022 for identification, determined the reference species to which the probe belonged. We found that increasing a, as a manipulation of similarity, improved generalization while reducing identification accuracy, conforming to the relationships reported above (Figure (5ha). Models with higher values consistently showed enhanced generalization, confirming the ability to manipulate this tradeoff through both training and threshold parameters (see Figure TO in the for the full tradeoff curves as function of E and alpha).
al.
Year similarity task in LLMs. We then evaluated three open-source large language models (LLMs)
(gemma-2b- 2b-it (Team et al. 2024), Llama-3. --Istuuttt (Grattafiori et al. (024) and Qwen2.5-7B- Instruct (Yang et al.) 2024)) ) on a similarity task requiring temporal discriminations on the scale of years. The models were prompted to answer questions of the following type "A was born in T1. B was born in 2- Who was born closest to p?"", where A, B are randomized names, a center year c is sampled in [1500, 1700), I c dx, 222 c 8x for dx E {20, 50, 100, 200) and p + 5p
8

---

Published as a conference paper at ICLR 2026
takes all years in |c 300, 300]. Figure shows the decision curves indicating the empirical probability with which each model responded with the correct answer. This shows that the models' year representations closely follow assumptions about resolution: all models showed decreased performance as probe dates moved further from reference dates, similar to what we observed with exponentially decaying similarities with noise ( 322) exp( -xd((=1;; + A (bottom row).
Spatial similarity task in VLMs. Finally, we tested the effects of resolution in two Vision-Language Models (VLMs) (gemma-3- 1-b-i- (Team et al.) 2024 Team 2025a) and Owwn2.55VVL-7B-Instte
Yang et al. 2024 Team 2025b)), on a visual spatial similarity task. Four different black shapes were presented
model in the four corners of the image (Figure 55). together with a red cross in random position. The model was tasked with indicating which black shape was closest to the red cross, and we recorded accuracy for each sampled position. Figure 53 shows that, once again, the models display clear resolution limits in their generalization capabilities, similar to those observed in the year task.
6 DISCUSSION
We have provided formal theory of the tradeoff between identification and generalization in systems constrained by finite semantic resolution, building on the formal framework of Frankland et al. (202I). Our closed form expressions reveal universal Pareto front determined by resolution scale and stimulus geometry, a fundamental limit that is obeyed in empirical tests of model architectures both small and large. Our analysis identifies the optimal resolution for generalization, at which semantic similarity functions tile approximately half of the representational space in discrimination tasks (Sorscher et al. 2022). Beyond this point, increasing resolution impairs identification as representations become too broadly generalized. Below it, representations are discriminable, but fail to capture meaningful similarities, thus compromising generalization. This offers an explanation for why both humans and state-of- state-of-the-art neural network models struggle with multi-objec reasoning, despite their vast computational resources and remarkable capabilities in other domains.
The spontaneous emergence of this tradeoff across architectures, from minimal ReLU networks to vision- lasoun-aagguage models, is consistent with our analyses and our empirical findings, that are unified under the hypothesis that finite semantic resolution constitutes an informationntheoretie constraint rather than implementation artifact. This, in turn, provides a rigorous mathematical foundation for understanding capacity limits in both artificial and biological systems.
Our theory also indicates how competing representational strategies of intelligent systems are tied one another: identification demands sharp. distinct representations, while generalization requires coarse, overlapping ones. This tension is echoed in neuroscience literature on representational efficiency (coding related items compactly) versus processing efficiency (handling multiple items
finctuning
b
year similarity was bocu in it. O was boru ss oy. Who was born olosest p?
spatial similarity task
64
black shape moow
pammod-222 2014
2/
To
25
Figure 5: Empirical resolution tradeoffs across realistic neural architectures. a. A CNN fine- tuned on bird recognition shows the tradeoff between species identification and generalization to phylogenetic similarity as a function of the weights generalization a and of the resolution e. b. LLMs tasked with comparing years of birth show different regimes of performances, compatible with the existence of an emergent finite resolution (~ 70-80 years). e. VLMs tasked with spatial proximity tasks show decreased accuracy beyond model-specific resolution scale. Details in appendix A.8

---

Published as a conference paper at ICLR 2026
jointly) (Petri al. 2024 2021 Lesnick al. 2020). Our analyses also provide formal explanation
for empirical observations in neural population coding (Cohen et al., 2020; Ganmor et al. 2013).
where semantically clustered "neural thesaurus" structures emerge optimal strategies under noise
constraints, connecting to earlier models of representational redundancy (Curto et al. 2013).
Limitations and future work. The present model focuses non-compositional representa- tions, which do not capture phenomena such as hierarchical syntax, analogical reasoning, or arith- metic-where representations are formed by systematic combinations of simpler parts (Lake and Baroni 2023, Fodor and Pylyshyn 1998). Extending our framework to compositional coding schemes remains an important future direction (we provide an initial approach possibility the SI, see Fig. 6 In addition, while we were able to directly demonstrate the presence of the tradeoff in the toy and CNN models, showing its presence in large language- vision models is still outstanding (despite we provided evidence for finite resolution in them, as also indirectly suggested by Modell et al. (2025).
Future work could further extend our results by: (1) using uynergy-reduddane decompositions (Proca et al.] (024) to examine how generalization shapes the joint encoding of multiple stimuli; (2) adopting techniques from mechanistic interpretability [Breska and Gavves (2024) to distill the similarity functions directly from internal representations; (3) developing resolution-basee diagnostic tools for optimizing neural architectures by targeting task-appropriate generalization intriiiiinnnnttttttooonn balance; and finally (4) testing whether neural manifolds from fMRI or clectrophysiology exhibit comparable resolution bounds, potentially establishing semantic resolution as a measurable link between neural geometry and behavioral generalization.
Reproducibility Statement. We describe our theoretical framework with complete derivations and provide detailed descriptions of all experimental settings, including architectures, datasets, and training procedures. Hyperparameters, random seed usage, and evaluation protocols are specified in the appendix. Code and data preprocessing scripts to reproduce all results can be found at the repository https: httpa//w....//ppppeeaaa////eeaaa
ACKNOWLEDGEMENTS
M.N. acknowledges the project PNRR-NGEU, which has received funding from the MUR DM 352/2022. G.P. acknowledges partial support by ERC Consolidator Grant RUNES (Grant no. 101171380) and the MSCA Doctoral Network BeyondTTheFdge (Grant no. 101 120085). F.V. ac- knowledges that this study was carried out within the FAIR Future Artificial Intelligence Research and received funding from the European Union Next-GenerationEU (PIANO NAZIONALE DI RIPRESA E RESILIENZA (PNRR) MISSIONE COMPONENTE 2, INVESTIMENTO 1.3 D.D. 1555 11/10/202, PE00000013) J.D.C. was supported by Vannevar Bush Faculty Fellowship administered by the Office of Naval Research.
REFERENCES
Declan Campbell, Sunayana Rane, Tyler Giallanza, Camillo Nicolò De Sabbata, Kia Ghods, Amogh Joshi, Alexander Ku, Steven Frankland, Tom Griffiths, Jonathan D) Cohen, et al. Understanding the limits of vision language models through the lens of the binding problem. Advances in Neural Information Processing Systems, 37: 17:113444 13316-11333466 2024.
Dongyu Gong and Hantao Zhang. Self-attention limits working memory capacity of transfomer-bant models. ArXiv, 2024. URL httptttttss:aiiiiieeee org. pttp:///dd///.....////// 2409 10715
Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind: Failing to translate detailed visual features into words. arXiv preprint arXiv:2407.06588. 2024.
Sunayana Rane, Alexander Ku, Jason Baldridge, Ian Tenney, Tom Griffiths, and Been Kim. Can generative multimodal models count ten? In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46, 2024.
Chenhui Zhang and Sherrie Wang. Good captioning bad at counting: Benchmarking gp on earth observation data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7849-7849, 2024.
10

---

Published as a conference paper at ICLR 2026
Martha Lewis, Nihal V Nayak, Peilin Yu, Qinan Yu, Jack Merullo, Stephen H Bach, and Ellie Pavlick. Does clip bind concepts? probing compositionality large image models. arXiv preprint arXhv:2212.10537. 2022.
George A Miller.
The magical number seven, plus minus two: Some limits on capacity for processing information. Psychological review, 63(2):81, 1956.
Steven J Luck and Edward K Vogel. The capacity of visual working memory for features and conjunctions. Nature, 390(6657):279-225 2819, 1997.
Nelson Cowan. The magical number 4 in short- memory: A reconsideration of mental storage capacity. Behavioral and brain sciences, 24(1):87-114, 2001.
G. E. Hinton, L. McClelland, and D. Rumelhart. Parallel distributed processing: Explorations in the microstructure of cognition. MIT Press, 1986.
Geoffrey E. Hinton. Learning distributed representations of concepts. In Proceedings of Eighth
Annual Conference of the Cognitive Science Society, 1986. URL http:::tt://// cS http://wwwww.....ot..ttt toronto
fami
Paul Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1-2):159-216, 1990. doi: 10.101660004-3702(90)) 90007-M.
Adina L. Roskies. The binding problem. Neuron, 24, 1999. URL http:p/uuuttttss//ow./.......
http://ww........eee.e.e.aaaaanS 2F i ott.a//ww..oooaoooooooooooooaaa0 3Fshowal183Dtr
Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial
neural networks. ArXiv, 2020. URL /ttttt/s://xxii../////////////// /2tp://df/2112...omm 00200 05208
Anne M Treisman and Garry Gelade. A feature-inteeratioon theory of attention. Cognitive psychology, 12(1):97-136, 1980.
Roger Shepard. Stimulus and response generalization: Deduction of the generalization gradient from trace model. Psychological Review, 1958a. [tR.hhtttss//pscceett http:///ww....pa..oo/rrroroooo//
1959-09346-031
Roger N Shepard. Toward a universal law of generalization for psychological science. Science, 237 1417 1987.
Roger N Shepard. Stimulus and response generalization: deduction of the generalization gradient from a trace model. Psychological Review, 65(4):242, 1958b.
Chris R. Sims. Efficient coding explains the universal law of generalization human percep- www.://.eencc.... tion. Science, 360, 2018. URL http:///wwww.............///////
aaq1118
Joshua B. Tenenbaum and Thomas L. Griffiths. Generalization, similarity, and Bayesian inference. Behavioral and Brain Sciences, 24(4) 6440 August 2001. doi: 10:10177:014555500000000
Nick Chater and Paul MB Vitányi. The generalized universal law of generalization. Journal of Mathematical Psychology. 47(3):346-366 2003.
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics, 6:483 495, 2018.
Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not all language model features one-dimensionally linear. arXiv preprint arXiv:2405. 14860, 2024.
Ziming Liu, Ouail Kitouni, Niklas S Nolte, Eric Michaud, Max Tegmark, and Mike Williams. Towards understanding grokking: An effective theory of representation learning. Advances in Neural Information Processing Systems, 35:34651-34663 2022.

---

Published as a conference paper at ICLR 2026
Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. The clock and the pizza: Two stories in mechanistic explanation neural networks. Advances neural information processing systems, 36:27223-27722 2023.
Adam Shai, Lucas Teixcira, Alexander Oldenziel, Sarah Marzen, and Paul Riechers. Transformers represent belief state geometry in their residual stream. Advances in Neural Information Processing Systems, 37:75012-75034, 2024.
Alexander Modell, Patrick Rubin- bubin-Delanchy, and Nick Whitcley. The origins of representation manifolds in large language models. arXiv preprint 25..:::5505582335. 2025.
Steven M Frankland, Taylor Webb, Richard L Lewis, and Jonathan D Cohen. No coincidence, george: Processing limits in cognitive function reflect the curse of generalization, Oct 2021. URL osf wttp:o/ppitsss/////ppppppp..ss.s
s wwwpssrsssyyyy//pp//a/ juxb/vyu
Gunnar Carlsson, Tigran Ishkhanov, Vin De Silva, and Afra Zomorodian. On the local behavior of spaces of natural images. International journal of computer vision, 76:1-12, 2008.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Duncan Luce. Individual choice behavior, volume 4. Wiley New York, 1959.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Haffiel-Doodds Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209. 10652, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, doi: 10.1 10.1109/CVP..201..90
pages 770-778, 2016.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech- UCSD Birds-200-2011 dataset. Technical Report T0-201120000000 California Institute of Technology, 2011.
Sudhir Kumar, Morgan Suleski, Jessica Craig, Anna Kasprowicz, Maxwell Sanderford, Mingfeng Li, Glen Stecher, and S Blair Hedges. TimeTTeee 5: An expanded resource for species divergence times. Molecular Biology and Evolution, 39(8).msac17) 74, 2022. doi: 10. m0.10///llbe///sacccc 174.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa. Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at practical size. arXiv preprint trXiv:2408.0M 18, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama herd of models. arXiv preprint 2rXiv:2407.21783, 2024.
An Yang, Baosong Yang, Beichen Zhang. Binyuan Hui, Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 1rX1v:2412.15115, 2024.
Gemma Team. Gemma 3. 2025a. URL Ut//ittpps///oo..gee/mmmmm/ee
Qwen Team. Qwen2.5-vl, January 2025b. URL htttp:://qenim.githhuue wwt.m//m..gihuu...ooooooo
gwen2. 5-v17
Ben Sorscher, Surya Ganguli, and Haim Sompolinsky. Neural theory for few-shot learning of naturalistic stimuli. Proceedings of the National Academy of Sciences, 119(1))ee112440019, 2022. doi: 00.0073/pmas.211241011
12

---

Published as a conference paper at ICLR 2026
Giovanni Petri, Sebastian Musslick, and Jonathan D. Cohen. An information -hetorrtoo--thorettic approach to reward rate optimization the tradeoff between controlled and automatic processing in neural network architectures. eLife, 13, 2024. URL https::/eeLfeescieneees org/
Geviewed-prepr- wwwprintsrr////ep 93251
Giovanni Petri, Sebastian Musslick, Biswadip Dey, Kayhan Ózcimder, David Turner, Nesreen K Ahmed, Theodore L Willke, and Jonathan Cohen. Topological limits the parallel processing capability of network architectures. Nature Physics, 17(5):646-655) 2021.
Michael Lesnick, Sebastian Musslick, Bíswadip Dey. and Jonathan D. Cohen. A formal framework for cognitive models of multitasking. PsyArXiv, 2020.
psyarxiv/1y2dn vl
Uri Cohen, Yue Chung, Daniel D Lee, and Haim Sompolinsky. Separability and geometry of object manifolds in deep neural networks. Nature Communications, 11(1):746, 2020. doi: 1411038/41-667-000-4555555
Elad Ganmor, Ronen Segev, and Elad Schneidman. thesaurus for neural population code. eLife, 2015. /elLatts///lllfes...eeee ortp/atticles//011.... pdf
Carina Curto, Vladimir Itskov, Katherine Morrison, Zachary Roth, and Judy L Walker Combinatorial neural codes from a mathematical coding theory perspective. Neural computation, 25(7):1891- 1925, 2013.
Brenden M. Lake and Marco Baroni. Human-like systematic generalization through meta-
learning neural network. Nature, 623, 2023, URt/ttpss: cwwp://wwwwwtt...coooooooooo/o wtw.c/ommaattesssscoo s41586- 06668-3
Jerry A. Fodor and Zenon W. Pylyshyn. Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 1998. URL https::/ http://wwww... www.scienced.eeccc www.nncewdeeeee.oeooooooooooeeee www..cootttttteeeeeeeeeeoeeeeeee
3bs/pp//0000227B0000
Alexandra M Proca, Fernando E Rosas, Andrea Luppi, Daniel Bor, Matthew Crosby, and Pedro AM Mediano. Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks. PLoS computational biology, 20(6) 1012178, 2024.
Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for ai safety a review, arXiv preprint arXiv:2404. 14082, 2024.
13

---

Published as a conference paper at ICLR 2026
A
APPENDIX / SUPPLEMENTAL MATERIAL
A.1 LLM USAGE
LLMs (ChatGPT) were used to aid in polishing the paper after writing.
A.2 TECHNICAL DETAILS
In this section, we formalize the technical aspects and assumptions required for the results of the paper.
We assume (M, d. v) be separable metric measure space equipped with the standard metric space topology, the Borel o-algebra Σ generated by balls in M and with probability measure This measure v. which such that ((M) 1, determines how we are sampling stimuli from the stimulus space M.
In the following derivations, we will make use of two objects:
bp(c) = (p): the measure of the ball radius e centered in p;
Sp, the push-forward measure on [0, oo) of the distance function in p. p. d(p, i.e., for any measurable subset of IR, Sy(E) v(d7'
Note that bp is the cumulative distribution function of Sp as Sp(( -0, E) = Sp(10, =)) = bp(=) and therefore it non-decreasing. We also have that (oo) lim (,) 1.
We assume that b, is an absolutely continuous function on every closed sub-interval of [0, oo) ie. such that for every > there exists 5 > 0 such that for any finite set of disjoint intervals (0n,31) Bt)....,. (aN, 3x)
N (3: 0.) <0 =
N E0, bp(a)) <e. 1=1
1=1
By Nielsen 1997) (Theorem 20.10), the absolute continuity of bp(()) implies that Sp is an absolutely continuous measure w.r.t. the Lebesgue measure μ.. This implies, Radon-Nikodym theorem, that S, admits a density f. Sp. = f fdp, with f(() (s) almost everywhere. In fact, we can think of absolute continuity stronger notion of continuity, as the fundamental theorem of caleulus for Lebesgue integrals (Folland (199), Theorem 3.35) tells that, on every interval [c, d]. by is almost everywhere diffcrentiable, and (p( (c) = f (r)d(((())
We now see how these assumptions allow to notably simplify the derivations of the probability of success in both similarity and identification tasks, while still not being too restrictive. In fact, most non-pathological cases of interest, like probability distributions with differentiable densities on manifolds, satisfy the assumption.
Lemma 1. Let X be the random variable of the correct answer the n-item similarity or identifica- tion task X = argmin {d(x), .)..... d(x,, p) If. for every p E M, bp absolutely contimuous on every closed interval (c, d] oC), then P[[ >

---

Published as a conference paper at ICLR 2026
Proof. Let [X] be the cardinality of the set X.
m
P([X) 1)
Σ
P(d(#1,p)
d(ak d) > d(z.,)).....d(an
p) d()1,p))
k=
(1)
P(d(E1,))= = -.. =d(#k: p)) -
Σ
d(),,p))
(2)
k=2
' P(d(z1.P) - d(z2, = r)d((r) -
n k
P(d(z1, () r)dp(()
2
k
(3)
Σ ( /. (p(tr))dp(r) 0,
(4)
where the last equality comes from the absolute continuity of Sp w.r.t as (((r)) = 0.
This result tells us that, under the assumptions, there is probability of that there are multiple correct answers to the similarity and identification tasks. Therefore, in the following derivations will always only have to deal with the case [X| =
A.3 PROOF OF THEOREM
A.3.1 SIMILARITY TASK
Proof. Let us derive the probability of succeeding in the similarity task in the case of items. The following proof will be a subcase of the more general one for n items but, given its complexity, it is useful to analyze this subcase separately.
Let 21, #2. be sampled independently from M according to the probability measure v. Let X (2 F2, (2, p) argmin d(r,,p). Notice that X can have three different values (E(1.2)
X X(21,22.p) {1}
if d(z11))) < d(z2,P) if d(x2,,p) < d(r1,p)
X X(11, (2)
X(21, = [1,2] if d(z11))) = d(x2.p).
In this last case, when Ti, 22 equidistant from p. any answer to the task will be correct. By Lemma[] [ we only need to focus on the first two as the probability that more than one answer is correct is O.
have that
2
P(Y = X) = Σ P(Y = i|x iP
(5)
1=1
Let us now rewrite the probability PP(Y = i| i) by conditioning over possible results of the samplings of 21, 22 and the probe p.
For this, given the independence assumption, we assume that the event (æ1, x2, p) is an element of the measure space (M3, 203) equipped with the standard product measure.
P(Y i| 1) / Ara
P(Y i|X (d)
i).
where d(( F2, p\ X = i) is the conditional measure the sampling of Tı, p given the event that X = which, by Bayes theorem, can be rewritten
1 [ x2, ilddv(z1) =idd=(z;)dd((22)dd/2
12
P(X = i)
where ([X,,,,22.P) = coincides with the conditional law of the (deterministic) variable X X2),,
2

---

Published as a conference paper at ICLR 2026
Replacing this in Equation S
we get
2
P(X = -(X=Y)=
P(Y = i|X = i, (r1, 22, p2 - (ddv(z))dr(z22ddv) (6)
M
The independence of the samplings of I1 and 22 means that all indices are equally likely to be the correct answer P(Y i|Xx = P( = j| j) Vi.j E {1.2}.
P(X Y) =
P(Y = (,,,,,
[x (d2))))dd(())d)( (7)
MM
= J/ J [ M M JegEMullzaa) g( g p)2,
g(r1. p)
dv
(8)
Given the fact that we are considering constant similarity functions g = ge:0( y) which depend only on the distance between x and y. g(x, y) = g(d(a, v)), we perform the following change of coordinates d(x1,p) p) TI. d(x2.P) r2.
P(X = =(X=Y)= = S/a dv(p) S dsp(ri) ( (52( g(rl) + g(r2)'
g(ri)
where Sp is the pushforward measure induced by the distance function from the probe p.
We decompose Equation (9) into two cases: a. when F1 > E and thus both items fall outside the resolution region of the probe p. and b. when F1 and thus the closest item falls inside.
a. In the first case, given that r2 > T1s we will have that both x1 and 22 are too far from the probe to be recognized similar, resulting in both numerator and denominator in Equation ( to be O. Here we adopt the convention 0/(0- + 0) = 1/2 to describe the model being maximally uncertain in its decision.
f dv(p) M
I dsp(ri)
I
-Jo dv(p) Secon Jm,.., (S,(())dS,,ra) -dvup))
(1) (r)( =
(E,20)
To compute this integral, we leverage the almost-eevrywhhere differentiability of bp and apply the fundamental theorem of calculus
Jemn Jon. (Sp(rr)ds,,rr)> =/r [(,))oo)
(1 (1- (1-b,(ri))dS,(11)
(10)
(1- (1-bb((r)))(((()dd)) (1-bb())))))((-))))))--00-)-""))
(1- (1-b)(()))
(1 (1- (1-bp(E)) bp(=))
(11)
2
2
b. When S E the first item will be considered to be similar to the probe g(ri) = 1, while the second can be both similar and dissimilar.
2 / dv(p) ( dSp(rz) I Jmr. dSp(r2) 1 g(2))
1
(12)
M
2 dv(p) f dsp(ri) I 2
dr2) +2 / du(p) / dsp(ri)
1
/ [G,0))
dSp(r2). (13)
paol
The term I, just like above, can be computed in the following way
S. du(p) (..)
I dSp(rı) I dSp(r2)
(14)
(m.s)
= =f du(p) I (bplE) (r(r)))),,(1)) 1.d
(15)
-1 bp(5)2 de(p)
(16)
3

---

Published as a conference paper at ICLR 2026
The term II is simply given by 2 f bp(E)(1 (p)))dd((p)
Summing together a. and b. we arrive at the following:
1 / (1 bx()))² + 22)( (oo()))(-bp(e)dddl
P(Y x)
(17)
M
(ppe)
(18)
M
We obtain the formula for the probability of succeeding the similarity task:
P(Y P(Y= = x)= = (b(e) (b(s)
(19)
A.3.2
IDENTIFICATION TASK
Proof. The identification task can be seen as subset of the similarity task, in which the probe uniformly picked among the input stimuli. This means that the correct response will be X ( 22. = {i E [1,2} 2; p} and both answers will be correct only in the case that æ1 22-
Retracing the first steps outlined in Appendix A.3.I, we find that the probability the model being correct will be
P(Y X) = /
Σ
P(Y 1 ((),,, (2,,
[]dv(xe (212)
M
pe (m)
(20)
I1) -I as (( g(222
(x1)).
(21)
Note that we used the fact that p with probability 1/2 and 22 with probability 1/2. Given that g(x, x) and, by the definition of metric space, d(x, y) 0 I y, we change coordinates d2) r and rewrite Equation (20)
as
Y)= S du(z1)
1 (0,cz) 1+ g(r) ds., (r).
P(X (X=
(22)
When E, the second item does not interfere with the probe p = 21 and the model will choose ø1 with certainty, while, if r I E, g(r) = 1 and it will instead be maximally uncertain.
S. du(z1) /
[0,00) 1 +r(r)) dS., (r) [ du(x1) /
dS,, (r) + du(zz)) L IdS,, (r).
(0,5) 2
(5,00)
(23)
1 - //1m be, (s)du(z1)+ + / (1- be, (E)) = (b(=))
(24)
2 M
A.4 PROOF OF THEOREM 2
The proof proceeds by retracting the proof of the noiseless case with some adjusted constants.
We start from the similarity task success probability, rewritten in Equation 9 Once again, the integral can decomposed into two cases: a. when F1 > E and thus both items fall outside the resolution region of the probe p and b. when r1 and thus the closest item falls inside.
a. When 22 > T1 > E, we have that g(r1) = =((r)) = and thus the ratio ((r1)/(o(rz) + g(r2)) = 1/2 resulting in the same term of Equation [11]
2/2
b. When ri < E and 72 < E both items are similar to the probe and thus we get the same contribution of the term (6) /2 in Equation (16.
4

---

Published as a conference paper at ICLR 2026
The only difference from the proof of the noiseless case is when "1 S E and 12 > In this case, the first item is similar to the probe while the second is not, but the noise erodes the probability of the model picking the first item. Therefore we get the following contribution to P(Y X).
e dv(p) dsp(ri) / Jee.No) M
S fo.el
1 1+ 1+ I+A /, bp(E)(1 bp(e))du(p)
(S2(
Putting all the terms together get
1
2
P(Y X) M
bp(e))
+ pp)())
I+ A (p(=)(1- bp(=))du(p)
(25)
2
-1 (bp(e) -du())))
(26)
JM
1 A
1 A
((bp()) (bp(c)3))
(27)
2 1+ A
For the identification task, we start from Equation 22 Now, when r > E, the second item is outside of the resolution region of 21 but the noise will still make the model's decision not certain.
PPYY- x) x)= =- dr(zz) S.am)) I+ g(r)
1 (T)
(28)
/ du(oi) I J(oc) jd (r) du(ai)
1
dSa, (r).
(29)
(5,20) 1 1+A
M
2 . bz, (2)dU((()1)) + I+A / (1 (1- bas (z. (=))d
1
A
2(1 + A) (b(=)).
(30)
1 A
M
A.5 PROOF OF THEOREM 3
Lemma 2.
n
(1 x)n- (1 x)
11
x (1-2))--
j
j=1
j=1
Proof. Let us call fa the left-hand side of the identity and In the right-hand side. We prove the result by showing that the generating functions of the series (fn)n (gn)n are equal.
Let us start with (fn)n.
1 -
1 x (1 22--
F(2) =
fnz
(31)
n=0
0=
j=l n=
Σ
8
11
(1 x
2m
+ k.
(1 )
2)("2at)
(32)
1=1 J
j
)=)
j=1
k=
1
1
22
2 (1
(33)
(1
+
1 2 32
==1
log
22
log
1-3
1-2ta2
(34)
- 2 22
1 + 22
where we used the generating function identity for the binomial, see (Graham et al. 1989) (Equation 5.56) and the power series expansion of log(1 x).
5

---

Published as a conference paper at ICLR 2026
Let us proceed in the same way for gn:
20 8
G(z)
Σ
In
(11 - =)"- -(1 - (1-x)*m) ))
(35)
n=0
n= ==1
Σ 11 x) -
Σ (1 - 2"
(36)
n=
R=j
00
(1 (1-x)~ -) Σ
(2 2T )n
Σ 22 - 2x)
(37)
n=
n=
8
(1-z)= (2 =zr)
(2 zr>
(38)
]
2 + 22
1 z + 2σ
j=1
2x 1
Σ
(z - zσ))
(39)
+ z
j=1
log (1
+ logg(1 - 2 + =xx
log
1-8
T-x
1-2+22
(40)
2 22
+ 22
A.5.1 SIMILARITY TASK
Recall that in the n-item similarity task, we are sampling independently n stimuli ....... In and probe p and we ask the model to find which among the stimuli is the closest to p. Recall that Lemma tells us that the probability of having more than a correct answer is 0.
Retracing the first steps in Appendix A.3 we find that
P(Y = X) = Σ
P(Y = =|= = i)p(X =i) i)
(41)
1=1
(42)
By the symmetry induced by the independence of the sampling, we see that IP(Y +|X
P(Y = j|X' = j) = P'Y = 1|X = 1) Vi,j j and P(X = i) = P(X = 1) Vi, and thus we can restrict the case when the closest stimulus the first one.
P(Y 1|X 1) / M
P(Y 1 1, (2 ..... Zn d) Tn p|x =1=
(43)
1 MMXM
kn, = P
P(Y = 1|X 1, ......,....)))
dv(x))
(44)
P(Y = X)
(45)
=n / J P(Y = 11x = (21, En. (x),... p) = 1]du(x11 ... du( )dv())
(46)
= n / du(p) I du(xz) J Sateen En g(z.,p) p)
g(01,p) dv(xz)) .. dv(xx).
(47)
Given that y is constant similarity function g(z, y) 30,0 v) which depends only on the distance between x and y,. we perform the change of coordinates d( p) + ra with Sp being the pushforward measure induced by the distance function from the probe p.
dv(p) / dS,(rn)
dSp(rn) g(ri)
P(X TT=Y) Y) = n
dSp(r2) ... [o,0
(48)
Σi= g(r.)

---

Published as a conference paper at ICLR 2026
We now consider two cases separately. a. If 71 > no item falls close enough to the probe and thus g p) = 0 v4 = I..... n and the model's 's response is random:
J dulp) (croe) dSp(ri) I ((110)) dSp(r2) ... [inm dSp(rn)
(49)
= =fa du(v) Jc (1 (r(1)))--dSpp( = dulp) / (c,00))) (1- (rddd()))))))
(50)
Notice now that by absolutely continuous implies that (1 - (r(r)))²=1 is absolutely continuous and, by Lebesgue's 's theorem, it is differentiable almost everywhere and the fundamental theorem of calculus holds (see Appendix A.2p. We thus deduce that
J/ dv(p) /
/ du(p) | - (p-))((1))**
(1,000 (1-
(51)
"
/ (1 (pb))))) dv(p).
(52)
b. ri S E, then the closest stimulus is similar to the probe g(rl) and write
n S dv(p) So asp(ni) S (Sp(r2) ... [(m..s) (S, 1+
(53)
Each item > 1 can fall either inside of (p) and contribute to the denominator of the decision function, fall outside. Given that the denominator only depends on the number of stimuli which fall in Bg (p) and not on their index, we can write Equation (53) as
n Sf dv(p)
/
(1p(
Σ
n n-
(bp()) (1))(())))
(54)
M
[0,5]
k=0
k
k: +
dv(p)
n
(1 - (z(s))
k / (bplE) - (r))))))) (rr)d(((r))
(55)
M
k
[0.e]
n 1
dv(p)
(1 - (1 (=))
(56)
k
k+1 k+1
n 1
1
dv(p)
- (1b (E)
(57)
k
(k: + 1)²
=f dv(p)
1 1 (m(
Σ
(58)
where the last is performed by re-indexing j j= k + 1 and applying the property of the binomial coefficient ()=) 1 (7). Applying Lemma 2 rewrite the result in more convenient form
m (1 - -bp())))" dv(p).
(59)
and summing Equation 52 with Equation
59
we get our final expression
bx())) m- (1 (6))
(C) = Eprer
+
72
k
7

---

Published as a conference paper at ICLR 2026
A.5.2 IDENTIFICATION TASK
Re-tracing the first steps of Appendix A.5.I and Appendix AA.322
P(X Y) M pE)
Σ
P(Y 1|X 1, (21,..... ... Fno p)) (11 ................................ Fn. 1]dv(ar)
dd((x))
n
(60)
= / dv(z1)
SAS g(01, 21) + 2 ii= ((;, 21)
g( 21)
dv(xz) ... dd(x)))
(61)
M
= Sx dv(a1) (0,00)
[ dSp(r2) ... f dSp(ra) 1+ -r=)((r;)
(62)
M
[0,00]
Just like we saw in the proof of the similarity task, here any stimulus different from the probe will contribute to the denominator of the decision function if and only it falls in (x1)). Moreover, the
decision function depends only the number of such stimuli and not which ones contribute to the denominator. Therefore, we can write
P(X = Y) = 1 dv(z1)
n-
n
bus (E) bo, (5))
(63)
h
A: ++1
k=0
= S dv(x1)
n
Σ
bzn (E) (1 bz, (x)
(64)
n i=1
j
1 (1 (1- (
E Epva
(65)
(b)(s)
where we used the property of the binomial coefficient (G=1) = (1).
A.6 PROOF OF PROPOSITION I
We want compute ps and pr for the uniform measure the flat circle M = [0, with (xx,y) min( I - y/. 1 I yl) for the linearly decaying similarity function with resolution E, g(r) (1 - 2). where (ax) = max(æ, 0).
First, note that for the uniform measure, we have that
(2
2e
if $ 1
(s) = (BB( - [
2 b(e), > 2
if
i.e. the length of the interval [-e,e] on the circle. Accordingly, we have that the measure S, is such that
==( / ((r)dp(r) 2n(E))
if E C [0.
Similarity task. We start from Equation 9 and, once again, consider the different cases. r1.72 > there is no difference from the constant case: the probe has similarity 0 with both æ1 and r2, therefore the model is maximally uncertain. This term will contribute (1 b(=))2/2 to P(Y X).
If r < s and 1/2 > s, there is no difference from the constant case as the probe similar to æ with no interference from zz. We get contribution of 2b(=) b(E)).
If ri
S,T2 S E, we need to compute
2 as(r)) /
d5(rz) g(ri) =8 f
r1/e
juee
(r..e)
+ g(r2) g((2)
10.11 (riee) (r. 1 - r1/e + 1 - -r2/c
(66)
= $1 (8 - ri) log(2)dp(r2) = _ log((2) = (2s)² log(2) = log(2)b(e)²
(67)
8

---

Published as a conference paper at ICLR 2026
1
12
1am
100
7dm
7
0.50
0.55
0.60
0.65
0,70
0.75
0.80
0.50
0.55
0.50
0.65
0.70
0.75
0.80
Generalization
Generalication
Identfication 6
Identfiicaion
2
5
3
Number dimensions
Number dimensions
Figure 6: Top row. Generalization ps and identification pr accuracy curves for inputs sampled from higher-dimensional torus equipped with & (left column) and &2 distance metric (right column) and linearly decaying similarity functions (depicted above). Bottom row. Generalization and identification accuracies as functions of the number of dimensions, for fixed to the maximum- achieving resolution of ps on the 1-dimension torus.
Putting together the three contributions, we get
P(Y x) 1 b(e) + 1 b(e)2² 2b(e) 2b(e)² + (log(2)((e)
(3/2))(2/ooo
Identification task. We from Equation (22 and consider two cases. If > then .2 does not interfere with the probe and thus the model will choose at with probability 1. Just like the constant case, we get contribution of b()).
If r < we need to compute
Ioca 1+ g() d5(r)=
1
f/- (0,e] 1 1 T/E
dp(r) 2og((2)= log(2)b(E).
(68)
In total, we get
P(Y x) X) 1- b(e) log(2)b(s) = 1 log(2)))(E).
A.7 DIMENSIONALITY ANALYSIS
Here we numerically check how the results change when the input space is multi-dimensional. We consider d-dimensional (flat) torus whose points are d-tuples x (.i,..., x.) [o, 1| Each coordinate thus lives on circular space. Following classical work describing similarity in multi- multi-dimensional spaces Nosofsky (1986). we consider the Minkowski metric on such space
1/p
d(z, v)
Σ min(ly zi), 1 ly: xz1)?
(69)
with = 1, and an agent implementing linearly decaying similarity functions g(r) = max r, 0) with respect to that metric.
In the top row of Figure we show Ps. Py curves as the dimension d of the space increases from to 8. We see how, for both E and &g metrics, the curve shift towards higher identification performances

---

Published as a conference paper at ICLR 2026
while staying approximately constant in generalization. Fixing single resolution value and checking how performances depend on dimension (bottom row of Figure (6) shows fast decaying generalization and fast increasing identification. We interpret this effect of the curse/blessing of dimensionality. In higher-dimensional spaces the volume covered by the similarity function (whose "radius" is fixed) decreases. This results in reduction in interference between representations, i.e. higher identification capabilities but, at the same time, the similarities are not wide enough to support generalization.
A.8 DETAILS ON NUMERICAL EXPERIMENTS
All the code used to produce the results can be found in https://anonymouss 4open ccienns//z/generalizaaiie
A.8.1 TOY MODEL
The architecture of the toy model we used is the following linear bias-less autoencoder with nonlinearity at the end
f(z) o Wx).
where a is the ReLU activation function (xx = max(z) x. 0) and W E Rmxx
In both the pure- nuunstsccnnt and semantic experiments, the inputs were chosen to be 50 one-hot vectors x = e, Vi I,...,m. m. The hidden space dimension was chosen be m 10.
The pure reconstruction experiment is performed by minimizing the MSE loss between input one-hot and its reconstruction through the network
Tree -ll
o(W
Σ lle - (W w;)]l
In the semantic case, the loss is built in the following way. Three different indices i, are picked randomly and their associated one-hots are built ei, ej, Ck. Then, we compute the ratio of similarities
o (w.) T wx) Dj = aw! wK) + (w) wx)
o (w) wk)
D, = ow wx) (w) wx)
The index & E {i.j} of the correct answer computed by taking the minimum between d(r;, my.) and d(Iz, where the distance function given as training input the form of distance matrix. The loss, finally, is computed by taking the Negative Log Likelihood Loss (NLL) between the distribution (D,, D,) and the one hot vector encoding the correct response.
Laain D:-
For all experiments, each epoch is made of 2000 samples, with batch size 128. The models are trained for 500 epochs with the Adam optimizer, with learning rate 0.0007 and weight decay.
Given that random vectors in high- diggehimensional space tend to be close to orthogonal, biasing the model towards high pı. we initialize the weight matrix with i.id. uniform in the interval [o, 2].
At each epoch, the model is evaluated by performing similarity and identification tasks. 1,000 triplets kk,j,k) (k { for the identification)
extracted, and the average Dj is recorded obtain the values of ps and lı shown in Figure The average similarity functions shown in the figure' 's insets are obtained as g:( a (w) wx) for every j E 1.....4 Leveraging the symmetry of the circular structure, each vector yi is circularly shifted so that the index goes to the center of the circle gi + g..
Finally, we take the average over ğ E Øi-
We show the distance matrices for the circle and line experiments, together with the full learned similarity matrices for single run in Figure 7
In Figure g moreover, we see the results of the three different trainings for three values of the neural network's latent dimension. As it increases, we how the model is able have less interference
10

---

Published as a conference paper at ICLR 2026
Distance matrix
Learned similarity matrix
Circle
10
30
Epoch
Epoch
Segment
30
10
Figure Visualization of the distance matrix (left) and the learned similarity matrices through training for the circle (top row) and the segment (bottom row).
latent dimensions
10 latent dimcession
30 latent dimensionus
10
stulesty
shulle
09
Identifiaaion tast
a.a
0.
A A.
0.3
0.7
0.7
0.7
0.6
Pure teparsittttionn
Claole
Seguent
0.9
Similarity test Ps
Similarity test Ps
Similarity test Ps
Figure 8: Different training trajectories of the toy model with different latent dimensions, visualized in Figure
between representations, signified by Pı being able to reach higher values. Visualizing the average learned similarity functions and estimating the noise value, we are able in all cases to predict the maximum Pr using Theorem 2
A.8.2
CONVOLUTIONAL NEURAL NETWORK FINE-TUNED ON EVOLUTIONARY DISTANCES AMONG BIRD SPECIES
Experimental setup. To test our theoretical predictions in a realistic computer vision setting, we fine-tuned ResNet-50 model (He et al. 2016) pre-trained on ImageNet. We used the Caltech-UCSD Birds-200-201 dataset (Wah al. 201 1). which contains 11,788 images of 200 bird species, paired with evolutionary distance data from the Time' Tree database (Kumar et al. (202).. The experimental design involved two tasks with consistent triplet-based evaluation format:
Identification task: Given images of reference species (x1, x22) and probe image, determine which reference species the probe belongs to.
Similarity task: Given images of two reference species (æ1, x2) and a probe species (p), determine which reference species is evolutionarily closer to the probe.
Using contrastive loss that encouraged embedding bird images closer to their evolutionary relatives, we fine-tuned the model using a composite loss:
L (1 ai) Lia Lsim;
11

---

Published as a conference paper at ICLR 2026
where Li is cross-entropy loss for species identification, and Siim aligns the embedding space with evolutionary distances. The parameter a controls the balance between identification and generalization objectives. During evaluation, distances, where distances below indicated similarity. This allowed us to systematically study the cemeralization-identiiittil tradeoff by varying both and c.
defined similarity using threshold feature
a
b
= 0.0
0.5
Evolutionary Similarity Matrix (Log-Enhrng Enhhnccd Colorma))
03
06
30
04
20m
5p40
Specise
Species
20
60
9.0
Species
Species
Spocios
Evolutionary similarity between species obtained from (a) bird phylogeny and (b) the feature vector similarities as tuned.
Figure 9:
Training details. We trained the model for 15 epochs using SGD with momentum 0.9, weight decay le 4, and an initial learning rate of 0.001, reduced by a factor of 0.1 when validation performance plateaued. To handle GPU memory constraints, we used batch size of 8 with gradient accumulation over steps (effective batch size 32). We tested a values ranging from 0.0 to 1.0 with several random seeds (42-46) to ensure robust results.
The birds dataset was split 64- 16-20% for training. validation, and testing, with an additional 15% of species held out completely out-of-distributin test data. The evolutionary distance loss (Csim) was implemented by computing pairwise distances in feature space and aligning them with normalized evolutionary distances derived from the phylogenetic tree. This explicitly encouraged the CNN to map visual features into a space that preserved evolutionary relationships as shown in Figure 9
Theoretical connections. Our experimental framework directly maps the theoretical constructs in Miller's 's Law. The identification task measures (probability of correct identification), while the similarity task measures Ps (probability of correct similarity judgment). The threshold corresponds to the resolution parameter in our theoretical framework, controlling the ball measure 6(s) that determines which items are considered similar.
Evolution during training. We monitored how the identification ceentiffcationggeneraalaaaztoooi constraints evolved during training by tracking both scores across epochs. With a (pure identification objective), models rapidly optimized for identification at the expense of generalization. As a increased, especially beyond 0.5, models traced distinct trajectories through (ps, P7) (or G-I) space, with higher a values showing earlier and more pronounced shifts toward generalization.
The final equilibrium position in G-I space was primarily determined by a, with higher a values reliably producing models with better generalization capabilities. Out-of-distributtn testing revealed that models with higher a values demonstrated substantially better generalization to unseen bird species, confirming that the similarity-based training objective promotes more robust feature learning that captures fundamental biological relationships rather than superficial correlations.
Results. As shown in Figure 5h, the bird CNN exhibits a clear tradeoff between generalization and identification. We expand these results in Figure 10 in which (a) shows how the G-I tradeoff is
12

---

Published as a conference paper at ICLR 2026
Acron higha Valkorand throatdb
Simierity Performance Landecape
Ideniification Perforanne Landscape
1:00
-018
-078
Abr Narehaaoo
-316
-16
lost Theemnenn
$7330
-130
Aghang
to
Similaity % Change (vs a+
Identiiiation % Change (ve axd)
590-
699-
Figure 10: (a) annnraaizzaton-ideentffiiiaei tradeoff parametrized by threshold resolution (s) for various a values, showing how different regularization strengths shape the Pareto frontier. (b) Per- formance landscapes showing generalization (leff) and identification (right) continuous functions of a and log(e). (c) Training trajectories in the G-I performance space for different values, show- ing how higher a produces increasingly horizontal progressions that prioritize generalization over identification during learning. (d) Percentage change in generalization (left) and identification (right) relative the pure reconstruction baseline (a 0), revealing a critical range of values where positive deviations occur for both tasks.
parametrized by threshold resolution E for different a values, with each curve tracing the performance E varies while remains fixed. Panel (b) presents the full performance landscapes, showing generalization (left) and identification (right) scores as continuous functions of both a and log(e). Panel (c) illustrates the training dynamics: as increases, the training trajectories become increas- ingly horizontal, indicating that the learning process prioritizes generalization improvements over identification accuracy. Panel (d) quantifies the performance deviations from the pure reconstruction baseline (a = 0) through heatmaps. For low thresholds (s < 20) and high thresholds (e 2 80), deviations remain minimal across all values. However, for intermediate thresholds (30 S I 50), we observe critical regime: higher a values yield substantial positive deviations in generalization
(up to 25% improvement) while identification shows moderate negative deviations (typically 10-20%
decrease), most pronounced in the 2 0.8 range. Notably, there exists narrow band around & = 40 where both landscapes show positive deviations for moderate a values, confirming our theoretical prediction that optimal threshold selection enables simultaneous enhancement of both generalization and identification beyond the pure reconstruction baseline.
A.8.3 LLMS PERFORMING DATE-OF OO-OOO--TEHHROO IDENTIFICATION VS SIMILARITY TASK
Evidence of resolution
Experimental setup. We investigated whether large language models exhibit semantic resolution when processing time information. We tested three models: gemma-2-2b-ir (Tam et al. 2024). Llama- 3.2-3B- (Grattation et al., 2024), Qwen2.5--BBInsttru (Bai et al. 2023) on the following task. The models are fed the system prompt "You are a useful chatber assistant. and are asked to respond to the prompt "A was born in x. B was born in Who was born closest to p? Answer with a single name."
The variables A,B,x,y and p generated in the following way:
1. A central year c is sampled uniformly from the of integers (1500, 1501, ......... 1699);
13

---

Published as a conference paper at ICLR 2026
Similarity task
Identification task
0.7
gemmma-2-2b-it
1.0
Llama-3 1.2 22-35--itttrut
5-7B
Chance
0.5
gemima-2-20-t
0.4
Liama: 3.2 3.2--8-iinstee I--instruct
Probability
0.4
Chance
0.2
0.0
0.0
20
40
60
100
20
40
60
80
100
Number of Inguts
Number of inputs
Figure 11: Similarity and identification performances of three LLMs on the interval of years [800, 1599]
2. For each value dr E {20, 50, 100, 200), we fix x = -ō and y = a +8w with probability 0.5 and + and y -ō with probability 0.5.
3. For each pair x.y chosen in this way, we run the prompt with every p = 8p, Öp {o 300, c 300).
4. The prompt with each value of P is 20 times, randomizing the variables A and B. which are two different names sampled from the list "Alice", "Bob", "Charlie", "David", "Eve", "Frank", "Grace", "Heidi", "Ivan", "Judy", "Karl", "Liam", "Mallory", "Nina", "Oscar", "Peggy", "Quentin", "Repert", "Sybil", "Trent", "Uma", "Victor", "Walter", "Xander", "Yvonne", "Zach", "Abigail", "Benjamin", "Catherine", "Daniel", "Elena", "Frederick", "Gabriella", "Henry", "Isabella", "Jack", "Katherine", "Lucas", "Mia", "Nathan", "Olivia" ].
the answer of the model belongs to the set of sampled names, then we check whether is equal to the name associated to the smallest year.
We repeat the process, sampling с 40 times and averaging the results. What we obtain a function from the probe displacement w.r.t. c, 6p E [-300,300 to the probability of the model decision function Ex[Di y)] E [0, 1].
Results. Figure 5p in the main text displays the probability of correct answers for both models across date displacements, Several observations support our theory:
1. Both models show high performance when probe dates are near reference dates (small displacements), but performance degrades as displacement increases.
2. The pattern follows our theoretical assumptions: the performance approaches chance level (0.5) when the reference years are close and the probe falls between them and when the reference years are far and probes is far form both.
Similarity and identification tasks
Experimental setup. We then performed similarity and identification tasks to gauge the perfor- mance of these three models as the number of inputs provided increases.
For each number of inputs
{2, 42.4.... ... 100), sample 1,000 prompts built in the following way
Similarity task: "Al was born in x1. A2 born in x2. "*...4" An was born in xn." Who was born closest to p? Answer with single name
14

---

Published as a conference paper at ICLR 2026
Figure 12: Examples of image inputs for the spatial resolution task.
Identification task: "Al was born x1. A2 was born in x2. "...."" An was born in xn. Who was born in p? Answer with single name.
Here A1,.. An are names sampled from a list of 200 names similar to the one described above, and x1, x1,...; xn are random integers in the interval (800, 1599). The prohe p is a random integer in the same interval for the similarity task and, for the identification tasks, it is randomly chosen from the set [xl,...,xn];
plot the performances obtained for the three models. Overall, we see that all models perform well on the identification task, with its performance decreasing with a small rate. definitely worse, never being greater
Results. In Figure
Instead, for the similarity task we see how the performances than 0.7 but decreasing in a much graceful way than the 1 1/n random chance.
Interestingly, if focus Gemma and Qwen, are able to qualitatively observe the same behavior of the theoretical model Figure 3 of the main text. In fact, it appears that Qwen is favouring generalization, resulting in good similarity task performance for low number of items but a steeper decrease for increasing n.. Gemma, instead, achieves close to chance similarity task performances when n is small but decreases less rapidly when increases. If we map this our theoretical investigation, it appears that Gemma is adopting smaller E than Qwen, a conjecture which is corroborated by the identification test performance decreasing faster for the latter.
A.8.4
VLM TASKS
To assess the presence of finite semantic resolution in vasion-language models (VLMs), designed several spatial similaritylidenificatiin tasks using synthetic images. Two VLMs were evaluated: gemma-3-12bii1 and Qwen2. VL--BBInnstuce responses, we also logged the scores (logits) of selected, task-depending tokens, to inspect how each model ranks different token choices before softmax.
Besides collecting the models' textual
Evidence of resolution
Dataset. We generated 1,000 images, each featuring four black stencils positioned at fixed loca- tions on a white background. The stencils, chosen randomly between square, triangle, heart, star, varied specific position across images. Additionally, each image included randomly placed red X. designated as the "target" (see Figure 12. We logged the distance between the target and each stencil.
Experimental setup. We each image to both Qwen and Gemma, together with query prompt: "The picture contains four black shapes: square, triangle, heart and a star. There is also a red x. Which black shape is the closest to the X? Respond with only the shape's name." " We logged the models' textual responses and the token scores for square, triangle, heart and a star. For each sampled location, recorded the model's output and computed an accuracy map. A smoothed version of this map was obtained by averaging over local neighbourhoods to reveal confidence gradients.
Results. Figure 5 (main text) shows the accuracy maps for both models. In both cases, we observe central region around each shape where the model is consistently correct, surrounded by
15

---

Published as a conference paper at ICLR 2026
transition zones where accuracy rapidly deteriorates. This behavior mirrors the emergence of finite resolution scale: when the red cross is placed sufficiently far from all reference shapes, the models increasingly unable to resolve which object is closest.
Moreover, the spatial structure the confusion regions reveals differences between models: gemma-3-12b exhibits tighter high-confidence core, while Qwen2. 5-VL-7B-Inst shows broader transition bands, suggesting differences their spatial encoding fidelity. The smoothed accuracy maps further support the hypothesis that VLMs implement distance-dependent proximity function with finite support, analogous to the semantic similarity functions described in the theoretical model (Section 2
Color similarity task
Dataset. We created 5,000 images, each containing between and 12 colored squares. Fach square was labeled with a unique letter, serving an identifier, as shown in Figure 13 The colors of the squares were generated using the HSV color model, where the hue (H) was assigned randomly, while saturation (S) and value (V) were maximized ensure vivid and bright colors.
C
(d)
Figure 13: Examples of image inputs for the color similarity task. Panels (a) and (d) represent two reference images, with four and twelve colours respectively. Given the reference image (a), query image for the identification task (color occurring in the reference image) is depicted in panel (b), and a query image for the similarity task (color not occurring in the reference image) is depicted panel (c).
Experimental setup. In this task, presented the models with pair of images and textual query. The first image, dubbed reference image, contained 4 to 12 labelled color squares, as described above. The second image (query image) displayed single, centered square, whose color was either one of the colors occurring in the reference image (identification task) or completely random one (similarity task). In both case, the query was: "In the first picture there are squares of different colors, labelled with uppercase letters. the second picture there is only one target square. Identify which square in the first picture is most similar to the target square in the second picture. Reply with the corresponding letter and nothing else." ". We logged the color of the target square and its similarity with respect to all colors occurring in the paired reference image. We also logged the model's textual answers and the token score for each single letter (possible answers).
Results. In Figure 14a we show the similarity task and identification task performances as func- tions of the number of input colored squares. In particular, we observe decreasing identification performance which, in both models, can be fitted using the theoretical curve of Theorem 3 (main text). The fitted parameter b(=) suggests the presence larger effective resolution for Gemma and lower one for Qwen.
To investigate this resolution, we gather, for each experiment, the scores each model assings to the letters associated to the wrong colors (thus excluding the most similar ones). together with their circular hue distance from the probe color, normalized to [0, 0.5]. We only take the scores associated to the wrong colors in order avoid the bias of the correct answer having always low distance.
We plot the model score as a function of the distance Figure 14p. Both model display an emergent resolution, with points with large hue distance being concentrated around fixed "noise" level.
16

---

Published as a conference paper at ICLR 2026
Task performances
Model scores
1.0
Similarity
1.0
Identification
KcM22
107
Gemma Ownn
4 0.5
2.5
Gemma
Geeen
ejz1-0
ejc1-0 025
25-
20 12
20
0.5
0.1 0.
0.4
6.5
800
0.2
62
8.3
34
Number of inguts
Number of inputs
distance
distaince
Figure 14: a. Similarity and identification probabilities for the color test explained in Appendix A.8.4 In the identification plot, the dashed curves are the theoretical curves of Theorem [ (main text) fitted to the data. b. Token score associated to the wrong responses, as function the hue distance from the probe color. The dotted black lines represent the resolution values b(s)/2 fitted using the theoretical resul of Theorem (main text) on the identification performances.
Moreover, the scores for Gemma display step-like shape suggesting that the learned similarity function may be similar to the constant similarity assumed in the theoretical analysis. Qwen, instead, shows more continuous decrease in score-similarity with distance, more in line with the results obtained in Appendix A.8.3 and associated to higher performances (Figure 14a).
17

---

Published as a conference paper at ICLR 2026
REFERENCES
Ole A Nielsen. An introduction to integration and measure theory. (No Title, 1997.
Gerald B Folland. Real analysis: modern techniques and their applications. John Wiley & Sons, 1999.
Ronald L. Graham, Donald Knuth, and Oren Patashnik. Concrete Mathematics. Aedison-Wesley,, 1989. ISBN 0-201-14236-8.
Robert M Nosofsky. Attention, similarity, and the idntiffcationncuttgoriaaaiinn relationship. Journal of experimental psychology: General, 115(1):39, 1986.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, doi: 10.1 10.1109/V9R..206.999
pages 770-778, 2016.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech- UCSD Birds-200-2011 dataset. Technical Report ENS-TR-2011-2 1-001, California Institute of Technology, 2011.
Sudhir Kumar, Morgan Suleski, Jessica M Craig, Anna E Kasprowicz, Maxwell Sanderford, Mingfeng Li, Glen Stecher, and S Blair Hedges. TimeTrree 5: An expanded resource for species divergence times. Molecular Biology and Evolution, 39(8)mssac174 2022, doi: 10. 100.///mlbee//ssaclll 174.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at practical size, arXiv preprint arXiv:2408.00116 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama herd of models. arXiv preprint wrXhv:240.22178. 2024.
Jinze Bai, Yixuan Yang. Yingqi Chai, Victor Ling, Aohan Yang. Zhiyuan Lei, Junyang Huang, Yong- gang Tan, Xiubo Liu, Zhijian Yang, et al. Qwen technical report. arXiv preprint arXiv:2309.1666 16609, 2023.
18


Reviews:
[{"review_id": "2NCOCLeZhm", "reviewer": "Reviewer_MQEJ", "summary": "The authors derive a theory for how a representation (for constrained resolutions) trades off between generalization and identification, a long known phenomenon from the cognitive science literature. In particular, this theory implies a Pareto frontier between generalization and identification performance. They empirically demonstrate that ReLU networks navigate such a Pareto frontier and find similar performance in a CNN finetuned on a mixture of an identification and generalization task. Finally, they show that LLMs and VLMs both show evidence of a finite resolution, a key assumption of their theory.", "questions": "- Would you expect an emergent tradeoff between identification and similarity performance if you only trained the CNN on either the identification or similarity task over different epochs (akin to the ReLU network case)?\n- Similarly, would you expect such a tradeoff e.g. in different pretrained models?\n- Could you discuss how your insights would apply to e.g. the tasks examined in Campbell et al. (2024)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This manuscript provides a well articulated contribution to the field, formalizing a tradeoff that had previously been empirically observed. The theory is well-presented and I think the ReLU experiments in particular provide helpful support for the existence of the noted tradeoff. I appreciated the detailed contextualization of this present work in the field. Finally, the paper is generally well-written and the figures are well-designed. Below are some additional parts of the paper I particularly liked:\n\n- The equations (3) and (4) are simply and immediately make the tradeoff intuitive.\n- I think it's very interesting that the ReLU networks show some emergent evidence of this tradeoff even though they are only trained on the similarity task, not the identification task.\n- Figure 4 demonstrates a good match between their (modified) theory and empirical observations\n- Their Proposition 1 demonstrates that this theory can extend beyond the (somewhat minimal) binary similarity measure case.\n- The detailed explanation of the different regimes and Fig. 2 are very helpful.", "weaknesses": "As noted above, I liked this paper. My primary concern is that the experiments in section 5 provide rather limited evidence of this tradeoff. Your suggestion (and the suggestion of the prior literature) that this is a universal tradeoff would suggest that it should be apparent even in models that weren't trained explicitly on the identification and similarity task you're measuring. The fact that models become better at identification/similarity as you're varying the parameter prioritizing one or the other loss function is maybe not particularly surprising (though I agree that it demonstrates that there is a tradeoff between those two functions). Moreover, the LLM and VLM experiments don't demonstrate a tradeoff but rather just show that resolution is limited overall. I thought the ReLU experiment was more compelling in this direction, as it trained the ReLU networks only on the similarity task and still demonstrated this emergent tradeoff over different epochs. Unless I'm missing something, I think acknowledging this limitation would be important.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3ylvbORVHe", "reviewer": "Reviewer_8DWP", "summary": "The paper investigates the tradeoff between representational fidelity and distinctness under \"finite semantic resolution\". It shows that if a model’s similarity function has finite “semantic resolution” ε, then its accuracy on generalization tasks (p_S) and on identification tasks (p_I) must lie on a universal Pareto front.\n\nIf a model's embedding/similarity has a finite-resolution \"floor\", we should expect identification to drop as one pushes for broader generalization, and vice versa. Handling n simultaneous items suffers a sharp ∼1/n collapse in identification capacity; i.e., multi-object reasoning should not scale linearly just by adding parameters.\nOne can choose ε (via architecture, temperature, or thresholding) to target the sweet spot where p_S is maximized (roughly when the “similarity ball” covers half the space).\n\nIn short, the paper provides a simple geometry+noise model one can use to set thresholds, pick temperatures, and anticipate how adding more items or pushing for “similarity-aware” training will impact identification.", "questions": "How is \"semantic resolution\" estimated in practice, and does its value depend on how the embeddings are scaled or normalized? That’s important because, otherwise, \"semantic resolution\" might just be a unit-dependent artefact rather than a stable, interpretable quantity.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Comprehensive background section with helpful literature review.\n\nThe authors take the time to carefully explain their setup, accompanying notation with helpful illustrative examples.\n\nAuthors tested both toy (allowing for theoretical analysis) and realistic models (allowing for confirmation at scale).\n\nText is well-written, figures are clear, elegant, and helpful.\n\nCode is provided showing how to reproduce the results and figures used the paper.", "weaknesses": "1. As mentioned by the authors in Limitations subsection, compositional representations and hierarchy were outside the scope of the study. Adding a small pilot on a compositional task to show if/why the current theory breaks would strengthen the paper.\n\n\n__Minor points__:\n\n2. The concept of “semantic resolution” feels somewhat over-introduced. Mathematically, it appears equivalent to a kernel scale or bandwidth that simply controls how similarity decays with distance. While the term is evocative and may carry intuitive meaning across domains, its use risks adding unnecessary jargon. I suggest that the authors clarify whether “semantic resolution” represents a genuinely new construct (i.e., beyond a kernel bandwidth) or simply reinterprets that familiar notion in \"semantic\" terms. A short comparison or restatement using standard terminology would make the paper easier to follow for readers from machine learning backgrounds.\n\n3. Although there is nothing technically wrong with the abstract, it is very dense and hard to unpack on a first read. The authors could make it a bit lighter in order to appeal to a broader, non-expert audience. For example, by simplifying a few long sentences and highlighting the main contribution more explicitly rather than embedding it deep within the paragraph.\n\nTypos:\n- Please fix the m-dashes in line 84.\n- and [an] additional one (line 97).\n- maximal unncertain[ty] (line 158).\n- m-dash in line 189.\n- m-dash in line 457.\n- m-dashes in lines 464--465.\n- please fix hyphen in line 481.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AmgsS6mExZ", "reviewer": "Reviewer_1bfK", "summary": "This paper proposes that neural systems with finite semantic resolution face a fundamental tradeoff between generalization (correctly judging similarity between different inputs) and identification (distinguishing exact inputs). The authors derive closed-form Pareto curves characterizing this tradeoff for systems that compare inputs via a decaying similarity function, showing that performance lies on a universal frontier independent of input space geometry. They extend the theory to cases with noise and multiple items, predicting a sharp $1/n$ capacity collapse in multi-input settings. They also provide extensive empirical evidence that semantic resolution acts as a general information constraint in complex systems", "questions": "Is there an optimal distance function? the paper motivates its choice of decaying similarity functions using Shepard’s law, stating that generalization should decay with distance. Prior work (e.g., Sims, 2018) has shown that such generalization gradients follow from efficient coding via rate-distortion theory. Would you clarify whether the similarity function is intended to model a learned or designed encoding independently of such optimization?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The central tradeoff (Thm 1) is formally derived under clear assumptions about similarity decay and finite resolution. The analytical Pareto frontier is a nice contribution: mathematically precise, easy to reason about, and interpretable in terms of task accuracy. \n\n2. The empirical sections demonstrate that real-world models qualitatively follow the predicted tradeoff curves\n\n3. this work links the G-I tradeoff to well-known cognitive constraints like binding failures, generalization gradients, and working memory, providing a cohesive narrative.", "weaknesses": "1. The core tradeoff is derived assuming specific forms of similarity decay and decision rules. Would the authors elaborate how universal or sensitive their conclusion is to the choice of decay function? In Discussion, the authors refer to “finite-resolution similarity” as a universal constraint, suggesting that the existence of the tradeoff is robust, even if the exact shape of the curve depends on the similarity decay. It is unclear whether in those larger models, the similarity function follows the linear decay or takes a different shape.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JNpMPHrFTy", "reviewer": "Reviewer_NGhr", "summary": "The authors study the tradeoff between generalization and identification through a notion of finite semantic resolution. They support their theory through a variety of convincing empirics in both a toy neural network and real-world models.", "questions": "For your empirics with the toy neural network model, do you have an idea why the model seems to learn a linearly decaying distance function? Since it sounds like the similarity task employed a conventional softmax output / cross-entropy loss, it seems like the natural distance function would be exponentially decaying?\n\nYou mention that increasing the decay rate in exponential distance will increase both generalization and identification. In your experiments with Transformers (which presumably employ an exponential distance hard-coded in softmax attention), did you find that increasing the decay rate boosts generalization and identification accordingly? Based on your prescriptions, to optimally employ these models, should we always increase decay rate at test time for maximal performance?", "limitations": "", "rating": 10, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is very well written overall. The exposition is clear, precise, and convincing. The theory is impressively predictive of both synthetic and real-world empirics. The central role of semantic resolution and its influence on the generalization/identification makes for a very compelling story. Well done overall!", "weaknesses": "See Questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)