From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
Authors: Chen Shani
Liron Soffer
Dan Jurafsky
Yann LeCun
Ravid Shwartz-Ziv
Venue: ICLR 2026 Poster (2026)
Keywords: Compression
Human and Machine Cognition
Information Theory
Concepts
Primary area: interpretability and explainable AI
TL;DR: Humans think in rich, expressive language; LLMs think in "zip files",  revealing a fundamental trade-off between compression and cognitive flexibility. Bridging this gap could improve LLMs’ conceptual reasoning and human alignment.

Abstract: Humans organize knowledge into compact conceptual categories that balance compression with semantic richness. Large Language Models (LLMs) exhibit impressive linguistic abilities, but whether they navigate this same compression-meaning trade-off remains unclear. We apply an Information Bottleneck framework to compare human conceptual structure with embeddings from 40+ LLMs using classic categorization benchmarks (Rosch, 1973a; 1975; McCloskey & Glucksberg, 1978). 
We find that LLMs broadly agree with human category boundaries, yet fall short on fine-grained semantic distinctions. Unlike humans, who maintain "inefficient" representations that preserve contextual nuance, LLMs aggressively compress, achieving more optimal information-theoretic compression at the cost of semantic richness. Surprisingly, encoder models outperform much larger decoder models in agreement with human categories, suggesting that understanding and generation rely on distinct representational mechanisms.
Training-dynamics analysis reveals a two-phase trajectory: rapid initial concept formation followed by architectural reorganization, during which semantic processing migrates from deep to mid-network layers as the model discovers increasingly efficient, sparser encodings.
These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences between artificial and natural intelligence. This highlights the need for models that preserve the conceptual "inefficiencies" essential for human-like understanding.

Published as a conference paper at ICLR 2026
FROM TOKENS TO THOUGHTS: HOW LLMs AND HU- MANS TRADE COMPRESSION FOR MEANING
Chen Shani
Liron Soffer
Stanford University
Tel Aviv University
@chani@stanfotee
1ironso@mail htt.:/mmaaaa..a.a.mmmmmmmmaeeeee taa.tauuaaa ac i1
Dan Jurafsky
Yann LeCun
Ravid Shwartz-Ziv
Stanford University jurafsky@staffordd.dd
New York University; Meta - FAIR yann www.nenyunuuccc.mmmm edu
New York University r Fs80200ny edu
ABSTRACT
Humans organize knowledge into compact conceptual categories that balance compression with semantic richness. Large Language Models (LLMs) exhibit impressive linguistic abilities, but whether they navigate this same compression- meaning trade-off remains unclear. We apply an Information Bottleneck frame- work to compare human conceptual structure with embeddings from 40+ LLMs using classic categorization benchmarks (Rosch, 1973a 1975; McCloskey & Glucksberg 1978). We find that LLMs broadly agree with human category bound- aries, yet fall short on fine-grained semantic distinctions. Unlike humans, who maintain "inefficient" representations that preserve contextual nuance, LLMs ag- gressively compress, achieving more optimal information-theorett compression at the cost of semantic richness. Surprisingly, encoder models outperform much larger decoder models in agreement with human categories, suggesting that under- standing and generation rely on distinct representational mechanisms. Training- dynamics analysis reveals two-phase trajectory: rapid initial concept formation followed by architectural reorganization, during which semantic processing mi- grates from deep mid-network layers the model discovers increasingly ef- ficient, sparser encodings. These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences be- tween artificial and natural intelligence. This highlights the need for models that preserve the conceptual "inefficiencies" essential for human-like understanding.
THE ENIGMA OF MEANING IN LARGE LANGUAGE MODELS
"The categories defined by constructions human languages may vary from one language to the next, but they are mapped onto common conceptual space, which represents a common cognitive heritage, indeed the geography of the hu- mind." CCrofi (2007) 139
Humans excel at organizing knowledge into concepts which are compact categories that achieve remarkable compression while preserving essential meaning (Murphy) 2004). A single word like "bird" compresses information about thousands of species, yet maintains critical semantic properties (can fly, has feathers, lays eggs). This hierarchical organization (robin - bird -> animal; 1976) represents fundamental cognitive achievement: balancing efficiency with semantic fidelity.
al.)
Large Language Models (LLMs) demonstrate striking linguistic capabilities that suggest semantic understanding (Singh et al. 2024 Li et al., 2025). Yet. a critical question remains unanswered: Do LLMs navigate the compression meaning trade-off similarly to humans, or do they employ funda- mentally different representational strategies? This question matters because true understanding, which goes beyond surface-level mimicry, requires representations that balance statistical efficiency with semantic richness (Tveerkky 1977: Rosch 1973b).

---

Published as a conference paper at ICLR 2026
To address this question, apply neck principles (Tishby et al. 2000) to systematically compare LLM and human conceptual struc- tures. We digitize and release seminal cognitive psychology datasets (Rosch] 1973b 1975| Mc- Closkey & Glucksberg, 1978), which are foundational studies that shaped our understanding of hu- man categorization but previously unavailable in readable form. These benchmarks, comprising 1,049 items in 34 categories with membership and typicality ratings, offer unprecedented empirical grounding to evaluate whether LLMs truly understand concepts as humans. It also offers much better quality data than the current crowdsourcing paradigm by relying on experts to design and execute the human experiments.
Theory (Shannon 1948) and Information Bottle-
Analyzing embeddings from 40+ diverse LLMs against these benchmarks, we uncover a fundamen- tal divergence: L.MS and humans employ different strategies when balancing compression with meaning. While LLMs achieve broad categorical agreement with human judgment, they optimize for aggressive statistical compression the expense of semantic nuance. Humans maintain "ineffi cient" representations that preserve rich, multidimensional structure essential for flexible reasoning.
This divergence manifests across three dimensions. First, LLMs capture categorical boundaries but miss fine-grained semantic distinctions like item typicality, central to human understanding. Second, our intormation-theorette analysis reveals LLMs achieve mathematically "optimal" compression- distortion trade-offs, while human categories appear suboptimal. Third, encoder models surprisingly outperform decoder models human alignment despite smaller scales, indicating that understanding and generation may require fundamentally different representational strategies.
Through analysis of OLMo-7B across 57 training checkpoints, we further uncover how these strate- gies emerge during learning: conceptual structure develops via rapid initial formation followed by architectural reorganization, with semantic processing migrating from deep to mid-network layers as models discover increasingly efficient encodings.
These findings challenge the assumption that statistical optimality equals understanding, The appar- ent "inefficiency" of human concepts may reflect optimization for adaptive flexibility. Our frame- work and newly-digitized benchmarks provide essential tools for monitoring this critical balance, guiding development toward AI systems that achieve not just compression, but comprehension.
RESEARCH QUESTIONS AND SCOPE
Previous work has explored conceptual representations of LLM through multiple lenses: relational knowledge (Shani et al. 2023; Misra et al.) 2021). interpretable concept extraction (Hoang-Xor -uaa ct al. 2024. Maeda et al. 2024). sparse activation patterns (Li et al. 2025), similarity preserving architecture Doumbouya et al. (2026), and embedded geometry, including hierarchical structures 2025). While insightful, these studies often lack a deep. quantitative comparison of the compression-meanine trade-off using information theory against rich human cognitive benchmarks.
(Park et al.]
Separately, cognitive science has applied information theory to human concept learning (Imel & Zaslavsky 2024: Tucker al. 2025 Zaslavsky al., 2018 Sorscher et al., 2022). For example, Zaslavsky et al. (2018) developed an Information Bottleneck framework for color naming efficiency,
later extended to animal taxonomies (Zaslavsky et al.) 2020). Yet these cogmitive studies typically proceed without connecting to modern LLMs, and tend to focus on a specific domain. One notable example is Wu et al. (2025), which examined abstraction transfer in humans and LLMSs using a behavioral and cognitive modeling level. Our work is different in the sense that it analyzes how information is preserved or distorted inside LLM embedding spaces under controlled clustering transformations.
These two streams, LLM conceptual analysis and cognitive information theory, rarely intersect. We bridge this gap through rigorous comparison of how LLMs and humans navigate the compression- meaning trade-off, grounding our analysis in established cognitive benchmarks. This leads to three research questions:
[RQ1] To what extent do LLM-emergent concepts align with human- defined categories?
[RQ2] Do LLMs exhibit human-like internal structure, particularly item typicality?
[RQ3] How do humans and LLMS differ when balancing compression with semantic fidelity?
2

---

Published as a conference paper at ICLR 2026
Our framework approaches each RQ through unified lens. [RQ1] examines the categorical align- ment, or how information is compressed into discrete groups. [RQ2] probes internal structure, which L objective to evaluate the integrated trade-off. This progression from compression to preservation to their balance mirrors the fundamental challenge both systems face: creating representations that simultane- ously efficient and meaningful.
means how semantic meaning is preserved within categories. [RQ3] employs
Figure [ overviews the data generation and analyses. Human data was collected by asking whether an item i (e.g., chair) is a good example of the category C (furniture). These ratings are aggre- gated into ranked similarity profiles for each category. Models generate analogous scores using their embeddings. We then compute three metries: [RQ1] Mutual Information to assess category recoverability, [RQ2] Spearman correlation to measure alignment with human typicality structure, and [RQ3] rate-distortion objective capturing the trade-off between representation complexity and meaning preservation.
Human Data
item
concept
Human Similarity Score
Is "chair" good example of the category "furniture"
human_ rrrman_smm(chir,, furniture)
5
1: Very Poor
2: Fair
3: Average
4: Good
5: Excellent
LLM Representation Pipeline Research Questions (RQs)
[RQ1] Clustering
Clustering
Mutual Information
GOOD
Mi(human concepts, LLM clusters)
LLM Embedding
[RQ2]
Cosine Similarity E(item)
Input: "This chair."
Cosine Similarity Spearman Correlation
Spearmanc (LLMam,, human
Raw Embedding Space
E(category)
[RQ3] Objective Function
Optimization Objective
Complexity Distortion
Figure 1: Overview of the data generation and analyses. Human data collected by asking whether an item (e.g., chair) is good example of the concept C' (furniture). These ratings are ag- gregated typicality scores for each category. Models generate analogous scores using their embed- dings. We compute three metrics: [RQ1] MI to assess category recoverability, [RQ2] Spearman cor- relation measure alignment with human internal typicality structure, and [RQ3) rate- distorito objective capturing the trade-off between representation complexity and meaning preservation.
3 BENCHMARKING AGAINST HUMAN COGNITION
Investigating LLM-human conceptual alignment requires robust benchmarks and diverse models. This section details both components.
3.1 HUMAN BASELINES: EMPIRICAL DATA FROM SEMINAL COGNITIVE SCIENCE
We draw three foundational studies that shaped understanding of human categorization. Un- like many noisy modern crowdsourced datasets, these classic benchmarks were carefully curated by experts, capturing deep cognitive patterns. We focus three influential works:
3

---

Published as a conference paper at ICLR 2026
Rosch (1973): This foundational work (Rosch 1973a explored semantic categories part of the research program leading to prototype theory (Rosch 1973b The theory posits that categories organize around "prototypical" members rather than strict, equally shared features. The dataset includes 48 items in eight common semantic categories (e.g., furniture, bird), with prototypicality rankings (e.g., 'robin" typical bird, 'bat' atypical).
Rosch (1975): Building prototype theory, Rosch 1975) further detailed how semantic categories are cognitively represented. This work provides typicality ratings for a larger set of 552 items across ten categories (e.g.. "orange" prototypical fruit, 'squash' as less so).
McCloskey & Glucksberg (1978): Investigated the "fuzzy" boundaries of natural categories, show- ing membership is graded rather than absolute (McCloskey & Glucksberg 1978). Covers 449 items in 18 categories with both typicality scores and membership certainty ratings (e.g., 'dress' is typical clothing, 'bandaid' less so).
While originating from different researchers, these datasets share rigorous experimental designs and provide data both category assignments and item typicality. We aggregated data from these studies, creating a unified benchmark of 1,049 items across 34 categories. This data, which we have digitized and made publicly available (Appendix C). offers high-quality empirical foundation for evaluating the human-likeness, likeness of LLMs
3.2 LARGE LANGUAGE MODELS UNDER STUDY
We analyze 40+ diverse LLMs spanning multiple architectures and scales (300M to 72B parameters) understand how conceptual representation varies across model design choices. We note that our analysis requires access to LLMs' embeddings, rather than just output. Thus, we are unable to use any closed-source frontier models such as GPT-5 and Claude.
Model Selection. Our study encompasses three architectural paradigms. Encoder models include the BERT family (Devlin et al. 2019; He et al.) 2021 Zhuang et al.) (021) and CLIP ViT text encoders (Radford et al. 2021). 1. Decoder models form the majority of our analysis: the Llama family
(1B-70B: Touvron et al. 2023a b) Grattafiori et al.
2024). Gemma variants (2B-27B: [Team et al.,
2024
2025), Qwen models (0.5B-72B) Bai al.) 2023 Yang et al. 2024). Phi series (Javaheripl
ct al.
2023 Abdin et al., 2024 Abouelenin et al. 2025). Mistral-7B (Jiang et al. 2023), GPT-2
(Radford al.
2019), and OLMo-7B (?). We also include classic static embeddings Word2Vec
Mikolov et al.
2013a b) and GloVe (Pennington et al. 2014) as baselines.
This diverse selection enables to disentangle effects of architecture (encoder vs. decoder), scale (300M to 72B), and training objectives (understanding vs. generation). Note that encoder-only models are less represented (and are smaller) since recent LLM development has prioritized decoder- only architectures. Complete model specifications appear in Appendix
Embedding Extraction. We extract representations at two levels capture different aspects of con- ceptual knowledge: (1) static embeddings from input layers (E matrix), capturing context-free lexi- cal knowledge directly comparable to isolated words in human categorization experiments; and (2) contextual embeddings from hidden layers using controlled prompts, revealing how context shapes conceptual structure across network depth.
This dual approach allows us to trace how concepts emerge from basic lexical knowledge to contex- tualized understanding. Critically, our results prove robust to prompt templates and pooling strate- gies (Appendix D Moreover, despite substantial vocabulary overlap between model families, nei- ther token count nor tokenization patterns correlate with our results (Appendix T).
Prototyye theory is only account of how humans form concepts; exemplar theory offers an alternative based on stored instances. We do not adjudicate between thcories, but use this framework because it provides structured data suitable for modeling, Our computational analysis is compatible with alternative accounts. Appendix I shows that polysemy in our data and cannot account for findings.
4

---

Published as a conference paper at ICLR 2026
4 A FRAMEWORK FOR COMPARING COMPRESSION AND MEANING
To quantitatively compare how LLMs and humans navigate the fundamental tension between com- pact representation and semantic richness, we develop framework that captures both aspects of conceptual organization. Our approach adapts Kate-Distortics Theory (Shannon 1948) and the In- formation Bottleneck principle (Tishby et al. 2000) to measure the quality of conceptual systems.
4.1 THEORETICAL FOUNDATIONS
Human concepts achieve remarkable efficiency: the word "bird" compresses knowledge about thou- sands of species into single category. yet preserves critical semantic information (can fly, has feathers, lays eggs). This reflects a fundamental trade-off that any conceptual system must navigate:
Compression: Grouping diverse items into manageable categories (fewer bits needed)
Meaning Preservation: Maintaining semantic coherence within groups
1948) formalizes this trade-off for lossy compression. Given data X and compressed representation X, RDT seeks encodings that minimize:
Rate-Distortion Theory (RDT; Shannon
R+ XD = (X; X)= XX)
(1)
where the rate (bits required), D is distortion (information lost), and À controls their trade-off.
Information Bottleneck (IB; Tishby et al. 2000) extends this by compressing X into Z while
preserving information about relevant variable Y:
min (X; - 3I(ZYY)
(2)
For conceptual representation, we lack an external relevance variable Y. Instead, "relevance" becomes internal semantic coherence, i.e., how well categories preserve within within-group group similarity. We thus combine RDT's geometric distortion with IB's informition--heerriiic compres- sion, yielding our framework where clustering C represents items X by minimizing both the infor- mation needed to specify items (compression) and the semantic spread within clusters (distortion).
Our adaptation:
4.2 THE L OBJECTIVE: QUANTIFYING THE TRADE-OFF
We formalize how clustering represents items through an objective that combines information- theoretic compression with geometric coherence:
C(X,::))=
T(X:C)
+9
ΣΣ
le
Ecll
(3)
X
cEC
Complexity: bits needed
Distartion: semantic spread
where 3 weights the relative importance compression versus coherence.
4.2.1 THE COMPLEXITY TERM: MEASURING COMPRESSION
Complexity quantifies how much information the clustering preserves about individual items through mutual information ((;;C.. Intuitively, if knowing an item's cluster tells us little about which specific item it is, compression is high (low complexity).
Given X items partitioned into clusters of sizes {[Ca|]:
1 X
Complexity(X, C) = T(X; log, |X|
Cal loga [Cal
(4)
CEC
This equals the reduction in uncertainty about item identity when told its cluster. Uniform clusters minimize complexity (one [X| cluster; maximum compression), while singleton clusters maximize it (no compression).
5

---

Published as a conference paper at ICLR 2026
4.2.2 THE DISTORTION TERM: MEASURING SEMANTIC COHERENCE
Distortion captures how well clusters preserve semantic relationships and meanings by measuring the average squared distance between items and their cluster centroids in embedding space (spread):
Distortion (, C)
1 X
Σ Cal.
(5)
CEC
where =² -d.1
Ed. Eesellee Eell2 is the variance within cluster c, and é. its centroid.
Low distortion indicates tight, semantically coherent clusters with similar embeddings. This geo- metric measure captures the intuition of "meaningful" categories: robins and sparrows cluster tightly as similar birds, while bats would increase distortion on downstream tasks categorized with them.
4.3 CONNECTING FRAMEWORK TO RESEARCH QUESTIONS
Our framework provides unified metrics for all three research questions:
[RQ1] Categorical Alignment: How do LLMs and humans partition semantic space? The Com- plexity term T(X; C*) directly measures this by quantifies how many bits needed to specify indi- vidual items given their clusters. Comparing (X; CHuman) with 1 (X; CLLM) reveals whether both systems create similarly-sized groupings with comparable compression rates. Higher mutual infor- mation means finer-grained categories; lower means broader, more compressed groupings.
[RQ2] Internal Semantic Structure: Do LLMs capture human-like typicality signal? The Dis- tortion term measures how well clusters preserve semantic coherence. We tested whether typical items cluster tightly near centroids while atypical items lie farther away. Low distortion with clear center-periphery structure indicates prototype organization that mirrors human cognitive structure.
[RQ3] Compression-Maanine Trade-off: How do different systems balance efficiency against se- mantic fidelity? The complete L objective reveals fundamental optimization strategies. By varying K (number of clusters) and computing C curves, we uncover system priorities: aggressive compres- sors rapidly achieve low L values by sacrificing nuance, while systems preserving semantic richness maintain higher to retain meaningful distinctions. The shape and level of these curves expose whether system optimizes for statistical efficiency cognitive utility.
AN EMPIRICAL INVESTIGATION OF REPRESENTATIONAL STRATEGIES
Building information-theoreet framework (Section 4) and established benchmarks (Sec- tion 3.1). we empirically investigate how LLMs and humans navigate the compression-mcanine trade-off. For each analysis, we examine both static embeddings and contextual embeddings (across all hidden layers), revealing how and when context shapes conceptual organization.
5.1 [RQ1] THE BIG PICTURE: ALIGNMENT OF CONCEPTUAL CATEGORIES
We first investigate whether LLMss form conceptual categories aligned with humans, which ex- amines how information is compressed into discrete groups (the complexity term in our framework).
Key Finding: Broad Alignment with Human Categories
LLM-derived clusters significantly align with human-defined conceptual categories, sug- gesting they capture key aspects of human conceptual organization. Surprisingly certain encoder models exhibit strong alignment, sometimes outperforming much larger models, highlighting that factors beyond sheer scale influence human-liie categorical abstraction.
We tested whether LLMs naturally organize our 1,049 items into categories resembling human conceptual structure. Token embeddings were extracted at two levels: (i) static embeddings from input layers (E matrix), representing context-free lexical knowledge; (ii) contextual embed- dings from all hidden layers, measured layer-wise to identify peak conceptual alignment. These were clustered using k-means (K matching human category counts) and evaluated against human
Approach:

---

Published as a conference paper at ICLR 2026
0.60
Model Family
05
Gemmma
04
Owinn
0.45
Pri
Laao
0.40
Mistral
Olmo
GPT
Drepseek
0.25
Derporve
0.20
100M
500M
1.58
3.58
108
258
708
Model Fartly
(log scale)
Figure 2: LLMs capture categorical boundaries (RQI AMI scores) but miss internal geometry (RQ2 Spearman correlations). Left: All 40+ models achieve above-chance AMI with human cate- gories, with encoder architectures (squares, circles, and Xs) matching or exceeding decoder models 100x larger (stars). Results show the layer with peak AMI score per model (see static and mean scores Figure A (). Right: Despite categorical success, models show weak correlations (p < 0.2 for most) with human typicality judgments, revealing divergent representational strategies. This di- vergence between capturing boundaries (compression) while missing internal structure (meaning) reveals how LLMs and humans fundamentally differ in their representational strategies. We note that encoder models align more than decoder models, but these correlations are still modest. Com- puted using the static embeddings, see full results in Tables 4 in Appendix N
categories using Adjusted Mutual Information (AMI), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI) metrics. NMI quantifies how much information is shared between the model-derived clusters and the human-labeled categories; AMI refines this measure by correcting for the amount of overlap that would be expected by chance; and ARI assesses the degree of agree- ment between the two partitions while explicitly accounting for random assignments, providing a complementary view of clustering accuracy.
Broad Categorical Agreement: All 40+ models achieve significant above-chance alignment (Fig- ure 2 (Left)). Even baseline, static embeddings show substantial alignment (mean AMV 0.45), which contextual processing enhances to peak AMI = 0.55. This confirms that LLMs encode human-like categorical boundaries. Full NMI and ARI results in Appendices K M
Surprisingly, BERT-large-uncasee (340M parameters) achieves AMI 0.60, matching or exceeding models 100x larger. Classic static Word2Vec and GloVe embeddings. despite predating modern architectures by years, reach AMI scores rivaling contemporary L.LMs' peak performance. This suggests that fundamental semantic structure emerges from relatively simple distributional learning, with encoder architectures particularly effective at capturing human-like categories regardless of scale.
Architecture Matters More Than Scale:
5.2
[RQ2] ZOOMING IN: FIDELITY TO FINE- GRAINED SEMANTICS
Having established broad categorical alignment, we now examine whether LLMs capture the internal semantic structure of categories. Specifically, we check how meaning is preserved within clusters.
Key Finding: Limited Capture of Semantic Nuance
While LLMs effectively form broad conceptual categories, their internal representations demonstrate only modest alignment with human-perceived fine-grat fine-grained semantic distinc- tions, such as item typicality or psychological distance to category prototypes. This suggests divergence in how LLMs and humans structure information within concepts.
We test whether LLMs encode human-like typicality signals, i.e., whether robins are more "birdy" than penguins. For each item, compute the cosine similarity with its category name using embeddings (e.g., 'robin' - 'bird'; cosine.sm (E(robin) E(brdd)) We compare these similarities with human typicality ratings using Spearman's correlation coefficient (Wissler 1905).
Approach:
7

---

Published as a conference paper at ICLR 2026
We employed two analysis approaches: (i) static-layer analysis using embeddings directly from the input layer: (ii) peak AMI layer analysis using contextual embeddings from the layer that maxi- mized AMI in RQI. For the peak AMI approach, extracted category embeddings by replacing items with category names in the same prompt template, ensuring consistent contextualization (see Appendix Foror template and pooling robustness analysis).
Weak Typicality Alignment: Correlations between LLM internal organization of concepts and human typicality are modest best D Tables i Appendix[N) Nj. Static embeddings show weak correlations: BERT achieves 0.38 (p < 0.05), while most decoder models fall below p 0.15. Even when statistically significant, these correlations indicate limited correspondence with human judgments. This shows that the internal concept geometries of models differ from those of humans, with representation -oreeuttttoo-oocuedd focused models aligning more closely.
Architectural Patterns: Several clear patterns emerge in how different architectures capture typ- icality. Representation
models (Word2Vec, GloVe) and most encoder models (both ViT encoders, BERT-large) demonstrate stronger static-layer performance than decoder-only models (Llama, Gemma, Qwen families). Static correlations range from p R 0.25-0.40 for representation- focused models versus p < 0.15 for most decoders.
This divergence likely stems from training objectives: models explicitly trained for representation learning appear more effective at capturing semantic category relationships in their embeddings, while modern decoder-only models, which optimized primarily for next-token prediction, show consistently lower static-layer correlations. The pattern holds across model scales, suggesting archi- tectural design matters more than size for capturing fine-grained semantic similarity.
Layer-wise Analysis Reveals Trade-off: Comparing static and peak AMI layers exposes archi- tectural limitation. Peak AMI layers, which are optimal for clustering, show systematically weaker typicality correlations than static layers. This pattern holds across model families: layers that best separate categories (RQI) poorly preserve cithin-category structure (RQ2). The implication is clear: current architectures encode different aspects of meaning at different depths, forcing applications to choose between broad categorization and semantic nuance.
The divergence between LLMs and humans reflects fundamentally different orga- nizational principles. Humans judge typicality through rich, multidimensional criteria: robins are typical birds due size, flight ability, song, etc. This creates graded categories with clear prototypes and cognitive structures that optimize flexible reasoning and generalization.
Interpretation:
LLMs, contrast, appear to encode flatter statistical associations between items and category la- bels. Although sufficient for categorization and fluent text generation, these representations miss the prototype structure that makes categories cognitively useful. This difference suggests that LLMs optimize for different objectives than human cognition, a hypothesis that we test directly in RQ3 by examining how each system balances compression against semantic preservation.
5.3
[RQ3] THE EFFICIENCY ANGLE: THE CEVPRESSION-MEANIE TRADE-ORE
Having explored categorical alignment (RQI) and internal semantic structure (RQ2), we now ad- dress our central question: How do LLM and human representational strategies compare when balaneing compression against meaning preservation?
Key Finding: Divergent Efficiency Strategies
LLMs demonstrate markedly superior information-theoretii efficiency compared human conceptual structures. Evaluated via our L objective, I.LMMderived clusters consistently achieve more "optimal" compression-meantie balance. Human conceptualizations, while richer, appear less statistically compact, suggesting optimization for cognitive flexibility over pure statistical efficiency.
We analyzed function (Equation 3] 1) and mean cluster entropy (S,). For LLMs, performed k-means clustering across various values to trace the full compression-meanin frontier.
Approach:
categories and LLM-derived clusters using our L objective
Results: Our analysis reveals three key patterns (Figure 3 full results in Appendix Q
8

---

Published as a conference paper at ICLR 2026
Clessic
Distortion Component
Deepseek
Bart
Gemma
Clessic
GPT
Deepseek
Uama
Mistral
GPT
Olmo
Mistral
Qwen
Olmo
VIT
Qwen
Humen
Complexity Component
Number of Clusters (K)
(a) Complexityddistorionn trade-offs.
(b) L comparison with human categories.
ef- ficiency while humans preserve semantic richness. (a) Encoder models (BERT, ViT, classic static embeddings) consistently achieve lower distortion than decoder models at any given complexity level. (b) All LLM-derived clusters achieve lower C values than human categories (dashed line), indicating more "optimal" compression-distorio balance. Data from Rosch (975).
Figure 3: Divergent optimization strategies: LLMs achieve superior information-
LLMs Achieve Superior Statistical Efficiency: Both measures reveal stark differences between LLMs and humans (Figure full results in Appendix Q
Higher human entropy. Human concepts consistently exhibit higher cluster entropy than LLM clus- ters at comparable K values, indicating less statistical compactness but greater internal diversity.
Lower LLM C scores. LLM-derived clusters achieve significantly lower & values than human cate- piarressiooddistottin
gories across all tested K (Figure 3p). Since lower L signifies more optimal balance, LLMs are demonstrably more "efficient" by this information-theorede measure.
Architectural differences. The Complexity-Distorion plot (Figure 3a) reveals that encoder models (BERT, ViT, classic static models) achieve superior trade-offs. Their distortion at any given com- plexity is lower compared to decoder models, across both static and contextual embeddings.
This divergence reveals fundamental differences in optimization pressures. LLMs,, trained on massive text corpora, develop maximally efficient statistical representations that minimize redundancy and internal variance. Although human con- ceptual systems may look suboptimal under cermmttion-thorettee measures, prior work indicates that they are structured to support goals such as flexible generalization and causal reasoning rather than maximal compression (Murphy) 2004). Thus, the differing pressures shaping human and LLM representations help explain this apparent suboptimality.
Statistical Optimality Versus Cognitive Utility:
Our analysis reveals that compression efficiency does not predict functional capability. We find no correlation between L scores and downstream performance (r -0.20, / 0.51 MMLU; Appendix S). This suggests that apparent human "inefficiency" reflects optimization for cognitive flexibility rather than statistical compression. While LLMs excel compact representation, they may sacrifice the semantic richness essential for human-lik understanding.
The consistent architectural patterns observed raise fundamental questions: Do understanding and generation require distinct computational strategies? The superiority of representation- focused mod- els suggests that current architectures may be conflating two fundamentally different cognitive tasks.
5.4 EMERGENCE DURING TRAINING: How DIVERGENT STRATEGIES DEVELOP
Having established that LLMs and humans employ divergent representational strategies, we inves- tigate how these strategies emerge. Analysis of OLMo-7B across 57 training checkpoints (1K to 557K steps, approximately 4B to 2.5T tokens) reveals how conceptual structure emerges.
Two-Phase Representational Development. Conceptual organization emerges via two phases (Figure (5). First, rapid concept formation (1K- 100K steps) establishes basic categorical structure, with AMI rising from zero approximately 0.45, achieving 80% of final alignment within just

---

Published as a conference paper at ICLR 2026
10% of training. Second, architectural reorganization (100K-500K steps) systematically migrates semantic processing from deeper layers toward mid-network while AMI continues to gradually im- prove. This migration from layer 29 to layer 23 occurs without sacrificing categorical aligmment, suggesting the model discovers more efficient internal representations. Moreover, this double-phase dynamics occurs when testing attention sparsity, effective rank, and L values. That is, all of these exhibit the same early rapid shift followed by a slower restructuring phase. This convergence across independent metrics indicates that the model is not merely improving categorical alignment, but reorganizing its internal representations toward increasingly efficient structure. See Appendix
Architectural Reorganization as Optimization. The upward migration of semantic processing hints that the model discovers increasingly efficient encodings. Early training relies on deep, memorization-heay representations, but as training progresses, the model shifts to distributed mid- network encoding. This reorganization may explain apparent "emergent" capabilities: they arise not from learning fundamentally new information but from more efficient internal organization of existing knowledge.
Implications. These dynamics demonstrate that the compresiion-rriented strategy observed in fully trained models develops from the earliest stages of training. The rapid initial alignment followed by efficiency-focusee reorganization suggests models are inherently biased toward statistical com- pression rather than semantic richness. Achieving human-like representations may require not just different final objectives but fundamentally different learning dynamics that actively maintain se- mantic diversity throughout development.
DISCUSSION AND CONCLUSION
We investigated how LLMs and humans navigate the compression-meanine trade-off in concep- tual representation. Using information--hheoreicc analysis of 40+ models against classic cognitive benchmarks, we reveal fundamental differences in their representational strategies.
Key Findings. LLMS achieve broad categorical alignment with humans (AMI z 0.55), successfully partitioning semantic space into recognizable categories. However, they fail capture the inter- nal structure that makes these categories cognitively useful, as typicality correlations remain weak (p < 0.2) across model families. Most strikingly, when evaluated on the ceappesssoon-maannnn trade-off, LLMs consistently achieve lower (better) L scores than human categories, indicating they optimize for statistical efficiency over semantic richness. This pattern holds across architectures, though encoder models surprisingly outperform decoder models in human alignment despite being orders of magnitude smaller. Training dynamics analysis reveals rapid category formation followed by architectural reorganization that shifts semantic processing from deep to mid-network layers, suggesting efficiency optimization continues throughout training.
Implications. These findings challenge the assumption that statistical optimality equals understand- ing. LLMs excel at their training objective, which is minimizing prediction error, but this drives them toward representations that sacrifice semantic nuances. Encoder models' superior alignment with human representations questions the current paradigm of unified, scaled decoder models, suggest- ing that language understanding and generation may require distinct architectures and rely on different processes. Our framework provides quantitative tools for monitoring the compression- meaning balance in future systems.
Conclusions. Our findings reveal an apparent paradox, showing that LLMs are simultancously better and worse than humans. This occurs because LLMS and humans employ divergent strate- gies: statistical compression versus semantic richness, likely reflecting different optimization pressures. While LLMs process billions of tokens efficiently, humans enable flexible reasoning and generalization. Progress toward human-like Al may require preserving the apparent "inefficien- cies" that support cognitive flexibility. We provide theoretical understanding, practical metrics, and high-quality digital benchmarks to develop more human-aligne representations. We encourage the community to utilize the data and metrics for future research towards making Al more human-like
10

---

Published as a conference paper at ICLR 2026
7 ETHICS STATEMENT
Our study relies exclusively on publicly available LLMs and digitized datasets from classic cognitive psychology experiments (Rosch 1973a 1975; McCloskey Glucksberg 1978). No new human subject data was collected, and all benchmark data we release have been properly attributed and curated to preserve research integrity.
We do not foresee privacy, security, or faimesss risks arising from our analyses. Our contribution is methodological and theoretical, focusing on representational trade-offs between humans and LLMs. Nevertheless, we acknowledge that insights into model-human divergences could influence how future systems are designed. We caution that optimizing solely for statistical efficiency without considering semantic richness may exacerbate risks of misinterpretation or oversimplification in socially sensitive applications.
We declare no conflicts of interest external sponsorship that could bias the reported findings.
REPRODUCIBILITY STATEMENT
We have taken several steps to ensure reproducibility. All digitized human categorization datasets used in our analyses are publicly released machine-readable form (Appendix B. Detailed model specifications, including architectures, scales, and hyperparameters, are provided in Appendix B.2, and we document embedding extraction procedures, pooling strategies, and prompt templates in Appendix B.3-B.4. Full experimental results, including layer-wise analyses, clustering metrics, and training dynamics across checkpoints, are reported the appendices (B.5-B. Our theoretical framework and derivations described in Section 4, with complete definitions and formulations provided to enable replication. We will release the code for dataset processing, embedding extrac- tion, and evaluation upon acceptance (to preserve anonymity).
ACKNOWLEDGMENTS
We grateful to teacher and mentor, the late Prof. Naftali Tishby, for his profound contri- butions to information theory, as well as for imspiring us with the depth, clegance, and excitement of this field. This work was supported in part by the Koret Foundation grant for Smart Cities and Digital Living.
11

---

Published as a conference paper at ICLR 2026
REFERENCES
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404. 14219, 2024.
Abdelrahman Abouclenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, et al. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras arXiv preprint arXiv:2503.01743, 2025.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.1660 2023.
William Croft. Radical construction grammar: Syntactic theory in rypological perspective. Oxford
University Press, USA, 2001.
DeepSeek-AL Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning,
2025. URLtpsts://arxt wwwp://ggg/aaaaaee 12948
Jacob Devlin, Ming-Way Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume 1 (Long and Short Pupers), pp. 4171-4186 Minneapolis, Minnesota, June 2019. Association for Com- putational Linguistics. doi: 10. 18113// 14233, URL bettps:/accaathhot
M1931423//
Moussa Koulako Bala Doumbouya, Dan Jurafsky, and Christopher D Manning. Tversky neural networks: Psychologically plausible deep learning with differentiable tversky similarity. In The Fourteenth International Conference on Learning Representations, 2026.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, al. The llama 3 herd of models. In Neural Information Processing Systems. Curran Associates, 2024.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding--chanced bert with disentangled attention. In International Conference on Learning Representattinss 2021.
Nhat Hoang-Xure Xuan, Minh Vu, and My T Thai. Llm-assisted concept discovery: Automatically iden- tifying and explaining neuron functions. arXiv preprint arXiv:2406. 08572, 2024.
Nathaniel Imel and Noga Zaslavsky, Optimal compression in human concept learning. In Proceed- ings of the Annual Meeting of the Cognitive Science Society, volume 46, 2024.
Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio César Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. Phi-2: The surprising power of small language models. Microsoft Research Blog, 1(3)3, 2023.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample. Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William Sayed. Mistral 7b, 2023. URL https:
Yuxiao Li, Eric J Michaud, David D Baek, Joshua Engels, Xiaoqing Sun, and Max Tegmark. The geometry of concepts: Sparse autoencoder feature structure. Entropy, 27(4))344, 2025.
Akihiro Maeda, Takuma Torii, and Shohei Hidaka. Decomposing co-occurrence matrices into in- terpretable components formal concepts. In Findings of the Association for Computational Linguistics ACL 2024, pp. 4683-4700, 2024.
Michael E McCloskey and Sam Glucksberg. Natural categories: Well defined or fuzzy sets? Memory & Cognition, 6(4):462447, 1978.
12

---

Published as a conference paper at ICLR 2026
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word repre- sentations in vector space. In Proceedings of Workshop at International Conference on Learning Representations (ICLR), 2013a. URL httph: https::/aaxii..ort htth/bbs/111000 3781
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeffrey Dean, Distributed repre- sentations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, volume 26, pp. 3111 1111-3119 19, 2013b.
Kanishka Misra, Allyson Ettinger, and Julia Rayz. Do language models learn typicality judgments from text? In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 43, 2021.
Gregory Murphy. The big book of concepts. MIT press, 2004.
Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. The geometry categorical and hi- erarchical concepts in large language models. In The Thirteenth International Conference Learning Representations, 2025.
Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In Alessandro Moschitti, Bo Pang. and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp- 1532-1543, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 3115/v1/D14-116/
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PmLR, 2021.
E Rosch. On the internal structure of perceptual and semantic categories. Cognitive development and the acquisition of language/New York: Academic Press, 1973a.
Eleanor Rosch. Prototype theory. Cognitive development and the acquisition of language, pp. 111- 144, 1973b.
Eleanor Rosch. Cognitive representations of semantic categories. Journal of experimental psychol- ogy: General, 104(3):192, 1975.
Eleanor Rosch, Carol Simpson, and R Scott Miller. Structural bases of typicality effects. Journal of Experimental Psychology: Human perception and performance, 2(4):491, 1976.
Eleanor Rosch. Natural categories. Cognitive psychology. ((3):328-350, 1973c.
Chen Shani, Jilles Vreeken, and Dafna Shahaf. Towards concept-aware large language models. In Findings of the Association for Computational Linguisties: EMNLP 2023, pp. 13158-13170, 2023.
Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379 423, 1948.
Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761 2024.
Ben Sorscher, Surya Ganguli, and Haim Sompolinsky. Neural representational geometry under- lies few-shot concept learning. Proceedings of the National Academy of Sciences, 119(43): c2200800119, 2022.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based gemini research and technology. arXiv preprint 4rXiv:2403.0822 2024.
13

---

Published as a conference paper at ICLR 2026
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXii:2503.19775 2025.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 02380...9777 13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023b.
Mycal Tucker, Julie Shah, Roger Levy, and Noga Zaslavsky. Towards human-lin emergent com- munication via utility, informativeness, and complexity. Open Mind, 9:418-451, 2025.
Amos Tversky. Features of similarity. Psychological review, 84(4):327, 1977.
Clark Wissler. The spearman correlation formula. Science, 22(558):309-31) 311, 1905.
S Wu, M Thalmann, P Dayan, Z Akata, and E Schulz. Building, reusing, and generalizing abstract representations from concrete sequences. In Thirteenth International Conference on Learning Representations (ICLR 2025), 2025.
Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 1rX11:2412.15115, 2024.
Noga Zaslavsky, Charles Kemp. Tery Regier, and Naftali Tishby. Efficient compression color naming and its evolution. Proceedings of the National Academy Sciences, 115531)77937-7942, 2018.
Noga Zaslavsky, Terry Regier, Naftali Tishby, and Charles Kemp. Semantic categories of artifacts and animals reflect efficient coding, In Proceedings of the Society for Computation in Linguistics 2020, pp. 80-81, 2020.
Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. A robustly optimized BERT pre-training approach with post-training. In Sheng Li, Maosong Sun, Yang Liu, Hua Wu, Kang Liu, Wanxiang Che, Shizhu He, and Gaoqi Rao (eds.), Proceedings of the 20th Chinese National Conference Com- putational Linguistics, pp. 1218-1227, Huhhot, China, August 2021. Chinese Information Pro- cessing Society of China. URL https: http://e//ellatttoooooo.ooog//g.
14

---

Published as a conference paper at ICLR 2026
A COGNITIVE INTUITION
A.I A COGNITIVE INTUITION OF THE COMPRESSION-MEANTE TRADBOFF
The compression- meaniing tradeoff refers to the cognitive tension between representing concepts with maximal efficiency (i.e., minimal information) and preserving the semantic richness needed for flexible generalization, inference, and communication. For instance, upon hearing the sentence "There was large brown Labrador barking loudly near the playground," person will often encode a simplified memory, such as "big scary dog near kids." " This is not to suggest that humans cannot recall the full sentence, but rather that we typically retain the most meaningful elements to enable efficient reasoning and generalization; without such abstraction and simplification, leveraging past experience for learning and prediction would be more difficult.
We acknowledge that our metrics, while capturing the cetemaation-comressiiiii trade-off and geo- metric efficiency in embedding space, do not directly measure all aspects of human-style conceptual abstraction or reasoning. Our aim is not to claim full equivalence with human cognition (nor do think anyone can or should make such claims), but rather provide a quantitative, interpretable proxy that highlights where LLMs and humans converge or diverge in how they compress and orga- nize semantic information. We view these metrics as one lens among many, and we are careful in the paper to frame our findings as providing insights into human-like patterns rather than definitive evidence of human-style conceptual processing.
A.2 SRANTTIVELY-NNSIRRE INDUCTIVE BIASES
Future models could more closely align with human conceptual structure by incorporating cogni- tively motivated inductive biases and representational mechanisms. Hierarchical and compositional structure could enable models to capture nested relationships between categories, reflecting the way humans organize knowledge from superordinate to basic-level concepts (c.g., animal - mammal + dog + Labrador). Feature- and relation-basee biases could help models focus on meaningful perceptual or functional attributes and relational patterns, rather than relying solely on statistical co-occurrence.
Additionally, theory- or causally grounded priors that draw on humans' intuitive understanding of how objects interact or behave, could constrain learning in complex domains and support more flexible generalization. Incorporating hybrid exemplar-rule approach, combining memory of spe- cific examples with abstracted rules, would further approximate human category learning. Modular architectures, in which specialized sub-networks handle different aspects of conceptual represen- tation, could enhance generalization and reduce interference between unrelated features. Finally, meta-learned priors distilled from symbolic program- based representations offer way to embed structured, human-like concept hypotheses directly into neural models, allowing them to generalize more like humans across novel situations.
Together, these inductive biases offer a path for models that not only compress information effi- ciently but also organize knowledge in manner that mirrors human conceptual richness, capturing graded typicality, family resemblance, and hierarchical relationships. While integrating these biases may trade some compression for fidelity, they provide an exciting opportunity to reduce meaning distortion and bridge the gap between statistical efficiency and human- like conceptual understand- ing-
B LIMITATIONS
While this study offers valuable insights, several limitations should be considered.
Our analysis primarily focuses on English; generalizability across languages with different
structures is an open question.
Human categorization data benchmark may not fully capture cognitive complexity and could introduce biases.
Our IB-RDT objective is applied to specific LLMs; other models representations might
behave differently.
Our analysis is limited to textual input and does explore image-based representations.
15

---

Published as a conference paper at ICLR 2026
Future work could address these by expanding to other languages, exploring alternative cognitive models, and testing these principles on different architectures or in real-world applications.
C DATASET ACCESS DETAILS
The aggregated and digitized human categorization datasets from Rosch (173a 1975): McCloskey Glucksberg (1978) are made available CSV format at: [URL deduced for anonymity; Data attached as Supplementary Material).
D LLM DETAILS
deberta-large. bert-large- roberta-large (Devlin al. 2019 ct al. 2021 Zhuang et al. 2021).
BERT family:
He
QWEN family: qwen2-0.3b, qwen2.5-0.5b, (wenl.550.55 qwen2.5-1.5b, qwen2- 1.5b, gwen1.5-1.5b, gwenl.5-4b. qwen2.5-4b. qwen2-7b, 7b, qwen1.5-14b, qwen1.5-32b, qwenl.5-
72b, qwen2.5-72b (Bai et al.
2023 Yang et al. 2024).
Llama family: llama-3.2-1 3.2-1b, Ilama-3.1 3.1a--.1-8b. 1-8b, llama-3-8b, llama-3-70b, 1ama-3.1170b
Tou-
vron et al.
2023a Grattafiori et al. 2024
Phi family: phi-1.5, phi-1, phi-2, phi-4 (Javaheripi al. 2023
Abdin et al. 2024
Aboue-
lenin et al. 2025).
Gemma family: gemma-2b, gemma-2-2h, gemma-7b, gemma-2-9h Team et al. 2025).
2024
Mistral family: mistral-7b-v0.3 (Jiang al. 2023
GPT family: gpt2, ged2-medium (Radford al.
2019
DeepSeek family: ReepSeck-RI--issill-eeeeeee
DeepSec-RI--Dssill-ONe BeepSeek-RR1-iie
BaabSsee--R--Diiil------ee 14B,
Llama-8B (DeepSeek-A] 2025).
OLMo family: Olmo-7b (?).
VIT family: Clip VIT-B/32, Clip VIT-B/16 (Radford et al. 2021).
Classic static embeddings: GloVe, Word2Vec (Pennington et al. 2014 Mikolov et al, b).
E CONTEXTUAL PROMPTS AND POOLING STRATEGIES
Contextual embeddings of LLMs require feeding words into the model through a prompt. Because tokenizers often split word into multiple tokens, and since some items in our datasets consist of two or more words, we face a design choice regarding how to aggregate token representations. In our methodology, we adopt average pooling over the actual tokens, ensuring that all subword pieces contribute equally. Figures + and 5 reveal that the average pooling strategy achieves consistent performance and demonstrates the tightest distribution, making it the most reliable choice for our research.
For prompts, we selected neutral template, "This is a {word) (with a trailing space), designed to minimize any additional semantic bias the target item. Figures 6 and 7 show this prompt to balance performance and consistency, making it ideal for bascline comparisons.
In this section, explore alternative pooling strategies and evaluate diverse set of prompt tem- plates across multiple models.
Pooling Strategies.
We compare four common approaches:
Avg all tokens representing the word
First representation of the first token
16

---

Published as a conference paper at ICLR 2026
Model Size Peak AMI by Pooling Strategy
AVG Pooling Stratagy
FIRST Pooling Strategy
Reberts laron
0-6
0.2
Histra vo 10.3-700
PM 22222221
Catama lara
Bert (5459)
sabarta tige 1355M)
Pork
Owant 5 500
Marva v0.2
Daberte harg 2006
Jame
5 12
100M
10 Pooling Strategy
100
2004
L0
102
SENTENCE Pooling Strategy
0.7
Gemin
Gemea
12.711
Eniz 12.700
Malnal
Viainal
0/2
Battleye: (340M)
Ulmma 3.2
Lamu Best large (5400)
Numbar of Paramatera
Number Parameters
Figure 4: Average pooling demonstrates consistent performance across different models, mak- ing the most reliable choice for future research. Each point corresponds prompt template applied to model.
Last representation of the last token
Sentence Avg
the full sentence embedding
Prompt Templates. To test robustness, we design eight templates spanning different linguistic framings:
"This is (word)."
"This (word]. " (with trailing space)
"The concept of [word} is"
When we think of {word], we consider"
"A typical [word} would be"
"Examples of (word) include"
"The categozy {word] contains"
"One kind of (word) is"
Models. We evaluate eight representative LLMs covering major architectures:
lant-aerge (BERT family)
deberta-large (DeBERTa family)
gemma-2-20 (Gemma family)
Llama-3.2-1BB (Llama family)
Mistral-7B-v0.3 (Mistral family)
phi-2 (Phi family)
Qwen2.5-1.5B (Qwen family)
roberta-large (RoBERTa family)
17

---

Published as a conference paper at ICLR 2026
Peak AMI Distribution by Pooling Strategy
0.6
0.5
Peak AMI
0.3
0.2
last
first
sentence_avg
avg
Pooling Strategy
Figure 5: Average pooling demonstrates the tightest distribution, indicating the highest consis- tency and reliability across different conditions. Performance Distribution by Pooling Strategy Box plots showing AMI distribution for each pooling strategy
Template Performance Distribution (AVG Pooling)
0.65
0.60
0.55
0.45
0.40
The (wer)) INo SPACE
This & (won). (SPACE]
The concapt of (sord) i
Bramples of (word) include
The calepery (wor)) containe
When thin (ward), we
typical (word) would
Conrall
Template
Figure 6: The neutral prompt template "This is word. ddemonstrates balanced performance and moderate consistency, making it ideal for baseline comparisons. Performance Distribution across various prompts.
F STATIC VS. CONTEXTUAL AMI EXPLORATION
To understand how LLMs develop conceptual alignment with human categories, we examine the progression from statie contextual embeddings. Figure X presents three complementary views of this progression across different model scales and architectures.
The left subplot shows Static AMI scores, which represent the conceptual alignment achieved by models' input embeddings before any contextual processing (i.e., the E matrix embeddings of the target word). These scores reveal that even the most basic level, L.LMs encode semantic infor-
18

---

Published as a conference paper at ICLR 2026
Template Performance Heatmap by Model (AVG Pooling)
Ueme-3322 3.2-18
1i6rra-78--0.33
Qwen2.5-1.58
Model
bere/erge-uncased
deberta-large
10.3
gemma-2-22-
0.45
phi-2
0.43
robarralarge-
This is (word) (NO SPACE)
Thes (word), (SPACE)
concept al (word.
When we think of (no.
typical (word) wou.
Exampies of (word) I.,
category (word]
One kind (word) L..
Template
Figure 7: The neutral template "This is {word] families, Heatmap showing template performance per model.
remains stable choice across
mation that supports human-like categorical grouping. Remarkably, static models like Word2 and GloVe achieve static AMI scores that rival the peak contextual performance of modern LLMs, suggesting fundamental conceptual structure captured early the learning process.
The middle subplot displays Average AMI across all layers, providing a measure of overall se- mantic representation quality throughout the network. This metric shows the typical performance model achieves across its entire depth, offering insight into how consistently different layers main- tain conceptual alignment. The improvement from static to average AMI demonstrates that contex- tual processing generally enhances rather than diminishes semantic understanding.
The right subplot reveals Peak AMI,, representing the optimal conceptual alignment achieved by any single layer. This metric identifies where in the network conceptual understanding is maximized, typically occurring middle-to-late layers before declining in the final layers. The progression from static peak AMI shows that contextual processing not only preserves but significantly enhances the conceptual alignment present in static embeddings.
Several key insights emerge from this multi-metric analysis. First, all models demonstrate above- chance alignment even in their static embeddings, confirming that basic semantic structure fundamental property of learned representations. Second, the consistent improvement from static to peak AMI across all model types suggests that contextual processing universally enhances concep- tual understanding rather than creating it de novo. Third, encoder architectures of different types (BERT, ViT encoders, and static models) achieve comparable superior performance to much larger decoder models, highlighting that architectural factors and pre-training objectives signifi- cantly influence conceptual alignment quality beyond mere model scale.
This analysis complements the main text findings by showing that LLMs do not simply achieve above-chance alignment with human categories but rather so through systematic progression from basic to sophisticated conceptual representations, with contextual processing serving as an amplifier rather than generator of semantic understanding.
We also tested the robustness our results to different clustering seeds (Figure 9 We found AMI to be highly stable across seeds, with negligible variation in the peak values and layer-wise profiles, indicating that our conclusions are not sensitive to the choice of clustering initialization.
Lastly, plot the peak AMI against the number of FLOPS per token and find no systematic corre- lation, suggesting that computational cost alone does not predict human-aligned conceptual repre- sentations (Figure
19

---

Published as a conference paper at ICLR 2026
Static AMI
Average AM]
Peak AMI
Familyy
Bert
Gemma
Owen
Stalic AMI
Averaggn AM
Peak AMI
Llama
0.3
Mistral
Olimo
0.2
GPT
VIT
0.1
Classic
Deepseek
0.0
100M
500M
1.58
3.5
100
258
708
100M
500M
1.58
258
708
100M
500M
1.50
3.58
108
258
708
Model Size (log scale)
Figure 8: LLMs begin with basic conceptual alignment in static embeddings and achieve pro- gressively stronger alignment through contextual processing. Three subplots showing model size (log scale) versus different AMI metrics. Left: Static AMI reveals baseline categorical structure. Middle: Average AMI across layers shows overall semantic quality. Right: Peak AMI demonstrates high conceptual alignment. The consistent improvement from static to peak AMI across all model types reveals that contextual processing enhances rather than creates conceptual understanding, with encoders (BERT, ViT encoders, and static models) achieving comparable or superior performance much larger decoder models.
Peak AMI Consistency by Model and Dataset
Rosch 1975
Mooce argo
Pest MM Score
Cipnc save yott.1
(Spea Sasto11
redertan Begge
Lseoa AB
Owend S 1.5
piiz
Mussos v5.
deterta
PeSk 21 Score
lave ponzbll
cip p0ib
dedecie Bayse
rnberdin Sagge
germii 2.2b
18 viu
Ulma
McCloskey 1978
Reek 188 Score
base pabitle
cig boudn bu
defec Eage
large orciaeed
Iddenta lange
Lama 2. 1D
imeed Ai
alock
Nuarel vt.3
mmme
Figure 9: AMI Scores are Robust Across Clustering Seeds. Peak AMI between model representa- tions and human categories remains highly stable across multiple random clustering initializations. Peaks in each plot consistent, indicating that the observed alignment patterns not sensitive to the choice of clustering seed.
20

---

Published as a conference paper at ICLR 2026
0.60
*
0.55
Model Family
Bert
0.50
*
Gemma
*
Qwen
Peak AMI
0.45
Phi
Liama
0.40
Mistral
Olmo
0.35
GPT
Deepseek
0.30
ViT
Classic
0.25
0.20
200M
1B
3B
7B
20B
50B
140B
FLOPS per Token (log scale)
Figure 10: Peak AMI versus computational cost. We plot each model's peak AMI against its FLOPS per token. There is no systematic correlation, indicating that higher computational cost does not necessarily lead to better alignment with human conceptual structure.
G MULTILINGUAL ANALYSIS
To further explore conceptual understanding across languages, we translated our dataset into Span- ish, German, Italian, and Russian using Google Translate's ' API and repeated our analyses with the same LLMs and methods. In RQ1, clear scale effect emerges for all non-English languages, while English shows no such trend (Figures 11 12 We interpret this English training data: larger models are more likely have been exposed to sufficient multilingual data, improving their conceptual alignment. In RQ2, all models struggle to preserve the internal geometry of human concepts across languages (Figure T3). RQ3 shows that non-English languages exhibit greater compression (Figure 14). consistent with our explanation for RQ1: smaller exposure to non non-Engiln English data leads to more compressed representations, reducing flexibility and interpretabil- ity.
consequence of limited non-
H TRAINING DYNAMICS
The OLMo analysis examines how semantic structure develops during training by analyzing 57 intermediate checkpoints from the OLMo-7B model, representing evenly spaced sampling (every 10K training steps) spanning from 1K 557K steps (covering approximately 4B to 2.5T tokens).
The analysis employs two complementary sampling strategies: representative sampling (6 check- points) captures major developmental phases IK. 101K, 201K, 301K, 401K, and 501K steps, while high-resolution sampling (57 checkpoints) reveals the inherent noise and fluctuations in training. Despite significant training noise, the overall semantic development follows stable, pre- dictable pattern captured by the representative sampling, shown in Figure I6 The complete training trajectory with all checkpoints is presented Figure
Moreover, this double- phase dynamics occurs when testing attention sparsity, effective rank, and values (Figures 18 Meaning, all of which exhibit the same early rapid shift followed by a slower restructuring phase. This convergence across independent metrics indicates that the model not merely improving categorical alignment, but reorganizing its internal representations toward increasingly efficient structure.
21

---

Published as a conference paper at ICLR 2026
0.6
0.5
0.4
Peak AMI
0.3
English
English (BERT)
MERT SALTO
0.2
HERT SREE
Spenish
Ralien
Itaiaa (BERT)
Russian (RFRT)
Engfish comelation
Spanish comeeaaiio
Ralian
Geraan corrlection
Ruesten comelation
100M
500M
3.5B Model Size (log scale)
1.5B
10B
258
708
Figure 11: Scale effect emerges for non-English languages. We compute peak AMI as a function of model size different languages: English (red), Spanish (blue), Italian (purple), German (yellow), and Russian (pink). While English shows no systematic scaling effect, the other languages exhibit clear positive relationship between model size and AMI. We interpret this as consequence of limited non-Englist training data: larger models are more likely to have been exposed to sufficient multilingual data, improving their conceptual alignment. Additional analyses across other RQs using the multilingual data support this hypothesis.
Garman
English
Spanish
SATO
P-CID
0.35
0.5
0.30
0.30
0.4
0.20
0.20
0.15
0.10
0.05
sowe)
tcale)
Italian
Russian
0.5
0.40
0.30
0.3
0.20
0.2
0.10
0.1
0.05
304
38C
3004 2006
508K
506
100K
2006
2006
Figure 12: Scale effect emerges for non-English languages. We compute peak AMI as a function of model size across different languages; English (red), Spanish (blue), Italian (purple), German (yellow), and Russian (pink). While English shows no systematic scaling effect, the other languages exhibit clear positive relationship between model size and AMI. We interpret this as consequence of limited non- oon-Eshlish training data: larger models are more likely to have been exposed sufficient multilingual data, improving their conceptual alignment. Additional analyses across other RQs using the multilingual data support this hypothesis.
22

---

Published as a conference paper at ICLR 2026
Spearman Correlation of Cosine Similarity and Typicality by Language
Language
Engish
Peak AMI Layer Spearman Correlation
Spenisl
0.4
German
Russlen
0.2
0.0
-0.2
VIT
Ownen
GPT
Mistral
Bert
Uama
Gemma
Okmo
Model Family
Figure 13: Preservation of internal conceptual geometry across languages. Models struggle to maintain the internal structure of human concepts in all tested languages, with similar patterns observed across English, Spanish, German, Italian, and Russian.
I DATASETS POLYSEMY
Scope. We quantify lexical ambiguity in our psycholinguistic stimuli by counting the distinct Word- Net synsets associated with each lemma. This polysemy score lets estimate how many alternative senses a model must implicitly conflate when produces single embedding for a word.
Why it matters. Consider bar, which can denote either flying mammal piece of sports equipment. The same vector must account for both senses. Aggregating semantically distant senses can blur the representation and thus confound model-human comparisons, especially in tasks that rely on fine-grained semantic similarity. Explicitly tracking polysemy allows us to verify that any performance effects we observe are not artefacts of lexical ambiguity.
Results Figure shows the distribution of polysemy scores. The majority of items are unambigu- ous (1-2 senses). but heavy-tailed minority (e.g. Running (52 senses), Saw (28), Block (28)) is highly polysemous. This suggests that our findings are due to real differences between the mod- els rather than polysemy-related artifacts. An additional 141 lemmas that are not WordNet were omitted.
J TOKENIZER ANALYSIS
Rational. The tokenizer of a model has a significant influence over the representations: segmen- tation rules (WordPiece vs. BPE), vocabulary size and special control tokens can inflate sequence length, skew frequency statistics, and shape eтrоr patterns. To ensure fair cross-model comparisons, we therefore (i) cluster checkpoints by the tokenizer they use and (ii) quantify how much those tokenizers overlap when applied to our datasets.
Procedure. Before computing overlap, we normalize the vocabulary, stripping tokenizer-specific characters; SentencePiece prefixes (-), GPT-style BPE space prefixes (G/g) and newline markers (C), WordPiece continuations markers (#H), and related block characters. After this clcanup, tokens differing only by such prefixes collapse to shared canonical form (e.g. _house, Chouse, and #house all become house). We then compute pair-wise Jaccard similarity on these cleaned vocabularies.
Table summarizes the core statistics and information regarding the tokenizer types, while Figure visualizes the resulting pairwise vocabulary overlap.
23

---

Published as a conference paper at ICLR 2026
Findings. We find that most tokenizer families share substantial lexical overlap, often exceed- ing 60%, suggesting a de-facto common token inventory across recent open-source models. First- generation BERT WordPiece (bert-large pert-laage-uncasee and bert-base-uncasedl are outlier, sharing un- der 16% of tokens with any other group.
Model Family
Mean Tokens Item
Vocabulary Size
Tokenizer Type
BERT
1.65
30K
WordPiece
DeBERTa & RoBERTa
2.30
50K
WordPiece
GPT
2.30
50K
BPE
Gemma
1.65
256K
SentencePiece (subword)
Llama
2.19
128K
BPE (SentencePicce/Tiittkee
Mistral
2.35
32K
BPE control tokens
Phi
2.30
32K
BPE (SentencePicce/Tiittketl BPE
Qwen
2.19
151.6K
Table I: Tokenizer statistics by tokenizer family. Mean Tokens/Items refer to the average of the tokens per item in our datasets. The columns Vocabulary Size and Tokenizer Type are properties of the tokenizer.
Model Clustering By Tokenizer Family
Llama: Llama-3.2- (representative), 1lama-3.1-8B, Meta-Llamaa3--88 1lama-3.1-70B.
Meta-Llama-3-4 3--BBB
Gemma: gemma-2-9b (representative), gemma-7b, gemma-2b, gemma-2-2b
Mistral: Mistral-7B-vO.33 (unique)
Phi: phi-2 (representative), phi-1, phi-1.5
RoBERTa: roberta-large (unique)
DeBERTa: deberta-large (unique)
GPT: gpt2-medium (representative), gpt2
BERT: bert-large-uncant (representative), bert-base-uncased
Qwenl.5-14BB, Qwen2-0.5B, Qwen2-7B, Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-32B, Qwenl.5-32B, phi-4
Qwen: Qwen1.5-0.5B (representative),
K ADDITIONAL CLUSTERING METRICS
To further validate our cluster alignment findings (Section 5. in addition to Adjusted Mutual Information (AMI) and the Normalized Mutual Information (NMI, we also computed the Adjusted Rand Index (ARI) for the k-means clusters derived from LLM embeddings against human-defined categories. ARI measures the similarity between data clusterings, correcting for chance. Like AMI, a score indicates perfect agreement and indicates chance agreement.
Across all tested LLMs, the ARI and NMI scores largely mirrored the trends observed with AMI, showing significantly above-chance alignment with human categories and similar relative model per- formances. Silhouette scores, while more variable, generally indicated reasonable cluster cohesion for both LLM-derived and human categories. Detailed tables of these scores are provided below.
These supplementary metrics reinforce the conclusion that LLMs capture broad human-like concep- tual groupings.
L MINI-CONTROLLED EXPERIMENT (MATCHED TRAINING DATA)
To evaluate the extent to which dataset differences might account for the architectural patterns we report, conducted fatched-family analyses involving the only model families that can be aligned
-hi-4 has different tokenizer than the rest of Phi family. The results of its tokenizer match the tokenizer of the Qwen family.
24

---

Published as a conference paper at ICLR 2026
in both training data: GPT, Pythia, Cerebras, and T5. While these comparisons cannot rule out all confounds, they substantially reduce the influence of dataset variation. Across matched settings, encoder models continue outperform decoder models, yielding higher AMI and lower L. This indicates that the architectural effects observed throughout the paper cannot be explained by differ- ences in training data alone.
M DETAILED AMI SCORES PER MODEL AND DATASET
Table provides granular view of the static AMI scores for each LLM across the three individual psychological datasets.
Dataset
Model
NMI
AMI
ARI
RRosch 1973c)
bert- lango-
0.19453
0.2011
0.11336
Rosch 1975
bert-large-uncased
0.16547
0.27324
0.2216
Mccyoskey & Glucksberg 1978
bert-laggeunncssed
0.12003
0.15934
0.06306
Rosch 1973c
FacebookA//roberttalaagg
0.1021
0.10666
0.03393
Rosch 1975
Facebook&A/rrobettallrrgg
0.12138
0.23938
0.14165
McCloskey & Glucksberg
1978
FacebookA//rrbertta-bae
0.06271
0.08873
0.03173
Rosch 1973c)
google-tS/tS-larg
0.16583
0.16855
0.03676
Rosch 1975
google-t3/t5-lagg
-0.03799
0.04179
0.00758
McCloskey & Glucksberg
1978
google-t3/t5-arge
0.06146
0.08825
0.0082
Rosch 1973c
googlefgemmma-2227b
0.08523
0.09065
0.04158
Rosch 1975
google/gemma-22222 2-227
0.04276
O. 0.10062
0.06244
McCloskey & Glucksberg
1978
googlefgemmma-2227b
0.07814
O. 0.10274
0.04364
Rosch 1973c)
google/gemma-2221
0.04029
0.04107
0.01212
Rosch 1975
google/gemma-
0.04529
0.14844
0.07596
McCloskey & Glucksberg
1978
2oogle/gemmm--22bb
0.09953
0.13593
0.06326
Rosch 1973c)
google/gemma-2299
0.1222
0.12757
0.06053
Rosch 1975
googlefgemma-2-96
0.07841
0.16126
0.09617
McCloskey & Glucksberg
1978
2oogle/gemmma-299
0.10879
0.13997
0.06439
Rosch 1973c)
google/gemma226
0.04336
0.04616
0.01593
Rosch 1975)
google/gemma-22
-0.00353
0.04483
0.01577
McCloskey & Glucksberg
1978
google/gemma- 2b
0.03472
0.05484
0.02142
Rosch 1973c)
google/gemma-771
0.04459
0.04547
0.01052
RRosch 1975]
google/gemma--1
-0.03055
0.02644
0.01506
McCloskey & Glucksberg
1978
google/gemma-781
0,03338
0.05724
0.02176
Rosch 1973c)
meta-llama/Llama-3112200
0.03008
0.03528
0.01936
RRosch 1975)
meta-
Satt-lllmaLLllmm-3333
-0,07026
0.02636
0.00392
McCCoskey & Glucksberg
1978
meta- Dae-llmmmLllammm-33
-0.04773
0.00972
0.00236
Rosch 1973c
Bettallama/LLlmma-...1
0.00473
0.00393
0.00023
Rosch 1975
meta- pettalllaaa/Llamaa 3-88
-0.03928
0.05489
0.01884
McCloskey & Glucksberg
1978
meta-llama/Llama-3.11
-0.02671
0.02208
6.00E-05
Rosch |ooshh19973 1973c)
meta meta-llams/Llama-33
0.01936
0.01567
0.00246
RRosch 1975)
meta
Sitt-llamallllmm-33220
-0,01876
0.05663
0.00782
McCloskey & Glucksberg
1978
meta-llama/LLamma33222
0.03625
0.06798
0.01352
Rosch 1973c)
meta- meta-llams/Llama-
0.03757
0.03537
0.00876
Rosch 1975
meta-llama/llama-3322222
0.01893
0.09619
0.03193
McCloskey & Glucksberg
1978
Dettallaaa/Llaam-3333
0.03914
0.07395
0.0202
Rosch 1973c)
meta- meta-llams/Metia netellaaass/cctt__aaaa33-3 -lam-33--8
0.02289
0.03133
0.01514
Rosch 1975
Hita-llmmaMMett--llana----
-0.06428
0.0185
0.00554
McCloskey & Glucksberg
1978
meta-llama/Meraa -lama-3--00
-0.04595
0.01068
0.00272
Roosh 1973c)
0,03512
0.02852
0.00225
RRosch 1975
meta- -aaaaMaa-----a---81
-0.06011
0.03694
0.00676
McCloskey & Glucksberg
1978
meta- SittlllamMMMctt-Llaaaee
-0.0355
0.0219
0.00676
RRoshh 1973c)
microsoft/deberta-laagge
0.03748
0.03909
0.01467
RRosch 1975)
microsott/deberta-lagg
0.16568
0.28993
0.20527
McCloskey Glucksberg
microsoft/debertt-langg
0.03217
0.06175
0.03019
[Rosch 1973c)
microsoft/phi-155
0.02102
0.01786
0.0075
[Rosch 1975)
microsottphi-1.5
0.03989
0.13887
0.04305
McCloskey Glucksberg
microsoft/phi-13
0.00895
0.05215
0.00639
25

---

Published as a conference paper at ICLR 2026
Rosch 1973c)
microsoft/hhi-
0.0249
0.01698
0.00133
(Rosch 1975]
microsoft/phii-
-0.03625
0.028T1
(0000
McCloskey & Glucksberg 1978
microsoft/phi-
-0.01148
0,03085
Rosch 1973c)
microsoft/hhi-2
0.03703
0.02968
0.00404
[Rosch 1975]
microsoft/phi-2
-0.03654
0.04227
0.03942
McCloskey & Glucksberg
1978
microsoft/phi-2
-0.00254
0.02531
0.00533
Rosch 1973c)
microsoft/phi-4
0.03075
0.03043
0.01076
[Rosch 1975)
microsoft/phi-4
-0.06737
0.00099
0001361
McCloskey & Glucksberg
1978
microsoft/phi-4
-0.01789
0.02705
0.00066
Rosch 1973c)
mistralai/Misttal-78---0
0.0425
0.03507
0.00357
Rosch 1975
mistralai/Mistra-7B--0000
-0.05018
0.01217
0.0177
McCloskey & Glucksherg
1978
mistalla//Mistra-788--...
-0.01264
0.03902
0.00931
Rosch 1973c)
Qwvnn(ween1.5-0.51
0.00148
-0.00225
0.00399
Rosch 1975
Qwen/ONwenl.340.5B
-0.01538
0.04833
0.0095
McCloskey & Glucksherg
1978
Qwen/Owenl1.5-0.58
0.02559
0.06023
0.00771
Rosch 1973c)
Qwen/Qwenl.5-1.8B
0,03397
0.03232
0.01034
Rosch 1975
Qwen/QWeenl.5.1.8B
-0.01129
0.05803
0.00683
McCloskey & Glucksherg
1978
Qwen/OWeen..5-1.88
-0.00541
0.03614
0.00538
Rosch 1973c)
Qwen/Qwenl.5-144
0.0372
0.02738
0.0028
Rosch 1975)
Qwen/Qwenl.5-T44B
-0.02604
0.05153
0.01211
McCloskey Glucksberg
1978
Qwen/Owenl.5-14B
0.00124
0.04136
0.00338
Rosch (9/3c)
Qwen/Qwenl.5-32B
0.02638
0.02436
0.00409
Rosch 1975)
Qwen/Qwen1..5-328
-0.03413
0.02526
+0,0665
McCloskey & Glucksberg
1978
Qwen/Qwenl.5-32B
-0.01991
0.02124
+0,0059
Rosch 1973c)
Qwen/Qwenl.5-44B
0,03803
0.04058
0.01742
Rosch 1975)
Qwen/Qwen1..5448
-0.03309
0.03988
0.01678
McCClskey & Glucksberg
1978
Qwen/Qwenl.5-4B
-0.03997
0.00548
+0,00088
Rosch 1973c)
Qwen/Qwenl.5-72
0.03697
0.02892
0.00144
Rosch 1975
Qwen/Qween..5-72B
-0.06184
0.02213
0.0017
McCloskey & Glucksherg
1978
Qwen/Qwenl..57228
-0.02022
0.02918
0.00297
Rosch 1973c)
Qwen/Qwen2-40558
0.02266
0.01923
0.00662
Rosch 1975)
Qwen/Qwen2-0.5B
0,0515
0.14571
0.04999
[cCCloskey & Glucksherg
1978
Qwen/Qwen2-00558
0.01508
0.04357
0.00643
Rosch 1973c)
Qwen/Qwen2-L.5B
0.02956
0.02779
0.00544
Rosch 1975
Qwen/QQee221155B
-0.03595
0.03443
-0.01099
McCloskey & Glucksherg
1978
Qwen/Qwen2-11558
0.01768
0.05407
0.01604
Rosch 1973c)
Qwen/Qwen2-7B
0.06424
0.06439
0.02067
Rosch 1975)
Qwen/Qwen2-7B
0.0333
0.09155
0.02832
McCloskey & Glucksberg
1978
Qwen/Qwen2-7B
0.05329
0.07599
0.01977
Rosch 1975c)
Qwen/Qwen2.5-003B
0.03165
0.03291
0.01029
Rosch 1975
Qwvnn/Qwen..5-0.58
-0.06534
-0.0196
1.01
McCloskey & Glucksberg
1978
Qwen/Qwen2.550055B
0.0062
0.04191
0.0054
Rosch 1973c)
Qwen/Qwen2.55-1.58
0.04838
0.0489
0.0129
Rosch 1975)
Qwen/QQwe2..-115B
0.03785
0.113
0.02761
McCloskey & Glucksberg
1978
Owen/Qwen2.5-11.BB
0.06166
0.08675
0.03162
Rosch 1973c)
Qwen/Qwen2.5-311
0.03882
0.0348
0.00465
Rosch 1975)
Qwen/Qwen2.5-3B
0.03977
0.10821
0.04302
IMcCloskey & Glucksherg
1978
Qwen/Qwen2.55338
0.03416
0.07307
0.02959
Rossh 1975c)
Qwen/Qwen2..-77B
0.0529
0.05051
0.01605
Rosch 1975)
Qwen/Qwen2.5-7B
-0.00905
0.03227
0.01044
(McCloskey & Glucksherg
1978
Qwen/Qwen2.55778
0.00222
0.02759
0.00551
Table 2: Mutual information measures (normalized mutual information, adjusted mutual informa- tion, adjusted rand index) per model per dataset. Aggregated results are shown in the main paper and the Figures in the Appendix.
26

---

Published as a conference paper at ICLR 2026
N CORRELATION BETWEEN HUMAN TYPICALITY JUDGMENTS AND LLM INTERNAL CLUSTER GEOMETRY
The following tables present the Spearman correlation coefficients (p) between human typicality judgments and LLM internal representations across different analysis approaches:
T Static analysis correlations using embeddings from the E matrix. This approach captures the baseline semantic relationships between items and categories without contextual processing.
Peak layer analysis correlations using contextual embeddings from layer that maximized AMI scores (as identified RQ1). This approach leverages the optimal layer semantic clustering to assess fine-grained semantic fidelity.
Table 4
Both tables present correlations across three cognitive science datasets: Rosch (1973), Rosch (1975), and
McCloskey (1978), with asterisks (*) indicating statistically significant correlations < 0.05). The modest correlation values across most models suggest limited alignment between LLM internal representations and numan-pereeived semantic nuances.
Dataset Correlation (Spearman p) Statiic Layer
Model
Rosch (1973)
Rosch (1975)
McCloskey (1978)
Deberta large (304M)
0.144
0.107
0.075
Bert large (340M)
0.378
0.275
0.250*
Roberta large (355M)
0.005
0.038
-0.029
Gemma (2B)
0.069
0.078
0.007
Gemma (2B)
0.236
0.119*
0.147*
Gemma (7B)
0.131
0.100
0.007
Gemma (9B)
0,280
0.135
0,199*
Gemma (27B)
0.112
0.122
0.161 *
Qwen1.5 (0.5B)
0.175
0.076
0.096*
Qwen2 (0.5B)
0.238
0.041
0.040
Qwen2.5 (0.5B)
0.212
0.027
0.037
Qwen2.5 (1.5B)
0.141
0.086
0.078
Qwenl.5 (1.8B)
0.172
0.134
0,154*
Qwen2 (7B)
0.036
0.087
0.040
Qwenl.5 (14B)
0.154
0.086
0.108
Qwenl.5 (32B)
-0.032
0.100
0.081
Qwen2.5 (32B)
0.035
0.105
0.084
Mistral v0.3 (7B)
0.076
0.152
-0.009
Llama 3.2 (1B)
0.301
0.056
0.039
Llama (8B)
-0.002
0.099
0.080
Llama 3.1 (8B)
0.004
0.108
0.081
Llama (70B)
0.148
0.161
0.155
Llama 3.1 (70B)
0.122
0.161 *
0.155*
Phi 1 (1.42B)
0.071
0.052
0,054
Phi 1.5 (1.42B)
-0,088
0.079
0.018
Phi 2 (2.78B)
-0.056
0.044
0.024
Phi 4 (14.7B)
0.079
0.086
0.097
T5 Large (770M)
0,235
0.259
0,178*
GPT-2 Medium (355M)
-0.032
0.063
-0.017
VYT-B/32 Text (63.1M)
0.527*
0.315
0.286*
VIT-B/16 Text (63.1M)
0,528
0,289
0,278*
Word2Vec (300D)
0.442*
0.349
0,437*
Glove (300D)
0.315
0.333
0.350*
Table Correlation between Human Typicality Judgments and LLM Internal Cluster Geome-
try. Spearman static-layer rank correlations between human-rated psychological typicality/distance (higher human scores less typical/more distant) and item-to-centroid cosine similarity (higher sim- ilarity = more central to LLM cluster). "p < 0.05.
27

---

Published as a conference paper at ICLR 2026
Dataset Correlation (Spearman () Peak AMI Layer
Model
Rosch (1973)
Rosch (1975)
McCloskey (1978)
Deberta large (304M)
0.277
0.107*
0.126*
Bert large (340M)
-0.120
0.148*
0.026
Roberta large (355M)
-0.011
0.038
-0,022
Gemma (2B)
0.127
0.092*
0.039
Gemma 2 (2B)
0.034
0.139*
0.103*
Gemma (7B)
0.004
-0.050
0.088
Gemma (9B)
0.047
0.110*
0.098
Gemma (27B)
-0.135
0.090*
-0.101
Qwen1.5 (0.5B)
-0.025
-0.064
0.122
Qwen2 (0.5B)
0.090
0.064
-0.077
Qwen2.5 (0.5B)
-0.012
0.121
0.072
Qwen2.5 (1,5B)
-0.114
0.084*
-0.028
Qwen1.5 (1.8B)
0.092
-0.044
-0.004
Qwen2 (7B)
0.004
0.049
-0.087
Qwenl.5 (14B)
0.032
0.091 *
0.043
Qwen1.5 (32B)
0.026
0.082
0,111
Qwen2.5 (32B)
0.045
0.079
-0.060
Mistral v0.3 (7B)
0.013
0.039
0.107*
Llama 3.2 (1B)
0.063
0.094*
0.070
Llama 3 (8B)
-0.045
0.138
0.084
Llama 3.1 (8B)
-0.096
0.130*
0.087
Llama (70B)
-0.108
0.023
0.050
Phi (1.42B)
0,216
0.185*
-0.029
Phi 1.5 (1.42B)
0.162
0.09g
0.015
Phi 2 (2.78B)
0.325
0.142*
-0.033
Phi 4 (14.7B)
0.079
0.129*
0.007
T5 Large (770M)
0,219
0,226
0,282*
GPT-2 Small (117M)
-0.046
0.118
0.010
GPT-2 Medium (355M)
0.077
0.109*
-0,029
VIT-B/32 Text (63.1M)
0.128
0.055
0.152*
VIT-B/16 Text (63.1M)
0.089
0.086*
0,127*
Word2Vec (300D)
0.442*
0.349*
0.437*
Glove (300D)
0.315*
0.333*
0.350
Table 4: Correlation between Human Typicality Judgments and LLM Internal Cluster Ge- ometry. Spearman peak-AMI layer rank correlations between human-rated psychological typical- ity/distance (higher human scores less typical/more distant) and item-to-centroid cosine similarity (higher similarity more central LLM cluster). "p < 0.05.
O TYPICALITY AND COSINE SIMILARITY [RQ2]
Figure 25 shows representative scatter plots illustrating the relationship between human typicality scores (or psychological distances) and the LLM-derived item-centroid cosine similarities for selected categories and models. These plots visually demonstrate the often modest correlations discussed in Section
Figure 26 shows the aggregated Spcarman correlation across model families and datasets. These conrelations are very weak and mostly non-significant.
P THEORETICAL EXTREME CASE EXPLORATION FOR C
In the case where IC = |X| (cach data point is cluster of size 1, so |Cal Ve E C), then H(X(C'= = Ensc 1 log, 1 0. The distortion term o² 0 for each cluster as the item is its own centroid. Thus, L= T(X: 3- H(X) HXX(C)= H(X) log, |X]. This represents the cost of encoding each item perfectly without any compression via clustering, and distortion.
28

---

Published as a conference paper at ICLR 2026
In the where |C| 1 (one cluster Cx contains all [X] data points, CC [ then [X|C) Ix |X] log, X logz X Thus, I(X;C) -XX)-H(X(C) log, X logz X This represents
maximum compression (all items are treated as one). The distortion term becomes 3 [X] [X] ox = 8 ox
where x is the variance of all items X with respect to the global centroid of X. So, L 0 9. 3. This represents the scenario of maximum compression where the cost is purely the distortion incurred by representing all items by single prototype.
Q
COMPRESSION FIGURES
Figure 2% depicts IB-RDT ohjective (C) vs. K. Lower indicates a more optimal balance hetween compression (I):;C) and semantic fidelity (distortion). Human categories (fixed K) show higher L values.
R COMPLEXITY-DISTRRTEE RATIO (ON THE IMPORTANCE OF B)
Figure 29 provides an additional sensitivity analysis in which we examinc the ratio between the distortion and complexity components of C as 8 varies. Across all three datasets, encoder models maintain flat profiles, indi- cating stable conceptual structure under compression, whereas decoder models exhibit stronger shifts, reflecting greater reallocation of representational capacity.
L OBJECTIVE VS. DOWNSTREAM TASK PERFORMANCE
Analysis of 13 instruction tuned models across 5 families (Qwen, Llama, Gemma, Phi, Mistral; see Table results) indicates statistical significance (r = - 00.202, = 0,508). This finding suggests that while the C objective successfully identifies models that compress semantic categories ahility does directly translate improved performance standard NI.P henchmarks. The lack of correla- tion implics that concept compression henchmark accuracy represent distinct aspects model capahility, with the former capturing semantic organization efficiency and the latter mcasuring general knowledge and reasoning abilities. We specifically chose instruction-ture models to ensure fair comparison on MMLU, as base models would likely perform poorly this instruction-folloiing benchmark. While analysis covers a diverse range model families and sizes, this represents a subset available models due the limited availability of instruction- sandection-tuee variants.
effectively, this compression
Model
Size
L Score
MMLU Score
Qwen2-0.5 5.nnn---.B-Instrucc
494M
1.930
0,433
Dwen2.55-0.5B-Instrue
494M
1.982
0.469
Llama-3.2 1.-
1.2B
1.876
0.454
Owen2.5-1..BB-Insite
1.5B
2.071
0.597
Gemma-2B-IT
2.5B
2.382
0.366
Gemma-2-2B-IT
2.6B
2.263
0,565
Phi-4-mini-instruct
3.8B
1.905
0.678
7.2B
1.714
0.603
Swen2-7B-Instrus
7.6B
2.319
0.700
Meta- Llama-3-8B IB-Instruct
8.0B
1.348
0.647
L.ama-3.1-88BImme
8.0B
1.320
0.679
Gemma-7B-IT
8.5B
2.372
0,512
Gemma-2-99-I7
9.2B
2.467
0.723
Table 5: No correlation between L objective scores and MMLU scores across different sizes and families. Table displays objective values vs. MMLU scores for 13 instruction tuned models. Correlation measured (r -0.202, p 0.508).
29

---

Published as a conference paper at ICLR 2026
Engiish
Scomsh
hallen
Uama
VIT
Number Clusters
humber Clusters
(a) a function of K for Rosch 1973a
Engtis
Spanies
Number Elucarn
Number Elinderx Russion
Nurtber Clustexs
German
Humber of Clustrs
Numbor of Clusters
(b) L as function of K for Rosch 1975
Number of Clutar 30 German
Number of Clusters (K) Russion
Mumber of Cuatera (K)
Hurus
Number
(c) L a function of K for McCloskey & Glucksberg 1978).
Figure 14: Compression of non-English representations. All non- non-English languages (Spanish, German, Italian, Russian) exhibit higher compression than English, with smaller models showing the greatest compression. This supports the interpretation that limited non-Ent non-English training data leads to less flexible and less interpretable representations.
30

---

Published as a conference paper at ICLR 2026
6.50g
0.522
3550
Layar
064
co 425
C4157
22
Peak AMI Score
0.4
03860
Peak Layer Index
28
2.52
0.2
100M
200M
300M
400M
500M
103M
200M
300M
4DOM
500M
Training Step
Training Step
Figure 15: OLMo-7B develops conceptual structure through two-phase dynamics. Left: AMI with human categories rises rapidly in early training then refines gradually. Right: Peak seman- tie processing migrates from deep (layer 28) toward mid-network layers during training, revealing architectural reorganization for efficiency. Representative checkpoints shown; full 57-checkpoint analysis in Appendix B.5.
Poak AMI
Peak Layer
3410
0.4
Peak Layer Index
Peak AMI Score
25
0.1
300M
200M
300M
400M
100M
200M
400M
500M
Training Step
Training Step
Figure 16: Left: OLMo-7B representations steadily strengthen during training: Concept rep- resentations develop rapidly at early steps, then refine more gradually over time. Right: Semantic processing shifts from deep to mid-network layers: The model undergoes two-phase dynamic initially moving semantic processing upward during rapid learning, then reorganizing to opti- mize efficiency while preserving performance. To improve readability, we present six representative checkpoints that capture the trend.
31

---

Published as a conference paper at ICLR 2026
0.322
18257 $500
Layer
0.5
Peak AMI Score
Peak Layer Index
28
24-
0.1
0.24
20
0M
100M
200M
300M
400M
500M
0H 103M
200M
300M
400M
500M
Training Step
Training Step
Figure 17: Complete OLMo-7B training trajectory across checkpoints: This high-resolution view reveals the inherent noise and fluctuations in training, with individual checkpoint measure- ments varying throughout the process. Despite this variability, the overall trend aligns with the stable pattern shown in Figure [6] demonstrating that representative sampling effectively captures the underlying semantic development trajectory while filtering out training noise.
Aggragation
Sartcass-Aggggeiion
Tecding
Mide
Takkin
(a) Effective rank across network layers.
(b) Gini coefficient across network layers.
Marhids
Tepiing
(c) Hoyer sparsity across network layers.
Figure 18: Network-level measures of representational structure across training. Each panel shows a different metric: (a) effective rank, (b) Gini coefficient, and (c) Hoyer sparsity computed across network layers. All three measures reveal the same two-phase developmental pattern ob- served in Figure 15 an early rapid change followed by slower restructuring, indicating coordinated reorganization of internal representations.
32

---

Published as a conference paper at ICLR 2026
Reach (2873)
Reach (1375)
Mccloskey (1978)
60
23
2.7
1.
1030
200c
100
3:00
1.00
Taining
Taitning
Figure 19: across training, Across all three human benchmarks, Olmo training checkpoints reveal the same two-phase developmental pattern observed in Figure [5 early rapid change followed by slower restructuring, indicating coordinated reorganization of internal representations.
Mean:
Median: 2.00
500
400
Number Words
300
200
100
10
20
50
[[t]
Number of Senses
Figure 20: Polysemy is not likely to influence our results as most of the words in our dataset are conerete nouns, which tend to be less polysemous. Histogram WordNet sense counts for the 943 lemmas in our benchmark. The dashed lines indicate the median and mean (2 senses, 3.47 senses, respectively).
33

---

Published as a conference paper at ICLR 2026
Normalized Tokenization Similarity Matrix
Uama
73.2
99.8
Genma
42.4
100.0
36.5
36.4
36.4
36.4
30.4
15.6
42.3
Mistral
59.1
30.6
100.0
60.8
00.8
00.8
00.8
11.6
59.1
Takenizer Groups
Sirilariy
RoBERTS
30.4
100.0
DeBERTS
73.2
38.4
60.8
100.0
100.0
100.0
100.0
8:1
73.4
GPT
73.2
36.4
60.8
100.0
100.0
100
100.0
73.4
98.8
42.5
59.1
154
100.0
Liams
Gemmaa
Micraa
Batema
GPT
Owen
Takenizer Groups
Figure 21: Substantial lexical overlap suggests that tokenization differences alone cannot ex- plain the observed performance variations in our experiments. Vocabulary overlap between different tokenizers; most tokenizer families share substantial lexical overlap.
Bert
Gemma
Qwen
0.275
Liama
Phi
NMI
Mistral
0.225
0.175
*
0.125
20
21
22
23
24
25
Model Size (log scale)
Figure 22: LLMM-derived Clusters Show Above-Chance Alignment with Human Conceptual Categories. Normalized Mutual Information (NMI) between numan-defined categories and clusters from static LLM embeddings. Results are averaged over three psychological datasets. All models perform significantly better than random clustering. BERT's performance is notably strong.
34

---

Published as a conference paper at ICLR 2026
0.14
Bert
Gemma
0.10
Qwen
Llama
ARI
0.06
*
Mistral
0.02
-0.02
20
21
22
23
25
Model Size (log scale)
Figure 23: LLM-derived Clusters Show Above-Chance Alignment with Human Conceptual Categories. Adjusted Rand Index (ARI) between human-defined categories and clusters from LLM static embeddings. Results are averaged over three psychological datasets. All models perform significantly better than random clustering. BERT's performance is notably strong.
0.45
Spearas of Cos Smilsr and Typoolly
0.40
Layet 5000 mAN
Model Ramily
GPT
0.25
Pyrise
100M
S00M
3.50 Model Sine Oog scale)
1.55
10B
250
700
Modal family
(a) Peak AMI function model size.
(b) Layer-wise Spearman correlation structure.
Figure 24: Evidence that encoder-decoder differences are not driven by dataset artifacts. We compare the only model families that can be matched in training data GPT, Pythia, Cerebras, and T5. While these comparisons cannot eliminate all possible confounds, they show that the architec- tural patterns we report are robust: encoder models consistently achieve higher AMI and lower L, suggesting that the observed differences cannot be explained by dataset variation alone.
35

---

Published as a conference paper at ICLR 2026
FrUit
Weather Phenomenon Mistral-7B-y
Vegetablc gemma-7b
Betwean-catagory Inter-category
10
Between category category
Human Typicality
3
Between-categee category
Inten-category
-1.00
-1.00
-0.50
0.00
0.50
1.00
Cosine milarity
weapon
Clothing bert-large-uncased
Purniture
Meta-Llama Litta-Llaaa--BBB
QwenZ-0.5E
Between colegory htte-celegory
cecegory
Berween-category
Inter category
category
Human Tyoicalty
Human Typicality
Typ
1
-0.50
0.50
3.00
Costre Similantv
Cosine Simlartty
Figure 25: Weak-to-No Correlation Between LLM Embedding Distance and Human Typicality Judgments. Scatter plot examples of the cosine similarity versus the human typicality of items belonging to the category compared to items from other categories.
0.5
Spearman Correlation
0.4
0.3
0.2
0.1
0.0
Classic
Phi
ViT
Bert
Olmo
Mistral
Deepseek
Gemma
Liama
GPT
Owen
Model Family
Figure 26: Weak and Mostly Non-Significant Spearman Correlation Values Between Human Typicality Judgments and LLM Cosine Similarity Indicating Different Structure Representing Concepts. Mean Static Layer Spearman correlation values across the models belonging to the same family and across the three datasets.
36

---

Published as a conference paper at ICLR 2026
Bert
Closaic
Deepseek
Gerrma
Liema
Olmo
Qwen
VIT
Human
0
10
30
50
100 200
400
100 200 300
Number of Clusters (K)
Figure 27: Static Embeddings Achieve a more "Optimal" Compression-Menning Trade-off' by the L Measure. IB-RDT objective (2) vs. across all datasets. Lower C indicates a more optimal balance between compression (I(X; C)) and semantic fidelity (distortion). Static embeddings sistently achieve lower L values than both human categories and contextual embeddings. The plots correspond to the three datasets in the following order: Rosch (1733a). Rosch (1975), McCloskey & Glucksherg (178)).
Bert
Classic
Deepsook
Gemma
Liama
Mistral
Olmo
Phi
VIT
300 Number of Clusters (K)
Compression-Meanin Trade-off by the L Measure. IB-RDT objective (C) vs. K across all datasets. Lower L indi- cates a more optimal balance between compression (I(X; C)) and semantic fidelity (distortion). Contextual embeddings outperform human categories achieve higher values than static em- beddings. The plots correspond to the three datasets in the following order: Rosch (1973a). Rosch 1975 McCloskey Glucksberg 1978).
Figure 28:
Contextual Embeddings Achieve Better-than-Humaal
Ditarton
Distortton
154
280
(a) Complexity-distoree ratio for (b) cumplexity-distortion ratio for (c) Complexity ratio for Rosch (1773a).
Rosch (1775).
McCloskey & Glucksberg (1978..
Ratios Show That Encoder-Models are Less Sensitive to Variations in B. For each dataset, compute the ratio between distortion (loss of numann-llgnna conceptual information) and complexity (representation size) across values the rate-distortion tradeoff parameter B. Encoder models yield consistently flatter profiles, indicating that their ken embeddings preserve conceptual structure even under increasing compression. Decoder models exhibit more pronounced shifts, suggesting that they redistribute representational capacity more ag- gressively as 8 increases, leading to higher sensitivity in this tradeoff.
Figure 29; Complexity-Diston
37


Reviews:
[{"review_id": "cu8W3YhVVp", "reviewer": "Reviewer_E9Bi", "summary": "The paper compares how large language models and humans organize conceptual categories, exploring whether models trade compression for meaning in a similar way to people. It digitizes classic psychology datasets on human categorization and analyzes embeddings from over forty models, both encoders and decoders. Using an information-theoretic framework inspired by rate–distortion theory, it quantifies the balance between information compression and semantic fidelity.", "questions": "- Metrics - Need more information about the Adjusted Mutual Information (AMI), Normalized Mutual Information (NMI), and\nAdjusted Rand Index (ARI) metrics.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- very interesting and important premise \n- Systematic, broad comparison covering 40+ models and also layer-wise comparisons.\n- Formulation/digitization of the datasets from cognitive science is a good contribution\n- Transparent discussion on the limitations", "weaknesses": "- The main question lies in how strongly to rely on the proposed metrics to infer “human-like” learning. The information–compression trade-off captures geometric efficiency in embedding space, but it is not clear whether this translates to human-style conceptual abstraction or reasoning.\n- Dependency on parameters and metrics - The authors themselves acknowledge that \"architectural design and pre-training objectives significantly influence a model's ability to abstract human-like conceptual information.\"\nIs cosine similarity to category names, too narrow to capture the richness of human judgments?\n- robustness to different similarity measures, clustering methods, and parameter settings is not tested.\n- How does current large scale frontier models perform under similar analysis ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F5V49dievw", "reviewer": "Reviewer_zYSK", "summary": "The paper investigates the relationship between compression and meaning in human and machine representations, proposing that large language models (LLMs) achieve broad categorical alignment with human judgments but lack fine-grained semantic distinctions. The authors frame this as a compression–meaning tradeoff: LLMs optimize for efficient representation (compression) at the expense of semantic richness, while humans maintain inefficient but more structured representations that support flexible reasoning.\nThe study evaluates a diverse set of LLMs across multiple benchmarks of human conceptual categories, comparing model cluster structures (via token embeddings and k-means) with human category structures. Analyses include rate–distortion metrics and mutual information between model-derived clusters and human-labeled categories. Key findings include that LLMs achieve near-optimal compression–distortion tradeoffs, whereas human representations appear suboptimal by information-theoretic measures. Additionally, encoder-only models better align with human judgments than decoder-only models, suggesting differences between recognition and generation mechanisms.", "questions": "* What precisely are the human categories, and how are they measured?\n* How is mutual information calculated between human and model representations, and between human categories and other human categories?\n* What prompts were used for contextual embedding extraction?\n* What exactly is meant by “compression–meaning tradeoff” in cognitive terms?\n* What are the exact prompts used to elicit contextual embeddings from the LLMs? And how much is it deviating from the classical behavioral experiments. \n\nReference:\n\n_Medin, D. L., & Schaffer, M. M. (1978). Context theory of classification learning. Psychological Review, 85(3), 207–238_\n\n_Wu, S., Thalmann, M., Dayan, P., Akata, Z., & Schulz, E. (2025).  Building, Reusing, and Generalizing Abstract Representations from Concrete Sequences. In The Thirteenth International Conference on Learning Representations (ICLR 2025)_", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "* The study compares a wide range of LLM architectures and sizes.\n* Compression and concept formation is a deep question at the intersection of cognitive science and machine learning.\n* This study links displines between information theory and the prototype theories of human concept learning.", "weaknesses": "* The paper takes prototype theory as a given framework for human concept representation, but this theory has been very controversial compared to Exemplar theory (Medin & Schaffer, 1978) as an alternative explanation. Modern cognitive models are usually hybrid and integrate prototype and exemplar components. The manuscript should acknowledge this longstanding debate and should not take for granted that the cognitive system for humans are prototype-like. \n\n* The paper lacks an introduction or figures about the background and the set up for the cognitive experiments that they compare LLMs with, and how they relate to compression-meaning tradeoff.  A schematic figure illustrating the overall task (compression–meaning tradeoff, human vs. model representation pipeline) is missing, especially considering the importantance to introduce the cognitive experiment clearly for this crowd of audience. \n\n*  The paper is difficult to read and repetitive, I would suggest to remove some of the colorful RQ01 blocks. \n- Fig. 1 (left): The figure has only a single square, so the rest of them are decoder architectures? \n- Fig. 2 (right): I would not term this as categorical success\n\n* Unsupported claims:\n    * Line 421: The statement that “human conceptual systems, though appearing suboptimal, serve distinct cognitive needs such as flexible generalization and causal reasoning” is unsubstantiated within the current analysis.\n    * Line 72: “Challenges the popular assumption that statistical optimality equals understanding” — what is the “popular assumption” here? Please clarify.\n\n* line 90: \"cognitive studies applying information theory to human concept learning without connecting to modern llms\", that is not true, see Wu et al. 2025. about prior cognitive modeling work applying information theory to human and LLM, in the context of rate–distortion tradeoffs.\n\n* fig 1 left: there is just one square\n* line 323: I would not term this as gradients", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "11pPiwihHy", "reviewer": "Reviewer_7Xee", "summary": "This paper conducts a rigorous analytical re-examination of the boundaries of “understanding” in large language models (LLMs). The authors demonstrate that current LLMs still struggle to master fine-grained semantic distinctions, which is the critical element shaping human-level comprehension. Besides, Information-theoretic optimality does not equate to human-level understanding. Maximizing compression efficiency diverges from achieving semantic alignment fundamentally. In addition, focusing solely on decoder-based language models is unlikely to improve alignment with human comprehension capabilities. These findings reveal a fundamental divergence in the strategies employed by LLMs and humans for understanding natural language: LLMs rely on statistical compression, while humans depend on semantic richness.", "questions": "1. Could the authors comment on how the main results might change if the distortion term in the $\\mathcal{L}$ objective were replaced with a more cognitively grounded metric, such as human similarity judgments or human-rated typicality scores?\n2. It would be useful to include a brief sensitivity analysis or supplementary report across several $\\beta$ values to illustrate whether the LLM–human divergence remains consistent under varying compression–meaning trade-offs.\n3. Could the authors clarify whether the proposed $\\mathcal{L}$ is intended to generalize beyond categorical tasks, or whether it should be viewed as specific to concept-formation settings rather than to other forms of understanding such as relational reasoning, compositional generalization, or contextual disambiguation?\n4. Would the authors consider adding a brief discussion on what kinds of inductive biases or representational mechanisms might help future models better align with human conceptual structure?\n5. Although the compression is measured at a bit level in the current formulation, could the authors discuss what theory or empirical understanding about the compression at token-level in language models that this framework could provide?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "(S1) The paper offers a theoretically motivated and quantitatively explicit framework that fuses Rate–Distortion Theory and the Information Bottleneck to provide a new perspective on the compression–meaning trade-off in LLMs. The use of semantic compactness as an internal proxy for meaning fidelity is original.\n\n(S2) The empirical analysis is well-executed and leverages high-quality, historically grounded human categorization datasets. Their public release greatly facilitates reproducibility and establishes a lasting benchmark for semantic understanding in language models.\n\n(S3) The evaluation spans widely of model architectures (encoder, decoder, and static word embeddings) and scales (from 300M to 70B parameters). The inclusion of both static and contextual embeddings, and the analyses across training checkpoints like OLMo-7B, provides a multi-angle validation of the proposed framework’s generality and interpretive depth.\n\n(S4) The paper is written with exceptional clarity and precision, presenting complex theoretical ideas in an accessible and logically coherent manner, making the idea easy to follow.", "weaknesses": "(W1) Although the analysis is thorough, one might consider including token-level efficiency statistics in future works, though this does not affect the validity of the current findings.\n\n(W2) The study could be further enriched by considering computational efficiency (e.g., token-level cost) as an additional axis in the compression–meaning landscape. Doing so may illuminate how efficiency interacts with semantic representation in practice.\n\n(W3) This study only scoped in English, it would be interesting as future work to examine whether similar compression–meaning trade-offs hold across multilingual models, given that linguistic granularity may modulate semantic richness.\n\n(W4) The study currently focuses on conceptual categorization, exploring how it extends to relational or compositional understanding could broaden its applicability and lasting its significance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HFWc4nEkm4", "reviewer": "Reviewer_X8mp", "summary": "The paper studies how LLMs and humans trade-off compression and meaning in conceptual representations. Building on Rate–Distortion Theory and the Information Bottleneck (IB), the authors propose an objective function to combines information-theoretic compression with geometric coherence. Using digitized, classic human categorization benchmarks and embeddings from many models (encoders and decoders), they report three core findings: (i) LLMs broadly align with human category boundaries, (ii) LLMs capture weak typicality gradients relative to humans, and (iii) LLM-derived clusters achieve lower L (greater compression efficiency) than human categories, suggesting humans preserve more semantic richness at the expense of statistical efficiency. The paper releases the digitized benchmarks and positions the framework as a diagnostic for monitoring compression–meaning balance.", "questions": "1. How exactly is the Distortion term computed for human categories, from typicality distances, membership uncertainty, or an inferred geometry? Please add a short methodological box that maps human ratings to the variance term and discuss limitations.\n2. If typicality is computed against category names, can you replicate with prototype centroids (learned per category) or descriptive definitions, and do encoder-decoder gaps shrink?\n3. With a common tokenizer/corpus and matched parameter counts, do masked-LM encoders still outperform autoregressive decoders on AMI and typicality?\n4. During the mid-layer migration phase, what happens to the complexity-distortion frontier layer-wise? Does the “efficiency” shift track changes in heads/FFNs (e.g., sparsity, attention concentration)?\n5. Can you test the framework on non-noun categories (events, relations) or multilingual replicas?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Originality. A clear, unified information-theoretic lens that ties together clustering complexity (compression) and semantic coherence (meaning), then uses it to compare humans and LLMs at scale. The encoder-decoder contrast is especially thought-provoking.\n2. Quality. Careful benchmark curation (classic cognitive datasets), comprehensive model coverage, and multi-angle evaluation (category alignment, typicality, and L-curves); The training-dynamics analysis with 57 OLMo-7B checkpoints is helpful.\n3. Clarity. The paper’s structure is easy to follow; figures that separate boundary alignment vs. typicality vs. L-frontiers help the reader parse what “efficiency” means in practice.\n4. Significance. The finding that humans are less efficient but more semantically rich challenges a common “optimality” narrative and invites new objectives/architectures for more human-aligned understanding.", "weaknesses": "1. Many correlations are modest. Please provide bootstrap confidence intervals, multiple-comparison controls, and seed variability for clustering and correlation estimates.\n2. The conclusion that humans are statistically suboptimal depends on the chosen geometry and L weighting. Most results fix $\\beta=1$; the paper should report sensitivity curves over $\\beta$ and discuss how human/LLM rankings vary under plausible trade-off weights. Also clarify how human distortion is computed from typicality/membership data to avoid mismatched metrics.\n3. Encoder-decoder gap interpretation. The architecture result is intriguing, but confounded by training objective differences (MLM vs. autoregressive), tokenizers, and pretraining corpora. A controlled study with matched data/tokenizers and frozen-head probes would strengthen the claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)