Evolution and compression in LLMs: on the emergence of human-aligned categorization
Authors: Nathaniel Imel
Noga Zaslavsky
Venue: ICLR 2026 Poster (2026)
Keywords: LLMs
information theory
semantics
Primary area: interpretability and explainable AI

Abstract: Converging evidence suggests that human systems of semantic categories achieve near-optimal compression via the Information Bottleneck (IB) complexity-accuracy tradeoff. Large language models (LLMs) are not trained for this objective, which raises the question: are LLMs capable of evolving efficient human-aligned semantic systems? To address this question, we focus on color categorization --- a key testbed of cognitive theories of categorization with uniquely rich human data --- and replicate with LLMs two influential human studies. First, we conduct an English color-naming study, showing that LLMs vary widely in their complexity and English-alignment, with larger instruction-tuned models achieving better alignment and IB-efficiency. Second, to test whether these LLMs simply mimic patterns in their training data or actually exhibit a human-like inductive bias toward IB-efficiency, we simulate cultural evolution of pseudo color-naming systems in LLMs via a method we refer to as Iterated in-Context Language Learning (IICLL). We find that akin to humans, LLMs iteratively restructure initially random systems towards greater IB-efficiency. However, only a model with strongest in-context capabilities (Gemini 2.0) is able to recapitulate the wide range of near-optimal IB-tradeoffs observed in humans, while other state-of-the-art models converge to low-complexity solutions. These findings demonstrate how human-aligned semantic categories can emerge in LLMs via the same fundamental principle that underlies semantic efficiency in humans.

Published as a conference paper at ICLR 2026
EVOLUTION AND COMPRESSION IN LLMs: ON THE EMERGENCE OF HUMAN-ALIGNER CATEGORIZATION
Nathaniel Imel New York University iime @nyu. edu
Noga Zaslavsky New York University noga z@nyu edd
ABSTRACT
Converging evidence suggests that human systems of semantic categories achieve near-optimal compression via the Information Bottleneck (IB) complexity- accuracy tradeoff. Large language models (LLMs) are not trained for this objective, which raises the question: are LLMs capable of evolving efficient human-aligned semantic systems? To address this question, we focus color categorization-ake key testbed of cognitive theories categorization with uniquely rich human data-an replicate with LLMs two influential human studies. First, we conduct an English color-naming study, showing that LLMs vary widely in their complexity and English-alignment, with larger instruction-tuned mod- els achieving better alignment and IB-efficiency. Second, to test whether these LLMs simply mimic patterns in their training data or actually exhibit human- like inductive bias toward IB IB-efficiency, we simulate cultural evolution of pseudo color-naming systems in LLMs Language Learning (IILLL). We find that akin to humans, LLMs iteratively re- structure initially random systems towards greater IB-efficiency. However, only a model with strongest in-context capabilities (Gemini 2.0) is able to recapitulate the wide range of near-optimal IB-tradeoffs observed in humans, while other state-of- the-art models converge to low-complexity solutions. These findings demonstrate how human-aligned semantic categories can emerge in LLMs via the same funda- mental principle that underlies semantic efficiency in humans.
method we refer to as Iterated in- in-Context
INTRODUCTION
As large language models (LLMs) become increasingly popular in everyday use, it is crucial to understand how their learning biases and representational capacities align with our own. Here, we investigate this by focusing
key aspect of human intelligence: the ability to organize informa-
tion into semantic categories (Croft 2002; Rosch 2002 Koch 2008; Boster 2005 Majid 2015)
Malt & Majid 2013). This phenomenon presents two major challenges for AI. First. systems of semantic categories (semantic systems, for short) exhibit both universal patterns and cross-language differences (Berlin & Kay 1969; Berlin 1992. Croft 2002), which LLMs must navigate. Second, LLMs are not grounded the rich physical and social environment that humans are (Rosch) 1975) Laboy 1973). and it is unclear how these differences affect their ability to learn human-all human-aligned mantic categories. Therefore, in order to understand whether LLMs can efficiently communicate with people and adapt changing environments and communicative needs in human-like man- ner, it is crucial to study whether LLMs are capable of structuring meaning according to the same principles that guide humans.
To address this open gap in our understanding of LLMs, we propose a novel theoretical and cognitively-motivaeed framework for studying semantic systems in LLMS. We build on the frame- work of [Zaslavsky et al.) (2018), which argues that languages efficiently compress meanings into words by optimizing the Information Bottleneck (IB) principle (Tishby et al. 1999), instantiated as tradeoff between the informational complexity and communicative accuracy of the lexicon. This framework has broad empirical support across human languages (Zaslavsky et al. 2018 2019 2021
Mollica al. 202 Zaslavsky et al. 2022). Furthermore, [mel et al. (2025) recently showed that drive for IB- [B-eficiency may be present in the individual inductive biases of human learners. While
I.LMs are trained on vast amounts of human language data encoded as text, they not trained with

---

Published as a conference paper at ICLR 2026
respect to the IB objective. This raises the question: are LLMs capable of developing IB-efficient human-aligned semantic systems?
We address this open question with an in-depth analysis in the domain of color-a key test case for categorization theories in cognitive science with rarely available human data as well as practical implication for human-LLM interactions (see Section 2.1 and replicate with LLMs two influ- ential human behavioral experiments (Figure [ First, we conduct an English color naming ex- periment (analogous to the human experiment of Lindsey & Brown 2014), designed assess the efficiency and human-alignment of the color naming systems of LLMs. Second, we conduct an iterated learning experiment of pseudo color-naming systems (analogous to the human expe of [f et al.] 2213), designed to reveal implicit inductive learning biases of LLMs by simulating a process of cultural transmission (see Section 2.3 For the latter, we extend /Zhu & Griffiths (2024)'s 's iterated in-context learning (I-ICI. paradigm to iterated in-context language learning (IICLL).
First, we show that many prominent LLMs struggle to capture the English color naming system, exhibiting wide range of complexities that are often lower than the complexity of English. However, with increased size and instruction- tuning, LLMs can achieve high English-alignment and IB efficiency. Second, using our IICLL paradigm, we show that LLMs that perform well in the naming task are not merely mimicking patterns in their training data but actually guided by a human-like inductive bias toward IB- efficiency. Specifically, we show that LLMS iteratively restructure initially random artificial systems towards greater IB-efficiency and increased human-alignment. However, among the models tested, only the model with strongest in-context capabilities (Gemini 2.0) is able to recapitulate the wide range of near-optimal IB-tradeoff's observed in humans, while other models converge low- complexity solutions. Finally, we show that Gemini can also develop structured category systems via IICLL in domain that is qualitatively different from color, suggesting that our result could potentially apply also in other domains.
Our key findings and contributions are summarized follows:
Taken together, our findings demonstrate how numan-aligned semantic categories can emerge in LLMs via the same fundamental principle that underlies semantic efficiency in humans. Importantly, neither humans nor LLMs are explicitly trained for optimizing the IB objective, suggesting that IB- efficiency, and optimal compression more generally, may emerge to support intelligent behavior.
2 BACKGROUND AND MOTIVATION
2.1 TALKING ABOUT COLOR
For decades, cognitive scientists have used color as an essential tool to study perception and cat- egorization. This in part due to the unprecedented amount of human behavioral data available for color, which includes research perceptual space, cross-linguistic semantic variation, cate- gory learning, and cultural evolution. Especially relevant to our study is the World Color Survey (WCS) dataset (Cook al., 2005), which contains color-naming data from 1 110 non- con-industrialized languages. Another relevant study by[Xu et al. (2013) demonstrated that the cultural transmission of initially random, artificial color-term systems humans leads to more regular systems that resem- ble those documented in the WCS dataset. To our knowledge, these two data resources- on actual cross-linguistic naming patterns and on the cultural evolution of category systems- are unique to the domain of color, making it an ideal domain for evaluating how well LLMS align with human behavior. Furthermore, studying color naming in LLMs also has practical implications. Generative AI models, used for tasks like image generation and online product searches, require grounded rep- resentations of color language. In order for such models to interact in ways we expect them to, is crucial to determine the extent to which state-of-the-an art LLMs have learned the meaning of color terms that actually align with human naming patterns.
While previous work has shown that human-aligned color representations can be recovered from LLMs Abdou et al. 2021 Patel Pavlick, 2022), there is limited research on model behavior in the context of real-world settings (namely. prompt-based interactions). Marjieh et al. (2024) showed that several recent instruction tuned models (specifically, GPT-3, its "ChatGPT" variants, GPT-3.5 and GPT-4 (Brown et al.. 2020; OpenAl et al. (024) and Mistral 7B Instruct (Jiang al. ) can recover the English and Russian color naming systems by prompting the models to label hex codes. In contrast, we test a large set of 39 models with varying sizes and training stages,
2

---

Published as a conference paper at ICLR 2026
(a)
(c) Iterated in-context language learning
Peaturea: [0.73, 0.13, 0.20]
=> Label: Tove
Reaturea: [0.0, 0,32, 0.291
L
Tabel; Peglu
(b) Color naming task
Peatures: [0.27, 0,29, 0.12)
*> Tabel: PegTo
sample data dg
What color
ICL
naming
1s this?
4
Maroon
q(w m)
d
What color
img
L2
this?
Ma
bxt
d2
Nhat color is this? [0.28, 0.23, 0.27]
L
Figure 1: (a) The standard WCS color naming grid (Kay et al. 2009). (b) Color naming task with humans and LLMs. Multi-modal LLMs can observe colors either via text or images. of the IICLL paradigm. At each generation an LLM is prompted with small dataset for ICL, d,-1, consisting of pairs of colors and pseudo labels sampled from the previous generation's 's language, L.-1. With these data in context, the LLM performs the naming task for the full space (a).
(c) Illustration
consider both textual inputs and image inputs (for multi-modal models), we analyze the LLMs' color naming systems through the lens of the IB framework, and we further assess their underlying inductive learning biases a cultural evolution process via iterated learning. Next, we provide relevant background on the IB and iterated learning frameworks below.
2.2 THE INFORMATION BOTTLENECK PRINCIPLE AND SEMANTIC SYSTEMS
The IB framework for semantic systems Zaslavsky et al. 2018). which we employ in this work, based the following communication model: speaker wishes communicate mental rep- resentation, or belief state, m E M, defined as probability distribution over world states u E U, by mapping to a word w W via a stochastic encoder ( (n).. A listener then receives and interprets the speaker's intended meaning by constructing an estimator m.w- In the case of color, for example, the world states l/ are given by a set of target colors (FigureThh. To account for perceptual noise, it is assumed that each m is Gaussian distribution over the perceptual CIELAB color space centered around corresponding target color (Zaslavsky et al.) 2018). That is, each color mentally represented by the speaker as Gaussian distribution and the listener's goal is to reconstruct the speaker's 's belief state over colors.
According this framework, in order to communicate efficiently, the speaker and listener must jointly optimize the IB tradeoff between minimizing the complexity of their lexicon and maximiz- ing its accuracy. Assuming the listener's inferences by an encoder qw)mm) Complexity in IB roughly corresponds to the number of bits required for communication, formally defined by I (M; W), the mutual information between speaker's mean- ings and words. Accuracy defined by I (W;U), the information that the speaker's words main- tain about the target world state, which also inversely related the KL-divergence between the speaker's mental state and the listener's inferred state, E.g [D] M [[]] HHarremoes & Tishby 2007 Zaslavsky 2020). An optimal lexicon, or semantic system, is one that minimizes the IB objective function
adapted to the speaker, the lexicon is defined
Folu) L, W) (W;
(1)
where / 2 O controls the complexity accurrac tradeoff. The solutions to this optimization problem define the IB theoretical limit of efficiency.
This framework generates precise quantitative predictions that have been gaining converging empiri- cal support across hundreds of languages and multiple semantic domains, ranging from perceptually-
3

---

Published as a conference paper at ICLR 2026
grounded domains such color (Zaslavsky al., 2018 2022) to higher-level conceptual domains such as household objects (Zaslavsky et al. [019]] Taliaferro et al. 2025) and personal pronouns (Za-)
slavsky et al., (02I). In addition, it has been successtully applied to studying emergent communi-
cation in artificial agents (Chaabouni et al. 2022 Tucker et al. 2022 Gualdoni et al.
2024
Tucker
et al. 2025).
The IB color naming model. As part of our evaluation of LLM color naming, we use the previ- ously published IB color naming model from Zaslavsky et al. 2018). The black curve in Figure shows the IB bound for color naming from this study, together with reproduction of their results showing that color naming systems across languages (from the WCS as well as English from Lind- sey & Brown 2014)) achieve near-optimal IB tradeoffs.
2.3 ITERATED LANGUAGE LEARNING
The second main theoretical framework we apply in this work is based on iterated learning (IL), IL a paradigm in cognitive science for simulating cultural transmission and eliciting prior inductive biases (Griffiths& & Kalish 2007 Kirby et al. 2008 Griffiths et al. 2008). In a typical II experiment, participants form chains of "generations." " At each generation 4, a participant is exposed to data d,-1 from the previous generation and then produces responses that become input for the next genera- tion. In iterated language learning (ILL), participants learn examples of pairs of stimuli and artificial labels during training period, after which they assign labels new, unlabeled stimuli in the same meaning space. In doing so, participants produce a full category system L,, which is sampled from to provide training examples to the next generation, and the process repeats. This process, which requires generalization from limited data, reveals learners' inductive biases for certain linguistic or category structures (Griffiths & Kalish 2007 Griffiths al. 2008). As shown in Griffiths & Kalish (2007), under certain conditions, namely that the IL agents are Bayesian who share priors and likeli- hood functions, this Markov chain converges to a stationary distribution over languages equal the learners' prior distribution p( L.). This makes the strong prediction that languages emerging from II. reflect the population's 's underlying inductive biases. Although this is an asymptotic characterization of IL dynamics, in behavioral experiments with people, researchers observe rapid convergence to highly non-uniform distributions of languages (Kirby et al. 2015).
Related work on ILL. Previous research in machine learning has investigated dynamics related to ILL.. In agent- simulations, |enn al. (2020) introduced the neural iterated learning (NIL) framework, and showed how it can lead to compositional language in neural network agents, and Carlsson et al. (2024) showed that introducing communication-basee training in NIL leads to IB- efficient color naming systems. These studies, however, did not explore LLMs as we do here. ZZu & Griffiths (2024) adapted IL LLMs with strong in-context learning capacities as a prompt-based workflow known as Iterated In-Context Learning (I-ICL) to elicit LLMS' implicit prior distribu- tions over aspects of world knowledge. I-ICL has recently been used to analyze cultural evolution in LLMs (Ren et al. 2024) and to compare visual and linguistic abstractions between LLMs and humans (Kumar et al. 2024). Here, we introduce iterated in-context language learning (IILLL, Figure (T) which goes beyond prior work in leveraging the strong in-context learning abilities of LLMs to replicate as closely as possible the experimental conditions of ILL studies with humans, enabling direct comparison to LLMs of their respective inductive biases, particularly with regards to semantic efficiency and alignment.
ILL of color naming systems. The empirical comparison for our IICLL color naming study is the IL data from Xu et al. (2013). In their experiment, participants were asked to learn and transmit novel systems of color terms across thirteen generations. We focus on their main results that include twenty iterated learning chains, each initialized with a random partition of the WCS grid. These chains vary in the number of allowed color terms, ranging from two to six, and four replications of each condition. Participants were shown a set of randomly selected colors generated uniquely for each chain, and paired with corresponding pseudo words. After training, participants were asked to label all 330 colors of the WCS grid. Xu et al. 2013)) found that over time, the IL chains become increasingly regular and resemble the color naming systems documented in the WCS dataset. More recently, Imel et al. (2025) found that these chains also converge to highly efficient systems along the IB bound. Figure 3 shows our reproduction of this finding (plotting only final generations).
4

---

Published as a conference paper at ICLR 2026
3 EXPERIMENTAL SETUP
Our goal is to test whether LLMs have an inductive bias toward IB-efficiency, as observed in hu- mans. To this end, we conduct two studies with LLMs: (1) an English color naming study assess their semantic alignment and communicative efficiency with respect to English speakers; and (2) a cultural transmission experiment of artificial color naming systems (using IICLL) elicit their inductive learning biases beyond patterns they may have seen during training.
Models. We consider 39 models across model families: Gemini (Google 2025), Gemma
(Gemma-Teee Team et al. 2025). Llama (Grattafiori al., 2024). Qwen 2.5 (Qwen al., 2025). Olmo 2 (OLMo-Teamm Team et al. )025) and GPT-2 (Radford et al. 2019). Within each family, we vary models along several dimensions. Specifically, to gain insight into the properties that may influence the models' behavior in our tasks, we consider models with different sizes, instruction-tunel versus base models, and text-based versus multi-modal models. For Olmo, also considered its learning dynamics by analyzing training checkpoints. For a full list of models, see Table Tin iin Appendix
Prompts. both of our studies, we provided instructions in the prompts to choose only from a fixed set of terms. The Gemini API supports controlled generation which makes this constrained classification task straightforward; for all open-weight models, we used log probability based scor- ing of the allowed terms continuation of the prompt. Further details and example prompts can be found in Appendix
Stimuli. We used the 330 color chips from the WCS grid shown in Figure These chips repre- sent a systematic sampling of color space and are standard stimuli in color naming research. Each chip is associated with precise coordinates in the CIELAB color space, which can be converted to sRGB coordinates. To present the color stimuli to the text-based LLMs, we encoded color using these numerical coordinates. For multimodal models, we generated a square colored image corre- sponding to the WCS chip's sRGB values, and passed this image together with the text instructions.
Evaluation. Following Zaslavsky et al. 2018), we use two main evaluation measures in our stud- ies. First, the efficiency loss of a semantic system is measured by its minimum deviation from optimality, defined as E = minst (Fo(a) - F3)} where F3 is the optimal value of F3 in Eq. Second, we measure the semantic (miisaaligmmnt between two systems by the Normalized In- formation Distance (NID) (Kraskov et al.) 2005) Vinh et al.) 2010). NID metric capturing the distance between two clusterings (in this case, induced by color naming categories), providing quantitative measure of structural similarity between the naming systems. Since NID is bounded in [0, 1]. we take NID measure of similarity, alignment. IB-alignment measures the simi- larity between a system and the nearest (e-fitted) optimal IB system. WCS-alignment measures the average alignment between a system and the WCS languages, and English-alignment measures the alignment between a system and English.
4 RESULTS
4.1 ENGLISH COLOR NAMING
We begin with the results from our English color naming study, in which we elicited color naming responses from LLMs with English color terms and then evaluate their IB IB-efficiency and alignment with the actual English color naming system from Lindsey & Brown (2014). As illustrated in Fig- ure we consider two variants of this task, a text-only variant where colors are presented as sRGB coordinates, and an image- variant where colors are presented as a color patch. More details on procedure can be found in Appendix B
The resulting systems are shown in Figure 2p, their IB tradeoff's are shown in Figure 2h. and their quantitative evaluation is shown in Figure 2 (see also Figures 7 and 9 in Appendix D We find that LLMS vary widely in their complexity and models achieving better alignment and IB IB-efficiency. No model aligns perfectly with the English system from Lindsey & Brown (2014), but Gemini-2.0 and Gemma 3 27B (inst.) approximate closely. The fact that so many state-of-the-art pretrained LLMs struggle to recapitulate the English
with larger instruction- -uned
5

---

Published as a conference paper at ICLR 2026
(a)
English
70
Gemini 2.0
Accuracy (bits)
14
Gemini
Gemma 27B inst.
Gemma
Liama
Qwen
Wooe
Llama 3.3 70B inst.
O mo English
Qwen 2.5 32B inst.
1
2
5
Complexity (bits)
(c)
Gemini
Gemma
Llama
Qwen
Olmo
GPT-2
0.6
Instruction tuned modal
Alignment
64
0.0
Complexity
comonn 5 $220
Remo- 3 3-30
owen 2.5 5.
Dwwn-2 5-2b
qawn-2 5-70
owen- 2-50d
owenn 5.30h
olmo 2mm0-2488
slmro 2000-2. 32t
000-2
Remo a
Figure 2: English color naming experiment with LLMs. (a) IB complexity-accuran tradeofffs achieved by instruction-tunes LLMss (see Appendix E for base models too). plotted in comparison to the English tradeoff (blue star) and the IB bound (black curve) from Zaslavsky al. (2018). Models vary widely in their tradeoffs, with larger models approaching the English point. (b) Color naming systems of English (Lindsey & Brown 2014) and pest-performing LLMs. Each system is shown by its mode map, i.e., it is plotted against the WCS grid (Figure Ta), where each chip is colored by the color-centroid of its modal category. (c) aaliimmee (top) and IB complexity (bottom) of all LLMs. Markers are the same as in (a), where black edge indicates the instruction tuned model and no edge indicates the base model. Across model families, size and instruction-tumi are associated with higher complexity and better alignment to English.
color naming system, even though these models are trained on massive amounts of English data and have billions of parameters, quite striking. While instruction tuning generally associated with better performance, it does not guarantee IB-efficiency or alignment to English, even for very large models (for example, Ilama 3.3 70B inst.). At the same time, we were surprised to find that some models-particnn
Olmo 2 32B (inst.) and Qwen 2.5 VL 7B (inst.) produced systems with category structure resembling not English, but instead other, very low-resource languages from the WCS (see Figure in Appendix E This suggests that although many LLMs struggle to recover the same particular distinctions as English speakers, they may still possess human color categories.
To better understand how LLMs acquire color categories during training, analyzed the learn ing trajectory of Olmo 32B (Appendix (F. aliiimseelgmmeettt only slightly increases during pre- training, and the most substantial improvement occurs in the second, instruction-tuniig stage. This

---

Published as a conference paper at ICLR 2026
IICLL trajectories of Gemini
IICLL trajectories of Gemma
Accuracy (bits)
Accuracy (bits)
English
English WCS data
WCS data
Human data
Human date
Gemma
IICLL trajectories of Llama
|ICLL trajectories of Qwen
Accuracy (bits)
Accuracy (bits)
3-
2-
Engiish
English
WCS data
WCS data
Human data
Human data
Liama
Owen
Complexity (bits)
Complexity (bits)
Figure 3: IICLL with LLMs converges to near-optimal IB solutions. The trajectories of Gemini 2.0 (upper left), Gemma 27B (upper right), Llama 3.3 70B (lower left) and Qwen 2.5 32B (lower right) are plotted on the information plane (same as FigureZJA), together with the IB tradeoffs across human languages EnCS+English) and human IL data. Small black dots correspond to random initializations of chains with varying number of categories, k {2, 3, 4, 6, 14). Thin blue lines correspond to the LLMs' IICLL trajectories. Gemini captures the complexity range observed across human languages, while the other models converge to lower complexity systems. All models are instruction-tuned.
finding is consistent with the results of Figure suggesting that instruction-tuning is important factor in the models' ability to exhibit English-like color naming systems.
Finally, explore the impact of the input representation, we ran additional analyses. First, we conducted a minimal pair analysis of the multimodal LLMs (Figure Appendix comparing systems that were generated with textual vs. image-based representations of color. Interestingly, presenting colors as images rather than textual sRGB coordinates does not improve the alignment or efficiency of larger models, and can even harm their performance, but it does seem to boost the performance of smaller models. Second, we tested the impact of using CIELAB coordinates instead of sRGB, which better captures human perceptual similarities between nearby colors. Consistent with previous findings by Marjieh et al. (202), we find that all models, including the best performing ones, struggled to align with English naming when colors are presented in CIELAB. This reveals key difference between how LLMs represent color and how humans do.
Taken together, our findings show that while many LLMs struggle to align with humans on a very simple, but pnvcettuallyggrounde
color-naming task, frontier models do exhibit human-aligned color naming systems. This, in turn, opens crucial question: is this behavior merely a reflection of imitating patterns in the models' training data, does it signify more intrinsic LLM inductive bias towards IB-efficiency in categorization?
4.2 ITERATED IN-CONTEXT LANGUAGE LEARNING (IICLL)
To investigate whether LLMs possess an efficiency bias that extends beyond learning the specific categories of language which they were trained, we turn to our second study, which simulates cultural transmission in LLMs. To this end, we sought to replicate the ILL color naming experi- ment of Xu al. (2013) closely as possible, using IICLL paradign shown in Figure Tp (see Appendix G for more details). considered only large, instruction tuned models that performed well in the English color naming task for our IICLL experiment: Gemini 2.0, Gemma 3 27B, Qwen
7

---

Published as a conference paper at ICLR 2026
(a)
(b)
Efficioncy loss
06
WCS-AAl Alignment
Allignmen
05
04
6.75
3-
2.50
02-
0.75
01-
10
12
$
10
12
00g
10 12
Generation
Generation
Generation
Gemini
Gemma
Owon
English
Figure 4: Across IICLL generations, emergent LLM systems become more efficient (a), more aligned with the optimal IB systems (b), and more aligned with human languages (c). Colored curves show the average across initializations and conditions, and the colored regions corresponds to the 95% confidence intervals.
2.5 32B, and Llama 3.3 70B henceforth, Gemini, Gemma, Qwen and Llama (see Appendix L for an analysis showing that smaller models struggle in IICLL to produce non-degenerate category systems).
Figure B shows the resulting trajectories of IICLL chains on the information plane. Gemini develops color naming systems that converge to similar range of near-optimal IB solutions the typological patterns of the WCS languages, as well as the final generation systems from the human IL chains from Xu al. (2013). There is also broad qualitative fit between its final generation IICLL systems and languages from the WCS (see Appendix D. The other LLMs also develop systems that converge highly IB-efficient solutions, although they appear be limited the lower range of complexity observed in the WCS.
Additional quantitative support for these observations is provided in Figure 4 These figures show that over generations, LLM systems become more efficient in their mapping of stimuli to terms (Fig- ure 4b) more similar to naturally occurring human color systems (Figure 4b), and more similar to optimal IB systems (FigureHp)) Furthermore, L.LM IICLL chains converge near the bound relatively quickly (after roughly four generations), parallel to human II. dynamics.
Strikingly, many trajectories from all models initially climb in complexity towards the IB bound before slowly evolving downwards alongside it. This suggests that the capacity to learn and transmit complex yet near-optimally efficient category systems, while strongest for Gemini, is present in all four LLMs. One factor that may drive the difference between Gemini and the other models is that IICLL task requires very strong in-context learning, models must integrate dozens of in-context training examples to generalize well. For example, the = 14 condition includes 84 examples, and in this setting most of the L.MSs immediately converge low-complexity solutions. However, given that the terms in our IICLL experiment are made up (pseudo) terms and that we give no indication the model that the stimuli are in fact colors, only that they have "features" (Figure [e), all four models show an impressive ability to evolve IB-efficient systems.
To assess whether the emergent LLM systems are non-trivially efficient and aligned WCS lan- guages, we conducted a rotation analysis (Regier et al., 2007) on the final, evolved color systems from our LLM experiments alongside the human data from Xu et al. (2013) (Figure II Appendix H.. This analysis involved rotating the color-label mapping along the hue dimension of the WCS color grid (i.e., along the columns of Figure [T) and evaluating the difference in their efficiency and alignment scores. We find that such rotations away from the actual emergent systems lead to a significant decrease efficiency and alignment for Gemini, while the results are less conclusive for the other models. Furthermore, we find that Gemini's is efficiency and alignment over generations are higher than the human IL trajectories, and are also higher than a baseline learner based on an al- ternative ceature-based clustering algorithm (Appendix M).. This suggests that Gemini truly exhibits an emergent inductive learning bias toward IB -eeefiiciency. even though as fas as we know it was not trained fine-tuned with respect to the IB objective function.
8

---

Published as a conference paper at ICLR 2026
(a)
(b)
Figure 5: (a) The Shepard circles stimulus grid, (b) Gemini IICLL chains for naming Shepard circles. Rows correspond to individual chains, initialized randomly. Each system is plotted over the stimulus grid, where colors correspond unique labels.
4.3 THE EMERGENCE CATEGORIES IN SHEPARD CIRCLES VIA IICLL
While many results in color have previously been shown to generalize to other domains (Zaslavsky ct al. 2019 202I, Mollica et al. 2021), it is challenging to conduct in other domains a full-seale analysis of the scope we performed for color naming, primarily due to the lack of high-quality data from both native speakers and ILL experiments. Here, nevertheless, we apply the IICLL paradigm in qualitatively distinct semantic domain provide initial evidence that results may indeed generalize beyond color.
To this end, considered synthetic domain of so-called "Shepard circles," classic conceptual space used to study how humans categorize multidimensional stimuli (Shepard) 1964). These are circles which vary in both radius and angle of rotation of an internal spoke. Following Carr al. (222)). we generated stimuli by taking eight evenly spaced values for each of the two dimensions, yielding 64 total stimuli (Figure (l)). For this preliminary investigation, we limit our analysis to Gemini and & = allowed labels. We found that presenting these stimuli as pairs of numbers (radius and angle) proved to be challenging. This perhaps unsurprising, because unlike color- which is frequently represented in various ways online in both text and images-ther is likely no text online that would allow the model to associate these numbers with meaningful perceptual features. To overcome this limitation and better mimic human perception, images of the stimuli. Four samples of IICLL chains are are shown in Figure 5b. Over generations, Gemini transmitted categories that became increasingly compact in their partitioning of the space, and distinguished regions based on both the radius and angle of the circles. This suggests that LLMs especially frontier multimodal models pooenniallyy have domain domain-general bias organize features into non-arbitrary, and increasingly regular, semantic categories. An important direction for future work is to test whether this emergent structure also supports greater IB-efficiency as seen in humans (Imel et al. 2025).
presented Gemini with
5 DISCUSSION
In this work, we combined a theory-driven approach, based on the IB principle, with cognitively- motivated experimental methods, based color naming and iterated language learning to study whether LLMs can acquire human-like inductive bias toward optimally-comppessed semantic rep- resentations, without being trained for this objective. To do this, first conducted in-depth anal- ysis of English color naming across 39 LLMss, and found that a surprising number of state-of-the-ar models fail to capture the English color naming system. However, some the most recent, larger, instruction-tundd models achieve high English-alignment and comparable IB tradeoffs. We then demonstrated that LLMs that do align well with the English color naming system are not merely mimicking patterns in their training data, but rather exhibit a more fundamental capacity to learn human-aligned, efficient color category systems. To this, we introduced Iterated in-Context Lan- guage Learning (IICLL) to simulate cultural transmission of category systems. Over generations of IICLL, LLMS tend to restructure
artificial category systems toward greater IB- efficiency and alignment to human naming systems. We also provide initial evidence that LLMs can develop structured categories over generations of IICLL a domain distinct from color, suggesting that results may apply in other semantic domains well. Taken together, findings suggest

---

Published as a conference paper at ICLR 2026
that LLMs capable of evolving perceptually grounded, human-like semantic systems, guided by the same IB IB-efficiency principle that underlies human languages. Importantly, neither humans nor LLMs explicitly trained for optimizing the IB objective, suggesting that IB IB-efficiency may emerge to support intelligent behavior.
Our empirical results open up several important questions for future research. First, while our work demonstrates that cultural transmission alone (via IICLL) can be sufficient pressure for some LLMs develop efficient, human-like category systems, more complete understanding of language evo- lution requires integrating functional pressure of language use, c.g., via communication. Therefore, an important future direction to extend our IICLL framework to incorporate communication as selective pressure, for example, by adopting models that explicitly integrate both transmission and communication (e.g.. Kouwenhoven et al. 2024). Second, the precise origins of the bias we observe in LLMs toward efficiency are unclear (for example, how might this bias emerge from properties of the training data, instruction-tuning, or model size), and investigating this is another important direction for future work. Lastly, it is important to extend our analyses across more languages and semantic domains.
REPRODUCIBILITY STATEMENT.
This work utilizes a previously published model and code Zaslavsky et al. (2018), made avail- able at https: //g //g/tttuuu thub. com wttp:///aaaaaooooooooooooaa code from Carr et al. (2020), made available from httpt::pp//// / com/ iwcarr/ shepard under CC BY 4.0 License. The ILL data from Xu et al. (2013) were obtained from the au- thors, and the WCS data are publicly available from the World Color Survey website, https: 7/1ingui itingusstics berkeley........ edd.
under the MIT License, and
Appendix [ provides list of the specific models used with their Hugging Face or Google API IDs, and Appendix J includes example prompts. getb.ggthub........... io/
Code for reproducing our experiments can be found at battps://infooee evolut ion wewporressionn.. 1mms| To support full reproducibility, the LLM data that was gen- erated for this paper is available upon request.
ACKNOWLEDGMENTS
This work was supported by Google Cloud Platform Credit Award and BSF Research Grant (2024318) to NZ. An earlier version of this work received the best paper award at the NeurIPS 2025 Workshop on Interpreting Cognition Deep Learning Models (CogInterp).
REFERENCES
Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, and Anders Søgaard. Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color. In Arianna Bisazza and Omri Abend (eds.), Proceedings of the 25th Conference on Computational Natural Language Learning, pp. 109-132, Online, 2021. Association for Compu-
tational Linguistics. doi: 10. 10.1886/11/22211 conlllcoolll... 1.9.
B. Berlin and P. Kay. Basic Color Terms: Their Universality and Evolution. University of California Press, 1969. ISBN 978-0-520-00442-8..
Brent Berlin. Ethnobiological Classification: Principles of Categorization of Plants and Animals in Traditional Societies, volume 185, Princeton University Press, 1992.
James S Boster. Categories and cognitive anthropology. In Handbook of Categorization in Cognitive Science, pp. 7061 Elsevier, 2005.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners, 2020.
10

---

Published as a conference paper at ICLR 2026
Emil Carlsson, Devdatt Dubhashi, and Tenyy Regier. Cultural evolution via iterated learning and communication explains efficient color naming systems. Journal of Language Evolution, 91-2)): 49 49-66, 2024. ISSN 2058-458X. doi: 10.1093/jole/1zae010.
Jon W. Carr, Kenny Smith, Jennifer Culbertson, and Simon Kirby. Simplicity and informativeness in semantic category systems, Cognition. 202:104289, 2020. ISSN 0010-0277. doi: 10.1016/j. cognition.2020. 104289..
Rahma Chaabouni, Florian Strub, Florent Altché, Hugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot. Emergent com- munication at scale. The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Richard S. Cook, Paul Kay, and Terry Regier. The World Color Survey Database: History and use. In Henri Cohen and Claire Lefebvre (eds.), Handdook of Categorization in Cognitive Science, pp. 223-241. Elsevier Science Ltd, Oxford, 2005, ISBN 978-00-08-044612-77 doi: 10.1016/ B978- 008044412-7750064--
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecom- munications and Signal Processing). Wiley-Interscience, USA, 2006. ISBN 978-0-471-24195-9,
William Croft.
Typology and Universals. Cambridge Textbooks in Linguistics. Cambridge University Press, Cambridge, 2 edition, 2002. ISBN 978-0-521-8088422 doi: 10.1017/
CBO978051 1809978551884079.. 1840579.
Gemma-Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Cas- bon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiao- hai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan- Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander kov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang. Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlic Chen, Charline Le Lan, Christopher A. Choquette-Choo, C. J. Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng. Dimitris Paparas, Divyashree Shivaku- mar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eu- gene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimezak-Plucinske Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wi- eting, Jonathan Lai, Jordi Orbay. Joseph Fernandez, Josh Newlan, Ju-ycong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Ni- lay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Ruben- stein, Phil Culliton. Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanezyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin. Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Pöder. Sijal Bhatnagar, Sindhu Raghuram Panyam. Sivan Eiger. Susan Zhang. Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evei, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo. Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Bar- ral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel. Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean- Baptistt Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier
11

---

Published as a conference paper at ICLR 2026
Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma Technical Report, 2025.
Google. Gemini 2.0: Flash, flash-lite and pro, 2025.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Ko- renev, Arthur Hinsvark. Arum Rao, Aston Zhang, Aurelien Rodriguez. Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes. Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee. Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lec, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang. Jiecao Yu, Joanna Bitton, Joe Spisak, Jong- soo Park, Joseph Rocca Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marein Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa. Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang. Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaelle Gold- schlag, Yashesh Gaur, Yasmine Babaci, Yi Wen, Yiwen Song. Yuchen Zhang. Yue Li, Yuning Mao, Zacharic Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An- nie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang. Beth Loyd, Beto De Paola. Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Bric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni,
12

---

Published as a conference paper at ICLR 2026
Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayaman, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habceb, Harri- son Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Flena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang. Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong. Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar. Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji. Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang. Rachad Alao, Rachel Ro- driguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Baitey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng. Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad lonescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Jia, Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary De Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The Llama Herd of Models, 2024.
Thomas L. Griffiths and Michael L. Kalish. Language Evolution by Iterated Learning With Bayesian Agents. Cognitive Science, (1(3):4414800 2007. ISSN 1551-6709, doi: 10.1080/
15326900701326576
Thomas L. Griffiths, Brian R. Christian, and Michael L. Kalish. Using Category Structures to Test Iterated Learning as Method for Identifying Inductive Biases. Cognitive Science, 32(1))68-107, 2008. ISSN 1551-6709. doi: 10. 1080/0364021000118000
Eleonora Gualdoni, Mycal Tucker, Roger Levy, and Noga Zaslavsky. Bridging semantics and prag- matics in information-theorrtic emergent communication. A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 11059-21077 Curran Associates, Inc., 2024.
Peter Harremoes and Naftali Tishby. The Information Bottleneck Revisited or How to Choose a Good Distortion Measure. In 2007 IEEE International Symposium on Information Theory, pp. 566-570 2007. doi: 10. 0.110991STT2200745557755
Nathaniel Imel, Jennifer Culbertson, Simon Kirby, and Noga Zaslavsky. Iterated language learning is shaped by a drive for optimizing lossy compression. In Proceedings of the 47th Annual Meeting of the Cognitive Science Society, 2025.
13

---

Published as a conference paper at ICLR 2026
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie- Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William EI Sayed. Mistral 7B, 2023.
Paul Kay, Brent Berlin, Luisa Maffi, William R. Merrifield, and Richard Cook. The World Color Survey. Center for the Study of Language and Information, Stanford, 2009.
Simon Kirby, Hannah Cornish, and Kenny Smith. Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language. Proceedings of the National Academy of Sciences, 105(31): (00:::068111111 1-16686, 2008. doi: 1..1073/pnas.07078335100
Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith. Compression and commu- nication in the cultural evolution of linguistic structure. Cognition, 141:87-102, 2015. ISSN
0010-0277. doi: coo.oio.ooonionn22111111..
Peter Koch. 85. Lexical typology from cognitive and linguistic point of view. In 2. Halbband Language Typology and Language Universals 2.Teibband, pp. 1142-1178. De Gruyter Mouton, 2008. ISBN 978-3-11-019426-5.
Tom Kouwenhoven, Max Peeperkom, and Tessa Verhoef. Searching for Structure: Investigating Emergent Communication with Large Language Models, 2024.
A. Kraskov, H. Stögbauer, R. G. Andrzejak, and P. Grassberger. Hierarchical clustering using mutual information. Europhysics Letters, 70(2):278, 2005. doi: 10. 10.0////////000-- 100-/00000--1-3-------
Sreejan Kumar, Raja Marjieh, Byron Zhang, Declan Campbell, Michael Y Hu, Umang Bhatt, Bren- den Lake, and Tom Griffiths. Comparing abstraction in humans and machines using multimodal serial reproduction. In Proceedings of the Annual Meeting of the Cognitive Science Society, vol- ume 46, 2024.
William Labov. The boundaries of words and their meaning. In Charles-James N. Bailey and Roger W. Shuy (eds.), New Ways of Analyzing Variation in English. Georgetown University Press, Washington, D.C., 1973.
Delwin T. Lindsey and Angela M. Brown. The color lexicon of American English. Journal of Vision, 14(2):17, 2014. ISSN 1534-7362. doi: 10.1167/14.2.17.
Asifa Majid. Comparing Lexicons Cross-linguisticall. In John R Taylor (ed.), The Oxford Hand- book of the Word, pp. O. Oxford University Press, 2015. ISBN 978-0-19.964160-4. doi: 10. 10011093oxfforddb//999000 196779999991111 1604.013.020.
Barbara C Malt and Asifa Majid. How thought is mapped into words. Wiley Intendisciplinary Reviews: Cognitive Science, ((6):583-977, 2013.
Raja Marjieh, Ilia Sucholutsky, Pol van Rijn, Nori Jacoby, and Thomas L. Griffiths. Large language models predict human sensory judgments across six modalities. Scientific Reports, 14(1):2144) 1445, 2024. ISSN 2045-2322. doi:
Francis Mollica, Geoff Bacon, Noga Zaslavsky, Yang Xu, Terry Regier, and Charles Kemp. The forms and meanings of grammatical markers support efficient communication. Proceedings of the National Academy of Sciences, 11844))cc02299911 18, 2021. doi: 10. 22.1073/paas.222599331
OLMo-Team, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bha- gia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Py- atkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannanch Hajishirzi. OL 2 Furious, 2025.
14

---

Published as a conference paper at ICLR 2026
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji. Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadert-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brock- man, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gib- son, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha GontijoLoops, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hal- lacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Ka- mali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim. Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis. Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy. Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz. Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning. Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Ra- jeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang. Cullen O' OeKeffe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder. Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Sel- sam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song. Natalie Staudacher. Felipe Petroski Such, Natalic Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Pre- ston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vi- jayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang. Jonathan Ward, Jason Wei, C. J. Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff. Dave Willner, Clemens Winter, Samuel Wolrich. Hannah Wong, Lauren Work- man, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojeiech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. GPT-4 Technical Report, 2024.
Roma Patel and Ellie Pavlick. Mapping language models to grounded conceptual spaces. In Inter- national Conference on Learning Representations, 2022.
Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 Technical Report, 2025.
15

---

Published as a conference paper at ICLR 2026
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners. Technical report. OpenAl, 2019.
Terry Regier, Paul Kay, and Naveen Khetarpal. Color naming reflects optimal partitions of color space. Proceedings of the National Academy of Sciences, (04(4):1436-144) 1441, 2007. doi: 10.1073/
Yi Ren, Shijie Guo, Matthieu Labeau, Shay B. Cohen, and Simon Kirby. Compositional languages emerge in a neural iterated learning model. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, and Danica J. Sutherland. Bias amplification in language model evolution: An iterated learning perspective. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomczak, and C. Zhang (eds.), Advances Neural Information Curran Associates, Inc., 2024.
Processing Systems, volume 37, pp.
Eleanor Rosch. Cognitive representations of semantic categories. Journal of Experimental Psychol- ogy: General, 104(3):192-25) 233, 1975. ISSN 1939-2222. doi: 10. 10.1037/0096-3/44 100.--3344.111...199
Eleanor Rosch. Principles of categorization. In Daniel J. Levitin (ed.), Foundations of Cognitive Psychology: Core Readings, pp. 251-270. MIT Press, 2002.
Roger N. Shepard. Attention and the metric structure of the stimulus space. Journal of Mathematical Psychology, 1(1):54-87, 1964. ISSN 0022-2496. doi: 10. 02.10160002-244664490000777
Maya Taliaferro, Nathaniel Imel, Esti Blanco-Eloriittt, and Noga Zaslavsky. Bilinguals exhibit se- mantic convergence while maintaining near-optimal efficiency. In Proceedings of the 47th Annual Meeting of the Cognitive Science Society, 2025.
Naftali Tishby, Fernando C Percira, and William Bialck. The information bottlneck method. Pro- ceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, pp. 368 468-377, 1999.
Mycal Tucker, Roger Levy, Julie Shah, and Noga Zaslavsky. Trading off utility, informativeness, and complexity in emergent communication. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances Neural Information Processing Systems, 2022.
Mycal Tucker, Julie Shah, Roger Levy, and Noga Zaslavsky. Towards Human-I Like Emergent Com- munication via Utility, Informativeness, and Complexity. Open Mind, 9:418 451,, 2025. ISSN 2470-2986. doi: 10. 10.1162/opmi.aaa 100188.
Nguyen Xuan Vinh, Julien Epps, and James Bailey. Information Theoretic Measures for Clusterings Variants, Properties, Normalization and Correction for Chance. Journal of Machine Learning Research, 11(95):2877-2544 2010. ISSN 1533-7928.
Comparison:
Jing Xu, Mike Dowman, and Thomas L. Griffiths. Cultural transmission results in convergence towards colour term universals. Proceedings of the Royal Society B: Biological Sciences, 280 (1758):20123073, 2013. doi: 10.1098/rspb.2012.330
Noga Zaslavsky. Information-Then Theooretic Principles in the Evolution of Semantic Systems. PhD thesis, The Hebrew University of Jerusalem, 2020.
Noga Zaslavsky, Charles Kemp, Terry Regier, and Naftali Tishby. Efficient compression color naming and its evolution. Proceedings of the National Academy of Sciences, 1 115(31);7937-7942, 2018. doi: 10.1073/pmas.180003 110052115..
Noga Zaslavsky, Terry Regier, Naftali Tishby, and Charles Kemp. Semantic categories of artifacts and animals reflect efficient coding. In 41st Annual Meeting of the Cognitive Science Society, 2019.
Noga Zaslavsky, Charles Kemp, Naftali Tishby, and Terry Regier. Communicative need in colour naming. Cognitive Neuropsychology, 37(5-6):312-33) 3244, 2020. ISSN 0264-3294. doi: 10.1080/ 02643294.2011
16

---

Published as a conference paper at ICLR 2026
Noga Zaslavsky, Mora Maldonado, and Jennifer Culbertson, Let's talk (efficiently) about us: Person systems achieve near-Optimal compression. In Proceedings of the 43rd Annual Meeting of the Cognitive Science Society, 2021.
Noga Zaslavsky, Karee Garvin, Charles Kemp, Naftali Tishby, and Tenry Regier. The evolution of color naming reflects pressure for efficiency: Evidence from the recent past. Journal of Language Evolution, pp. Izac001, 2022. ISSN 2058-458X. doi: 10. 10.1003/jole//zac001.
Jian-Qiao Zhu and Thomas L.. Griffiths. Eliciting the Priors of Large Language Models using Iterated In-Context Learning, 2024.
A IB COMMUNICATION MODEL AND THEORETICAL BOUND
IB communication model The IB framework is based on a basic communication model that con- siders inventory of words W to communicate about space of meanings, MM. The specific model we apply in this paper is the previously published IB color naming model of Zaslavsky et al. (2018). In this model, meanings are assumed to be mental representations formally defined as probability distributions m[ over world states u E U. In our setting, the set of world states U/ is taken to be the 330 color chips from the WCS grid, and meanings over colors are grounded in the CIELAB perceptual space such that each target color referent, Ue E U, represented as a Gaussian distribution my(u) centered around ue. Mcanings are drawn from an information source, p(m), which characterizes how often each meaning needs to be communicated. The need distribution over meanings in this model, p(m), was estimated using the method of least- informative priors (see Zaslavsky et al. (2020) for an extensive evaluation of this need distribution). Note that, in our IICLL experiments, we sample stimulus-we word pairs uniformly for training data for each generation, instead of biased sampling from this prior. This was done in order to replicate the procedure in Xu et (2013).
beliefs over world states U,
Given meaning m p(m), a speaker produces signal using stochastic production policy, also called an encoder, q(w(m)) and then a listener interprets the signal by reconstructing an estimated belief state (u) Σm ((m|w)m(()) Note that this form of listener interpretations corresponds to Bayesian listener. While this form is assumed here for simplicity, it not an actual assumption but rather derivation from the theory (see the SI of Zaslavsky et al. (2018)).
IB theoretical bound A semantic category system in this framework corresponds to a stochastic encoder. (w[[m). which maps meanings to signals. An optimal semantic system, according to the IB principle, is an encoder that satisfies a tradeoff between its informational complexity and communicative accuracy. Complexity, also known as information rate (Cover & Thomas 2006), is defined by the mutual information between speaker meanings and signals,
g(w|m) q(w)
(M; Σ p(m)g(ulm))l log
which captures the number of bits, on average, that is required to encode meanings with signals. Maintaining low complexity corresponds to using fewer bits for communication. i.c., achieving high compression rate, which in turn, can be translated to affording smaller lexicon size. For example, minimal complexity can be achieved by compressing all possible meanings into a single signal. This, however, will result in very poor accuracy. Accuracy in IB terms, is quantified by
UW: I(M; Eg D [
(3)
which corresponds to maintaining a language that is informative about the speaker's intentions. Maximizing accuracy, as expressed in 3 amounts minimizing the expected distortion between speaker and listener meanings, i.e., the Kullback-Leible (KL) divergence [[M[|]] We say that language's semantic system is efficient to the extent that its encoder q minimizes the IB tradeoff:
(4) where B is a free parameter controlling the tradcoff between pressure to minimize complexity and pressure to maximize accuracy. The solutions to this optimization problem define the IB theoretical limit of efficiency, which means that no system can lie above this theoretical bound (see Fig. 3
Falg] = WM)
(W;WU) s.t. 11
17

---

Published as a conference paper at ICLR 2026
B ENGLISH COLOR NAMING PROCEDURE
The color naming task was conducted by prompting model to label a color chip, for all 330 color chips from the WCS stimulus array, one stimulus time, in randomized order. Unlike the prompts in our IICLL study, we do not provide the model with access to previous interactions. Each prompt was of the form: "What color this [ <color coordinates> Please provide only single label from the
You may only use
one of the following allowed labels: ['Red', 'Blue', list just provided. Do not give any explanation." " When prompting models with only, the color coordinates were sRGB triples. The allowed set of labels were fourteen basic English color terms corresponding to the modal terms from Lindsey & Brown (014).
C ENGLISH COLOR NAMING WITH CIELAB INPUTS
To address the generalizability of our findings beyond sRGB inputs, we replicated our English color naming task using CIELAB triples as the input features for the prompt text. CIELAB is percep- tually uniform space and so provides a principled color space to contrast with sRGB. We restricted to testing the four models that performed best from original study (which used sRGB-encoded stimuli).
Figure 6 iillustrates mode maps of the resulting systems for models when prompted with CIELAB- encoded colors. We found that performance generally degraded across all models when providing features CIELAB triples: they struggled significantly to align with established English color boundaries, and the resulting category systems were considerably noisier compared to the sRGB- based outputs. This outcome is consistent with previous research by Marjich al. (2024) which noted that other frontier models failed to produce coherent English color category systems when prompted with CIELAB triples. Based on this observation, we restricted our IICLL study to the sRGB feature space to assess the models' inductive bias under ideal conditions.
Gemini 2.0 Flash
Gemma 3 27B (inst.)
Qwen 2.5 32B (inst.)
Llama 3.3 70B (inst.)
Figure 6: Mode maps for the best-performing models from our English color naming study, when prompted with CIELAB triples.
18

---

Published as a conference paper at ICLR 2026
D MODELS
Model
Size (B params)
Instruction Tuned
Multimodal
Hugging Face Google API
gemini- 1.5
n/a
Yes
Yes
gemini-1.5-faa
gemini-1.55 1.5-8b
8
Yes
Yes
gemini- 1--mni--.5-flas--81
gemini-2.0
n/a
Yes
Yes
gemini-2.0-flast
gemma-3-JD 3-11b
No
No
google/gemmma-3-1b
gemma-3--lbit
Yes
No
google/gemma---1hor
gemma-3-4b
Yes
google/gemma-33405
gemma-3-4b-it
4
Yes
Yes googlc/gemma-3344h.
gemma-3-12b
12
No
Yes google/gemmma331226
gemma-3-122-it
12
Yes
Yes google/gemma--3122be
gemmaa3-27b
Yes
google/gemma-3342
gemmaa--27b-it
27
Yes
Yes
google/gemma--3227bb
gpt-2
0.224
No
No
openai connmii-ommmity/gp222
gpt-2-medium
0.355
No
No
openai sennni--oommmnttygttt-mmdddee
gpt-2-large
0.774
No
openai-oommunittyyg2--aaa.
Ilama-3.118h
No mett-llama/L_aama3331--8
Luma-3.1-8h-inns.
8
Yes
No
1lama-3.2-1b
1
No
No
meta-Ilama/I Jama-332211
Dama-3.2-Ib-insttuu
Yes
No Beta-llama/Lllmm-333.
3.2--1BIInstrucc
1ama-3.2-35
No -laaammm/L/mm-32-3333
No meta-Ilama/Llamme
3
Yes
llama-3.3-70biimstuu
70
Yes
No metallama//Llmaa-333777
olmo-2-76
7
No
No allcnai/OL.Mo-2278
olmo-2-7b-
Aimo-2-bbinnstruct
7
Yes
No alicmai/OLMMo---B--nnnucu
olmo-2-133b
No allenai/OLLMo-2-133B
1omo-2--.bbinnttrue
13
Yes
No allenai/OLMMo-2-1BBBBmma
olmo-2-32b
32
No
No
allenai/OLMoo-2322B
olmo olmo-2-322 4-32b-instruct
32
Yes
No
allonai/OLMMo---2BB-nnsss
Instruutc
qwen-2.5 2.5--.55115
2
No
No
Qwen/Qween:255-15B
qwen-2.5-14
2.5-1.5b-inntrucc
2
Yes
No
Qvenn(Qwen-255-155---1m
qwen-2.5-3b
3
No
No
Qwcn/Qwen-2255-3B
qwen-2.5-3b-instruct
3
Yes
No
Dvenn/Quem2225-3BBBnssree
gwen-2.5-7b
7
No
No
Qwen/Qwen-2.55778
qwen-2.5-7b-insttu.
Yes
No
Qwenn/Qwen=225-78-Innntt
qwen-2.5-vl-7biin..
7
Yes
Yes
Qwen/Qwen-2.5-VL-
wwen-2.5-v1.32b-insstuc
32
Yes
Yes
qwcn-2.5-14b
14
No
No Qwen/Qwen-2.5-144
(wen-2.5-14b-instrru
Yes
No Dvem/Quuen2255-144nnntt
qwen-2.5-325
No
No Qwen/Qwen-2.5-3222
(wen--2.5-2b-iinstruct
32
Yes
No Quenm/Qweem2255-22D-nnntiu
Table 1: Full list of models used in naming study. Models used in IICLL task bolded.
19

---

Published as a conference paper at ICLR 2026
E
NAMING SYSTEMS FOR ALL MODELS
4
Accuracy (bits)
Gemini
2
Gemma
Llama
Qwen
1
Olmo
Gpt-2
English
0
3 Complexity (bits)
7
Figure 7: IB complexity-accuracy tradeoffs achieved by text-based LLMs. Same as Figure 2 but including base (not instruction-tuned) models. Points with black edge indicate models that instruction-tuned
3
Gemini Gemma Qwen
2 10
Mode size (8 parameters)
Accuracy (bits)
101
2
Modality txt img
x 10"
4 10º
0
1
2
3
4
5
6
7
Complexity (bits)
Figure 8: IB complexity-accurant tradeoffss achieved by instruction-tumee multimodal LLMs. Points without color filling indicate systems resulting from providing colors as images, rather than via sRGB coordinates in text. Each minimal pair is connected by line. Interestingly, there appears be a cutoff at around 3 bits of complexity for which models that already perform well with text-based prompting do not improve alignment to English when prompted with image instead.
As can be seen in Figure many of the models performed poorly on the English naming task, with many base (not Instruction-tunedt models and some smaller instruction-tunel models failing to produce naming systems with any coherent category structure. Figure shows that these systems are distributed widely across the information plane, with only larger, instruction- very recent models achieving similar IB-efficiency tradeoffs to English speakers. Interestingly, some models show similar category structure not to English, but to languages in the WCS. For example, Olmo- IIsttuutt and LL--- Mayoruna (see the final row Figure 13), as well as some of the systems to which Gemini-2.0 converges generations of IICLI. This suggests that although some larger, instruction tuned
(when presented with images) bear resemblance to
20

---

Published as a conference paper at ICLR 2026
Gemini
perrir-200
gemeik-5
gemer--....
Gemma
gemma-3-1l-t
4b-tt
gamma 3-3
3--7--1
gemma-3-12
gemma 1.40
gemrin 3 3-12b
pomma 3-27b
Llama
lana 9 instruct
Lama 3.2 36 instrctt
Ilama 3.1. d6 instruct
Iama 3.9 700 instrcct
lamra.32.:66
Qwen
qwen22 3-3.5-5..66000 3..bblrdtruct
queen2 5-3b----b-ttc
Qwe 5-2b-instrot
queen2 5 7b-instrut
qwee-2 1---------steee Instrcct
owen-255 32b-isstutt
owen255:1.3
Qwenn-2 5-36
owan-225517
2wen 2-144
Swen 2.5 32b
Olmo
alma- dedrnircct
0rroo inliirrt Inttrct
cimc-2-75
clmo 13b
2-bo2-32b
GPT-2
spt-2
gpt-3-medum
gzt-2-arge
Figure 9: Mode maps for all models. The suffix "iimg for multimodal models indicates that images were presented as the color stimulus, rather than sRGB coordinates.
21

---

Published as a conference paper at ICLR 2026
models and multimodal instruction-tune -uned models fail recover the English naming system, they may still possess some general bias towards color category structure that is aligned with humans.
22

---

Published as a conference paper at ICLR 2026
F OLMO TRAINING TRAJECTORY
As model checkpoints are available for the base Olmo 2 32B (OLMo- Team et al. 2025 model throughout its training, we repeated our English color naming study throughout different stages of its model training. The training procedure involved two main phases; pre-training on up to 6 trillion tokens, and second training stage using curated dataset of 843 billion tokens, including high- quality, academic, and instruction-tune tuning data. This also involved training on 100 billion and 300 billion token samples and then averaging the final checkpoints
process called model souping.
Furthermore, we found that hinting to the model that the coordinates were in fact sRGB values when the prompt was changed "What color could improve model nates>> ! To aid in our analysis of the base pretrained Olmo is this rgb=[ <color models, we used this form prompting in order to see what structure is achievable principle.
Figure 10 shows that there is a slight increase in scores towards the end of stage 1 training, and large jump in performance occurs after just 1000 steps of stage 2 training. This is particularly interesting instruction-tuning data, given that found instruction-
given that stage included pretraining tuning was strongly associated with higher alignment and complexity across other models.
23

---

Published as a conference paper at ICLR 2026
A
Stage
2.0
Stage
model
15
5008
Complexity
1.0
0.5
0.0
200000
400000
600000
800000
0.175
0.150
English- English-Alignmen Aahh-liigmmet
0.125
0.100
0.075
0.050
0.025
0.000
200000
400000 Training step
600000
800000
B
stage 1
ADCE
stage 2
Figure 10: (a) Trajectory of English-alignmen and complexity of Olmo 2 32B systems the course of training. (b) Mode maps of Olmo systems over the course of training.
24

---

Published as a conference paper at ICLR 2026
G ITERATED IN-CONTEXT LANGUAGE LEARNING PROCEDURE
Procedure We aimed to replicate the procedure Xu et al. (2013). The initial training data for the first generation of IICLL consisted random pairings of colors and nonsense terms, with sample size six times the allowed vocabulary size. For vocabulary sizes ranging from 2 to 6 terms, the initial training data was sourced directly from Xu al. (2013) facilitate direct comparison. A 14-term condition was also included for comparison with English (Lindsey & Brown 2014), with its initial mappings randomly generated
was not part of the original study.
Next, we performed a sanity check to ensure that the LLM could recover the information from the training data. This involved presenting each stimulus from the training set to the LLM one at time, in random order and without any history of previous interactions, and verifying that the model could recover the correct labels as provided in the prompt. With the exception the most resource- intensive k 14 condition, all models achieved a stable comprehension accuracy of ~ 80% or greater throughout generations. The & = 14 condition served as an extreme-case sanity check, resulting in highly unstable performance (based a failure to retrieve the required 84 examples; see the final paragraph of this section). Only Gemini occasionally passed this threshold; we elected report the k = 14 condition results illustrate the point of divergence in model capability.
Following this, we began the 'production phase", in which we prompted the model to provide a label for an unseen color stimulus. This was repeated for every item in the WCS stimulus array, presented in a randomized order, including those stimuli that were part of the initial training set for that generation. During this production phase, we also monitored the LLM's consistency with the labels provided in the initial training set. While perfect adherence was not crucial since deviations may reflect the models' inductive biases, we observed that consistency with the training set remained much higher than chance in the first generations and quickly rose to ceiling.
In addition, to promote response coherence and crudely mimic short-term memory influences found in human experiments, each subsequent prompt was augmented with sliding window of the 10 most recent user-model interactions (see Appendix K for details on window size selection and its impact).
Once labels were obtained for all stimuli the production phase, this full set of stimulus-label pairs constituted the current generation's 's complete color category system. This marked the completion of one generation. To initiate the next generation, new training set was created by randomly sampling stimuluswwort pairs from the just-completed generation's system, with the sample size being six times the number of allowed words in the vocabulary. This entire process was repeated for up to 13 generations, (which was the length of chains in Xu al. (2013) or until two successive generations yielded degenerate systems (where all stimuli are mapped to a single term).
25

---

Published as a conference paper at ICLR 2026
H
ROTATION ANALYSIS
(a)
0.05
IB-Allgnmmet
WCS--Allgnme Al--lllggmettt
0.00
Efficiency
0.10
0.00
0.05
-0.05
-0.10
-0.10
-0.05
-0.15
-0.20
-19
-19
10
Rotation
Rotation
Rotation
Gemini
Gemma
Qwen
Llama
Human
Figure 11: Rotation analysis for final systems of all models' IICLL chains and human ILL chains from Xu et al. (013). The r-axis denotes rotation along columns (hue dimension) of the WCS grid, while the y-axis is the difference (A) in mean (a) efficiency loss, (b) alignment IB optima, or (c) alignment to languages in the WCS. Each colored curve is the average across initializations and conditions, and the colored region corresponds the 95% confidence interval around the mean (assuming normal distribution). In (a), A > means that the actual systems are more efficient that their rotated counterparts, whereas (b) and (c) A means that the actual systems are more aligned.
Gemini
English
F 10
==
7=
r=
r
Figure 12: Rotation examples. Hypothetical variants for a final generation system of IICLL with Gemini 2.0 in the l 14 condition, and the English system from Lindsey & Brown (014). Variants are obtained by rotating the color naming system in the hue dimension across the columns of the WCS stimulus palette.
corresponds to the actual system, = corresponds to a shift five columns to the right, and = -5 corresponds to shift of five columns to the left.
26

---

Published as a conference paper at ICLR 2026
I
EXAMPLE CHAINS COMPARED TO WORLD COLOR SURVEY LANGUAGES
3=3
K=4
K=5
K=6
K=14
Wobe
Mayoruna
Jicaque
Paya
Chiquitano
Figure 13: Mode maps for Gemini-2.0 systems generations of IICLL. Bottom row shows the language from the WCS dataset that is most similar to the final generation's 's system.
27

---

Published as a conference paper at ICLR 2026
J PROMPTING
In both of our studies, we provided instructions in the prompts to choose only from a fixed set of terms. The Gemini API supports controlled generation which makes this constrained classification task straightforward. For all open-weight models, we used log probability based scoring of the al- lowed terms as a continuation of the prompt to implement constrained choice. We used this decoding approach because after initial exploration of a brute-force method of repeating the prompt up to ten times until the model generated an output from the allowed set of terms, we found that only Llama 1-st-uuutttcct could generate usable output (and in this case produced comparable hehavior). For our probability- -based decoding, we used the default generation configurations loaded from the Hug- ging Face Transformers library, which include a decoding temperature of 0,6 and top.p cop.p sampling threshold of 0.9.
J.1 EXAMPLE PROMPT FOR ENGLISH COLOR NAMING TASK
"What color this 10.73579176. 0.13100809, 0.20245084] You may only one of the fellowing allowed labels: ["Red", "Blue 'Yellow "Green" " "Orange". "Purple" " "Pink", "Brown" "Black', "White". 'Gray', "Peach". 'Lavender" "Maroon ']. Please provide only a single label from the list just provided. Do not give any explanation."
J.2 EXAMPLE PROMPT DURING ITERATED LEARNING
Training During training, the instructions were:
"role"; "user",
"content":
"type"
"text".
"text" ": " Features: 10.73579176, 0.13100809, 0,20245084] -> Label: Tovo
Features: [0.0, 0.32875953, 0.29290289] -> Label: Feglu
Fentures: [0.27329472, 0,29777161, 0.12539189] >> Label: Fegln
Features: [0. 18075972. 0,20165954. 0.14769053) >> Label: Narp
Features: [0.77448248, 0.32302429, 0.52727771
Label: Zarn
Features: [0. 40.45905556. 0.30562009, 0.57919747] >> Label: Mib
Features: [0.71954112. 0.66114241.
>> Label: Mib
Features: [0. 10.11340154, 0,21466098. 0.10314594] -> Label: Tova
Features:
15.99 0.732415, 0.63294812] >> L.bbe:: Tovo
Features: [0. (0.777022815 0,83377671, 0.0] -> Label: Blim
Features: [0. 9400423.. 0.73594053, 0.83545817] >> LLbbll: Blim
Features:
77780255, 0,4893714, 0.71447577] Label: Narp
Features: [0. 89354841. 0.92711068. 0.55762157] >> Label: Zarn
Features: J0. 18901356.
0.139809561
>> Label: Blim
Features: (0.26099322. 0.1368575,
0.35608507] -> Label: Zarn
Features: [0.0, 0.67091016, 0.50450556] <> Label:
Features: 10.86141792, 0.28265837, 0.0] >> Label: Feglu
Features: [0.0 0,42196565, 0. 0.49203637]
<> Label: Mih
Features: 80047349. 0,92499868, 0.91404717]
Label: Mib
Features: [0. 17.27606817 0.14437327.
0.29090483]
>> Label: Zara
Features: [0.91415567. 0.59970537
0.67293472] -> Label: Narp
Features: [0.8590007, 0.65998705,
0.0] -> Label: Feglu
Features: [0.45183228. 0,37479448, 0.08237481] >> Label: Zarn
Features: 10.91680269. 0.63513813.
0.0] -> Label: Feglu
Features: [0. 8994021,
0.89968419,
0.89916582] <> Label: Blim
Features: 10.58159027. 0.030559. 0.08701426] >> Label: Blim
Features: [U. 10.84117237. 0.91203955.
0.918751831 -> Label: Feglu
Features:
12205525, 0.66759353.
0.39283492] -> Label: Tava
Fentures: [0.8 10.81106541, 0.67994708.
0.0] -> Label: Blim
Features: 10.81051796. 0.77667486.
0.88512298] >> Label: Feglu
Features: 10.47551109. 0.20309021.
0.05617448]
Label: Mib
Features: [0. 70626646, 0,35443549, 0.64213025]
>> Label: Mib
Features: [0.36974002. 0.20933179. 0.50495342] >> Label: Zarn
Features: 10.0. 0.44248437, 0.46499573) -> Label: Tave
Features: (0.29926292, 0.15509473. 0.11356989]
-> Label: Feglu
Features:
[0.0, 0,33817158, 0.21461567]
>> Label: Mib"
"rale": "assistant",
"onteent":
28

---

Published as a conference paper at ICLR 2026
29

---

Published as a conference paper at ICLR 2026
Generalization During generalization sessions, the prompt included (i) the entire training (ii) up to 10 previous user- aser-assistant interaction pairs to serve as a form memory history of the conversation, and (iii) the current stimulus being queried from the language model. For the history, previous interactions appeared in the form:
"role": "user". "content";
"type": "text". "text": "Based
preceding examples . Do not give any explanation, and limit your response to exactly one word from
that describes
this list labels: [ ('Narp', "Feglu", "Mib', 'Blim "Zarn ']. Feeaurre::
[0.59395637, 0.24607302, 0.5432978] -> Label:
"role": "; "assistant", "content"; "Mib"
"role": : "user". content";
"type": "text",
"text "; "Based on the preceding examples , what is the label that best deseribes this?
Do give any explanation, and limit your response exactly word from this list of labels: [ I'Narp". "Tovo" "Feglu", "Mib'. "Blim" "Zar 'I.A nFeatures: 10.73535226, 0.11620602. 0.27974255 -> Label:
"role": "assistant". "conteat": "Blim"
For the current stimulus being queried for a label, the presentation was the form:
"ooee: "user", "cntent""
"type": "text",
"text" "Based the preceding examples. what is the label that best deseribes this?
Do not give any explanation, and limit your response to exactly one word from list labels: I'Narp'. "Tovo" 'Feglu 'Mib'. "Blim" 'Zarn I nFeatures:
[0.73535226, 0.11620602, 0.27974255 -> Label:
"role": "assistant". "content":
30

---

Published as a conference paper at ICLR 2026
K SLIDING WINDOW CONVERSATION HISTORY
During the production phase of our IICLL experiments, each response from the model was saved and added sliding window of N previous user-model interactions, excluding the initial training data. After preliminary explorations with window sizes of 0, 10, 20, and 50, we determined that window size of 0 led degenerate category systems more frequently (see Figures [4 and [T]. while the results from a window size of 20 and 50 did not sigmificantly differ from those obtained with a window size of 10. Consequently, we the window size 10 for all IICLL experiments. This window was included in the prompt for subsequent stimuli, presented after the initial training data and before each new stimulus, aiming to promote coherence in the model's responses and to crudely mimic the influence of short-term memory that human participants would possess in Xu al. (213))ss experiments.
Figure 14: Randomly initialized IICL chains L.ama-3.3-700--mnss with history window of
31

---

Published as a conference paper at ICLR 2026
Figure 15: Randomly initialized IICL chains with Gemini-2.0 with a history window of 0.
32

---

Published as a conference paper at ICLR 2026
L IICLL WITH SMALLER MODELS
We restricted our main Iterated In-Context Language Learning (IICLL) analysis to models that per- formed well on the initial English naming task focus specifically on the evolution towards non- trivial IB-efficiency. This prioritization was based on our expectation that models lacking sufficient inference power, coherence in color naming, or in-context learning capacity would be unable to sustain non-trivial category structure under IL pressure.
In order confirm this empirically for models with weak initial performance, we ran supplementary IICLL simulations using four models that performed modestly at the English naming task: Gemma 4b, Llama 3.2 3B, Qwen 2.5 3B, and OL. 7B (all instruction-tunedd These simulations covered the five vocabulary-size conditions (k = 2 through 6) used in the experiments of et al., (2013).
The trajectories of these chains are depicted in Figure T6 Across these chains, only the Gemma model converged to two-word system (in the k 6 condition), while the remaining 19 chains all converged to degenerate, single-word systems. This confirms that smaller models with limited in- ference power in-context learning capacity tend to collapse to degenerate systems when subjected to the noisy transmission pressure of iterated learning.
IICL trajectories
IICLL trajectories of
Olmo (7b)
Gemma (4b)
Accuracy (bits)
Accuracy (bits)
3-
2-
English WCs data
English WCS data
almo-2-7o-instruu
gomma-3-40--4
IICLL trajectories
IICLL trajectories
Llama (3b)
Qwen (3b)
Accuracy (bits)
Accuracy (bits)
2-
Eng ish
English
WC5 data
WC5 data
Ilama 3.2 instrcct
owen 2.5 36 instruct
0
Complexity (bits)
Complexity (bits)
Figure 16: A sample of IICLL trajectories with smaller models from the English naming task. Compare to Figure Blin in the main text.
33

---

Published as a conference paper at ICLR 2026
M NEAREST NEIGHBOR BASELINE FOR IICLL
To test whether the semantic category systems that emerge from IICLL are merely simple approx- of baseline simulations using Nearest Neighbor (NN) classifier in place of an LLM in the IICLL task. In this setup, the NN classifier predicted the category label for each test stimulus based solely on the sRGB features of the provided in-context training examples.
imations of feature-based clustering, we conducted
We analyzed performance in the most challenging condition (k 14 allowed terms). The re- sults, presented in Figure 17, show that while the leading open-weight L.MSs achieve significantly worse alignment compared to the NN baseline, the leading frontier model, Gemini 2.0, performs significantly better than this baseline all evaluation metrics (efficiency, IB-alignment, and WCS alignment). other words, Gemini's s evolved categories are significantly more human-like and more IB-like than simple nearest-neighbor clustering of the input examples.
This result suggests that the ability to acquire human-like, optimalyy-compressed semantic rep- resentation is (1) not trivial because it is capability only emergent in the most advanced frontier model, and (2) it cannot explained by a simpler nearest-neighhor process that only maintains contiguous partitions regardless of their efficiency and IB-allinmmnt.
(a)
(b)
Efficiency loss
nementt
WCS. Aliinment
0.5
b.
01
2.
12
10
12
0.g
10
Generation
Generation
Generation
Gemini
Gemma
Qwen
Lama
NN-rgb
English
Figure 17: Evolution of LLM systems restricted to the k = 14 condition, shown in comparison to the Nearest Neighbor sRGB baseline classifier (NN-rgb). While many models struggle achieve alignment to IB and human languages comparable to that of the Nearest Neighbor classifier, Gemini achieves greater efficiency (a), alignment with optimal IB systems (b), and alignment with human languages (c) across IICLL generations. Compare to Figure 4
34


Reviews:
[{"review_id": "2bI296EZgN", "reviewer": "Reviewer_nH1w", "summary": "This paper investigates the categorisation behaviour of LLMs for colour, comparing it with human generated color categories in many languages. It adopts an information bottleneck analysis, testing whether generated colour categories optimally trade off accuracy and complexity. The paper shows that LLMs can differ dramatically in their alignment to English, and the complexity of their categories, with larger models tending to perform better. Second, the paper investigates an iterated learning scheme in which LLMs receive an initially random colour categorisation and over several rounds of in-context classification, converge to a new categorisation. This categorisation can then be assessed for its position on the IB tradeoff plane and compared to human languages. Results show that iterated in context learning progressively improves IB efficiency over iterations, and the strongest models show a similar range of solutions as in humans. Finally, the paper shows that similar ideas can apply beyond the domain of colour, by investigating the categorisation of a class of simple shapes that differ in orientation and size.", "questions": "How does k-means or another clustering method on the same features compare to the behaviour of the LLMs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper presents a thorough investigation of colour categorisation behaviour in LLMs. The extent of the experiments, range of models considered, and variety of training hyper parameters investigated (such as training epoch, instruction tuned or not, text vs image prompted, etc) paint a particularly complete picture of the full space of models' abilities in this area. \n\nThe results show that, surprisingly, only the most capable models show colour categorisation behaviour matching English. Further, only these models can develop diverse pseudo labels that span the IB tradeoff curve across complexities, similarly to humans. \n\nThe basic observations are extended to 'Shepard circles', a particular set of visual stimuli, to show that the proposed ideas are not specific to colour naming.\n\nThe paper contains an excellent review of prior work, and motivation for studying colour behaviour as a highly studied instance of human categorisation behaviour.\n\nThe experiments appear to be done to a high standard, and the plots are clear and easy to follow.", "weaknesses": "There were several points that I struggled to understand which may reflect my own limited knowledge of the area:\n\n-The clusterings seem to more or less subdivide the colour space into a set of contiguous regions with approximately equal size, as one might expect by running k-means on the feature vectors (sRGB values) or any other clustering technique. If this is so, then it may be quite easy for a system to have an implicit bias that would place it near the IB curve--it just needs to approximate k-means or similar. If this is right, then statements like \"LLMs can acquire a human-like inductive bias toward optimally-compressed semantic representations, without being trained for this objective\" become less remarkable. It is easy to see how in context learning might approximate clustering, and this implicit bias has no clear human-specific content. \n\n-The observation that Gemini 2.0 can recover English alignment from rounds of iterated learning could be consistent with the model inferring that the feature values are sRGB values, and aligning pseudo labels with real colour names. That is, in-context learning may be operating in a ‘task inference’ mode in which it guesses that the features correspond to colours, and then assigns pseudo labels real colour names accordingly. This mechanism would mean that iterated in-context learning is just another measurement of whether an LLM is aligned to English colour categories, not that something fundamental about how its inductive bias aligns with humans.\n\n-The experiments present colours as sRGB triplets, but many other colour spaces are possible. Does the choice of colour space features affect the results?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HOdoBtPS7g", "reviewer": "Reviewer_5Lw8", "summary": "The authors investigate large language models’ linguistic capabilities to partition semantically the colour domain and study the efficiency of the resulting systems in terms of the IB framework. Further, they investigate their capabilities to evolve inefficient semantic systems towards more regular categories ((again, in the IB framework sense), both in the colour domain and in the less humanly intuitive Shepard circles one. They show that while many models struggle to recover an efficient colour system, many times exhibiting trivial, low complexity categories, the ones that show efficient, more human-like systems also show the capabilities to evolve a non-efficient artificial system towards more efficient ones via in-context iterated language learning", "questions": "What do you think might be the role of communication in evolving these systems towards more efficient ones? Do you think that an innate preference for informative, non-trivial systems (which previous efficient communication literature argues is a consequence of language use via communication) is present in the evolution of these artificial languages because of the data these models are trained on?\n\nHow do you think discrete domains (e.g. kinship) would influence these results? One could argue that continuous domains like colours and Shepard circles might be simpler to efficiently partition for both humans and large language models than more discrete ones.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper’s biggest contribution is to study an important question in cognitive science, namely the emergence of IB-efficient linguistic systems, in the context of large language models.  The paper is sound, well-written and very easy to follow. The experiments are well designed and multiple models, as well as multiple modalities, are systematically tested. The author’s extension of the in-context iterated learning paradigm of Zhu and Griffiths (2024) is a natural and effective way of studying efficient communication questions in the context of large language models, while the author’s replication of the human experiments from Xu et al. (2013) with large language models offers us an additional perspective on iterated learning as a paradigm to evolve artificial linguistic systems. The authors’ results are both well documented and well presented visually, and are also in line with efficient communication theory. \n\nWhile previous literature has analysed large language models’ capabilities of recovering human-like semantic systems in the colour domain, or the role of iterated learning in large language models, to my knowledge, this is the first comprehensive study of large language models from an efficient communication point of view.", "weaknesses": "While the authors analyse a large number of different families of models and, inside those,  models of many different sizes, they restrict their iterated learning analysis to only the best-performing models at the previous task. It would have been interesting to see if iterated learning might have improved their ability to evolve degenerate systems towards more IB-efficient ones. Similarly, the preliminary work on Shepard Circles is lacking a more quantitative analysis similar to the results in Section 4.2, while a figure showing the evolution of the LLM systems like Figure 3 would have helped quantify the evolution of these systems.\n\nA more general feedback is that the authors do not analyse the possible role for large language models of language use via communication in evolving inefficient linguistic systems toward more human-like ones. While Imel et al. (2025) showed how learnability might be the principal pressure that leads towards efficient systems in the colour and Shepard circles domain, an analysis on the role of both cultural transmission via iterated learning and language use via communication would be a very interesting avenue for future work. More specifically, the framework of Kouwenhoven et al. (2024) might be of interest for the authors. \n\nSmaller points to address would be to fix the citations in the appendices, while some figures (cf. Figure 3) are grainy if zoomed in.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZgOZTljLry", "reviewer": "Reviewer_1kNB", "summary": "The present paper investigates whether LLMs are capable of evolving efficient human-aligned semantic systems in the domain of color categorizatzion. The authors replicate two previous human studies on LLMs and find that they vary in their complexity and English-alignment. Larger instruction-tuned models achieved better alignment and IB-efficiency. Finally, they simulate LLMs on cultural evolution of pseudo color-naming systems and find that LLMs iteratively restructure initially random systems towards greater IB-efficiency.", "questions": "It is interesting that instruction-tuned models achieve better alignment and IB-efficiency. Is that merely an reflection that these models are better at the task? Could one take, for instance, an instruction-tuned and a base model that achieve the same accuracy and look at whether they differ in terms of alignment and IB-efficiency?\n\nTo what extend is the observeation that (some) LLMs are IB-efficient a reflection of the fact that they are simply trained on human language?\n\nThe authors write \"IB-alignment measures the similarity between a system and the nearest optimal IB system. WCS-alignment measures the average alignment between a system and the WCS languages, and English-alignment measures the alignment between a system and English.\" Are these measures defined somewhere?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Overall, the paper was well written and easy to follow. It builds on prior work with sound methodology, but extends and adapts it for LLMs. The authors consider a large set of models, including models with different properties, different checkpoints, as well as open-source and frontier models.", "weaknesses": "The main weakness from my perspective is that novelty and contribution is rather limited. The authors take an existing task and evaluation method and simply run LLMs on it. That might have been enough a few years ago, but it is not  anymore (at least in my opinion). The extension to ICCLLL, which would be the main novelty of the paper, is also fairly straight-forward.\n\nWhile the authors show some results on how their findings would generalize beyond the color naming domain, more could be done to establish a robust and general pattern. \n\nEven if one would find that LLMs are IB-efficient in their naming systems, would would that mean? What implications would come with that? Or, vice versa, if it were not the case, what would the implications be? From my perspective, these questions deserve some discussion because at present it is a bit unclear to me why I should care.\n\nMinor: p2 rarely -> readily?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)