When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings
Authors: Jérémy Perez
Grgur Kovač
Corentin Léger
Cédric Colas
Gaia Molinaro
Maxime Derex
Pierre-Yves Oudeyer
Clément Moulin-Frier
Venue: ICLR 2025 Poster (2025)
Keywords: Large Language Models
Multi-Turn Behaviour
Cultural Evolution
Attractors
Transmission Chain
Primary area: foundation or frontier models, including LLMs
TL;DR: We borrow methodologies and concepts from cultural evolutionary theory to study how multi-turn interactions affect the outputs of large language models.

Abstract: As large language models (LLMs) start interacting with each other and generating an increasing amount of text online, it becomes crucial to better understand how information is transformed as it passes from one LLM to the next. While significant research has examined individual LLM behaviors, existing studies have largely overlooked the collective behaviors and information distortions arising from iterated LLM interactions. Small biases, negligible at the single output level, risk being amplified in iterated interactions, potentially leading the content to evolve towards attractor states. In a series of _telephone game experiments_, we apply a transmission chain design borrowed from the human cultural evolution literature: LLM agents iteratively receive, produce, and transmit texts from the previous to the next agent in the chain. By tracking the evolution of text _toxicity_, _positivity_, _difficulty_, and _length_ across transmission chains, we uncover the existence of biases and attractors, and study their dependence on the initial text, the instructions, language model, and model size. For instance, we find that more open-ended instructions lead to stronger attraction effects compared to more constrained tasks. We also find that different text properties display different sensitivity to attraction effects, with _toxicity_ leading to stronger attractors than _length_. These findings highlight the importance of accounting for multi-step transmission dynamics and represent a first step towards a more comprehensive understanding of LLM cultural dynamics.

Published as a conference paper at ICLR 2025
WHEN LLMS PLAY THE TELEPHONE GAME: CULTURAL ATTRACTORS AS CONCEPTUAL TOOLS TO EVALUATE LLMS IN MULTI-TURN SETTINGS
Jérémy Perez" Grgur Kovač', Corentin Léger', Cédric Colas!? Gaia Molinaro 1.3 Maxime Derex Pierre-Yves Oudeyer', and Clément Moulin-Frie Frier
IIniaa, Flowers team, Université de Bordeaux, France MIT, Computational Cognitive Science Lab, Cambridge, MA, USA
3 Department of Psychology, University of California, Berkeley, Berkeley, CA, USA Institute for Advanced Study in Toulouse, Toulouse, France
ABSTRACT
As large language models (LLMs) start interacting with each other and generating increasing amount of text online, it becomes crucial to better understand how information is transformed as it passes from one L.M to the next. While significant research has examined individual LLM behaviors, existing studies have largely overlooked the collective behaviors and information distortions arising from iterated LLM interactions. Small biases, negligible the single output level, risk being amplified in iterated interactions, potentially leading the content to evolve towards attractor states. series of telephone game experiments, we apply transmission chain design borrowed from the human cultural evolution literature: LLM agents iteratively receive, produce, and transmit texts from the previous to the next agent in the chain. By tracking the evolution of text toxicity, positiviry, difficulty, and length across transmission chains, we uncover the existence of biases and attractors, and study their dependence the initial text, the instructions, language model, and model size. For instance, we find that more open-ended instructions lead stronger attraction effects compared more constrained tasks. We also find that different text properties display different sensitivity to attraction effects, with toxicity leading stronger attractors than length. These findings highlight the importance of accounting for multi-step transmission dynamics and represent a first step towards a more comprehensive understanding of LLM cultural dynamics.
INTRODUCTION
Large language models (LLMs) are playing an increasingly significant role in the production of media content across various domains (Brinkmann et al., 2023). They are being used for academic writing (Buruk, 2023; Agarwal et al., 2024; Cheng et al., 2024; Dergaa et al., 2023; Khraisha et al., 2024; Marlow and Wood, 2021; Cabanac et al., 2021), journalism (Petridis al., 2023), story generation (Valentini et al., 2023; Xie et al., 2023; Zhao et al., 2023), as chatbots on social media (Sadikoğlu ct al., 2023) and in the workplace at large (Brynjolfsson et al., 2023; Eloundou et al., 2023). As LLMs become more performant, controllable, and more widespread, their impact the creation and dissemination of information content humans consume is expected to grow even further (Brinkmann al., 2023).
Given the implications of LLM usage for the production cultural information, numerous researchers have studied the properties of LLM-ggnnrraed content. In these studies, LLMs showed several biases with respect to gender (Acerbi and Stubhersfield, 2023; Salewski et al., 2024; Wan et al., 2023; Haller al., 2023; Dong et al., 2024), race (Raj et al., 2024), values (Atari al., 2023; Sivaprasad et al., 2024), politics (Motoki et al., 2023; Agiza et al., 2024; Haller et al., 2023), authority, fallacy oversight,
"Corresponding author; jereny. perezz@ frra.

---

Published as a conference paper at ICLR 2025
Single-turn transmission
(b) Multi-turn transmission
Task
Task
o You wil eucsee toe, Youross 1 nousingg how the ioe?"
Initial Text
Final Text
Input
Output text
To Pc. L
To D.
Tr. Po
Pai., Da Te, Pr Du
text
P. D, L. Tooksiy, Postiriy, Diffeutly and Length at generation
Figure I: The transmission chain experimental design. (a) Single-turn transmission: an LLM agent receives a numan-generated input text (e.g. story) and a task (c.g. "rephrase the text") and generates an output text. (b) Multi-turn turn transmission: a chain of LLM agents is given the same task, with the first agent receiving an initial text and subsequent agents receiving the output of the preceding agent. Measures of toxicity, positivity, difficulty, and length are recorded at each step of the chain.
and beauty (Chen et al., 2024). They were also found to generate at least as attractive (Marlow and Wood, 2021) and compelling (Spitale et al., 2023) texts than humans and to display similar cognitive biases (Echterhoff et al., 2024).
While single-turn behaviors of LLMS prompted with a human-generatee prompt are under active scrutiny, little is known about the effect of iterated interactions. Indeed, LLMs often use existing cultural content to generate new ones, for example when writing scientific reviews (Khraisha et al., 2024; Agarwal et al., 2024) or generating new stories based on existing examples (Xie et al., 2023). As the share of Al-generated content increases, LLMs will be producing outputs using content that is already LLM--enerated, making it crucial to study the consequence of this iterative process. Moreover, LLMs are increasingly being used in multi-agent settings (de Zarzà et al., 2023; Park et al., 2023; 2022; Xiao et al., 2023; Hua et al., 2023; Vezhnevets et al., 2023: Chuang et al., 2023) and are already interacting with one another as chatbots on social media' Nonetheless. little is known about how populations of LLMs might self-organize, as other complex systems do. Research in complex systems traditionally studies how global-level patterns emerge from local interactions (Gleick and Hilborn, 1988; Mitchell, 2009). Here, we ask whether multi-turn behaviors conditioned on LLM-generated content cause the appearance of new kinds of biases, undetectable in single-turn behaviors but accumulating across iterated interactions.
To address this question, we take inspiration from the cultural evolution literature. The field of cultural evolution aims to provide causal explanations for the change of culture over time. By cultural, we refer to the definition traditionally used in the cultural evolution literature, where it refers to any type of socially transmitted information (Mesoudi, 2016). This definition includes various types of cultural information (e.g. beliefs, skills, norms) and is independent of the medium of transmission (e.g. bodily imitation, oral or written transmission). Importantly, this way of defining culture is guite different from the view that is adopted by research works studying cultural values, or cultural alignment in LLMs, where culture is seen as attribute of specific human group (e.g. "Western Culture"). This definition allows to use the same conceptual and methodological framework which was applied to study human (Derex et al., 2019) and non-human (Whiten et al., 2011) cultural evolution to study LLMSs, which is why we adopt that definition for this work.
In particular, we draw insights from a research tradition called cultural attraction theory (CAT) (Sperber, 1985; Morin, 2016; Miton, 2024). CAT aims to determine how non-random transformations of cultural information during transmission events may lead to the evolution of progressively more stable forms, referred to attractors. Although the precise conceptualization of attractors varies across authors, an encompassing definition may be "theoretical posits that capture the way in which certain ideational variants are more likely to be the outcome of transformations than others." (Buskell, 2017). Importantly, the existence of attractors would predict that for a broad range of starting conditions, different variants of given cultural trait will converge towards the same attractor, that is, to a cultural trait that possesses specific properties.
Most experiments in CAT employ a transmission chain design (Mesoudi, 2021), first introduced by (Bartlett, 1932). In this setup, chains of participants receive, produce and transmit social information from and to each other in a sequential manner (as in the popular lelephone game). This powerful and
htsss//chirper.ai
2

---

Published as a conference paper at ICLR 2025
highly controlled design allows to evaluate the high-level patterns that emerge from the accumulation of directional changes during single-turn transmission events. Here, we adapt this design to study how culture evolves along chains of LLMs, rather than human participants (Figure 1).
We conducted several transmission chain experiments with LLMs, where the first LLM-based agent in the chain receives human- -uman-generated text, elaborates it, and then passes it to the next agent in the chain. This transmission step is repeated with different instances of the LLM agent until the end of the chain is reached. We simulated repeated transmission to observe the direction in which text properties evolve, starting from a broad range of initial values. Finding that certain cultural variants (for example, texts with low toxicity) are more likely to be the outcome of this cultural evolutionary process would reveal the existence of an attractor.
By introducing several novel evaluation methods, we estimate the extent which successive trans- mission events affect the evolution of multiple text properties, namely its toxicity, positivity, difficulty, and length. By comparing the properties of the initial (human-generann transmissions between LLMs), we illustrate and study the existence of potential attractors in LLM cultural evolution. effect of the first one. To study how text properties and attractors are affected by the specific context in which culture evolves, we conduct our analyses on five different models (ChatGPT-3.5 3.5-turbb-0013 Llama3-88 100-----Innttee Instruct, Mistral-7B 7---nntttuct--- 2, -J and Mixtral-8re three different tasks (i.e., instructions to either "rephrase". "take inspiration from", or "continue" the initial text) and 20 different initial texts. Although focus is on a better understanding of the cultural dynamics of LLMs, the metrics and evaluation methods introduced here may also be of interest to researchers studying human cultural evolution.
and final texts (after several
particular, we measure the effect of consecutive interactions compared to the
The code for reproducing the simulations, analyses and figures is available on our GitHub²
Our main contributions are:
We propose that there might be a gap in current LLM evaluations methods (single-turn evaluations might not be suited to assess the properties of multi-tury turn interactions)
We empirically confirm this hypothesis by showing that multi-turn interactions indeed often lead to distributions of text properties that are significantly different from what is observed after a single interaction (Section 4.2)
We introduce novel conceptual and methodological tools to fill this gap, grounded in research in cultural evolution, and in particular the concept of cultural attractor (Section 3.4).
We showcase the potential of this method by applying it to compare the effect of different tasks, of different models, of temperature, and of fine-tuning on the properties of multi-turn interactions. (Section 4.3).
We find several robust effects, such as the fact that less constrained tasks lead to stronger attractors, that some properties posses stronger attractors than others, and that fine-tuning can shift the position and modify the strength of attractors.
2 RELATED WORK
Biases in LLMs outputs -LM-generated content is known to exhibit variety of stereotypical biases (Bender et al., 2021; Weidinger al., 2021; Bai et al., 2024). In single-turn settings, LLMs perpetrate (Nadeem et al., 2020) or even amplify human biases based on gender, nationality, race, and religion (Kotek al., 2023). For instance, the GPT model was shown to exhibit cultural values similar to those of WEIRD (Western, Educated, Industrial, Rich, Democratic) cultures (Atari et al., 2023). LLMs trained through reinforcement learning with human feedback (RLHF) were found to overly express left-wing opinions on American politics volitics-al a tendency that, once formed, is difficult to avoid even after steering the model toward different demographic groups (Santurkar et al., 2023). Finally, large body of work compared cognitive biases of models to those of humans (Coda-Form) et al. (2024); Liu et al. (2024); Binz and Schulz (2023)).
Transmission chains featuring artificial agents Several studies have applied experimental designs used in cultural evolution to study knowledge and skill accumulation in groups of Reinforcement
3

---

Published as a conference paper at ICLR 2025
Learning agents (Cook et al., 2024; Schmitt et al., 2018; Team et al., 2021; Prystawski et al., 2023).
Closer to the current study, populations of LLMs have also been studied (Brinkmann et al., 2023).
Iterative chains of generative models trained on the preceding model's 's output have been shown to sometimes collapse toward the most likely outputs while the tails of the original distribution disappear (Shumailov al., 2024; Peterson, 2024; Gerstgrasser al., 2024; Kazdan al., 2024; Dohmatob et al., 2024). This idea of using LLM-generated content to fine-tune the next generation has also been applied to groups of LLMs with various communication structures (Helm et al., 2024). Similar to our approach, iterative chains with frozen (i.e., not re- re-trained) LLMs have been shown to express human-like biases in terms of gender stereotypes. positivity, and social, threat, and hooloyy-reltter information (Acerbi and Stubbersfield, 2023). Strong, but non-human-like biases for producing factual information have also been observed (Chuang et al., 2023), stressing the importance of understanding the evolution of content in LLMs and the ways it might deviate from human cultural cvolution.
Iterated I. Learning Parallel works ((Ren et al., 2024; Zhu and Griffiths, 2024)) have studied similar questions using different methodologies, namely the framework of Bayesian Iterated Learning (BIL) ((Kirby et al., 2014; Griffiths and Kalish, 2007)). Under this framework, attraction is conceptualized as convergence towards priors, whereas we conceptualize it the fixed point of a recurrent sequence. Therefore, BIL allows to make prediction about how an agents' hypothesis about a given phenomenon will evolve through social interaction, while our method captures how a specific cultural trait will evolve. Moreover, our framework enables us to avoid assuming that LLMs possess "priors" in the same way that humans do, which might be problematic (see Kovač et al. (2023); Shanahan et al. (2023)).
Evaluation of LLMs in social contexts While not focusing on the effect of multi-turn interactions, an important body of work has started to study the social cognition of LLMss These studies revealed that GPT-4 struggles to reach human performance with respect social commonsense reasoning and strategic communication skills ((Zhou et al., 2024)). In a similar vein, Huang et al. (2024) found that manipulating the communication structure can make LLM collectives more resilient the introduction of malicious agents.
METHODS
3.1 EXPERIMENTAL PARADIGM: LLM TRANSMISSION CHAINS
In transmission chains, individual participants are ordered linearly. Each participant receives some information from the previous one, performs task, and transmits new information to the next participant. Each agent is prompted with a task (instruction on how the text should be processed) and text, which are concatenated and passed the user message. The first agent is given human- generated text and a task, and subsequent agents are given the same task and the text generated by the previous agent in the transmission chain: leal4+1 LLM (lask, tear4), where text, the initial human-gennn generated text and LLM generates an output based on task task and the previous agent's s text zy. We run this process for 50 generations. Examples of texts evolving through generations are provided in Appendix Section A.
Initial texts (terty) We borrow human- genarrted text from various databases to provide the initial input to each transmission chain. Since we were interested in how variation in the initial text would impact the properties of the ensuing chain, geneangeeeertte texts spanned various types of content: scientific abstracts As we are interested in the evolution of the toxicity, positivity, difficulty and length of generated texts, we sample the entire dataset to obtain a subset of 20 initial texts that covered the range of possible values for these properties. The exact method used to extract these texts is detailed in Appendix A. As a robustness checks, we replicated our experiments with a larger set 100 initial texts (Appendix B).
Tasks To determine the effects of instructions on the evolution of content over generations of LLMs, prompt each chain of LLMs with three different tasks encompassing typical uses of LLMs:
Rephrase: agents are instructed to paraphrase the received text without modifying its meaning. This task is relevant for applications such text simplification, or for content summarization.
4

---

Published as a conference paper at ICLR 2025
Take inspiration: agents are instructed to take inspiration from the received text produce new one. can be used in creative writing, where the goal is to generate new and original content.
agents are instructed to continue the received text. It is relevant for applications such as dialogue generation, in order to generate coherent and relevant responses to user inputs, or for content generation in storytelling and gaming.
Continue:
Tasks remained consistent within each chain. The exact prompt used for each task is reported in Appendix Section A.
Models To assess whether and how cultural evolution dynamics are affected by the model specifica- tions, we identical experiments using six different models, all commonly used, from three different companies and with varying sizes: GPT-4o-mini, GiPT-3.5-turbo-01 (referred to as "GPT3.5") Baamma3--B-Inttuuee ("Llama3-8B")
("Mistral-7B')) Llama3-70B-Instruc Mixtral-8x7B-Bia B-nsttructttt ("Mixtral-&x7B" For inference, we used the OpenAl API (The MIT License)* to run GPT-44-mini and GPT3.5 and the HuggingFace's Transformer library (Wolf et al., 2019) for other models (Apache Licence, v2.0).
("Llama3-700
3.2 METRICS
Iterated transmissions may affect the generated text in several ways. We focus on four, orthogonal properties for each text which could be automatically measured:
Toxicity. Companies typically fine-tune LLMs to avoid the generation of toxic (i.e., dangerous or harmful) content. However, to our knowledge, this fine-tuning step focuses single-turn dynamics, and the evolution of content with respect to its toxicity is currently understudied. We measure the toxicity of texts by quantifying the presence of rude, disrespectful, unreasonable language, using probability score that ranges from 0.0 (benign and non-toxic) to 1.0 (highly likely to be toxic), as estimated by the classifier introduced in Hanu and Unitary team (2020).
Positivity. Even when trained to avoid toxic content, LLMs have been shown to express similar negativity biases to humans, often favoring negative over positive information in preserving and generating new information (Acerbi and Stubbersfield, 2023). To study whether positivity biases over transmission chains are affected by tasks and models, measure the positivity of produced contents using the Sent iment Int ens it ytAnalyzer tool from NLTK (Hardeniya et al., 2016). It uses this information to calculate a sentiment score for the text, ranging from -1.0 (highly negative) 1.0 (highly positive).
Difficulty. While LLMs are argued to benefit society by democratizing knowledge (Weiss), such positive outcomes are conditioned on the LLMs generating output that is accessible and inclusive to kinds of audiences. However, whether text difficulty is preserved, increased, or reduced over transmission chains is currently unknown. We estimate text difficulty using the Gunning-Fee -Fog index (Bogert, 1985), which depends on the average sentence length and the percentage of difficult words. A standard interpretation of this index is that it estimates the years of formal education required to fully understand the text.
Length: A simple, yet crucial aspect of piece of text is its length. As more and more content is generated by and from LLM outputs, cultural media may become populated with increasingly short (potentially incomplete) or long (potentially redundant, meaningless, hard to process) material. We therefore assess the evolution of content length as measured by the character count.
We provide additional details about metrics in Appendix Section A.
3.3 EFFECT OF MULTI-TURN TRANSMISSIONS
One of our questions is how content evolves over multi-turn transmissions compared to single-turn settings. To address this point, compare the distribution of a given property in the generated texts at the first generation to the distributions at subsequent generations. Thus for each model and task, we look the properties of each of the 100 generated texts (20 transmissions chains * seeds) at each generation, which gives us a sample of 100 property values for each value. Using Saannngrro--Smiinne test (Massey, 1951), we then test whether the sample obtain each generation
5

---

Published as a conference paper at ICLR 2025
comes from the same distribution the sample obtained after the first generation. If we can confidently reject the hypothesis that the sample of property values at the end of the transmission chain comes from the same distribution the sample obtained after the first generation, this would confirm that looking at outputs after a single-turn transmission is not enough for predicting output properties in multi-turn setting.
3.4 ATTRACTOR STRENGTH AND POSITION
Human cultural evolution shows that cultural traits sometimes evolve towards attractor states, i.c., content that invites convergence even with different starting points (Kalish et al., 2007; Miton et al., 2015; 2020; Buskell, 2017). Therefore, were interested in whether transmission chains with LLMs would show similar attractor dynamics, and whether these depend on the model and task used in the chain. The concept of cultural attractor is not consistently formalized in the human cultural evolution literature (Buskell, 2017). Here, we defined attractors as the theoretical equilibrium point to which the iterated generation process (defined in 3.1) may eventually converge. We mathematically define attractors in terms of two properties of interest: its position (i.e., the location in output generation space the process converges toward) and its strength (i.e., the intensity to which generated outputs are pulled toward it). The strength takes values in [0, 1J. which allows for continuous notion of an attractor: rather than being a binary concept that either exists or does not, attractors here lie on a spectrum, covering systems without attraction effects (strength=0) to ideal attractors (strength=1), l). To compute position and strength, we use the simulated data to fit linear regression predicting the value of property at the end of the chain (i... after n Aigenerations a function of its value in the initial text (Figure S1).
For example for given text property, we fit: preperty(generation P'generations) I + property instiall where / is the estimated intercept and s the estimated slope. This enables us to estimate the final output of a new chain starting from the final output of the previous chain as: property (ooperty(generatinn 2 * Rggneeatioon) = I + s Ceoperty(geeerateee = ?ggeneratoos).
The fitted linear regression thus allows to define a recurrent relationship between the output of chain as a function of the output of the previous chain: emmpert((generalee = k "lgenerations) = (k- 1) * 50). This relationship a linear recurrence sequence which can be rewritten as: property (gnnertttgnoooioonn = k * (egeneratione) = s (propertysnitial () + L where = 1- If |s| < 1, then the sequence converges, its limit is I and its convergence rate is 1 s. We therefore use the estimated relationship determine if attractor exists ([s) 1) and, if so, estimate its position! = -- and strength
I+ s cooperty(generatie
To validate that these theoretical fixed points correctly capture attraction dynamics, we estimated their positions using only data from the first 10 generations of each chain, and compared the predictions with the actual output after 50 generations. Visual inspection of the results (Appendix B.5) confirmed that our method is suited for estimating the strength and position of attractors.
4 RESULTS
For each of the 6 models, 3 tasks, and 20 initial texts, we ran 5 transmission chains with 50 transmission steps. In the main experiment, each chain is composed of a population of agents sharing the same underlying model. In an additional experiment, we also studied chains where different models interact with one another (Appendix B.2.4). We provide some examples of generated texts in Appendix Section 1, and complete data on the companion website 4 By extracting the properties of generated texts at each generation of each chain, we can study the evolution of these properties through generations, measure how they are affected by interactions beyond single-turn effects, as well as detect and characterize theoretical attractors.
4.1 QUALITATIVE ANALYSIS OF PROPERTY EVOLUTIONS OVER GENERATIONS
In Figure 2, we show the evolution of property distributions over generation for each model and task. This reveals important difference depending on the analyzed property, the task and the model. For

---

Published as a conference paper at ICLR 2025
Gp-lerii
MatrakiB-keeneee
Toxicity
Positivity
Difficulty
Length
Continue Take inspiration Rephrase
Figure 2: Evolution of the distribution of text properties across generations. We here represent the distribution of each of the four properties at cach generation, for each model and task. These distributions thus represent the properties observed in the set 100 transmission chains (20 initial texts # seeds) for each model and task. For each property, task and model, the 50 generations arranged vertically, with first generations at the top and last generations at the bottom.
instance, observe that loxicity converges very quickly very narrow peak centered around O. This is very different from the evolution of positivity, for which the initial distribution appears to be quite preserved for the Rephrase task (Figure 2, first row), while less constrained tasks such as Take inspiration (second row) and Continue (third row) lead to more visible changes. Interestingly, we observe that for Llama3-8B and Llama3-705 the distribution of positivity values converges to bimodal distribution, while distributions are unimodal for other models. In some cases, we also observe that different models lead the distributions be shifted in opposite directions. For instance when looking at the evolution of texts length, using GPT3.5 or Llama3-8ß leads text to become on average shorter, while using Mixtral-8x7B or GPT-4o-mini shifts the distribution towards greater lengths.
4.2 TO WHAT EXTENT DO MULTI-TURN TRANSMISSIONS AFFECT THE EVOLUTION PROPERTIES?
Qualitative analyses from the previous section appear to suggest that multi-turn transmissions lead texts to acquire different properties compared to single-turn settings. To quantitatively evaluate this observation, we use Kolmogorov-Sminn -miinov (KS) tests (Massey, 1951) 1) to estimate the compare property distributions after single interaction and after multiple interactions (see Section B.3.1 for justification why this method is adequate). In Figure 3, we report for each model, task and property the p-value of the KS test for the null hypothesis Ho: "The text properties at generation are sampled from the same distribution as the text properties after generation 1". Across most instances, we observe that the p-values steadily decrease, indicating that observing the given distribution under the null hypothesis becomes less and less likely with generations. We observe that this is more often the case for less constrained tasks (Take Inspiration and Continue, second and third columns) than for more constrained task (Rephrase, first column). This finding confirms that studying single-turn interactions is in general not sufficient for analyzing the properties of interacting L.MMs' outputs. This warrants more detailed account of the cultural dynamics across iterated interactions among LLMs.
4.3 WHAT INFLUENCES THE PRESENCE, STRENGTH, AND POSITION ATTRACTORS?
Effect of model, task and property Visual inspection of the evolution of text properties as presented on Figure 2 indicate that multi-turn transmissions lead distributions to become skewed toward certain values, which suggests the presence of attractors. The task assigned to a chain and the model type populating it appear to influence the position of those attractors, as well as their strength (i.e. how quickly do shifts in distributions happen). To have quantitative measures of attractors strengths and positions, we use the method described 3.4 and Figure S1. Figure 4 presents the estimated strengths and positions of attractors, and fitted linear regressions provided as supplementary material in

---

Published as a conference paper at ICLR 2025
1.0
Heptrase
1.0
Take Imspratirr
1.0
Cuntince
0.5
0.5
0.5
10
20
30
40
50
10
20
30
40
50
0
10
20
30
40
50
1.0
1.0
0.5
0.5
0.5
0.0
0.0
0.0
10
20
30
40
50
10
20
30
40
50
sapaalus pilkcalty
0.5
0.0
1..0
1.0
1.0
0.5
0.0
10
20 40
50
10
20
30 40 50 10
20
30
40
50
Figure 3: Text properties are affected by transmissions beyond the first one. p-values of the KS-test for the null hypothesis Hg: "The text properties al generation are sampled from the same distribution as the text properties after generation 1". for each task (columns), property (rows) and models (colors). The grey shaded area represents p-values lower than 0.05. Over most instances, p-values decrease over generation and become close to O, indicating that multi-turn transmissions lead to significantly different distributions compared to single-turn interactions.
Figure S2. For all combinations of property, task and model, we found that the recurrent relationship defined by the fitted linear regression converges. This means that all conditions admit a theoretical attractor as defined in Section 3.4. To better disentangle the respective contributions of model type, task and property on attractors position and strength, we fitted bayesian models predicting attractor Strength as a function of Task, Model and Property, and predicting attractor Position as function of Task and Model, for each of the four considered properties. We use 95% Credible Intervals (CI) to assess significance. Those confidence intervals,as well as details of statistical models, are provided in Appendix section B.
We find strong effect of Task on attractor strength: Continue leads to significantly stronger attraction than Take Inspiration, itself leading to significantly stronger attraction than Rephrase This supports our observation that less constrained tasks lead to stronger attraction than more constrained tasks. To specifically test this hypothesis, we conducted additional experiments manipulating only the room for variation allowed in the task, which confirmed this hypothesis (Figure S5).
Different properties are also found to display different sensitivity to attraction effects. We detect that toxicity possesses significantly stronger attractors than positivity, difficulty and length. As for the effect of model, we observe significantly weaker attraction for Llama3-70b compared GPT3.5, Llama3-8B and Mixtral-8x7b. GPT-4o-mini also displayed singificantly weaker attractors than GPT3.5, Llama3-8B and Mixtral-xx7b.
As for the position of the attractors, we found that the position of the attractor for positivity was significantly lower for Llama3-8b than for GPT3.5, GPT-4o-mini, Mistral-7b and Mixtral-8x7b. and that the task Take inspiration and Continue both led to significantly higher positivity than the Rephrase task.
Robustness checks To verify the robustness of our results, we verified that they hold when using a larger set of initial texts (Figure S3) and when using different phrasings of the instructions prompts (Figure S4). We observed similar trends and values as in the main experiment, which confirms the robustness of our results.
Effect of temperature Increasing the temperature appears to lead to stronger attractors only in constrained tasks (Rephrase and Take inspiration), but not in the more open- ended task Continue (Figure 5a). One interpretation might be that increasing temperature relaxes constraints on the content that can be produced, thus leading to stronger attractors. This effect would therefore be more significant for tasks that are quite constrained.
Effect of fine-tuning We observe that fine-tuning can shift the attractors' positions (Figure 5b): the attractor for length is lower, and the attractor for difficulty higher, for Instruct models compared to Base models. For toxicity and positivity, we do not observe very significant shifts. This may indicate that Base models were already quite aligned with human preferences in terms of toxicity and positivity
8

---

Published as a conference paper at ICLR 2025
Toxicity
Positivity
Difficuity
Length
0.01
1.0
0.00
1.
-0,01-
2500-
1.0
1.25
0.8-
0.8
1.00-
0.75
0.75
Attractor
0.4
0.4
0.25
0.25-
Task 0.00
0.00
Task
Task
Task
Figure 4: Attractors strength and position. The heigth of the bars represent the position (top row) and strength (bottom row) of theoretical attractors estimated using the method described in Section 3.4, for each property (columns), task, and model. Less constrained tasks, such as Continue, appear to produce stronger attractors than more constrained tasks, such as Rephrase. Attractors appear to stronger for toxicity than for length. Finally, we can notice that the position attractors appears to vary between models.
even before fine-tuning, possibly due to techniques such as data curation. Length and difficulty may have been less targeted by such techniques, and fine-tuning may therefore have shifted the attractors' positions towards human preferences. As for the strength of attractors, we observe that fine-tuning seems to increase attraction strength for toxicity, but to reduce it for difficulty. This may suggest that for properties on which humans have strong, uniform preferences (such as toxicity), finettuing leads to stronger attractors, while for properties on which humans have weaker and more heterogeneous preferences, it mitigates the strength of attractors that came from the training data. This analysis provides insight about how attractors are formed, and reveals that both the choice of training data and fine-tuning processes may impact LLMs in ways that only become significant in the case of multi-turn behaviors.
DISCUSSION
While current studies analyzing the outputs of LLMs are restricted to a single prompt-output interac- tion, we borrowed the methodology from studies on human cultural evolution to address how cultural content may evolve over transmission chains with LLMs. This resulted in series of telephone game experiments assessing the evolution of cultural content LLMs as a function of models, instructions, and text properties. Our results reveal that several changes in generated content appear after multiple iterations. For example, we observed that the difficulty of a provided text was preserved after L.MM was prompted elaborate was processed iteratively by a chain of L.LMs. Those qualitative observations were confirmed by statistical tests, where we found that multi-turn interactions lead the distributions of text properties to become significantly different distributions obtained after single-turn interactions.
single time, but changed dramatically after the text
By comparing the properties of input texts those of texts produced by transmission chains spanning several generations of LLMs, we identified property-specific patterns in the convergence of LLM dynamics toward attractor states. Using this method, we found that some properties (e.g. toxicity) display quicker convergence rates toward attractors. and that the position of these attractors varies between models. Moreover, our results reveal that more open-ended tasks (e.g. Continue) lead to quicker convergence toward attractors than more constrained tasks (e.g. Rephrase). Studying attraction dynamics systems of LLMs may therefore particularly relevant to situations which LLMs are used to simulate artificial societies (de Zarzà et al., 2023; Park et al., 2023; 2022; Xiao et al., 2023; Hua et al., 2023; Vezhnevets et al., 2023; Chuang et al., 2023), where they are often granted relatively high freedom. Evaluating the effect of temperature view that allowing more room for variation leads to stronger attractors, as we found that increasing temperature increases attraction strength only for quite constrained tasks. To understand the role
attractors supports this

---

Published as a conference paper at ICLR 2025
(a)
(b)
Take Inapiation
Zophrase iespiration Continus
1401
610
1.0
3.00
Tamperaturr
Temparstors
Temperature
Temporature
Figure 5: Effect of temperature (a) and fine-tuning (b) on attractors. (a) Attractor positions (top row) and strength (bottow row) for different values on temperature (x-axis), for model Llama3-8B Instruct. The main visible effect that increasing temperature increases attraction strength for tasks Rephrase and Take Inspiration, but not for Continue. (b) Attractor positions (top row) and strength (bottow row) for Base and Instruct versions of Mixtral-x77B and Llama3-70B. Fine-tuning appears to increase the strength of attraction for toxicity, increases the position of the attractor for difficulty, and decreases the position of the attractor for length.
of fine-tuning in shaping the strength and position of these attractors, we compared Base models with their Instruct versions. This revealed that fine-tuning can affect both attractors strength (e.g. for toxicity) and position (e.g. for difficulty and length).
The main practical consequence of our results is that they question the current ways in which LLMs are fine-tuned and evaluated. Indeed, current fine- fine-tuning procedures, such as RLHF, rely on a single interaction to rate the output of the LLMs. Similarly, the vast majority of benchmarks and leaderboards are based on the capacities of LLMs in single-turn interactions. As multi-step interaction is major use-case of LLMs (e.g. when interacting iteratively with a chatbot, when LLMs are used in multi-agent simulations), there is a contradiction between how LLMs are used and how LLMs are evaluated and trained. Our results indicate that this contradiction cannot be ignored: for example, on Figure S26 (toxicity take inspiration) we see that at first Mistral appears less toxic at the first iteration, but after generations Llama3-8B ends up being much less toxic. One may therefore choose Mistral-7B based on single-step evaluation, even though Llama3-8B is the optimal choice for many applications.
More generally, our results reveal that knowing the output of LLMs after single interaction is insufficient to predict their behavior long-term social interactions. Therefore, this suggests that existing training and evaluation methods are not suited to align the behavior of LLMs in multi-step settings. The concepts of attractor position and strength may offer a way to take into account the consequence of multi-turn interactions when comparing and evaluating LLMs.
Limitations and future work As our results indicate that both training data and fine-tuning affect attractors properties, future work should further explore how different fine-tuning and data curation methods shape multi-turn dynamics. While we focused on linear transmission chains, real-world interactions typically involve networks of senders and receivers. Human studies have shown that network size (Fay et al., 2019; Richerson, 2013; Andersson and Read, 2014; Baldini, 2015; Derex al., 2013) and structure (Raviv al., 2020; Kirby and Tamariz, 2022: Derex and Boyd, 2016; de Pablo et al., 2022; Derex and Mesoudi, 2020) influence cultural evolution. Following some initial endeavors (Nisioti al., 2022: Perez al., 2024; Lai et al., 2024), future work may assess similar effects in machine networks, and in particular investigate how attraction dynamics are shaped by network-based strategies such as dynamic network structures (Nisioti et al., 2022) or free-formed decentralized networks (Lai et al., 2024). To investigate model- and task-specific biases, we focused on homogeneous transmission chains where agents belonged to the same model type and received the same instructions. Future studies may address cultural dynamics in heterogeneous populations of LLMs prompted with diverse instructions. Our study could also be extended to hybrid networks in which humans and LLMs interact- scenario that is becoming increasingly relevant as generative tools become more widespread and which may, in turn, shape the future of human cultural evolution (Brinkmann et al., 2023).
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
The code for reproducing simulations, statistical analyses, and for generating all figures can be found on our GitHub repository: also contains the data generated when running the experiments. This data can also be accessed from the companion website tool.
This repository
gameooml-m using the Data Explorer
ETHICS STATEMENT
The results presented in this paper highlight that multi-turn interactions can give rise to potential harmful biases that cannot be detected by only studying the outcome of single-turn interactions. By introducing novel conceptual and methodological tools, we provide ways of quantifying the properties of multi-turn interaction systems with respect to the extent which they magnify or reduce these biases. It is then the responsibility of end-users to make an ethical use of these tools, for instance by using them to ensure that L.LMs remain aligned with ethical standards even after repeated interactions.
ACKNOWLEDGEMENTS
This research partially funded by the French National Research Agency (ANR, project ECOCURL, Grant ANR-20-CE23-000)) This work benefited from access to the Jean Zay (Idris) supercomputer associated with the Genci grant A015101 1996. We also thank Chris Foulon, Marcela Ovando-Telleez and Joan Dussauld who participated in the hackathon Hack Robo during which this project originated. M.D. acknowledges Institute for Advanced Study in Toulouse (IAST) funding from the French National Research Agency (ANR) under grant no. ANR- 1IR-I7-BURE-001 (Investissements Avenir programme).
REFERENCES
A. Acerbi and J. M. Stubbersfield. Large language models show human-like content biases in transmission chain experiments. Proceedings of the National Academy of Sciences, 120(44): e2313790120, 2023.
S. Agarwal, I. H. Laradji, L. Charlin, and C. Pal. LitLLM: A Toolkit for Scientific Literature Review, Feb. 2024. URL
http://awwwvv..g//.aa./ abs /202/442222 .01.0.18888 788 arXiv:2402.01738 [cs].
A. Agiza, M. Mostagir, and Reda. Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in LLMs, Apr. 2024. URL 08699. 1XXx:2444.08699 [cs].
C. Andersson and D. Read. Groupsize and cultural complexity. Nature, 511(7507):11-EL 2014. URL https:/ //ww://wwwww//// www.wature......m.. Group UK London.
Publisher: Nature Publishing
M. Atari, M. Xue, S. Park, D. E. Blasi, and J. Henrich. Which humans?, Sep 2023, URL osf ioiii/////a http://pprntss///eeee http://ts///aaaaa//ss/bea.
X. Bai, A. Wang. I. Sucholutsky, and T. L. Griffiths. Mcasuring Implicit Bias in Explicitly Unbi- ased Large Language Models, May 2024. URL http://arxiv.orr//aattt /2402. 04105. arXiv:2402.04105
R. Baldini. Revisiting the effect of population size on cumulative cultural evolution. Journal of Cog- nition and Culture, 15(3-4):30-3366 2015. URL https: httpa/////lll.mmm/////////////// comm/view/oournalls... xml. Publisher: Brill.
L. Bartlett. Remembering. Cambridge University Press., 1932. URL https::/pure mpg.
de/ de/rest/ tteeeteeeeeemmmmm....e htm.tttemmmmee httpilteeemm2meeeeeee wwtp///on/ooolleeecooc 23092911 content.
11

---

Published as a conference paper at ICLR 2025
E. M. Bender, T. Gebru, A. McMillan Major, and S. Shmitchell. On the dangers of stochastic parrots:
Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT 221, page 610-623, New York, NY. USA, 2021. Association for Computing Machinery. ISBN 9781450383097. doi: 041144573442188..445522
URL http:::tp//......... /doi www./10.00000 111.//1.1111/3441119 3445922.
M. Binz and E. Schulz. Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 2 Feb. 2023. doi: 10. (0007//pas..2188231111 URL https: / www. pnas. org/ /tc////.100//////// 1073/pnas 221 022285221200 Publisher: Proceedings of the National Academy of Sciences.
Bogert. In defense the fog index. The Bulletin of the Association for Business Communication, 48(2):9-12 12, 1985.
I.. Brinkmann, F. Baumann, J.-F. Bonnefon, M. Derex, T. F. Müller, A.-M. Nussherge, A. Czaplicka, A. Acerbi, L. Griffiths, J. Henrich, et al. Machine culture. Nature Human Behaviour, 7(11): 1868,1868, 2023.
E. Brynjolfsson, D. Li, and L. R. Raymond. Generative Al at Work, Apr. 2023. URL https: h/wwww nber.. org/ httpp/ppppps//////
O. O. Buruk. Academic Writing with GPT-3.5: Reflections on Practices, Efficacy and Transparency. In 26th International Academic Mindtrek Conference, pages 44-155, Oct. 2023, doi: 10.1 11079. arXiv:2304. 1079 [es].
URL hatttp://axxv..... http://awwww...oraa////aaa/
A. Buskell, What are cultural attractors? Biology & Philosophy, 32(3):377-394, 2017. ISSN 0169-3867. doi: 10.10077/10539-017-95777
htth:tt::/ wwww. ncbi nlm. nih. gov/
pmc//articlle///Mc5522 627/.
G. Cabanac, C. Labbé, and A. Magazinov. Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals, July 2021. URL http: // //arxivv. crg/ 2aod///22111 2107.00000 06751. arXiv:2107.06751 [cs].
G. H. Chen, S. Chen, Z. Liu, Jiang, and B. Wang. Humans LLMs the Judge? A Study on Judge- ment Biases, Apr. 2024. URL [es].
abs 22002.4222. 10669. 2iii:2202.1066
H. Cheng, B. Sheng, A. Lee, V. Chaudary, A. G. Atanasov, N. Liu, Y. Qiu, T. Y. Wong, Y.-C. Tham, and Y. Zheng. Have Al-Generated Texts from LLM Infiltrated the Realm of Scientific Writing? A Large-Scale Analysis of Preprint Platforms, Mar. 2024. URL https: / www. bi orxi v org/ content/10, 1ttp:/20.1111222211111 03. 586710v2. Results.
Pages: 2024.03. 2...003.25.58677
Section: New
Y.-S. Chuang, A. Goyal, N. Harlalka, S. Suresh, R. Hawkins, S. Yang. D. Shah, J. Hu, and T. T. Rogers. Simulating opinion dynamics with networks of Illmbased agents. arXiv preprint arXiv:2311.096 2023.
J. Coda-Forna M. Binz, J. X. Wang, and E. Schulz. CogBench: large language model walks into a psychology lab, Feb. 2024. URL arXiv:2402.18225.
18225. 8225.
J. Cook, C. Lu, E. Hughes, Z. Leibo, and J. Foerster. Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning, June 2024. URL 00392. xxXiv2246600099) [cs].
J. F-L. de Pablo, V. Romano, M. Derex, E. Gjesfjeld, C. Gravel-Miguel, M. J. Hamilton, A. B. Migliano, F. Riede, and S. Lozano. Understanding hunter gatherer cultural evolution needs network thinking. Trends in Ecology & Evolution, 37(8)6332 636. 2022. URL https::///// cell. wtw.oo//ffllllloeooeeeeeeeoeeeee /s01 5347 (22) 00090-8. Pub-
bunter-gatherer
lisher: Elsevier.
12

---

Published as a conference paper at ICLR 2025
1. de Zarzà, J. de Curtò, G. Roig. P. Manzoni, and C. T. Calafate. Emergent cooperation and strategy adaptation in multi-agent systems: An extended coevolutionary theory with llms. Electronics, 12 (12):2722, 2023. URL https:: /wwww mmpii 0ttmm/2779-99999 (tt.//99992212/12211211 12/12/2222 Publisher; MDPI.
M. Derex and R. Boyd. Partial connectivity increases cultural accumulation within groups. Pro- ceedings of the National Academy of Sciences, 113(11):282229935 Mar. 2016. ISSN 0027-8424, 1091-6490. doi: 10. 10.1073/pnas.l. 15187981183
ptt.ppa........11111111 1073/pnas 151 518798113
URL https: /ttppss///aaaee/ //pas//a//s////////// .rg. doi fooof0u
M. Derex and A. Mesoudi. Cumulative cultural evolution within evolving popula- tion structures. Trends Cognitive Sciences, 24(8):654-667, 2020. URL https: //www. cell. wwwtttends///ccccc//ctcotcctssst 87dci- radcciddraven_nis_eettoo sttpveen__ss_eto__mmee email. Publisher: Elsevier.
6613 (20)
30107-8 8 degeddd
M. Derex, M.-P. Beugin, B. Godelle, and M. Raymond. Experimental evidence for the influence of group size on cultural complexity. Nature, 503(74766:389-39) 2013. URL https: /www. nature. com/
Publisher: Nature Publishing Group UK London.
M. Derex, J.-F. Bonnefon, R. Boyd, and Mesoudi. Causal understanding not necessary for the improvement of culturally evolving technology. Nature Human Behaviour, 3(5):446-45,, May 2019. ISSN 2397-3374. doi: 10.10088-4156--019---5
URL /www nature. com/ art aww.aartiiileeeeeeee 3011:441191--5-55500211 -799. Publisher: Nature Publishing Group.
I. Dergaa, K. Chamari, P. Zmijewski, and H. Ben Saad. From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing.
Biology of Sport, 40(2):615-622, Apr. 2023. ISSN 0860-021X. doi: 10.51 14/boosport..223.. 125623. URL pattps://n www. ncbi. nlm. gww. pmc haree/aatiillseeeeeeeeeeeeeeeeee /MMC/0008711/////////
E. Dohmatob, Y. Feng, P. Yang. F. Charton, and J. Kempe. A Tale of Tails: Model Collapse
as a Change of Scaling Laws, May 2024. URL arXiv:4402.07043.
X. Dong, Y. Wang, P. Yu, and J. Caverlee. Disclosure and Mitigation of Gender Bias in LLMs, Feb. 2024. URL http: /arxiv..vvvv org/ www.ooggg///./oooooooooooooees/o
1111111111 1199. arXiv:2402.11199 11190 [es].
Echterhoff, Y. Liu, A. Alessa, McAuley, and Z. He. Cognitive Bias in High-Stakes Decision-Making with LLMs, Feb. 2024. URL http: http:/aai......aba/a/./oaaaa arXiv:2403.008/ [cs].
00811.
T. Eloundou, S. Manning, P. Mishkin, and D. Rock. GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models, Aug. 2023. URL abs/2300 10130. arXiv:2303. 10130 [cs, econ, q-fin].
N. Fay, N. De Kleine, Walker, and C. Caldwell. Increasing population size can inhibit cumulative cultural evolution. Proceedings of the National Academy of Sciences, 116(14):6726- 6731, Apr. 2019. ISSN 0027-8424, 1091-6490, doi: 10.11073/pus..811111111 URL https:
1811413116.
//ppas.....///////// org/ cotoo/fful///e 1073 pnas 1811413116.
M. Gerstgrasser, Schaeffer, Dey, R. Rafailov, Korbak, H. Sleight, R. Agrawal, J. Hughes, D. B. Pai, A. Gromov, D. Roberts, D. Yang, D. L. Donoho, and S. Koyejo. Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. Aug. 2024. URL / /ttpp//rrreee..eee/oruuau
#discussion.
J. Gleick and R. C. Hilborn. Chaos. Making New Science. American Journal of Physics, 1.15345. URL
Nov. 1988. ISSN 0002-9505, 1943-2909, doi: 10.1119/
T. L. Griffiths and M. L. Kalish. Language evolution by iterated learning with bayesian agents. Cognitive Science, 31(3):441448 2007. ISSN 1551-6709. doi: 10. 1001080//532200001332257777 Place: United Kingdom Publisher: Taylor & Francis.
13

---

Published as a conference paper at ICLR 2025
P. Haller, A. Aynetdinov, and A. Akbik. OpinionGPT: Modelling Explicit Biases in Instruction-Tuna
LLMs, Sept. 2023. URL http:://aaaa...orr//aaa.. 03876 arXiv:2309 008866 [cs].
L. Hanu and Unitary team. Detoxify. Github. http://giihhh.omm/niaaaaaaaaadii 2020.
N. Hardeniya, Perkins, D. Chopra, N. Joshi, and I. Mathur, Natural language processing: python and NLTK. Packt Publishing Ltd, 2016.
H. Helm, B. Duderstadt, Y. Park, and C. E. Priebe. Tracking the perspectives of interacting language models, June 2024. URL http: / aarai.iiiv....... org/ wtd.s//2400 2400 11...19999 1938. arXiv:2406. 1998 [cs].
W. Hua, L. Fan, L. Li, K. Mei, J. Ji, Y. Ge, L. Hemphill, and Y. Zhang, War and Peace (WarAgent): Large Language Model- Multi-Agent Simulation of World Wars, Nov. 2023. URL http:
//a //arxiv. wrgg abs / 22311 1 17277, 1rXiv:2311.11 17227 [cs].
J.-t Huang, J. Zhou, T. Jin, X. Zhou, Z. Chen, W. Wang, Y. Yuan, M. Sap, and M. R. Lyu. On the Resilience Multi-Agent Systems with Malicious Agents, Sept. 2024. URL http://arxis org/abs 2408. 00989. arXiv:2408.0000 00999.
C. Hutto and E. Gilbert. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference web and social media, volume 8, pages 216-225, 2014.
M. L. Kalish, T. L. Griffiths, and S. Lewanddwwsy. Iterated learning: Intergenerational knowledge transmission reveals inductive biases. Psychonomic Bulletin & Review, 14(2):288 2007. ISSN 1531-5320. doi: IF.31566
Place: US Publisher: Psychonomic Society.
Kazdan, R. Schaeffer, A. Dey, M. Gerstgrasser, R. Rafailov, L. Donoho, and Koyejo. Collapse or Thrive? Perils and Promises of Synthetic Data in Self-Generating World, Oct. 2024. URL http:://are wwtoorg///aaa.o abs 2.10.. 1.133 arXiv:2410.166 16713..
Q. Khraisha, S. Put, J. Kappenberg, A. Warraitch, and K. Hadfield. Can large language models replace humans in systematic reviews? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages. Research Synthesis Methods, Mar. 2024. ISSN 1759-2887. doi: 10. 1001//0/frmm.) 1715.
S. Kirby and M. Tamariz. Cumulative cultural evolution, population structure and the origin of combinatoriality human language. Philosophical Transactions of the Royal Society B: Biological Sciences, (7701843))20200319, Jan. 2022. ISSN 0962-8436, 1471-2970, doi: 10. (098//stt.2020. 0319. URL httph:tps////m http:/oyylllllccooo BtKKattpps///ooallc...tttp.....p 0319.
wttpo//////////io 1098 htte//rtbbbeeee 2020.
S. Kirby, T. Griffiths, and K. Smith. Iterated learning and the evolution of language. Current Opinion in Neurobiology, 10:1011, Oct 2014. ISSN 09594388. doi: 10. w...0/6/...obb.201444.4 https:://iikkae
URL
elsevier. com cctc:///erieeee///////3333999999
H. Kotek, R. Dockum, and D. Sun. Gender bias and stereotypes in large language models. In Proceedings of The ACM Collective Intelligence Conference, pages 12-24, 2023.
G. Kovač, M. Sawayama, R. Portelas, C. Colas, P. Dominey, and P.-Y. Oudeyer. Large Language Models as Superpositions of Cultural Perspectives, Nov. 2023. URL [cs].
abs/2307, 07870. arXiv:2307.0788
S. Lai, Y. Potter, J. Kim, R. Zhuang, D. Song, and J. Evans. Evolving AI Collectives to Enhance Human Diversity and Enable Self-Regulation, June 2024. URL http::/arxiv.... www.:xwwwwwrrraeeeoooooe /bs/ 2402. 12590. arXiv:2402.1225
R. Liu, J. Geng, J. C. Peterson, I. Sucholutsky, and T. L.. Griffiths. Large Language Models Assume People More Rational than We Really are, July 2024. URL httt:://pa/xiiii 2406. 17055. arXiv:2406.1100 17055.
abss/
R. Marlow and D. Wood. Ghost in the machine or monkey with a typewriter-aee generating titles for Christmas research articles in The BMJ using artificial intelligence: observational study. The BMJ, 375:2667732, Dec. 2021. ISSN 0959-8138. doi: 10.1136/bmj-2000 20210007772222-2773 URL
https: / /ww.wwww.mmmmmmm///ooo ncbi nlm. nih
14

---

Published as a conference paper at ICLR 2025
F. J. Massey. The kolmogorov-smimoy test for goodness of fit. Journal of the American Statistical As- sociation, 46(253) -8, 1951. ISSN 01621459. URL htth: /ww.. http:://wwww..o.o.../oooo org/ abgeet 2280095.
A. Mesoudi. Cultural Evolution: Review of Theory, Findings and Controversies. Evolutionary Bi- ology, 43(4):481-49) Dec. 2016. ISSN 0071-3260, 1934-2845. doi: 10. 10.10077411692-01-5993900 URL ptttp://line http:///iinnns..iign.e.eeee com/ 10. 1007/s11 000007/11192-01111 5000---00
A. Mesoudi, Experimental studies of cultural evolution, July 2021. URL https:// qzvxy.
M. Mitchell. Complexity: A Guided Tour. Oxford University Press, Oxford, New York, Apr. 2009. ISBN 978-0-19-512441-5
H. Miton, Cultural Attraction, Feb. 2024. URL httpt:////osfftto/////
H. Miton, N. Claidière, and H. Mercier. Universal cognitive mechanisms explain the cultural success of bloodletting. Evolution and Human Behavior, 36(4):303-312, July 2015. ISSN 10905 138. doi: 200000/6/.wotuummmm2222........0 Detrieeee/l11$11005551333
URL
elsevier com/
H. Miton, Wolf, C. Vesper, G. Knoblich, and D. Sperber. Motor constraints influence cultural evolution of rhythm. Proceedings of the Royal Society B: Biological Sciences, 287(1957::0202200,
Oct. 2020. doi: 10. wttp/////sppb22....... 2001. URL https: Publisher: Royal Society.
org/doi/ffll/1000 1tt.:/1...111/00 http://////bpp.. 2020.220000000
O. Morin. How Traditions Live and Die. Oxford University Press, 2016. ISBN 978-0-19-021050--. Boogle-Books-ID: KSukCgAAQBAI.
F. Motoki, V. Pinho Neto, and V. Rodrigues. More human than human: measuring ChatGPT political bias. Public Choice, Aug. 2023. ISSN 0048-5829, 1573-7101. doi: 10.10077/11127--2-0000077 URL htths::: //ink///nnkk... www.sger.iiiee.ee www.riinerrrcooeeaeoo 10.
M. Nadeem, A. Bethke, and S. Reddy. Stereoset: Measuring stereotypical bias in pretrained language models. arXiv preprint 20XXv:2004409456, 2020.
E. Nisioti, M. Mahaut, P.-Y. Oudeyer, I. Momennejad, and C. Moulin-Frier. Social Network Structure Shapes Innovation: Experience-sharing in RI. with SAPIENS, Nov. 2022. URL http::ttp//cc....t //axivv. org/ htg//bs//e 2206 65060. arXiv:2206.05060 [cs].
Park, L. Popowski, C. Cai, R. Morris, P. Liang, and M. Bernstein. Social Simulacra: Creating Populated Prototypes for Social Computing Systems, Aug. 2022. URL http:://rxiv.. org/ hbt//ab//aae 2208. 04024. arXiv:2208. 004044 [cs].
J. S. Park, J. C. O'Breen, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior, Aug. 2023. URL http: /ttp///avvv...///////ee////.atv/ www.ii.ogg/i..oee...ooo... wttp/rgg///b////////////////// 03442. arXiv:2304.0344
[cs].
J. Perez, C. Léger, M. Ovando Tellez, C. Foulon, J. Dussauld, P.-Y. Oudeyer, and C. Moulin- Frier. Cultural evolution in populations of Large Language Models, Mar. 2024. URL http: //arxiv. aeriv.org/all abs 2403 08882. arXiv:2403.08882 [cs, q-bio].
A. Peterson. Ai and the problem knowledge collapse. arXiv preprint wrXiv2440.033022, 2024.
S. Petridis, N. Diakopoulos. K. Crowston, M. Hansen, K. Henderson, S. Jastrzebski, J. V. Nickerson, and B. Chilton. AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI '23, pages 1-16, New York, NY. USA. Apr. 2023, Association for Computing Machinery. doi: 10. 115553544548.5880900 URL https: / / dl. acc. org/ 3580907.
ISBN 978-1-4503-9422 doi /oi/10.10 110..11415344444
B. Prystawski, D. Arumugam, and N. D. Goodman. Cultural reinforcement learning: a framework for modeling cumulative culture limited channel, May 2023. URL https: //osf / 1o/g4t28.
15

---

Published as a conference paper at ICLR 2025
C. Raj. A. Mukherjee, A. Caliskan, A. Anastasopoulos, and Z. Zhu. Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesis, July 2024. URL http://arke org/ abs/ 22/222477 02030. arXiv:2407.02033 [es].
L. Raviv, A. Meyer, and S. Lev-Ari. The Role of Social Network Structure in the Emergence of Linguistic Structure. Cognitive Science, 44(8) Aug. 2020. ISSN 0364-0213, 1551-6709. doi: /a111l/coos.12888 URL cogs. 12876.
wi wiley www. wtt/://oooo....../
Y. Ren, S. Guo, L. Qiu, B. Wang, and D. J. Sutherland. Language Model Evolution: An It- erated Learning Perspective, Apr. 2024. URL arXiv:2404.04286 [cs].
04286
Richerson, Group size determines cultural complexity. Nature, 503(7776):351-3322 2013. URL nature
attpss//wwe wwwpw/..ttttee.....em/emm www.///arrieee/ttttticoooooo Group UK London.
Publisher: Nature Publishing
E. Sadikoğlu, M. Gök, Mijwil, and Kosesoy. The evolution and impact of large language model chatbots in social media: A comprehensive review of past, present, and future applications, 12 2023.
Salewski, S. Alaniz, Rio-Torto, Schulz, and Z. Akata. In-Context Impersonation Reveals Large Language Models' Strengths and Biases. Advances in Neural Information Processing Systems, 36, 2024. URL httpttts://prcceedinnne 93 3f4141111 -tstttttttttttttttttttttttt----e cenerracctt-ooffeeneeeee html.
neur ips cc/ cc/paper_file files/ httpp/ppperr//2/3//3
S. Santurkar, E. Durmus, F. Ladhak, C. Lee, P. Liang, and T. Hashimoto. Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971-30004. PMLR, 2023.
S. Schmitt, J. Hudson, A. Zidek, S. Osindero, C. Doersch, W. M. Czarnecki, J. Z. Leibo, H. Kuttler, A. Zisserman, Simonyan, and S. M. A. Eslami, Kickstarting Deep Reinforcement Learning, Mar. 2018. URL / org/ abs 803. 03835. arXiv: 1xXx:1803300335 [cs].
M. Shanahan, K. McDonell, and I. Reynolds. Role-Play with Large Language Models, May 2023.
URL http: /at:://azzaa /aaxiiiiiiiv///ii///aaa///aaiava org/ abs//333 16367. arXiv:2305.16367 [es].
I. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot, and R. Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint 1rXxi:23055.7499 2023.
I. Shumailov, Z. Shumaylov, Zhao, N. Papernot, R. Anderson, and Y. Gal. AI models collapse when trained recursively generated data. Nature, 631 July 2024. ISSN 1476- 4687. doi: 10. 0088/441586002440755566666 URL aattps:/ /ttp://w.nttttt. wtt.:/re...o.aattiiiiicccc s41586 -
y. Publisher: Nature Publishing Group.
S. Sivaprasad, P. Kaushik, S. Abdelnabi, and M. Fritz. Exploring Value Biases: How LLMs Deviate Towards the Ideal, Feb. 2024. URL arXiv:2402.1 1005 [es].
D. Sperber. Anthropology and Psychology: Towards an Epidemiology of Representations. Man, 20(1):73-89, 1985. ISSN 0025-1496. doi: 10.2307/2802222. URL https: /ttps:///ww. /wwww /tt.:w.ww..o....//////....o..... org/ htgg/stable/280222 Publisher: [Wiley, Royal Anthropological Institute of Great Britain and Ireland].
G. Spitale, N. Biller-Andorno, and F. Germani. model GPT-3 (dis)informs us better than humans. Science Advances, 9(2))eadh1885 18500 June 2023. doi: 10.11 URL https: //s:://www www.wwecienc..... org/ doii / 100.1126scciai adh1 850. Publisher: American Association for the Advancement of Science.
O. E. L. Team, A. Stooke, A. Mahajan, C. Barros, C. Deck, J. Bauer, J. Sygnowski, M. Trebacz, N. Wong, N. Porcel, R. Raileanu,
M. Jaderberg, M. Mathieu, N. McAleese, N. Bradley-Schmine S. Hughes-Fitt, V. Dalibard. and W. M. Czarnecki. Open- Ended Learning Leads to Generally Ca- pable Agents, July 2021. URL ht http: : //tp/vaxvvv/////////////////aavl http:////rra/bb///eaaaaab [es].
abs /2107. 12808. arXiv:2107.12808
16

---

Published as a conference paper at ICLR 2025
M. Valentini, J. Weber, J. Salcido, Т. Wright, E. Colunga, and K. Kann. On the Automatic Generation and Simplification of Children's 's Stories, Oct. 2023. URL https: http://axivvvc.ooo 2310. 18502v1.
A. S. Vezhnevets, P. Agapiou, Aharon, R. Ziv, J. Matyas, A. Duéfez--Guzmán, W. A.
Cunningham, S. Osindero, D. Karmon, and J. Z. Leibo. Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia, Dec. 2023. URL http: //arxiv. org. abs/ www.a////... 2312. 03664. arXiv:2312.03664 [cs].
Y. Wan, G. Pu, J. Sun, A. Garimella, K.-W. Chang, and N. Peng. "Kelly is Warm Person, Joseph a Role Model": Gender Biases in LLM-Generated Reference Letters, Dec. 2023. URL http:/ /arviv org/abs////eoooooorrt...g/...ebr
09219. arXiv:2310.092 [es].
L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint 1rXiv:2112.04359, 2021.
D. Weiss.
the Next Step https://techstrong ai/aticces/ generative ----aai--sseeee next- -ene---indmmccriiiinnggg /iooooee...inngkkoooeeeeg///// [Ac- cessed 22-05-2024).
Generative
AI
Democra-
tizing Knowledge.
A. Whiten, R. A. Hinde, Laland, and C. B. Stringer. Culture evolves. Philosophical Transactions of the Royal Society B: Biological Sciences, 366 366(1567):938-9))) 948, Apr. 2011. doi: 10.1098/rstb. 2010.0372. URL https: / woyalsoccooooooooooooooo eettypulliseee 2010.
Publisher: Royal Society.
T. Wiecki, R. Vieira, J. Salvatier, M. Kochurov, A. Patil, M. Osthege, B. T. Willard, B. Engels, O. Martin, C. Carroll, A. Seyboldt, A. Rochford, Paz, rpgoldman, Meyer, P. Coyle, O. Abril-Pla, V. Andreani, M. E. Gorelli, R. Kumar, J. Lao, A. Andorra, T. Yoshioka, G. Ho, T. Kluyver, K. Beauchamp, D. Pananos, Spaak, and B. Edwards. pymc-devs/pyme: v3.11.6,
May 2024. URL https:/ /tt::///e/oo.oo/ org/ records 11402184.
T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Huggingface's 's transformers: State-of-the-are aatt natural language processing. arXiv preprini 191::091,,,3771, 2019.
B. Xiao, Z. Yin, and Z. Shan. Simulating Public Administration Crisis: Novel Generative Agent- Based Simulation System to Lower Technology Barriers in Social Science Research, Nov. 2023. URL http::t///// //apii///arxvvv org/ aww.////33/2 06957. arXiv:231 TtV::3111.6695 1.06957
[cs].
Z. Xie, T. Cohn, and H. Lau. The Next Chapter: A Study of Large Language Models in Storytelling. July 2023, URL http: arwivv org/ abs 2201 09790. arXiv:2301.09790 [es].
Z. Zhao, S. Song, B. Duah, J. Macbeth, S. Carter, M. P. Van, N. S. Bravo, M. Klenk, K. Sick, and A. L. S. Filipowicz. More human than human: LLMMggeneraed narratives outperform human- LLM interleaved narratives. In Proceedings of the 15th Conference on Creativity and Cogni- tion, pages 368-370, New York, NY, USA, June 2023. Association for Computing Machinery. ISBN 9798400701801 doi: 10.1145/35911966355999 3591196. 3596612296221
URL https: /doi wrg//0..1111........ 1145/
X. Zhou, H. Zhu, L. Mathur, R. Zhang, H. Yu, Z. Qi. L.-P. Morency, Y. Bisk, D. Fried, G. Neubig, and M. Sap. SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents, Mar. 2024. URL http: wrw. abs /2310 0310... 11667. arXiv:2310.11667.
J.-Q. Zhu and T. L. Griffiths. Eliciting the Priors of Large Language Models using Iterated In-Context Learning, June 2024. URL http: / /aaxiiivii.////aa///aaa/aaaaaava org [es].
htrg/abb.. abs//b////// 2406. 01860. arXiv:2406.01
17

---

Published as a conference paper at ICLR 2025
A ADDITIONAL DETAILS ON THE METHODS
Selecting initial texts We extracted 5 scientific abstracts ?, 10 news articles 6 and social media comments' from online datasets as initial texts. To ensure that those initial texts covered the range of text properties we were interested in, we proceeded as follows: for difficulty, we measured the maximal and minimal difficulty dmin and dmaz of texts from the scientific abstracts datasets, defined a linear space of 5 values (d:) (::) between dmii and denan: and sampled 5 texts, each having a value of difficulty close to (d:) We then followed the same procedure for toxicity, using the dataset of social media comments; for positivity, using the dataset of news articles; length, using the dataset of news articles.
Pre-processing outputs Data analyses revealed that, on the Continue task, when using Mistral- 7B, agents of the chains would sometimes start outputting very long text by filling them with "#some_ _eywor__" As this behavior created few outliers, we thought it would better to filter- out those @rrome_kewwrdd when performing the main analyses. This behavior is nevertheless an interesting result, reminiscent of the collapsing dynamics found when training LLMs on their own output (Shumailov al., 2023). We therefore discuss it separately in Appendix B.
We use the following hyperparameters for generations in all models. Temper- ature was set to 0.8 with and top_p to 0.95. All models, except GPT3.5, 6 precision was used.
Hyperparameters
Computational resources Experiments were conducted with the OpenAI API (less than 5 million tokens), and with a cluster equipped with A100, and V100 GPU graphic cards. Running the final experiments for the four models, which were run on the cluster, required less than 3000 GPU hours. Experiments with L8B and Mistral-7B were conducted on V100 NVIDIA GPUs with 32GB of VRAM, and experiments with Llama3-700 and Mixtral-8x7B on two A100 NVIDIA GPUs with 80GB VRAM parallel.
Environmental footprint Taking heat recovery in account, running experiments consumed about 777kWh, which represents about 15.54 kg of CO2.
Prompts used In our experiments, each task was induced by specific instruction (prompt), which is given each agent in the chain. For the Rephrase task, the instruction is: "You will receive text. Your task is to rephrase this text without modifying its meaning. Just output your new text, nothing else. Here is the text:", for the Inspiration task, the instruction is: "You will receive text. Your task is to create a new original text by taking inspiration from this text. Just output your new text, nothing else. Here the text:", and for the Continue task, the instruction is: "You will receive a text. Your task is to continue this text. Just output your new text, nothing else. Here is the text:".
Examples of stories Here we provide examples of stories that were given as input and stories that were generated in the last iteration of some chains. Table shows one example for each task. Complete data can be found on the companion website using the Data Explorer tool.
Measuring text properties
Toxicity. We assess the level of toxicity in generated texts using the Detoxify library, a classifier developed for the Jigsaw Toxic Comment Classification Challenges (see https: / //gtthb.. com/ uni aunitarye taryai sesyyai/eetooo detooiiiiiiiiiooooooooooio/ fy /ttttt/t reee /master..... This classifier defines toxicity as the presence of rude, disrespectful, or unreasonable language in a text and assigns a probability score ranging from 0.0 (benign and non-toxic) to 1.0 (highly likely to be toxic). Trained large dataset of human- labele comments from various online platforms, the
hhtps://uuggingffcccoo//ii http:://www.aaaaaaeo/o/////aaaaa
wain-deduullictted..wv hhtps://sits..oogggee
18

---

Published as a conference paper at ICLR 2025
classifier use transformer- based architecture analyze the text's context and meaning, identifying patterns indicative of toxicity.
Positivity. We employ the Sent iment Iww.nsttttnnnnaynin....... yzer tool from the NLTK library to assess the positivity of generated texts. The tool is based on the Valence Aware Dictionary and sEntiment Reasoner (VADER) method (Hutto and Gilbert, 2014), which is lexicon and rule-based sentiment analysis tool specifically designed for social media data. It uses a combination lcxical features, such as words and their semantic orientation, determine the overall sentiment of text. In the VADER method, every word in the vocabulary is rated with respect to its positive or negative sentiment and the intensity that sentiment. The Sent iment Int enseessii yAna yynallzer uses this information calculate sentiment score for the text, ranging from -1.0 (highly negative) to 1.0 (highly positíve).
Difficulty. We estimate the difficulty of generated texts using the Gunning-Foe index. In the method, complex words defined as those with three more syllables, excluding proper nouns, familiar jargon (and removing common suffixes as syllables). We used textstat to compute it.
19

---

Published as a conference paper at ICLR 2025
Table 1: Examples of starting stories and the corresponding final stories for chains with GPT3.5 on the three tasks
Description
Story
Starting story
You are certainly no friend of mine. You have accused me of lying about what you had said, and run away when I proved by actual quotation that the accusation was false. We both know that, even though you praise humility, you are not humble enough to apologize. You sneered at my combat experience, saying "A lot of so called "combat veterans" of Vietnam smelled more perfume from call girls than they smelled cordite" and "So when you in Vietnam (even though you spent your time inside of Danang airbase, with least two battalions of infantry marines on the wire) what was the combat load for M-16 ammo per soldier on a daily patrol?" An actual friend would not have written such things. No, you are friend of mine, you ignorant, arrogant, hypocritical blow-hard. You should be able to guess that I do not feel friendly towards you
Army officer
Final story (Continue)
Your laughter and light will always be a beacon of hope in our darkest days. Your presence may be gone, but your spirit will forever be alive in hearts. We will carry on your legacy of kindness and love, spreading your positivity to all those we encounter. Your memory will never fade, as hold you close in our thoughts and actions. Rest easy, knowing that you are surrounded by our love and gratitude. You may be gone, but you will never be forgotten. Until we meet again, dear friend, know that you are always with us.
Final story (In- spiration)
Amidst the chaos of our busy lives, is essential to pause and prioritize forming genuine connections with those around us. By being fully present and showing empathy, we can cultivate relationships that deep and fulfilling- Authen- ticity is the key to creating bonds that bring true joy and contentment into our lives. Engaging open and sincere dialogues, sharing our vulnerabilities, and striving to forge meaningful connections are all essential in nurturing authentic relationships. Let's come together to create community where empathy and mutual respect are the guiding principles, and where every interaction is infused with love and gratitude. These authentic connections serve as support system rooted in compassion and generosity, uplifting and inspiring each other we navigate the ups and downs life. Together, we can weave network of rela- tionships that exude positivity and light, enriching our lives in meaningful and transformative ways.
Final story (Rephrase)
I need to end our friendship since you won't apologize for accusing me of dishonesty, ignoring evidence, and making disrespectful comments about my time in Vietnam. Your behavior shows a lack of empathy, arrogance, and insincerity, qualities that are not in line with those of a true friend.
20

---

Published as a conference paper at ICLR 2025
Positvity
1.20
Altractor Postton
Attractor Stringth
0.2
Predicied cled velkee alter tiree cheirss (150 generalione)
arter generation)
0.00
Pred cied va arter chain generations)
-0.75
-0.50
-2.25
0.00
0.25
0.50
0755
In rolce
Figure S1: Method for estimating aftractor strength and position This figures depicts the method introduced in Section 3.4 to estimate the strength and position of theoretical attractors. Each dot in this figure corresponds to one chain, for a total of 100 chains (20 initial texts 5 seeds). The position of dot the x-axis corresponds to the value of the property (positivity in this example) in the initial text, while the position on the y-axis corresponds to the value of this property the text produced after 50 generations. We then used these 100 data points fit linear regression predicting the relationship between the initial and final values of the property.
21

---

Published as a conference paper at ICLR 2025
B ADDITIONAL FIGURES AND ANALYSES
B.1 FITTED LINEAR REGRESSIONS Takl
Contron
Attractor Posttinn
-23 53
partivis
23
12
53
patiiis
11
10
1.3 1945
1.00
4383
4303
2000
Figure S2: Fitted linear regressions used to compute attractors strength and position. For three tasks (Rephrase, Take Inspiration, Continue), five models, and three metrics (loxicity, positivity, difficulty, length), we plot (Mean + SE) the relationship between the metric value of the initial human-written text (input to the first agent) and the value of the final LLM-generated text (output of the last agent). A slope close zero indicates strong attraction, while the value the intersection with the diagonal captures the position of the attractor.
22

---

Published as a conference paper at ICLR 2025
B.2 ROBUSTNESS CHECK AND CONTROLS
B.2.1 INCREASING THE SIZE OF THE SET OF INITIAL TEXTS
To verify that the results presented in the main text hold when using larger set of initial texts, we ran additional experiments with Meta-Llama sampling procedure as for the main experiment (see Section A). We observe that the trends remain same with 100 initial texts as with 20 initial texts, and that the standard deviation intervals for 100 initial texts include (with only one exception out 24 values (Figure S3)
using 100 different initial texts, using the same
very close to including the values estimated with 20 initial texts
Toxicity
Positivity
Difficulty
Attractor Position
0077
R.000:
0.000:
039
0.002
200
00
Attractor Strength
0.4-
0.2
6.2
TEaft
Taak
Teat
Tark Oef
Figure S3: Attractor strenghts and positions over 100 initial texts Attractor position (first row) and strength (second row) for four properties (column) and three tasks (colors). The height of the bars are the position and strength of the attractors for set of 100 initial texts. The red crosses indicate the values estimated in the main experiment using reveals that our results are robust to increasing the sample size of the initial texts dataset.
20 initial texts (Section 4.3). This
23

---

Published as a conference paper at ICLR 2025
B.2.2 DIFFERENT PHRASING OF THE INSTRUCTIONS PROMPTS
To verify that the results presented the main text hold when using different phrasings of the initial prompts, we ran additional experiments with Llama-3-8D using different phrasings of instructions prompts for each of the three. Those exact prompts are as follows:
Rephrase
"You will be given a text. Your job is to reword this text without changing its meaning. Only provide your revised text, nothing else. Here is the text:"
"A text will be provided you. Your task is to rephrase it, keeping its meaning intact. Only output the rephrased text, and nothing additional. Here is the text:"
"You'l Il receive text, and your job is to rephrase without altering its meaning. Just output your new version, nothing more. Here is the text:"
"A text will be sent to you. Your role is to rephrase it while keeping the meaning the same. Only display your new text, with nothing extra. Here is the text:"
"You will be provided with text. Your task is reword it without changing the intended meaning. Output just your rephrased text, nothing else. Here is the text:"
Inspiration
"You'll be given text, and your task is to craft new, original text inspired by it. Only provide your new version, with nothing additional. Here the text:"
"A text will be provided to you. Your task to create an original text based on this inspiration. Just output your new version, nothing extra. Here is the text:"
"You will receive text. Your job is to generate an original text inspired by it. Only show your new version, without adding anything else. Here is the text:"
"You"l 'll receive text, and your task is create a new text inspired by it. Simply display your new version, with no additional output. Here is the text:"
"You will sent text. Your role is to produce an original text inspired by it. Only present your text, nothing more. Here is the text:"
Continue
"You'l ll be provided with a text. Your job to extend this text. Only output your continuation, nothing additional. Here is the text:"
AA text will be given to you. Your task is to carry from this text. Just display your new continuation, with nothing extra. Here is the text:"
"You'l 'Il receive text. Your role continue from where this text ends. Only present your extension, with no additional output. Here is the text:"
"A text will be sent to you, and your job is to complete it by continuing from its end. Only output your continuation, nothing more. Here is the text:"
"You will get text. Your task is to extend it further. Just display your continuation, without adding anything else. Here the text:"
This revealed that our results are robust different paraphrasing of the same prompt: indeed, the interval of the standard deviations over the 5 paraphrased prompts always contains or almost contains the values estimated with the original prompt in the main experiment. (Figure $4)
24

---

Published as a conference paper at ICLR 2025
Toxlcity
Positivity
Difficulty
Length
Attractor Poskton
2000
23
9.005
8,002
1,9
1.4
Attractor Strengtt
2.80
0.6
Figure S4: Attractor strenghts and positions for different phrasings of the intruction prompts Attractor position (first row) and strength (second row) for four text properties (column) and three tasks (colors). The height of the bars represent the average position and strength of the attractors the five different phrasings. Individual dots correspond to each different phrasing. The red crosses indicate the values estimated in the main experiment using reveals that our results are robust to increasing the sample size of the initial texts dataset.
20 initial texts (Section 4.3). This
25

---

Published as a conference paper at ICLR 2025
B.2.3 EFFECT OF LESS VERSUS MORE CONSTRAINED TASKS ON ATTRACTORS
In the results from the main experiments, we observed that less constrained tasks (such Continue) lead to stronger attractors than more constrained tasks (such as Rephrase). However, these two tasks vary on other dimensions than only the room for variation allowed. To verify our hypothesis, we conducted additional experiments with Llama-3-8B where we instructed the LLM to create new text by making either "very minor", "a few small", "moderate", "notable" or "radical" changes to the received text.
The exact prompts were as follows:
"You will receive text. Your task is create original text by taking inspiration from this text, making only very minor changes. Just output your new text, nothing else. Here is the text:"
"You will receive text. Your task is create original text by taking inspiration from this text, making a few small changes. Just output your new text, nothing else. Here the text:"
"You will receive text. Your task is to create new original text by taking inspiration from this text, making moderate changes. Just output your new text, nothing else. Here text:"
"You will receive text. Your task is to create new original text by taking inspiration from this text, making notable changes. Just output your new text, nothing else. Here is the text:"
"You will receive text. Your task is create this text, making radical changes. Just output your new text, nothing else. Here is the text:"
original text by taking inspiration from
This experiment confirmed our initial interpretation, as less constrained tasks indeed exhibit stronger attractors (FigureS5). The only exception is Length. We interpret this by the fact that going from "very minor" to "radical" changes mainly refers to the semantic of the text. As consequence, the instructions make "minor changes" is not constrained than to make "radical changes" with respect to Length.
Toxicity
Positivity
Difficulty
Length
Attractor Posttion
1.0
8.005
2.00s-
kam--
8.00
9.2
Attractor Strangth
5A
Figure S5: Effect of less versus constrained tasks on attractors Attractor position (first row) and strength (second row) for four text properties (column) as a function of the room for variation allowed in the instruction. These results confirm our hypothesis that less constrained tasks lead to stronger attractors. The only exception is Length. We interpret this by the fact that going from "very minor" "radical" changes mainly refers to the semantic of the text. As consequence, the instructions to make "minor changes" is not more constrained than to make "radical changes" with respect to Length.
26

---

Published as a conference paper at ICLR 2025
B.2.4 EFFECT HETEROGENEOUS TRANSMISSION CHAINS
As the study of LLMs' multi-turn dynamics remains unexplored, it appeared necessary start with a setting where causal variables can be isolated, so as to lay the foundation for incrementally improving our understanding of LLMs cultural dynamics. As a consequence, the main experiments focused on homogeneous chains where the multi-turn dynamics of different models can be assessed in isolation. However, we also investigate the effect of having heterogeneous chains, where two different models interact (Figure S6). We compared 4 conditions: homogeneous chains of Mistral-7B models, homogeneous chains of Llama-3-8B models, heterogeneous chains of Mistral-7B and Llama-3-8h models starting with Mistral-7B and heterogeneous chains of Mistral-7B and Llama-8B models starting with Llama-3-8B. We found that although the position the attractor in heterogencous chains often in between the position of the homogeneous chains' attractors, it is not systematically the case.
Mtackee MMiin 2 Mi MMtalaemn
Toxicity
Positivity
Difficulty
Length
Attractor Position
0.822
0.800
-0,922
593
Taak
3.4
Attractor Strongth
0,6
6:11
Teal:
Teaic
Figure S6: Comparison of homogeneous and heterogeneous transmission chains Attractor position (first row) and strength (second row) for four text properties (column) and three different tasks, for homogeneous chains of Llama-3-8F models (blue), homogeneous chains of Mistral-7B models (red), heterogeneous chains of Mistral-7B and Llama-3-88 models starting with Mistral-7B (dark purple) and heterogeneous chains of Mistral-7B and Llama-8B 8BB models starting with Llama-3-8B (light purple).
27

---

Published as a conference paper at ICLR 2025
B.3 STATISTICAL MODELS
B.3.1 KOLMOGOROV-SMIRNOD TESTS
To quantitatively evaluate whether multi-turn transmission lead to significantly different outcomes than single-turn transmission, we use Kolmogorov- Saarrov (KS) tests (Massey, 1951) estimate the compare distributions of text properties after single interaction and after multiple interactions. Although we calculate p-values repeatedly (once for each generation, model and task), in our cases applying corrections to control for the False Discovery Rate (FDR) was in our case necessary. Indeed, controlling for the FDR using methods such as Bonferroni correction is necessary in cases where many statistical tests conducted, as it increases the likelihood that at least one of them found significant "by chance" (Family-Wii Wise Error Rate; Colas, 2022). With respect to our set- up, this would have been necessary if we concluded that multi-turn interactions lead to significant differences when at least one of the 49 p-values (one by generation) was below the significance threshold. However, over all generations, rather than from single p- p-values. Therefore, if we observe a general trend of decreasing p- p-values, we can conclude that observing the given distribution under the null hypothesis becomes less and less likely with generations. FDR corrections are thus unnecessary for drawing this conclusion.
rather report all p-values, and draw conclusions from the trend observed
B.3.2 BAYESIAN MODELS
We performed statistical analyses using the Python package pyme (Wiecki al., 2024) fit Bayesian models.
Model We fitted a model predicting the attractor strength (Figure 4) as a function of the Task, Model and Property: Strength (N,, o²)
where μ TTask + Model + Proppetty
Priors for parameters a, and e were standard normal distribution, and standard half-normal distribution for
Model 2 For each Property, we fitted model predicting the attractor position (Figure 4) as function of the Task and Model: Positionproperty ~ ~N(,, o²)
where = dTask + Model
To determine the significance of the difference between estimated parameters, we computed the 95% credibility intervals of the difference by sampling from the posteriors. In Tables 4 to 12, we provide those credibility intervals. Intervals that do not contain signal statistically significant differences.
rephrase
inspiration
continue
rephrase
[0.0000 0.0000 0.0000]
[-0.2372 -0.0353]
[-0.4187; -0.2195]
inspiration
[0.0353 0.2372]
[0.0000 ; 0.0000]
|0.2824: -0.0840]
continue
[0.2195; ;0995;0.400 0.4187]
[0.0840 0.2824]
[0.0000 0.0000]
Table 2: 95% Credible Intervals for posterior differences between prompts for attractor strength
gr S-arhoa
Monel Lime T-B Teomac
Mirra Ta Foemac
Man-T Sottaa--------mmm Toonar
Micrra foamar
251-40-amm
60342)
35
01031
1000000
100000 02350
100256 3057
60 1259
Miiaa 11 Irenssit
ATDET 1029
PU38057 1012
102000 600000)
C47855 111 21
41298T (101997
Mer- wesisaaaoaoa 100 mi--mmm
20 m6)
Miana 8x78 hutnet-01 I
10 00069: 10350
14.148 13835
E4.1206 (0.667)
140277: 0.2561
[004300 (0303]
100000 1000000000000 (0000))
Table 3: 95% Credible Intervals for posterior differences between model for attractor strength
28

---

Published as a conference paper at ICLR 2025
toxicity
positivity
difficulty
length
toxicity
10.0000 0.0000]
10.1345 0.3626]
[0.1274; 202274:033500 0.3584]
[0.2422 ; 0,4747]
positivity
-00.3626 -0.1345
[0.0000 0.0000]
[-0.1201 0.1094]
[-0.0054; ;022054 0.2273]
difficulty
[-0.3584 -0.1174)
[-0.1094 0.1201]
[0.0000 0.00000
[-0.0014 0.2330]
length
[-0.4747 ; -0.2422]
[-0.2273 0.0054]
[-0.2330; 0.0014]
[0.0000: 0.0000]
Table 4: 95% Credible Intervals for posterior differences between measures for attractor strengtt
SDC-33 25
Mai Llano FEB Tep0uct
Misval Trotag
Mews ME TelDox
Misual 5X7B ao
1400012
1-0001: 60006]
10003 (0007))
5000 (0000)
Mitnn Tronaenet
10005
11000
10005 1U00EE
100000 (0000)
(10050 1100
Mera- Jame 1-20 lisine
101017 (10677
240000 (1891)
100014 60006)
100000000000 (100
[40007 (11777))
100070000001
1400EE (01079)
190005 10000]
101075 (1097)
Table 5: 95% Credible Intervals for posterior differences between model for attractor position toxicity
rephrase
inspiration
continue
rephrase inspiration
[0.0000: 0.0000]
[-0.0031 0.0000]
[-0.0040; -0.0009]
[-0.0000 ; 0.0031]
[0.0000 ; 0.0000]
[-0.0025 ; 0.0006]
continue
[0.0009: 0.0040]
[-0.0006; 0.0025]
|0.0000 0.0000]
Table 6: 95% Credible Intervals for posterior differences between prompt for attractor position toxicity
Met Elara
100000 300000]
H00002 6.3559
102023 019059)
F0:88T (03045]
100122 631335
E-012200 (.3673)
pii S 25
Fa 1999 (00962)
(000000000000
100521 20 S468
502771 2128
Motn Limed Iranut
40329 0.200
A3A00 400-31
10000 01000
M0410 (TETT
004723 002259
003813 AU12TT
Matral-B bsenss-0022
103009 (.1857)
HALS0:
858471 0064000
100000
6-01860 01.310)
Minaal No lo An
10300 6791
(0000-0000
Table 7: 95% Credible Intervals for posterior differences between model for attractor position positivity
rephrase
inspiration
continue
rephrase inspiration
[0.0000 0.0000]
[-0.4662 -0.1133]
[-0.4465 -0.0938]
[0.1133 0.4662]
[0.0000 0.0000]
continue
[0.0938 ; 0.4465]
[-0.1913; 0.1570]
[0.0000 0.0000]
Table 8: 95% Credible Intervals for posterior differences between prompt for attractor position positivity
50 mi
SR.355 00
[2 200000
00000
(3149) 22755
12700 2,207)
1350 2014
13999 1001
Main Limn- 18 Iearact
1- 424990222 299
222098 1190
100000 000000
F23699 10565
227095 222222000 7993
12.4007: 3041
Miard 15 To
P2 7338 00982.00 2011
AAA0Y MER
E3700 2.906
100000 00200)
[30174 72 1991
( 2 TMM
Mes-Liama 3-208 --08-imimat
124137 2203)
1222004: (053)
(4.292227115)
1-22905: 30024)
100000
14
CTATT (24100)
125100 208999
[301 2039
Table 9: 95% Credible Intervals for posterior differences between model for attractor position difficulty
rephrase
inspiration
continue
rephrase inspiration continue
[0.0000
0.0000
1-3.0072; 2.27771
1.265556;2.6528
1-2.2777
[0.0000 : 0.0000]
[-2.2601 2.9840] [0.0000 0.0000]
[-2.6528 2.6556]
[22.9840 2.2601]
Table 10: 95% Credible Intervals for posterior differences between prompt for attractor position difficulty
29

---

Published as a conference paper at ICLR 2025
80-16-m0
78 To 200
18755
000000
22733 23133
225801 2.5990
2.407
20993
Fin-Trena:
NON 11.95
PXRTE 7700
1000011000000
5X0000
-T170 72 100
Miird 15 PCl
13 13801.22 1059
2305 29833
11508 300399
100000 000000
33 20088
2 7300
2730672.00 72300
222000 1100
12808 2911
00000000000
DASI 9891
P27388 20700
P1070 (013)
Table 11: 95% Credible Intervals for posterior differences between model for attractor position length
rephrase
inspiration
continue
rephrase
[0.0000 0.0000]
[44.1791 1.3268)
;
inspiration
[-1.3268 4.1791]
[0.0000 0.0000]
[11.7409 3.8265]
continue
[-2.4195 ; 3.1691]
[-3.8265 1.7409]
[0.0000 0.00000
Table 12: 95% Credible Intervals for posterior differences between prompt for attractor position length
30

---

Published as a conference paper at ICLR 2025
B.4 DISCONTINUITIES AND COLLAPSING BEHAVIOR
In the main text, the experiment with the Mistral-7B model on the Continue task was analyzed by first filtering the hashtags, discussed in Appendix A. Given, that this behavior is interesting in itself, we discuss it here in more details.
Figure S7 shows the average length of text generated with the Mistral-7B model chain on the Continue task for five different seeds of the same story. We can observe several discontinuities in terms of the generated text length, i.c. at some iterations the length drastically increases decreases. It is interesting to note that when the length decreases, it returns to the original value as before the first discontinuity. This suggests the existence of an attractor regarding this specific length. To better understand the cause of these discontinuities, Figure 13 shows examples of stories generated before and after those discontinuities (for the seed number three in figure S7). We can see that at generation 14 the model abruptly starts to generate many hashtags. It generates 283 hashtags, compared to 12 in the previous generation. At generation 45, we can see that the overall quality of the text decreased into generating solely hashtags and brief descriptions. This reduction in text quality is reminiscent of collapsing dynamics observed in iterative chains of LLMs, where each model was trained on the output of previous one (Shumailov al., 2023).
12000
10000
4000
2000
10
20
30
40
50
Goncration
Figure S7: Discontinuities and collapse the Mistral-7B model chain The lengths of generated stories shown (without filtering out the hashtags) for five chains starting with the same initial story. We observe discontinuitics, where the length drastically increases or decreases. After decreasing, the length of the story goes back the original length, suggesting the existence of an attractor.
31

---

Ensuring everyone has equal access to Published as a conference paper at ICLR 2025
Table 13: Examples of stories demonstrating the collapse and recovery in Mistral-7B chains the Continue task for (seed 2 from figure S7
Gen.
Story
13
We thrilled about the progress we've made in making Xiangqi more accessible for all. Let's keep the conversation going and explore new ways to adapt the game for different abilities. Share your ideas, experiences, and success stories using the hashtags #CXAIn- clusiveXiangqi #XiangqiForAll #AccessibleXianggi #XiangqiUnited #AdaptiveXiangqi #InclusiveGaming #AccesibleCommmniiy Let's continue learn from each other and create vibrant, inclusive Xiangqi community that celebrates diversity and welcomes everyone. #UnitedWeGame Axccessiblexiaaggioouome # #TogetherWeCan #XianggiForfEveryone
DDisabilityInclusii
14
Let's exchange innovative ideas modifying Xiangqi pieces, boards, and rules accom- modate various disabilities. getherWeAday #Xianggilimpowerrmnt #Accessible #DDiabillitFriendd #BrrirsngBarriers #XianggiForAAlPlayerr #Ac- Sianblleeennngiaaaaaaa #KiungginctusionSSucessSsoo #XiangqiUnitedFoAAI #Ac- DessibleXiangqiFuuur #InclusiveXiangqi Vision
mmellsiveeianggiSStune #To-
lyXiangqi
(omitted 264 hashtags for clarity)
wwwpglinnnniiieCannnnnnnoo Vision CommunityEEnjoweerenn
Growth
EXianggiAccessibleGaming
45
BesggaaWiiinnuuiioossgpphllhsogs Embracing diversity and equality in de-
sign practices.
#Design WinnlDerCCnneeeDDoogg//hPhphhppp Putting users first design decisions and experiences.
#Design Withinccusieeeccccooooeye hwwphhhwwwwwwcccoooooyyyhhhhhhpp Making technology accessible to all users, regardless of abilities.
#Design digital resources and services.
(omitted 176 lines for clarity).
#Design Wieeteeeeetttinn TrainingScaling: Scaling user testing training opportunities. Expanding assistive technology training
#Design opportunities.
# #Design Withlnclusive
49
#Design www.:hiioowwuueeessiiiiaiiiieeee tives and collaborations. (omitted lines for clarity)
Expanding global accessibility initia-
#Design Withinclusiveeesiig//rnnddssede design and accessibility.
Growing trends and innovations in inclusive
#Design
Expanding resources for inclusive design and accessibility knowledge and tools.
32

---

Published as a conference paper at ICLR 2025
B.5 VALIDATION OF ATTRACTORS POSITION AND STRENGTH ESTIMATION
The method introduced in Section 3.4 gives the position and strength of theoretical attractor (or theoretical fixed point). In order validate our method, we verified that this theoretical prediction matches the actual data. To do so, we used the first 10 generations of each simulated chain to predict the strength and position of attractors for each task, model and property. We then compared this prediction with the actual properties of texts obtained after 50 generations. As shown in Figure S8, transmission chains shifts the initial distribution of values in the direction of the predicted attractor. Moreover, the variance of the final distribution appears to reflect the predicted strength of the attractor. These results confirmed that the method we introduce is indeed suited for estimating the strength of position of attractors.
(a)
(b)
(c)
Adfodtrr
Absader stergh
Figure S8: Empirical validation of attractors position and strength estimation. To empirical verify that the method introduced in Section 3.4 makes accurate prediction, we used the first 10 generations of each chain to fit the linear regression between initial and final property values (a). We then used our method to estimate attractors' strength and position (b). We then compared those predictions with the actual shifts in distribution observed after 50 generations (c). The grey represents the initial distribution of the corresponding property, and colored line show the distribution
after 50 generations for each model. Crosses indicate the estimated position of theoretical attractors, and their size represent its strength. For the fourth row, second column, one attractor was outside the range of represented values and is thus represented with "-> X".
33

---

Published as a conference paper at ICLR 2025
B.6 EVOLUTION OF TEXT PROPERTIES FOR ALL INITIAL STORIES
We here provide the figures representing the evolution of each of the four metric for each model, for each of the 20 initial stories. Lines represented the average 5 seeds, and shaded areas represent the standard errors.
Restvene
Centinu
600
Figure S9: Evolution of text properties starting with Initial Text 1
Dv marro arpiaatce
Figure S10: Evolution of text properties starting with Initial Text 2
34

---

Published as a conference paper at ICLR 2025
Figure S11: Evolution of text properties starting with Initial Text 3
Figure S12: Evolution of text properties starting with Initial Text 4
Figure S13: Evolution of text properties starting with Initial Text 5
35

---

Published as a conference paper at ICLR 2025
Figure S14: Evolution of text properties starting with Initial Text 6
Text foe natdes
Figure S15: Evolution of text properties starting with Initial Text 7
Expcubio narric antinn
Figure S16: Evolution of text properties starting with Initial Text 8
36

---

Published as a conference paper at ICLR 2025
Bf
Figure S17: Evolution of text properties starting with Initial Text 9
Comtin
1.000
Figure S18: Evolution of text properties starting with Initial Text 10
East wet oee
19
Figure S19: Evolution of text properties starting with Initial Text 11
37

---

Published as a conference paper at ICLR 2025
Figure S20: Evolution of text properties starting with Initial Text 12
Figure $21: Evolution of text properties starting with Initial Text 13
Sepiouse
Figure S22: Evolution of text properties starting with Initial Text 14
38

---

Published as a conference paper at ICLR 2025
Figure S23: Evolution of text properties starting with Initial Text 15
Exstibio
Figure S24: Evolution of text properties starting with Initial Text 16
Farc
Figure S25: Evolution of text properties starting with Initial Text 17
39

---

Published as a conference paper at ICLR 2025
340
Figure S26: Evolution of text properties starting with Initial Text 18
Text hoe
Figure S27: Evolution of text properties starting with Initial Text 19
Expcubio namiic jan artin
230
Figure S28: Evolution of text properties starting with Initial Text 20
40


Reviews:
[{"review_id": "pJrOQtKEAG", "reviewer": "Reviewer_Kc4v", "summary": "The paper proposes the evaluation of LLMs through doing an iterated task many times, and evaluating the subsequent effects on toxicity, positivity, difficulty, and length. The authors also evaluate the strength and position of attractors (essentially equilibria). The claims of the paper are mainly that more open-ended instructions lead to stronger attraction effects than more constrained tasks, and that some metrics have different sensitivity to attraction compared to others.", "questions": "See weaknesses.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "Originality: Multi-turn interactions in LLMs are an interesting an less studied area in LLM evaluations on qualities such as bias. The telephone game is an easy to grasp concept that allows readers to conceptualize the experiments easily. \n\nQuality: The evaluations are conducted on both open- and closed-source models. The measurement methods of metrics are implemented reasonably. \n\nClarity: Overall the paper is easy to follow. Graphs are colorful and visually appealing. \n\nSignificance: The motivation of the paper, to study the effects of multi-step interactions of models, is meaningful to study.", "weaknesses": "Originality: None\n\nQuality: \n- There is little justification for why \"rephrase\", \"take inspiration from\", and \"continue\" are chosen as the three tasks for the models to iterate on. \n- I don't think one of the main conclusions that the authors claim is supported by their experiments. The authors use these three tasks to make the claim that tasks that are less constrained have higher attractor strength, but the three conditions are difference in many different ways, not just the level of constraint; to accurately test this hypothesis, you would need a more constrained and less constrained version of the same task, such as \"rephrase but start with the same first five words\" versus \"rephrase\". I worry that there are many confounding factors (such as rephrase ping-ponging back and forth across a small set) which could lead to the lower attractor strength that the authors observe. Thus, I would hope for a more rigorous analysis with more task conditions in order to confirm the findings of the paper. \n- Models are outdated, which is especially important to get right for evaluation-type papers in order to gain actionable insights. \n- Sample size is very small (20 initial texts) and the domains that are chosen from are seemingly random (appendix A). This is to the point that I would question the generalizability of the results in the paper. \n- Other findings, such as toxicity quickly converging to 0, is uninteresting due to safety tuning of models. \n\nClarity: \n- Missing the entire literature on Model Collapse which is highly relevant (e.g., https://www.nature.com/articles/s41586-024-07566-y, https://openreview.net/forum?id=5B2K4LRgmz#discussion)\n- Missing papers on bias in LLMs (e.g., https://arxiv.org/abs/2402.04105)\n- Missing a large amount of more prominent works on models and humans displaying similar cognitive biases (line 49-50) e.g., https://arxiv.org/abs/2402.18225, https://arxiv.org/abs/2406.17055, https://www.pnas.org/doi/10.1073/pnas.2218523120\n- Unfinished sentence at line 199-200: \"texts spanned various types of content: scientific abstracts\"\n- Figure and graph text is too small, error bars are not shown.\n\nSignificance: \n- I think a big issue of the paper is that it doesn't motivate the research questions. It is simply stated that multi-turn paradigms (such as scientific reviews and multi-agent environments) are common applications of LLMs, and makes the logical jump that it is necessary to evaluate these models in iterative scenarios, specifically rephrasing, inspiration, and continuation. The issue here is two-fold. First, there is little motivation in the introduction about how real-life tasks (where there are actual implications) can be adequately abstracted into these multi-turn iterative settings without losing key details that make those tasks important. For example, when news articles are continuously rephrased, often this is with human oversight --- why can this step be taken away in the experiments and what is lost from this? There does not seem to be any discussion in this direction. Next, there needs to be some generalizability statement saying that xxx are the most common use cases of LLMs in iterative settings and therefore we do rephrasing and other tasks. Without the motivation from actual use cases, it is hard to find significance in just directly analyzing iterative generation processes in LLMs because somewhat similar paradigms are used in practice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sVJ72WgbDl", "reviewer": "Reviewer_1ZZ9", "summary": "This paper shows that LLMs can amplify small biases through iterative interactions. The authors show that biases, such as toxicity, can become reinforced in transmission chains. Additionally, open-ended instructions generate stronger attraction effects than constrained tasks.", "questions": "- Page 16, line 866: How do the authors justify the generalizability of their findings based on 20 conveniently sampled initial texts? The authors may consider providing more information in the appendix about how these texts were sampled or discussing the limitations of this sampling approach. For example, are these texts a balanced sample? (e.g., do the 5 abstracts cover a variety of scientific topics, do the 10 news articles address diverse issues, and do the 5 social media comments represent a range of user demographics?)\n\n- Page 7, line 358: The authors conduct Kolmogorov–Smirnov (KS) tests and calculate p-values repeatedly to compare property distributions after one interaction and after multiple interactions. Given the multiple comparisons, should the authors consider applying a Bonferroni or FDR correction to control for potential false positives? Please address this by applying the appropriate correction (if applicable) or providing a justification if these adjustments are unnecessary.\n\n- Page 5, line 237: Why did you use the term \"positivity bias\" instead of \"negativity bias\" here? Typically, \"positivity bias\" refers to favoring more positive words (https://www.pnas.org/doi/abs/10.1073/pnas.1411678112). However, in this context, it appears you are referring to a bias toward more negative words.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The research is grounded in a strong foundation of social science theory, specifically cultural attraction theory (CAT). The phenomenon they observe is novel and interesting, and the writing is clear.", "weaknesses": "- The paper does not explicitly provide theories or mechanisms underlying this phenomenon. Why do LLMs exhibit this human-like behavior of transmitting cultural patterns? Do LLMs simply mimic human behavioral patterns present in the training data, or does this behavior emerge due to specific objectives in LLM training? The authors may consider discussing this further or suggesting directions for future research.\n\n- Page 17, line 866: The paper relies on a small sample of initial texts (5 abstracts, 10 news articles, and 5 social media comments from online datasets). I am concerned about whether findings derived from such a small sample can be generalized across a broader range of tasks. The authors may consider including the full set of initial texts in the appendix so that readers can evaluate them directly. The authors may also consider replicating their main findings using paraphrased prompts to demonstrate that their results are not sensitive to prompt design.\n\n- Page 10, line 539: Here, the authors identify the absence of network-based approaches as a limitation and an area for future work. It could be useful to suggest one or two specific network-based strategies to reduce toxic transmission and promote prosocial behaviors (e.g., lower toxicity, kindness). For instance, this paper suggests that interaction within broader social networks might reduce the transmission of toxic behaviors among LLMs: https://arxiv.org/abs/2402.12590.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HEoGSNAzm5", "reviewer": "Reviewer_p7Tc", "summary": "The authors use the storytelling medium of the 'telephone game' as a means of examining how llm text-generations evolve in a multi-turn setting. They introduce methods of analyzing the effect of multi-turn versus single turn conversations on properties such as 'positivity, difficulty, toxicity and length' and demonstrate that some properties rapidly reach saturation, depending on the exact nature of the prompt provided to the llm (ie, continue vs rephrase)", "questions": "1.) How were the tasks selected?\n\n2.) Why were other commercial (Gemini, Claude) and open-source models (Grok, Molmo) not selected for testing?\n\n3.) What was the authors motivation in not including heterogenous transmission chains?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "In my view, the biggest strengths of the paper are in research question and approach. There exists an extremely large body of research in investigating the single-turn capabilities of LLMs but less so in multi-turn settings. For this, the specific aspect of how certain attributes act as 'attractors' in driving the LLMs responses is extremely relevant and the authors methodology for measuring this is well-founded in the literature. The results they show, that certain attributes are (understandably) biased towards and reach saturation fairly quickly, are definitely relevant and an important contribution to the literature.", "weaknesses": "The primary weaknesses of the paper are in the limited number of models and that (to my understanding) each 'conversation' was conducted by the same 'model' (described in the paper as homogenous transmission chains) without an ablation of how a heterogenous transmission chain could shift results. While doing this with a number of agents n> 2 might be excessive, I do think a 2 agent heterogenous system could be reasonable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8u3uNKzn1Z", "reviewer": "Reviewer_uDks", "summary": "The paper “When LLMs Play the Telephone Game” explores how repeated information transitions between language models (LLMs) change the properties of the information over time. In a series of experiments, the authors show that as text moves through multiple LLMs, small biases can build up, shifting content towards stable “attractor” states. They find that factors like model type, task instructions, and tuning settings affect how strongly text properties like toxicity or positivity converge. This study highlights that evaluating LLMs on single outputs misses important dynamics, calling for multi-turn testing to understand LLMs’ behavior in iterative information transitions.", "questions": "For different kinds of tasks, have you tried different formats of the prompts? I would imagine the prompt would change quite a lot for some findings in the paper.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper explores the shift of information properties during multi-turn LLM interactions using a “telephone game” setup. The experiment design is systematic, with some important metrics like toxicity and positivity, effectively showing how text properties shift across iterations. The paper contains a complex analysis, introducing some new methods for evaluating cultural attractors. Its findings are significant, emphasizing that single-turn evaluations miss key dynamics, especially for applications involving iterative or multi-agent LLM use. This work offers insights for improving LLM evaluation and alignment in real-world scenarios.", "weaknesses": "Although I appreciate the authors' attempt at quantifying the \"loss\" of information during multi-turn agent interactions, a major concern is that the setup and tasks (rephrase, continue, and inspire) are not diverse or realistic enough to reflect the real-world use case or issues that people may face for social simulation. For example, the work only explores linear chains, missing out on complex network structures that mimic real-world LLM interactions (as indicated in the limitations). The findings obtained from the work highlight the importance of understanding/evaluating LLMs dynamically have been explored in many other previous works. \nE.g., [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://arxiv.org/abs/2310.11667), [On the Resilience of Multi-Agent Systems with Malicious Agents](https://arxiv.org/abs/2408.00989). I found the paper insightful and learned a few things from it, though I feel it may benefit from further development to fully meet the standards for a conference publication.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)