Generative Monoculture in Large Language Models
Authors: Fan Wu
Emily Black
Varun Chandrasekaran
Venue: ICLR 2025 Poster (2025)
Keywords: monoculture
bias
alignment
Primary area: foundation or frontier models, including LLMs

Abstract: We introduce {\em generative monoculture}, a behavior observed in large language models (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for a given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g., LLMs more often produce efficient code), the dangers are exacerbated in others (e.g., LLMs refuse to share diverse opinions). As LLMs are increasingly used in high-impact settings such as education and web search, careful maintenance of LLM output diversity is essential to ensure a variety of facts and perspectives are preserved over time. We experimentally demonstrate the prevalence of generative monoculture through analysis of book review and code generation tasks, and find that simple countermeasures such as altering sampling or prompting strategies are insufficient to mitigate the behavior. Moreover, our results suggest that the root causes of generative monoculture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.

Published as a conference paper at ICLR 2025
GENERATIVE MONOCULTURE IN LARGE LANGUAGE MODELS
Fan Wu', Emily Black?t, Varun Chandrasekaran"
1 University of Illinois Urbana-Chane -obaaa-Champaign New York University i Equal advising
(fanw6, varunceeillinine edu, emi emilyblackenyu.ed.
ABSTRACT
We introduce generative monoculture, behavior observed in large language mod- els (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g.. LLMs more often produce efficient code), the dangers are exacerbated in others (e.g.. LLMs refuse to share diverse opinions). As LLMs are increasingly used high-impact settings such as education and web search, careful maintenance of LLM output diversity is essen- tial to ensure a variety of facts and perspectives are preserved over time. We exper- imentally demonstrate the prevalence of generative monoculture through analysis book review and code generation tasks, and find that simple countermeasures such altering sampling prompting strategies behavior. Moreover, our results suggest that the root causes of generative mono- culture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.
insufficient to mitigate the
INTRODUCTION
Large language models (LLMs) show promise due to their emergent abilities (Wei et al. 2022ah and state-of-the-art performance on sev- eral NLP tasks (Bommasani et al.) 2023). How- ever, concerns have been raised about the in- creasing reliance on LLM-based systems with insufficient testing, and how they impact soci- ety (Anwar et al.) 2024] Wang et al. [2023). Re- cent evidence has shown that LLMs have dan- gerous tendencies: they convincingly return in- correct information (Dahl et al. 2024| Li et al. 2023a Zhang et al. 2023), produce toxic lan- guage Abid et al. [021]] Wen et al. 2023), and
1.0
10:35.1.00
1.0
108.000
06608
0.8
0.75.0885
0.8
(3406)
[0.35,0775]
0.6
(0.35.0:55]
0.6
[0.0.]]
[0.0.35]
0.4
0.2
0.2
S0C
gen
SFC
gen
Figure 1: (Left)) Comparison of the range average- per-book sentiment scores for book reviews generated by LLM (gen) and by human reviewers from the Goodreads dataset (src). Note the generated reviews have a much smaller range, as they are overwhelmingly positive. Model: L1 amaam-22 2-cchtt. (Right) The spec- trum of the mean pairwise Jaccard similarity among the algorithms of coding solutions. Note the generated code covers a narrower range of algorithms. Model: GPT-4
can effectively propagate misinformation (Bar- man et al. 2024 Sun et al. 2024).
In this paper, we focus on different concern: that for a given prompt and task, LIMs do not faithfully represent the diversity of potential responses available their training data. We call this behavior generative monoculture: given some task (c.g., generating book reviews) and data attribute (c.g. sentiment), generative monoculture refers to the narrowing of the probability distribu- tion of the considered attribute from source data (i.e., available human-wrine written book reviews part of the training data) to the generated data (i.e., A.M--generate book reviews).
As preview, for book reviews (Fig. Left)), compare the diversity in sentiment of Goodreads reviews (Wan et al. 2019) (i.c., src) very likely a portion LLM training data (Achiam et al. with I.M --M-eenerated reviews (i.e., gen). The range of mean sentiment scores per book
2023

---

Published as a conference paper at ICLR 2025
across gen book reviews is much narrower than that in the sre: in the experiment pictured, the average sentiment score for gen reviews are mostly over 0.85, whereas the the average sentiment score for sIc reviews over the same books have wider range from zero to one. For the code generation task (Fig. [ Right)), the range of algorithms employed in (correct) solutions to a given coding problem (i.e., gen) was much less varied than a sample of human answers (src) available on the web (Li et al., 2022): : we show this through the range of Jaccard similarity of the algorithms employed in sets of wwrtten (src) and LLM-generated (gen) responses a coding prompt.
Through the rapid adoption of LLMs such as ChatGPT, CoPilot, and Devin across education, code generation, and day-to-day information gathering, generative monoculture can harm society through loss of information, creativity, and intellectual diversity. For example, students asking LLMs ques- tions about class material to get help researching for an essay may have their opinions formed with- out exposure to a sufficiently wide subset of available information; this will allow for certain opin- ions to die out over time. Concretely, reduction in diversity of sentiment displayed above may lead to the loss of arguments from negative opinions on controversial books, potentially crucial for historical or literary context or a nuanced understanding of a book's 's contributions.
Generative monoculture could even lead to security threats, depending on the application: soft- ware engineers across the globe relying on ChatGPT and CoPilot receiving similar code generations which do not reflect the true diversity of methods to solve a given problem may lead to similar code vulnerabilities across several large tech companies. Indeed, as we preview in Fig. T and show in detail in $ LLM output exhibits are less diverse than human-written solutions in the training data (e.g., by employing a narrower array of algorithms), which could lead to similarity across a wide range of code bases, leading in turn to repeated vulnerabilities (Perry et al.. 2023 Pearce et al.) 2022). However, in this case, LLM outputs not reflecting the full diversity of available coding ex- amples can be positive: LLM outputs over-represent correct and efficient solutions. We present a nuanced picture of generative monoculture: while it can highlight the optimal portion of human- written data for some attributes, its pervasiveness across generation tasks and attributes may cause harm without careful intervention.
In this paper, we (1) define the concept of generative monoculture, and compare it to prior work around related topics ($ 2 and 7); (2) introduce a paradigm for measuring generative monoculture in LLMs ($ 3 (3) show experimental evidence for the prevalence of generative monoculture across variety of application areas (book reviews and code) and (open-source and proprietary) LLMs, and provide some evidence for what may exacerbate generative monoculture, such as alignment tuning (Ouyang et al. 2022), ($ 5 and 6); and (4) show the (in)efficacy of several methods abate generative monoculture: changing temperature, sampling, and prompting techniques ($5)and and 6).
2 DEFINING GENERATIVE MONOCULTURE
We broadly characterize generative monoculture as a distribution shift from source data (i.e., human- written training data) to model generated data (i.c., model outputs) for a specific task, such as gen- erating reviews for books or solutions to coding problems. This can be formalized using measures of statistical dispersion applied to various task-specific attributes.
Definition 1 (Generative Monoculture). For a given task, let Pare denote the probability distri- bution of the source data, Pgen denote the probability distribution of the -.Le-eeeerated data, denote a function extracting attributes from data (such as sentiment, or algorithms used in code), and Dispersion(-) denote a dispersion metric (e.g., entropy). Then we define generative monocul- ture as the condition where Pgen is statistically narrower than Pare. namely: Dispersion(h(2n Pgen) < Dispersion(h(11 |a ~ Paze).
Note, Parel Pgen can be the distribution of human-writtentn wwritten/mooell-geennnin responses, for a given task, conditioned on one specific given prompt (which we refer to as the conditional distribution), or the distribution of human-written written/mooell stapte///modll-geeeaeeeeee responses for any possible prompt in a con- sidered domain (which we call the unconditional distribution).
This phenomenon signifies shift towards less varied outputs. We emphasize that the investigation of generative monoculture is intrinsically task-dependent, as the attributes of interest differ across tasks. In addition, as we often do not have access to the source distribution in practice, we approxi-
2

---

Published as a conference paper at ICLR 2025
mate it using source dataset (Dazc), comprised of subset of the training data of the LLMs. Simi- larly, we approximate the generated distribution through a dataset generated by the model gen)).
Generative Monoculture, Human Preference, and Alignment: Generative monoculture can cause LLMs to over-emphasize numan--referre areas distribution for a certain data attribute; this is often desired behavior. For example, result in having a narrower distribution of code correctness or efficiency biased towards correct, fast, and low-memory code. We conjecture this consequence of alignment procedures such as reinforcement learning with human feedback (RLHF) Ouyang et al. (2022).
demonstrate in 6 generative monoculture can
However, when a tendency stemming from human preference bleeds beyond its intended use- a preference for positive sentiment affecting outputs that need or should not be positive- seemingly advantageous behaviors can prevent an equally important goal: maintaining diversity of opinion and expression. Further, along data attributes which do not have clear "preferred area" of the distribution, generative monoculture can limit the scope of methods, topics, or ideas expressed.
3 MEASURING GENERATIVE MONOCULTURE
We outline general approach to measuring generative monoculture in LLMs. In particular, follow- ing Definition| |] we outline steps to construct Dere and yeen and compare their diversity through extracting data attributes and calculating dispersion metrics. We illustrate our approach in Fig. 2
3.1 DATA CURATION
For given task, we aim to create a source dataset that is likely to have been used in training the LLM we wish to investigate. Training data for most LLMs a closely guarded secret. While recent work (Oren al. 2023) describes how dataset contamination can be determined, such approaches are (a) riddled with false positives, and (b) computationally expensive. Thus, we often take an educated guess (based on dataset popularity and ease of use) in ascertaining if given dataset is likely training dataset candidate.
Formally, define the source dataset Dare fy: sraHemN where (a) q. is problem instance within a task (e.g., name of a book for which a review has to be written), and (b) srC; = {src, Jyre]] is a set of n4 human-written answers to the given prompt &i (e.g., a set of ni of book reviews for that particular book). In practice, we utilize existing datasets likely to be used during LLM training, and perform filtering and sub-sampling to obtain our Dexc.
To create the model-generated dataset D Dgea. for each sample q we prompt the LLM (M) we wish to evaluate, m. times to generate a set of responses, gen; = (gen)) Here, gen? + (P(Puss(gu) (q:, kwargs) is the response obtained in the j-th call of M, where (a) Pask denotes the task-specific formatting prompt that wraps the sample qi, and (b) kwargs denotes the generation keyword arguments (e-g., temperature) that specify the sampling strategy. Across both Dare and D Dyen we select or generate large enough number of responses per qi to ensure variety.
1. Data curation
2. Attribute extraction
3. Metric calculation
responses
corpus
SIG;
queries
the (stce))))e))
Li
Attribute extractor
A
Dispersion metrics
gen:
(ha(gon)) 1
E [N]
Figure 2: An overview of the procedure.
3.2 ATTRIBUTE EXTRACTION
For a given task, we identify and compile a list of attributes that are of interest from the perspective of preserving diversity. This is subjective task, but focus metrics which target understanding the
3

---

Published as a conference paper at ICLR 2025
content of LLM output (e.g., book review sentiment and topie; code time complexity and algorithms used), as opposed to more general metrics of output language quality such as saliency, fluency and coherence. More importantly, we need to ensure that extraction functions reproducible-wwe we outline our tests to ensure these qualities in $ and the Appendices anddd For example, care must be taken to use L.Mss for attribute extraction, as they are known to be biased towards their own responses (Xu et al. 2024). For a given attribute A, extraction function A takes a string tgt to obtain the attribute value ht Note that tgt can either be sre, or gen.- The extracted attribute can either be contmuouus/categorical variable or of other more complicated types, depending on the nature of the attribute.
efficient, accurate, and
3.3 METRIC CALCULATION
As the last step, compute metrics the extracted attributes. Given set of responses, dispersion metries aim to capture their breadth or coverage of these attributes. We describe those used this paper below and in Fig.
Dispersion Metrics. We introduce dispersion metrics suited to different data types.
A. Distribution of the mean. For ordinal or continuous attributes, calculate the mean over the conditional distribution that is, we calculate the metrics over each sre; or gen; (e.g. reviews for given book), and show the distribution of this mean over all & for certain task (e.g. over all books). While the mean itself does not directly measure dispersion, the distribution of the mean values sheds light on dispersion: concentrated mean values indicates smaller dispersion of the data. The advantage is that it not only describes dispersion, but also the qualitative tendency of the attribute (e.g. bias towards positive/negative sentiment), which cannot be captured otherwise.
B. Entropy and standard deviation. For categorical attributes, we measure dispersion using entropy over the conditional distribution. For continuous attributes, we use standard deviation over the con- ditional distribution to quantify the dispersion of values around the mean, providing the variability.
C. Mean pairwise similarity. For attributes that are not easily characterized as categorical or contin- uous, we adopt specific similarity metrics catered to the data type. We then calculate the pairwise similarity values for the conditional distribution-i.. calculate the mean similarity value for all the pairs of elements within each src:/gen, then show the distribution for all N samples. We use:
Mean pairwise Jaceard index: Given two sets of categorical variables and B, the Jaccard index, (A, B) IAnBl/)AUB], measures similarity between them. An example of such set could be set of several algorithms inferred from a piece of code. A higher mean Jaccard index indicates a higher overall similarity between the set, and consequently, lower dispersion.
2. Mean pairwise cosine similarity: Given two collii-dmmesionall embeddings é and e2 obtained via sentence embedder (Reimers & Gurevych 2019), calculate their similarity via cosine similarity, i.e., (c(e1, (2) Ielll A higher mean cosine similarity indicates a higher similarity and lower dispersion.
3. Mean pairwise fingerprint similarity: For tasks related to coding or computer programs, sim- ilarity is based on the overlap of selected hash values (or fingerprints) generated by Winnow- ing (Schleimer et al. 2003). We adopt an existing open-source tool COPYDETECT (Lingenfel- (er). which takes in a set of programs and returns the pairwise similarity scores for all programs in the set. We then calculate the mean value of these pairwise similarity scores as indicator of the similarity for the set of programs. A higher mean fingerprint similarity indicateshigher structural and syntactical similarity of the code.
In addition to the dispersion metrics, we consider one other approach--vsuualizine the top modes of unconditional distributions for certain attributes, c.g., topics. This helps identify areas of emphasis in src and gen distributions, as well as the tendency of change across distributions.
4 MITIGATING GENERATIVE MONOCULTURE
To attempt to mitigate generative monoculture, test four methods known increase LLM output diversity: increasing the temperature T, top-p parameter, setting a temperature decay, and changing
prompts. More details are in Appendices A and C.6
4

---

Published as a conference paper at ICLR 2025
Temperature T. This determines the dispersion of the probability distribution the next token: increasing the temperature leads to a more flat probability distribution and increases the likelihood of sampling from less probable tokens, resulting in more diverse generations.
Top-p. This controls the randomness the generations by limiting the range of tokens considered. Specifically, it considers the smallest subset (consisting of the top probability tokens) whose cumu- lative probability exceeds the threshold p. A smaller p encourages the model to sample from a more focused set of likely tokens.
Decaying Temperature. We choose the starting temperature 7 10.0 and follow a linear schedule for temperature decay, over the course of 50 time-steps (i.e., from the 1-st output token to the 50-th output token), with an ending temperature T = 1.2. The method is inspired by Carlini et al. 2021))
Prompts. Tuning the specific content and framing of the prompt can steer the model's output more effectively (Brown et al. 2020; Sclar et al. 2023) and significantly impact the diversity of the gener- ated text. We use crole-plyyin or impersonation (Salewski al. 2024). which instructs the model to produce the output in the persona of a specific person, and expect it to induce more personalized and varied responses.
5 EXPERIMENTAL SETUP
In this section, we describe our experimental setup for measuring and mitigating generative mono- culture for two tasks, namely, generating book reviews and code solutions. We provide details for datasets, LLMs used, and most notably, the data attribuute and metrics considered. We open source code at htttttt://ggitiiee
com/ MMM/G//MMLLLMMMMMMM
5.1 GENERATING BOOk REVIEWS
Data Curation: For D use the Goodreads dataset (Wan et al. 2019), which contains multiple books with several reviews each. We perform filtering and sampling to ensure reliable attribute extraction (see Appendix [B.1), and craft final dataset of N 742 books with English titles, and Vi, n2 = 10 reviews per book such that the review length is between 300 and 700 words.
To obtain D gon used the following LLMs:
(a) Ll alaam--i -2-33b (Touvron et al.) 2023)
Attribate
Data type
Level
Metric
(henceforth referred to as ama -2). (b)
semiment
categorical
C
mean,entrapt entropy
mean,
1-ama-2-13h chat (Touvron et al. 2023)
review
Book
eategorical
C
entropy
(henceforth referred
Llama chat).
U
distribation visualization
(c) Vicuna-13b-v..5 (Chiang et al., 2023
wording
eategorical
entrogy
(henceforth referred as Vieuna- 133b, (d) GPT-3. 1.3..5-turbo-ii iiarro--nssuuctt
conrectness
categorical
mean
efficiency
cotegorical
C
entropy
(0914) (uyang et al.] (022) (hence- forth referred to as GPT-3.5) and (e) GPT-4-turbo (0125) (Microsoft) 20244 Achiam et al.) 2023) (henceforth referred to as GPT-4). We performed nucleus sampling (Holtzman al.) 2019) with var- ious sampling parameters: (a) temperature T E 0.85, 0.8, 1.0, 1.2, 1.5). and (b) top- E {0.90, 0.95, 0.98, 1.00). We also experi- mented with two candidates for Puski prompt (1) "Write a personalized review of the book titled (title):". and prompt (2) "Write book review for the book titled (title) as if you are (person):"" Prompt (2) was chosen as LLMS are known to generate more diverse responses when instantiated with persona (Salewski et al. 2024). We list the names of the 10 persons we considered in Appendix B.2 For comprehensiveness, we experimented with three more groups of prompts and report the results in Appendix C.7l For each combination of LLM, sampling parameter, and prompt, we independently sampled from the LLM 10 times to generate responses. We filtered out low-quality (generated) reviews by examining their perplexity (see Appendix C.T). This is to
(complexity)
U
distribation visualization
Coding
efficiency (runtime)
contimaous
C
mean, standard devintion
fiagerprint
mean figerprint simillairy
code summary
embedding
C
mean pairwise cosine similarity
(1u8))
code summary (categorical)
categorical
c
mean pairwise Jaccard index
Figure 3: A summary of the scenarios, the attributes consider, their data types, and the corresponding analysis levels as well as metrics. and U stand for conditional and unconditional distributions.
5

---

Published as a conference paper at ICLR 2025
ensure that the data used for analysis represents well-formed and coherent text, thereby improving the reliability of our findings. Thus, Vi, m, I 10.
Attribute Extraction: We want attributes that capture both the semantics and syntax of book views, representative of the key thematic and linguistic clements. Most importantly, while these attributes are not exhaustive, their extraction is reliable and efficient.
1. Sentiment indicates whether a review is positive (praising the book) or negative (criticizing the book). We employ a fine-tuned sentiment classifier (HuggingFace b) as the attribute extractor which accepts text and returns a prediction in {0,1]: 1). This model has been downloaded ~ 5.4 million times, and reaches an accuracy of 91.3 % on the dev set of SST-2 (Socher et al. 2013)
2. Topic refers to the themes discussed in a review (Wallach 2006 Alghamdi & Affalqi 2015
We leverage BERTopic [Grootendorsi] (022) pre-trained on Wikipedia (HuggingFace which assigns one topic to each review out of total of ~2,000 topics.
3. Word choice captures the lcxical diversity in a review. To quamtify this, we produce a frequency table of the unique words (sce Appendix B.3 and immediately have the number of unique words.
For sentiment, we calculate mean and entropy for the conditional distribution. For topic, we calculate entropy for the conditional distribution as well as visualize the unconditional distribution of topics across all reviews, focusing on the top 10 classes. Finally, for word choice, we calculate count and entropy of the unconditional distribution.
Metric Calculation:
5.2 GENERATING CODE SOLUTIONS
Data Curation: For D Derc. chose the CodeContests dataset (Li et al. 2022), competitive programming problem dataset where each problem comes with multiple correct and incorrect solu- subset (N 100) of level-A problems (easy problems) on Code-
tions. We limited the scope forces (CodeForces). and the language of the solutions to python3. More details in appendix[D.]] For each problem in the subset, we randomly sampled Vi, nı = 20 correct solutions from all of the mi solutions for that problem.
To obtain Dgen: we use: (a) GPT-4, and (b) Cl
(Anthropic 2024). We did not use open-source LLMs, as these were not able to generate correct solutions for the problems we chose. More details are in Appendix [E.4] We performed nucleus sampling (Holtz- man et al. 2019) with various sampling parameters: (a) temperature T E (0.5, 1.0). and (b) top-p (0.9, 1.0). We used only one candidate for Pusk i.e., "Please read the below problem description and generate a python code to solve the problem (problem descriptions nothing else." While we experimented with providing the LLM with persona i.e., asking the LLM to pretend to be a "grandmaster in solving competitive programming problems", the resulting accuracy was lower (see Appendix [E.2) For each combination of LLM, sampling parameter, and prompt, we produce Vi, m. 2 20 generations such that at least 20 of the generated solutions were correct (details in Appendix [.2)) We instantiated this by keep generating samples and measuring their correctness, until at some point all problems reached at least 20 correct solutions; we then stopped. This gave us l 100 for GPT-4 and 200 for Claude-
Please only generate code and
Attribute Extraction: We consider the following attributes which characterize different aspects of code. We rely G 5 for extracting some of the attributes; we manually verified the extracted attributes and confirmed their quality is high (see Appendix E.3
1. Correctness refers to whether piece of code correctly solves the given problem and passes all the test cases. We measure accuracy as the ratio of correct solutions among all solutions (details in Appendix D.3). quantify the quality of wowuunmmooolll-eeeeeeee
solutions.
2. Efficiency is crucial for scalability (Huang al. 2024). This is measured through: asymptotic time/space complexity and runtime efficiency. We prompt GPT-3. 5 to infer the big O time and space complexity (MacNeil et al., (022)), and execute the code on test cases measure runtime and memory usage (see Appendix D.4)
3. Fingerprint provides insights into the structural and syntactical uniqueness of each code segment.
As stated in Section 3.3
we use the COPY tool for this.

---

Published as a conference paper at ICLR 2025
4. Code Summary (textual) explains the functionality of the code. Prior work has demonstrated the effectivvness of GPT-3. code understanding (Nam et al. 2024). Thus, we use it to pro- duce text-based summaries, and a description, functionaliet
algorithm and dat structure (prompt for this task is in Appendix D.5). To compare the similarity for these text summaries, produce their embeddings using the a 1-Min 1M-16--22 C).
model (Hugging-
5. Code Summary (categorical) reflects the techniques employed the code through categorical tags, as used the Codeforees website. We prompt GPT-3. 5 assign rags to code segment by providing it a set of tags to choose from (prompt for this task is in Appendix D.5p. We obtain per code segment. We similarly prompt GPT-3. 5 choose from list of gori thms and data structures.
Metric Calculation: For correctness, calculate the mean value i.e., accuracy the ditional distribution. For efficiency (asymptotic complexity), we calculate: (a) entropy for the conditional distribution, and (b) plot the histogram for the unconditional distribution. For runtime efficiency, we calculate mean and standard deviation for the conditional distribution. We measure the following over the conditional distribution:
(a) fingerprints, where we calculate the mean pairwise fingerprint similarity; (b) code summary (textual), where we calculate the mean pairwise cosine similarity in their embedding space; and (c) code summary (categorical), where we calculate the mean pairwise Jaccard index.
RESULTS AND TAKEAWAYS
[0.0.35]
(0.35,0.55)
(0.55.0.75)
(0.75.0.85]
[0.85.0.95]
(0.95,1.00)
(a) Sentiment (varying sampling)
(c) Sentiment (varying models)
1.0
1.0
0.8
(e) Topic (unconditional)
0.6
333d(handmaidd.o noveeli.
0.4
275, (twrones.novelle
15: (gracen,.bb..)
666: (66:ffo oon bestselee.))
0.0
7-12
0.055
1-12
7-3
1-12
0.0
7-
2-3.0
7-13
p-10
(tales forktales..)
(b) Sentiment (varying prompt)
(d) Topic (lemperature decay)
100: (novels.
1.0
549: writee.....
2=1p 3.0
369: 369:enovel.co
S/C
0.8
7=12,p-03
Density
L.
(ratinggrrattd
7=12,p-08
7=12,p 0.98
2208: (inrtntt.
7=12,p=100
0.4
0.0
0.1
0.2
0.2
Percentage
0.0
T-00
D-
7-12
T-12
Entropy
p-09
S-10
Figure 4: (a-c) stacked barplots for the mean sentiment scores under varying sampling parameters, prompts, and models. For these plots, in each bar, darker hues (bottom) represent lower scores while lighter one (top) denote higher scores. See the legend for the value range of each hue. In subfigure (b), (1) and (2) refer to two prompts as introduced in Section [S.] In subfigure (c), (a-c) refer Llama L.lam-2-chatt Vicuna-138 3b, and m1ama-2. (d) kernel density estimation (KDE) for the entropy values calculated on the conditional distribution of the topics. (e) unconditional topic distribution for top-10 topics. For all the subtigures, mark the sampling parameters in them; unless marked with (1-2) (a-c), the results obtained on 1.1 amma-2-cha under prompt
(1). These subfigures show that the model-generated reviews are overwhelmingly positive cover narrower range the topics per book; moreover, there distinctive under- and over-representation of the topics covered overall.
Guide: We present our results on measuring, and attempting to mitigate, generative monoculture in Fig. and [5] We display results mainly in three formats: (a) stacked bar charts, where different hues correspond to different value ranges as indicated in the legend; (b) histograms (or grouped bar charts), reflect the probability mass of categorical variable; and (c) kernel density estimation

---

Published as a conference paper at ICLR 2025
(KDE) plots, to reflect the estimated probability density of continuous variable. We note that, in the code results, for all plots except that evaluating accuracy, we restrict to correct solutions.
Takeaway 1: Monoculture Exists and is Severe, Within and Across LLMs. As shown in Fig. and there exists significant narrowing from the source to generation distribution in all attributes considered for both scenarios, book reviews and coding.
Notably, for book review, proprietary OpenAl LLMs (CPT-3.5 and GPT-4) demonstrate even more severe monoculture compared with the open-source Llama family LLMs (see Fig. Fig.9in Ap- pendix C.3 for more details). Particularly, for both GPT-3.5 and GPT-4, 4, 100% of the samples have average positivity falling in (0.95, 1.00] under prompt (1), and 98.9% and 97.6% under prompt (2). For coding, similar reductions in diversity can be seen: in Fig. e we see increased simi- larity in natural language descriptions of LLM-generated code solutions, and Fig. (1) shows the Jaccard similarity of the generated solutions in terms of the inferred a algor thms, with the ma- jority of problems displaying high similarity across generated solutions. Of particular interest, the plagiarism scores of the LLM-generated code are extremely high (Fig. (b)), compared to the source solutions which achieve an utterly zero plagiarism score for all the problems. We examine a few pairs of examples and their plagiarism scores in Appendix E.
Takeaway 2: LLMs tend to produce Human-favorable Generations. Our results show that LLMs tend to over-represent parts of the attribute distribution that are preferred by humans: hu- mans largely prefer text with positive sentiment (Dodds et al. Augustine et al. [201 Boucher & Osgood 1969) as well correct and efficient code, and researchers have specifically infused these preferences into LLM assistants through preference tuning (e.g. RLHF) (Ouyang et al.] [202]]
Bai et al.) 2022 Roziere al. 2023). Fig. aa) and Appendix C.5 show that LLMs produce over- whelmingly positive generations. Fig. 5 as well as Fig. 22 and 23 in the Appendix reveal that
A.Me-genrrteed code segments (a) are over 2 more accurate than the average human solutions, (b) enjoy an overall lower asymptotic time and space complexity, and (c) use less runtime and memory during execution. This may just be the intended consequence of RLHF, which explicitly optimizes the LLM towards producing human-favored responses in its objective, as guided by reward model trained on human preferences.
However, as our results show, this implies loss of diversity guided by human preferences, which, only naively understood and enforced, could lead to unwanted consequences if going unnoticed. One example of the unintended artifacts is the under- and over-represented topics (Fig. 4e)); the topic group 15 which contains keywords "rob" and "kill" etc. is significantly under-represented, likely consequence of RLHF alignment tuming.
Takeaway 3: RLHF Hurts Diversity the Most. Llama chat is obtained via performing RLHF tuning (Ouyang et al. 2022) the pre-trained (PT) L1a-a22. Similarly, Vicuna-13 3b is obtained via supervised fine-tuning (SFT) on the PT 21ama-2 (Chiang et al., (023). Comparisons on these LLMs (see Fig. cc.. as well as Fig. 8 in Appendix C.4) show that the PT LLL -eneeraed reviews are much more similar to the source. The PT LLM 6lama-2 2 has 5.9% of samples with av- erage sentiment values falling in the range of (0.95,1.00), which is much closer the source percent- age of 2.8% than 44.7% for Vicuna-De - 13b and 82.1% for Llama-2 2ataa-2ccha... Vicuna- 3b also shows better diversity than Llama- 2- thi consistent with findings suggesting RLHF reduces output diversity compared with SFT (albeit with different metrics) (Kirk et al. 2024).
Takeaway 4: Naive Mitigations are Insufficient. Changing the sampling parameter (inereasing T and p) and using a more diversity- diversity-inducing prompt (e.g., prompt (2) for book reviews) can reduce the gap (see Fig. a) and Fig. 5). For example, using prompt (2) reduces the percentage of the most positive range from 82. 1 % to 58. 1 % in Fig. b) for 7 = 1.2, p = 1.0. However, the gap is still large. More results in Appendix (Figures 13 and 14, and Appendix show similar conclusions.
We attempted two other strategies: (a) picking higher temperature, and (b) leveraging decaying temperature scheme (see $ 4.. Results in Appendix C.2 show that the gap still remains big even at such high randomness. Furthermore, for larger T. we notice significant degradation of the generation quality as a result of the increased randomness. In T we present the average fraction of valid generations for L1ama-22 2-chat and Vicuna- 3b under various sampling parameters. The table shows that the valid number of generations rapidly drops as the randomness increases, particularly at T 1.5; the implication is that such high randomness setting cannot be adopted for practical use.
8

---

Published as a conference paper at ICLR 2025
[0.0.2]
[0.2,0.4]
[0.4,0.6]
[0.6,0.8]
[0.8,1.0]
0=0.5, 0.9
T= =
===1.0,==03
==110,p=1.0
(a) Accuracy
(c) Time complexity (unconditional)
(e) Description
0.6
0w111
Density
2
0.4
Otolog m1
0.4
0.6
0.8
1.0
p=1.0
0.0
0.2
0.4
0.6
Mcan Pairwise Similarity
Percentage
(l) Algorithms
(b) Plagiarism score
(d) Time complexity (conditional)
1.0
1.0
0.8
0.8
0.8
Percentage
Percentage
0.6
0.4
0.2
0.2
0.2
0.0
0.00
0.25
0.50
0.75
1.00
0.0
0.4
0.8 Entropy
1.2
1.6
2.0
0.0
5t 0.0.5
p=09
p=1.0
T
T=10
p=0.9
T=
pm1.
Pairwise Similarity
Figure 5: (Left) (a) stacked barplot for accuracy and (b) probability mass along with KDE for plagiarism scores. (Middle) Time complexity: (c) histogram of the (unconditional) distribution of the asymptotic plexity
probability mass for the (conditional) distribution entropy values. (Right) Selected code summary: (e) KDE plot for the mean pairwise cosine similarity seores for "description" as natural language and (ff stacked barplot for the mean pairwise Jaccard scores for "algorithms" as categorical values. Overall, the model-ge model-generated solutions are more accurate and efficient, display higher description similarity to each other, and
span of algorithms. (More results in Appendix E..
Table 1: The average fraction of valid generations (out of total of 10) for two models under various
sampling parameters (lemperature, top-p, and prompt-denoted (1) and (2)). We regard "valid" generation text of perplexity value & 20-as support, we present high perplexity samples in Appendix C.I We observe that the number drops as the randomness increases (along the increase both T and top-p values). As a
reference, GFT-4 achieves an average ratio of 1.000 at T = 1.2 and = 1.0.
I.aama-2
2
0.50
p-
0.95
0.96
p- 1.00
p--
0.95
2- 0.08
p. 1,00
-13b-chat
(1)
(7)
(1)
(2)
(1)
(1)
(2)
(1)
(2)
(1)
(1)
1,000
10000
1.000
1,000
0.5
0.903
0.900
0.944
0.084
1.0
1.000
1.000
0.999
1,000
0.999
1,000
0.998
0.999
1.0
0.951
0.962
0951
0,958
0992
0.943
0.915
0197
1.5
0.91
0.988
0.930
0,883
0,743
0.649
0.512
0.394
1.5
0.871
0.90
0.835
0.876
0.766
0.540
0.650
0.767
Though prior work showed promise for decaying temperature to encourage diversity while main-
taining quality (Carlini et al. 2021), this too failed to achieve higher diversity (see Fig. 4()) and Ap- pendix C.6).
7 RELATED WORK
Diversity and LLMs. Santurkar et (2023) demonstrate that LLMs do not give representative opinions to polling questions when compared to the general U.S. population. Our work focuses on the narrowing of diversity in LLM output from its human-written training data-while Santukar er al. demonstrate a narrowing in diversity from actual human survey respondents (and not training data). Additionally, our work proposes a general framework for measuring monoculture, Padmmku- mar & He (2023) demonstrate that using LLM assistance can lead reduced diversity in human- written argumentative essays when compared to essays written without LLM assistance. While they mention that this is partially because the models themselves do not produce diverse output, they do not focus on the narrowing of diversity from LLM training data to LLM- -LM-genertted data. Finally, Zhang al. (2024) propose approach to fine-tune LLMS to output desired target distri-
nondeulture.Padmakuu

---

Published as a conference paper at ICLR 2025
butions, and [Sorensen et al. (2024) outline an alignment framework to emphasize pluralism to work towards creating models which express a variety of opinions and perspectives. While these are cer- tainly related our work, generative monoculture
phenomenon extends beyond differences in opinion, and expresses the narrowing of any number of task-specific attributes, from code correct- ness to topics covered to many others. One common thread across many of these works, which our work adds to, is that current alignment practices-
RLHF- harms output diversity.
Other Notions of Monoculture. Our notion of generative monoculture relates to, but differs from, other notions of monoculture in the AI literature. For example, algorithmic monoculture (Kleinberg & RRghavann 202T) and outcome homogeneity (Bommasani et al., 2022) describe the societal state where many decision iecision-making actors rely on the same underlying algorithms to generate (classifica- tion or ranking) predictions, from the perspective of cecisionnmmkknng actors and individuals subject to those decisions respectively. These works show that algorithmic monoculture is sub-optimal for both decision-makiin decisions, as repeated outcomes across models leave little room for algorithmic recourse. In contrast, generative monoculture focuses on documenting the phenomenon of individual LLMs narrowing the diversity of their output in relation their source data-for for example, only returning positive book reviews about a controversial book. We do, however, document in this work that generative mono- culture exists to similar extents and in similar directions across variety of available LLMs, (e.g.. Llama, Vicuna, ChatGPT-4) leaving open the possibility of concerns brought up by Kleinberg & Raghavan 2021). but in a generative context.
actors (due to correlated failures across models) and for those subject model
We evaluate models trained on human-curated data, whereas model collapse evaluates models trained iteratively on synthetic data (either fully, or mixed with human data). In settings of model collapse where the model is trained only on synthetic data (Shu- mailov al. 2023] Taori & Hashimoto 2023), it is understandable that the generation quality is low. In contrast, our work shows that the generation quality is good (e.g,, model generates correct coding solutions), but the "diversity" in generations is low. Unlike model collapse which converges to the mean of the distribution (Shumailov et al.) 2023), our observation is there's 's an emphasis on specific part of the distribution which is not necessarily the mean. In this way, the work is tangen- tially related to collapse, but is not special case of it (as the collapse phenomenon necessitates the distribution to match the mean with many rounds).
Connections to Model Collapse:
8 CONCLUSION AND LIMITATIONS
In this work, we introduce the concept of generative monoculture, a phenomenon where LLMs nar- row the diversity of their output relative their source data for given task. We experimentally demonstrate its prevalence across text and code generation tasks, and show the difficulty in miti- gating the behavior. Our work has limitations: first, we did not analyze the full training set of the LLMs we study due to time and compute restrictions, as the corpora are large and often proprietary. Further, as we note in $ 3 measuring monoculture is difficult as selecting attributes subjective, and the attribute extraction process is sensitive to the reliability of extraction techniques. (We ver- ify our own attribute extraction techniques in the appendix). Further, while generative monoculture itself can have unfair consequences by enforcing the suppression of minority opinions, mitigating monoculture without extreme care could lead to the proliferation of harmful ideas or even toxicity by allowing for representation of the entire distribution of source text. We look forward to future work mitigating monoculture while maintaining low levels toxicity and other dangerous behavior.
ACKNOWLEDCEMENTS
This research was supported in part by the Accelerating Foundation Models Research grant from Microsoft.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Abubakar Abid, Maheen Farooqi, and James Zou. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference AI, Ethics, and Society, pp. 298- 306, 2021.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2333.08774 2023.
Rubayyi Alghamdi and Khalid Alfalqi. survey of topic modeling text mining. Int. Adv. Comput, Sci. (ppS, 6(1), 2015.
Anthropic. Claude-3 language model.
2024. Accessed:
2024-04-08.
Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, et al. Foundational challenges in assuring alignment and safety of large language models. arXiv preprint
arXiv:2404.09932 2024.
Adam Augustine, Matthias R Mehl, and Randy Larsen. A positivity bias in written and spo- ken english and its moderation by personality and gender. Social Psychological and Personality Science, 2(5):508- 11008-51,,200 2011.
Stefan Baack. A critical analysis of the largest source for generative ai training data: Common crawl. In The 2024 ACM Conference Fairness, Accountability, and Transparency, pp. 2199- 2208, 2024.
Yuntao Bai. Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0586,, 2022.
Dipto Barman, Ziyi Guo, and Owen Conlan. The dark side of language models: Exploring the potential of iims in multimedia disinformation generation and dissemination. Machine Learning with Applications, pp. 100545, 2024.
Steven Bird and Edward Loper. NLTK: The natural language toolkit. In Proceedings of the ACL Interactive Poster and Demonstration Sessions, pp. 214-217, Barcelona, Spain, July 2004. Asso-
ciation for Computational Linguistics. URL https: htt///alaantttt http:///aaaaatthoooo.o.o ptg/
Rishi Bommasani, Kathleen A Creel, Ananya Kumar, Dan Jurafsky, and Percy S Liang. Picking on the same person: Does algorithmic monoculture lead outcome homogenization? Advances in Neural Information Processing Systems, 35:3663-3678 2022.
Rishi Bommasani, Percy Liang, and Tony Lee. Holistic evaluation of language models. Annals of the New York Academy Sciences, 1525(1):100-146, 2023.
Jerry Boucher and Charles E Osgood. The pollyanna hypothesis. Journal of verbal learning and verbal behavior, 8(1):188, 1969.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901, 2020.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Vors Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 2/), pp- 2633-2650, 2021.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. Vicuma: An open-source chatbot impresssing gpt-4 with 90%* chatgpt quality, March 2023. URL https: 7/1 //mmmss.orr/bbll//02333333 vicicuu

---

Published as a conference paper at ICLR 2025
CodeForces, Codeforces.
URL https://codeeooces.ccoo com/
Accessed: 2024-03-20 at
https://code codefooces........oooooo com/
OpenAl Community. Web chat default temperature for gpt-3.5 and https://commin openai
-a -eeerulll-eeppprttee---roo gpt 1-pt--3--amm 3-3--nd-44 and-4/
1 5 2023. accessed: 2024-04-08.
Matthew Dahl, Varun Magesh, Mirac
Suzgun, and Daniel E. Ho. language models are per- https: wtw.aaa...ttatoo.ddeddeeeeeeeeee a pervasive
Hal-
lucinating law: Legal mistakes with large
vasive,
Jan
2024.
URL
hallucinating 1aw law-legal casggalmmisaaaee -arre--r-e-e-e Language Sanguage--modisee models are
Peter Sheridan Dodds, Eric M Clark, Suma Desu, Morgan R Frank, Andrew J Reagan, Jake Ryland Williams, Lewis Mitchell, Kameron Decker Harris, Isabel M Kloumann, James P Bagrow, et al. Human language reveals universal positivity bias. Proceedings of the national academy of sciences, (12(8)22389-2399 2015.
Maarten Grootendorst. Bertopic: Neural topic modeling with class-based tf-idf procedure. arXiv preprint arXiv:2203.05794 2022.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case neural text degeneration. arXiv preprint arXiv:190..09730 2019.
Dong Huang, Jie M Zhang, Yuhao Qing, and Heming Cui. Effibench: Benchmarking the efficiency of automatically generated code. arXiv preprint arXiv:2402.02037, 2024.
HuggingFace.
Maartengr/berrtpiic wikipppee ho//Maarteedd////Sarpppiiiiiiiis
URL https://huggingfaee Accessed: 2024-03-20 at https:
//ugg_/nffeecco///aacccs///////p wikiippeeiiiiiieiiiii Wikipedia
HuggingFace.
english, b.
URL
httpss//hogggnngaacc.cco//ddddsd Ac-
distilbert base- uncased- finet ota.aaaeeeeiiitttnaaaaeeee2pppp1
cessed:
2024-03-20
HuggingFace.
Beeteenneestaarrroorrrrr//aaalaa Accesedd: 2024-03-20 at https://
Accessed:
Michael Kerrisk. time(1) linux manual page, 2023. URL https://mann.... /inpx
mar: wwwpppaaaaaaaaa... wtw.eeie...... html
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of RLHF on LLM generalisation and diversity. In The Twelfth International Conference on Learning Representations, 2024. URL.
htttps://ppenrr Lew net
Jon Kleinberg and Manish Raghavan. Algorithmic monoculture and social welfare. Proceedings of the National Academy of Sciences, 1 (012))22011400 1340118,1110 2021.
Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian- Yun Nie, and Ji-Rong Wen. HaluEval: A large-scale hal- lucination evaluation benchmark for large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 6449 6464, Singapore, December 2023a. Association for Computational URL
Linguistics. doi: 10.18653/v 20118653/v//22:3 200223.mmllpmaaiie 2023. emn www.nplmaaaammmmmmm mwm.main... 397
org/
Raymond Li. Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Mare Marone, Christopher Akiki, Jia Li. Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprini 230..22305.06161. 2023b.
12

---

Published as a conference paper at ICLR 2025
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d' d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Push- meet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-level code generation with alphacode. Science, 378(66244 2022. doi: 10.1126/science..t abq 158.
URL https: www. www.scccciicggceee.e.eosos http://///s..s..oao...o.o..o///a
wwwp://///cinncc.iiaiccaee abg1158
Bryson Lingenfelter. blingenf/copyydeteet:: Code plagiarism detection tool. URL https:// Accessed: 2024-03-20 at lhttps://githb:
github. Seshuu..coo/blllnngnet/r/yyeeeed com b1 httpb//inngenncooyyeee.ec.
Stephen MacNeil, Andrew Tran, Dan Mogil, Seth Bernstein, Erin Ross, and Ziheng Huang. Gener- ating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Sehearch--oluumee 2, pp. 37-39, 2022.
Microsoft. Openai models azure openai service en en-us www.as/seaaaaaaeeeeeeeeea http://wwww..e.//ea////aeoeoaee mmt//mmoels...mml 2024. accessed: 2024-04.
attps://leara.iicrrer
com/
08.
Daye Nam, Andrew Macvean, Vincent Hcllendoom, Bogdan Vasilescu, and Brad Myers. Using an Ilm to help with code understanding. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE), pp. 881-881 IEEE Computer Society, 2024.
OpenAl. Chat completions openai api. https: http:::/pplatoooo openai wwt.:/cocccooooooeeoo openai.com//ocss. api wweeereeneechat/ceeeeeeeeee
2024. accessed: 2024-04-08.
Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori B Hashimoto. Proving
test set contamination in black box language models. arXiv preprint arXiv:2310.17623, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Vishakh Padmakumar and He He. Does writing with language models reduce content diversity? arXiv preprint arXiv:2009.05/96 2023.
Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Gavitt, and Ramesh Karri. Asleep at the keyboard? assessing the security of github copilot's code contributions. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 754-768. IEEE, 2022.
Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. Do users write insecure code with ai assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 2785-2799, 2023.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 11 2019. URL https: :/appii//arxiv..
orq/ brq/abs/1908 10084
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat. Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. Code llaaa: Open foundation models for code. arXiv preprint arXiv:2308 12950, 2023.
Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. In-context im- personation reveals large language models' strengths and biases. Advances Neural Information Processing Systems, 36, 2024.
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? arXiv preprint arXiv:2303.175 2023.
13

---

Published as a conference paper at ICLR 2025
Saul Schleimer, Daniel S Wilkerson, and Alex Aiken. Winnowing: local algorithms for document fingerprinting. In Proceedings of the 2003 ACM SIGMOD international conference on Manage- ment of data, pp. 76-85, 2003.
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sen- sitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.1 13244, 2023.
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget, 2023.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language pro- cessing, pp. 1631-1642. 2013.
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christo- pher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, et al. A roadmap to pluralistic alignment. arXiv preprint arXiv:2402.050770 2024.
Yanshen Sun, Jianfeng He, Limeng Cui, Shuo Lei, and Chang-Tiee Lu. Exploring the deceptive power of iin-generated fake news: A study of real-world detection challenges. arXiv preprint arXiv:2403. 18249, 2024.
Rohan Taori and Tatsunori Hashimoto. Data feedback loops: Model-driven amplification of dataset biases. In International Conference on Machine Learning, pp. 33883-33920. PMLR, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.0922 09.88888.88,,, 2023.
Hanna M Wallach. Topic modeling: beyond wordss. In Proceedings of the 23rd international conference on Machine learning, pp. 977-984, 2006.
Mengting Wan, Rishabh Misra, Ndapa Nakashole, and Julian McAuley. Fine-grained spoiler detection from large-scale review corpora. In Anna Korhonen, David R. Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Conference of the Association for Computational Lin- guistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. 2605- 2610. Association for Computational Linguistics, 2019. doi: 10.18653/V1/P19- 1248. URL https:://oi..rz/ 1tt.:///......1155////////1/3130
Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. arXiv preprint arXiv:2306.1 1698, 2023.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 2022a.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in 2022b.
neural information processing systems, 25:24824-24882
Jiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang, Chengfei Li. Jinfeng Bai, and Minlic Huang. Un- veiling the implicit toxicity in large language models. In Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing, pp. 1322-1338, 2023.
Wenda Xu. Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, and William Yang Wang, Perils of self-feedback: Self-bias amplifies in large language models. arXiv preprint arXiv:2402. 1436, 2024.
Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, Zico Kolter, and Daphne Ippolito. Forcing diffuse distributions out of language models. arXiv preprint arXiv:2404. 10859, 2024.
14

---

Published as a conference paper at ICLR 2025
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's ' song in the ai ocean: a survey on hallucination in large language models. arXiv preprint
2023.
15

---

Published as a conference paper at ICLR 2025
CONTENTS
Introduction
2
Defining Generative Monoculture
3
Measuring Generative Monoculture
Data Curation
3.2
Attribute Extraction
Metric Calculation
Mitigating Generative Monoculture
5
Experimental Setup
5.1
Generating Book Reviews
5.2
Generating Code Solutions
6
6
Results and Takeaways
7
Related Work
8 Conclusion and Limitations
10
Sampling Parameters
17
Additional Details: Book Reviews
18
B. ] Construction of the Source Dataset
18
B.2 Names of the Celebrities Used in Prompt
18
B.3 Text Processing for Analyzing Wording Choice
18
Additional Results: Book Reviews
18
Filtered-out Reviews
18
Generation Results at Higher Randomness
................................
19
Topic Shifts
19
Pre-trained Model
19
Results of OpenAl models
22
C.6
Mitigation via Temperature Decay
22
Evaluation on Additional Prompts
23
8
More Results
24
D
Additional Details: Coding
27
Restriction to Level-A Problems
27
D.2
Correctness Testing: Autojudge with Testcases
27
16

---

Published as a conference paper at ICLR 2025
D.3
Measuring Accuracy
27
D.4
Measuring Runtime Effccincy
27
D.5
Prompting GPT-3. Generate Code Summary (both Text Descriptions and Cat-
egorical Values)
28
Additional Results: Coding
28
E,
Examples for Plagiarism Scores
28
Attempts on Varying Prompts
28
Human Annotations for the Quality of LLM Summary
................................
31
E,4
Failure Results with Open-Source Models
32
E.S
A Complete Set of Results for the Experiments on GPT
32
E.6
Claude
32
Compute Resources and Data Licenses
32
Compute
32
F2
Licenses
37
G
Extended Investigations
37
Influence of Model Size on Monoculture
37
G.2
Influence of Length Sentiment Scores
38
More Training Data Control & Connections to Monoculture
38
G 4 Evidence of GoodReads in CommonCraw]
39
APPENDIX
We open source our code at hhttps://githu htt.uuu..com/GMM//eeee https/utu...comm/MMe1///////0
A SAMPLING PARAMETERS
1. Temperature: Concretely, the temperature parameter T determines the "dispersion" of the prob- ability distribution over the next token. Mathematically. the probability of a token w being generated exp(s(u[==/TT) is given by the softmax function: P(w (x) Ew exp(e("'x)/7T) where s(wlz) is the unnormalized log-pro prepbbbbbblltyy of the token w given the context x. Increasing the temperature leads a more flat probability distribution and increases the likelihood of sampling from less probable tokens, result- ing more diverse generations. This can also be understood from the perspective of entropy of the next token, H(W)= = - P(w[x) log P(w[x), where it can be seen that the increase of entropy directly follows.
The top-p parameter Holtzman et al. 2019 (p (0, 1)) controls the randomness of the generations by limiting the range of tokens considered. Specifically, considers the smallest subset (consisting of the top probability tokens) whose cumulative probability exceeds the threshold A smaller encourages the model to sample from a more focused set of likely tokens, while a larger p allows sampling from broader range and thus increases randomness (p 1 basically means restriction on the vocabulary).
2. Top-p:
Different platforms have different default values for T and top-p. 107-3.5-turbo-iin (0914) web version adopts T 0.8 Community (202)). The default values in OpenAl APIs are T 1.0 and p 1.0 OpenAA 2024).
17

---

Published as a conference paper at ICLR 2025
3. Generation Length: This parameter (max_new wew.kkknsstkt dictates most how many tokens the model should generate before it stops. We used 500 for book review generation and 2048 for code generation.
B ADDITIONAL DETAILS: BOOK REVIEWS
B.1 CONSTRUCTION THE SOURCE DATASET
To ensure picked popular books that the LLMs know about, filtered the books according to the number of reviews they have. Constraining the number of reviews to be between 1,000 and 2,500, obtained 750 books. We further filtered out books with non-English titles; conducted this filtering because some downstream LLMs (c.g., sentiment classifiers) are not multilingual, and result can not analyze the generated non-English reviews. To ensure high quality of the reviews, we further filtered the review by length, constraining the length of each review to be between 300 and 700 words. After this step of filtering, we sampled 10 reviews per book. Eventually, we obtained a dataset of 742 books, where each book comes with 10 reviews.
B.2 NAMES OF THE CELEBRITIES USED IN PROMPT 2
We prompted GPT-4 to provide a list of celebrities suitable for writing diverse book reviews. The list of names are as follows: Trevor Noah, Janelle Monáe, Yuval Noah Harari, Serena Williams, Reshma Saujani, Neil deGrasse Tyson, Margaret Atwood, David Attenborough, Malala Yousafzai, Jordan Peele.
B.3 TEXT PROCESSING FOR ANALYZING WORDING CHOICE
We first concatenated all documents and converted them to lowercase to standardize the text. We then expanded contractions (c.g., "don't" "do not"). We then tokenize the text with word tokeni i ze () from the NLTK library Bird & Loper (2004), and removed the punctuation. We continued by filtering out non-alphabetic tokens to focus solely on words. Lastly, we lemmatized tokens via WordNet Lemmat commmaiizer () to their base forms, aiding in consistent frequency analysis. These steps are essential for minimizing textual noise and ensuring the reliability of our word fre- quency assessments.
C ADDITIONAL RESULTS: BOOK REVIEWS
C.1 FILTERED-OUT REVIEWS
We established perplexity threshold of 20 to filter out low-quality reviews. To validate our choice, we randomly sampled reviews with perplexity scores at different intervals and manually inspected them. For elarity, instead of presenting the entire review text, selectively extracted chunks that exemplified the low-quality nature of the sampled reviews.
Perplexity in (20, 25]: This page-turning and fast-paced gripping story makes excellent use both tropes and novel ideas by employing both like, instance, dystopian setting which uses an exaggeratory ruminating setting so typically found in series belonging to this genre, only here the author's unique skill at weaving such plot-lines together in a novel that feels fresh more like Harry Potter but with an 800word length page each chapter instead of JUST 200word paragraph the end of an excrutiatingly slow chapter.
Perplexity in (25, 30]: That and artist Adrian Alphona bringing the world-sprawling amazing action and gorgeous characters from beautifully rendered backgrounds in color by vivid and dazzling color is what makes the graphic novel in itself even more delightful. With each color panel bringing an explosion of color onto every page no panel left the same no character or page lacking the same amount of energy vivid and gripping from the moment turned on. the illustrations are bright beautiful detailed it truly does it live upto its tag line and then some hero like other a book like other definitely worth the dive.
18

---

Published as a conference paper at ICLR 2025
Perplexity in (30,35) One would wonder if she lived some of this amazing journey to unhidden holes in the ground or lived the stories and characters by the holes they created one will see how amazingly her fantastic works has a way connect the reader a place which feels real and fascinating! Holes by Loues Erdrich has beautifully constructed a magical and a world which transports and takes readers like on a journey thrifting us so many magical and real places which we might not have an opportunity if it wasn't it for the eyes and minds eye of Louise Erdrech.
Perplexity in (35, 40]: It made me reflect for deeper appreciation both the power of memory in creating some literature and art's most significant works cultural touchstone masterpieces such as those listed that our lead female artist was known to adore and how that and the themes that author a most wonderful story teller by the way, by way of characters that wanted or did want to spend more time with and learn the most in depths exploration of, and not all can reach it. And so deeper appreciation for the of storytelling from great narratives by great narrators? The Little Paris Bookp shop does achieve astounding success that regard, I am thrilled I'd like shout out to any and All ear shot about
C.2 GENERATION RESULTS AT HIGHER RANDOMNESS
We experimented with even higher randomness at T = 1.5 motivated by the observation that in- creasing the randomness does help increase the diversity. However, high randomness, there is still a huge gap between the source and the generations.
show in Fig. 6 even
Moreover, notice significant degradation of the generation quality result of the increased randomness. We present in Table the average number of valid generations for two models under various sampling parameters. The table shows that the valid number of generations rapidly drops as the randomness increases, particularly at T = 1.5; the implication is that such high randomness setting basically cannot be adopted for practical use.
C.3 TOPIC SHIFTS
We present Fig. 7 the results of the unconditional topic distribution. Across all three settings for comparison, we observe highly similar distribution shift signified by the over- and under- representation of certain topic groups. Concretely, the topic group 15 with the keywords "grace, rob, nick, anna, house, discovers, realizes, killed, confronts, killer" are under-representte, poten- tially because certain words like "rob" and "kill" are eliminated from the output as a result of RLHF, On the other hand, the topic group 333 with the keywords "handmaid, novels, novel, writers, tale, book, books, literary, novellas, synopsis" and the topic group 666 with the keywords "nonfiction, bestseller, novelist, autobiography, novels, memoir, author, memoirs, paperback, novel" are over- represented. The over-representation of topic group 333 in Vi cuna 1eee-12 1.bb is much more severe than aml chatt, can be seen from the 3rd subfigure; this could be because higher exposure relevant materials during its fine-tuning.
C.4 PRE-TRAINED MODEL
The aligned model is obtained from performing supervised fine-tuning followed by RLHF alignment tuning on the basis of the pre-trained model Touvron al. (2023). We compare the performance of the aligned model llama-2- 1-3--hhh with the pre-trained model llama-2-131 and present the results in Fig.
From the 1st and 2nd subfigure, we observe that the pre-trained model is much more diverse than the aligned model; the pre-trained model even very close to the source data (see the dark purple result tagged as T 1.2,p 1.0 (c)).
The 3rd subfigure delivers two messages. First of all, there still exists divergence from the pre- trained model and the source in terms of the covered topics. We attribute this divergence we observe between the pre-trained model and the source data to the difference between the source data we use (the Goodreads dataset) and the ground-truth training data (a much broader corpus which we have no information about). We regard the Goodreads dataset as a proxy of the ground-truth, but intrinsically Goodreads is smaller and does not fully accurately represent the groundtruth training distribution, especially in nuanced attributes like the unconditional topic distribution measured on all samples. Second, the direction and trend of changes in the aligned model is not completely the
19

---

Published as a conference paper at ICLR 2025
[0,0.35]
(0.35,0.55]
[0.55,0.75]
[0.75,0.85]
[0.85,0995]
(0.95,1.00)
LJama-2- 2-13b (1)
0.9, varying T
0.95, varying T
0.98, varying T
1.0, varying T
1.0
1.0
1.0
1.0
0.8
0.0
0.0
0.0
0.6
0.6
0.6
0.6
p=00 p=07
p-03 p.03
0- 0.5 p p-035 pm055
p- 0.0 pM 0.35 --030
p-10
2-10 p=10 p-10
T' 0.5, varying P
T
0.8, varying P
T
1.2, varying P
T
1.5, varying P
0.0
7-08
7-13
0.0
0005 p-000
1 0.0 p-032
7-09 p-030 p-032
p=033 p-032
Vicuna-13b (2)
0.9, varying T
p 0.95, varying
p 0.98, varying
1.0, varying
0.0
0.0
0.0
p-03 p-02
p- 6.0 p-
p-000 p-00 p-038p--2
p-10
2010
p-30
0.5, varying P
0.8, varying P
1.2, varying P
1.5, varying P
1.0
X
X
0.4
0.4
0.4
0.
0.2
0.2
0.2
0.0
0-00 F-03
0.0
Tw01
7-08
0.0
7-12
7-
1-32
T=12
0.0
7016
Figure 6: Mean sentiment scores across different models (Llama -2-catt, Vicuna-1333 and prompts ((1) and (2)), and varying sampling parameters (temperature T and top-p). We do observe increase of diversity along the increase of generation randomness (i.c., increasing T and p), but the gap remains high between the source and even the highest generation randomness setting. the top of the setting with concerning low number of valid
Moreover, put black cross number of generations (see Table [ for a concrete explanation); they mostly only occur for the few highest randomness settings.
20

---

Published as a conference paper at ICLR 2025
MC
T-12,pp0.35 -T=12pp0008
-7-12,p-10
-T-08,--3311 -1=08,p=0312
ST-12,p-1.01
--
-T-12,p-1a (a)
-1=
-f=12.p=100)
-1=12.p=10
Varying sampling
Varying prompt
Varying Model
323 handmaid_ novels
novels
15, gracee_ebb
ronfiction
30 tatettlleeaaae folktales
novelists
548 451 writer
64_ rettinggretedd
2208 novelist literature
0.0
0.1
0.2
0.0
0.1 Percentage
0.2
0.0
0.1
0.2
Percentage
Percentage
Figure 7: Grouped bar charts of the unconditional distribution of the topic (i.e., the distribution of the topic over all the samples). (Left):: generated by Ll amaamma- 22-aaa2-chha at different generation kwargs, where fixed T 1.2 and varied p (0.90, 0.95, 0.98, 1.00). The prompt used is (1) as introduced in Section 5.1 (Middle): generated by Llama 2 2-chat under different prompts (1)
and (2). (Right): generated by models (a) Ll alama- 2 -cchat and (b) Vi Vicuna-2 -1bb using the prompt (1). Across all three subfigures, we observe highly similar distribution shift signified by the over- and under-represee redeeseepresnntation of certain topics.
[0.0.35]
[0.35.0.55]
(0.55.0.75)
(0.75.0.85)
[0.85,0.95]
(0.95.1.00)
T- 1,2,p - 0.9 (a)
T-
0.9 (c)
T 122
1.0 (a)
T- 1--1220-1.00
Sentiment
Topic
Topie distribution
1.0
0.8
275 por.ccrramns.cooo novel....
0.8
0.6
15:ggracsrrt. reb.)
0.6
Density
(ronfiction testochor.)
0.4
0.4
24t (451 wither)
0.2
0.2
393 (nowd (cout.)
0.0
G8 Cor motes 2200 (potel [tteaturs))
p=09
p=0.
p=10
1-10
0.0
(8)
(6)
[a]
641
Entropy
Percentage
0.0
0.1
0.2
Figure 8: Comparison between (a) the aligned model Llama-2-chat and (c) the pre-trained model Llama-2, under prompt (1), temperature T 1.2, and varying top-p parameters. We present subset of the results (sentiment, topic, and topic distribution), corresponding Figure 7 The figures show that the pre-trained model enjoys much better diversity than the aligned model, and is much closer the source.
21

---

Published as a conference paper at ICLR 2025
[0]0.35]
[0.35,0.55]
(0.55,0.75]
(0.75,0.85]
[0.85,0.95]
(0.95.1.00)
==1.2.p=0.9
7= 1.2.,
0.95
1=12.0m 0.98
122
1.0
Sentiment
Topic
Topic distribution
0.8
handrmail
205,
GPT-33.-instruct
0.8
0.6
ponfirtar Destooler
0.6
Density
0.4
0.4
0.2
0.2
330, Lovel
0.0
2200 noveller Eowatare
Entropy
Percentage
0.0
0.1
0.2
SrC
GPT-3.5-instruct (1)
OPT-3.5-instruct
GPT-4 (1)
GPT-4 (2)
1.0
0.8
332_hardraakt
GPT comparisons
275,
0.8
0.6
0.6
Density
nonficsar
0.4
takn,
100,
0.2
0.0
novelis
11) (2)
()
0.00
0.05 0.10
0.15
Entropy
Percentage
Figure 9: Plots for OpenAI models. (Upper row): GPT-3.5 under prompt (1), T 1.2, and varing p. (Lower row): comparing GPT-3.5 5 and GPT-4 at T = 1.2, = 1.0 and the two propmts (I) and (2). We present subset of the results (sentiment, topic, and topic distribution), corresponding to Figure The figures show that both GPT-3.5 and GPT-4 display severe GeMo, which cannot be effectively mitigated via varying the sampling parameters or using diversity- inducing prompt.
same as the aligned models, e.g., the topic group 15 containing "rob" and "killer" is not inhibited as much as in the aligned models, potentially due RLHE.
C.5 RESULTS OF OPENAI MODELS
We repeat the same set experiments on the proprietary OpenAl models GPT-3.5-crrbo-nnstnn (0914) and GPT-4-turbo (0125). We present the results in Fig. 9
The sentiment results (in column 1) suggest that both models are overwhelmingly and invariably positive in their reviews, across various sampling parameters and prompts.
The results of the conditional topic distribution (in column 2) reveal similar tonelusions-there is significant deviation between the generated reviews and the source reviews across varying sampling parameters (see upper low) and for both GPT--355-nnstrs and GPT-4 (see lower row). Nevertheless, using a diversity-innuuing prompt, i.c., prompt (2) does lead to increased diversity (see lower row).
The results of the unconditional topic distribution (in column 3) also demonstrate the distribution shift, though in slightly different way than observed for Ilama family models (see presented in Ap- pendix C.3). Concretely, the topic group 333 does not experience a significantly representtainnnn while the topic group 30 does. As discussed previously, we attribute the difference in the nuanced topic distribution mainly to the difference in the training data.
C.6 MITIGATION VIA TEMPERATURE DECAY
We describe the decaying temperature scheme. Concretely, we choose the starting temperature T = 10.0 and follow linear schedule for temperature decay, over the course of 50 timesteps (i.e.,
22

---

Published as a conference paper at ICLR 2025
[0,0.35]
[0.35,0.55]
[0.55,0.75]
[0.75,0.85]
[0.85,0.95]
(0.95,1.00]
SrC
T= 1=1.2,p1.00 (1)
decay T,p 1.0 (1)
-==12pp=100
decay T,p (2)
Sentiment
Topic
-lama-2-chat,1 p 1.0
Vicuna-
1.0
(1ama-2-chat,
1.0
Vicuna-13b; p 1.0
1.0
0.8
0.8
0.8
0.6
0.6
0.8
0.4
0.4
02
0.2
0.2
0.0
T-12
06/94
T-
0.0
40
T-
DACW
7- 3.2
0.0
2
0.0
2
Figure 10: Effect temperature decay during decoding time for two models u-lam-2-chh and cucuma11bb, with top-p parameter 1.0 and two prompts (1) and (2). We compare the fixed temperature (T 1.2) with the decaying temperature (decay 7) and we present the results for sentiment and topic. The figures reveal the ineffectiveness of the temperature decay method.
from the 1-st output token to the 50-th output token), with an ending temperature T = 1.2. The method is inspired by Carlini et al. Carlini al. (2021) which reported this scheme as one sampling method that induce diversity and high quality outpu
We present the results in Fig. TO For both the sentiment and the topic, we see that the fixed temper- ature scheme achieves higher diversity compared to the decaying temperature scheme.
C.7 EVALUATION ADDITIONAL PROMPTS
[0,0.35]
(0.35.0.55)
(0.55,0.75]
(0.75.0.85]
[0.85,0.95]
(0.95,1.00]
1.0
1.0
1.0
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
Te08
Tw Tell
0649 mM0
dralvve
685 180
resli psitee
Figure 11: Stacked barplots for the mean sentiment scores. Left and middle for Llama-2-13b- chat and right for GPT-4. The label "celeb" corresponds Prompt (2) in the main submission; the prompts for other labels can be found below. The all black bar for the label "negative" means that all the generations for this prompt achieve extremely low scores (refer to the huc legend).
Aside from prompts (1) and (2) introduced in the main paper in $ 5.1 with three groups of prompts for comprehensiveness. Group 1 [specifying detailed roles):
we additionally experimented
Prompt (role): "Write book review for the book titled (title) as you are {role}:", where role ("teenage fantasy enthusiast", "critical literature pro- fessor", "romance novel lover", "tech-savvy sci-fi geek", "history buff". "casual weekend reader", "book club moderator". "non-fiction aficionado". "poetry appreciator", "mystery thriller addict")
Prompt (creative): "Write a book review for the book titled {title), from the viewpoint of different personas, such as 'aspiring
We refer
ottp:///ccccccaaaaaaaaaaaaaaaaaa Traininr-inn Baaiing--att--eeee La rge Leg--aagauge--odeeeeeee
blob/mal n/ ract wewppeeeeeeeeeeeeeeeeeeeee rature wetttur__eccy___eee saaccy.pp) py for an implementation of the temperature decay sampling scheme for HuggingFace models.
23

---

Published as a conference paper at ICLR 2025
Llama-2-chat (1)
Llama-2-chae (2)
Vicuna-13b (1)
Vicuna-13b (2)
8.97
8.88
3.98
8.09
9.00
9.01
9.02
9.03
9.02
9.01
9.02
9.03
8.90
8.90
8.92
8.02
9:30
9:15
9.23
9.27
9:14
9.19
9.24
9.32
9:14
9.20
9.27
9.32
9.00
9.11
9.17
9.24
9:24
9.37
9.51
9.69
9.28
9.02
56
9.75
9:27
9.37
9.47
9.56
9.21
9.31
9.39
9.49
p.000
p
p.005
0.00
1100
p=09
p-0.5
p-0.06
p.0.0
p-00
p=005 p
0.36
p=1.0
p=0.3
0.00
p.0.5
p=10
Figure 12: Heatmap of the entropy of the (unconditional) distribution of the word choice in model-generated reviews. We present results of two models (1-lam-2--chtt and Vicuna-133 and two prompts ((I) and (2)). As a reference, the entropy of the source is 9.99, higher than all the entropy values of model generations.
writer', 'history enthusiast', 'teenage sci-fi fan', or 1 "career-focused parent', etc. Be creative!"
Group 2 [Steering attributes such as sentiment]:
Prompt (negative):
"Write a negative review the book titled
(title):"
Prompt (neutral): "Write neutral review of the book titled (title):"
Prompt (positive): "Write positive review of the book titled (title):"
Group 3 [Other variations]:
Prompt (critical): "Write critical review of the book titled (title):"
We conducted the additional experiments on a) Ltlama-2-33b-cha former, we evaluated two sampling parameters (T = 0.8,p = 0.9 for relatively low randomness, and T = 1.2,p 1.0 for high randomness); for GPT-4, 4, we considered only the high randomness parameter and only the prompt (critical) due to the long runtime. For each combination of model, prompt, and sampling parameter, we generated n 10 reviews.
and b) GPT-4. For the
We present the results in Figure II and discuss the takcaways below.
Prompt (critical) achieves the best diversity among all. However, there still signifi- cant gap from the src, which holds true for both models.
Prompt (role) achieves a similar level diversity compared to Prompt (person), showing that specifying diverse set of detailed roles do not bring improvement over speci- fying a diverse set of celebrities. Giving the LLMM extra freedom to explore using different roles (i.e., (i.e., Prompt (creative) however, leads
worse diversity.
3. Using sentiment words (i.c., negative, neutral, positive) can indeed steer the sentiment of the generations very effectively. Yet clarify that the model being a good instruction follower does not directly translate into it preserving diversity, without good instruction giver. Moreover, as per our definition, we care about diversity of multiple attributes. Explicitly steering one attribute insufficient for achieving diversity across numerous attributes. More generally speaking, the issue resides in-what we can control/steer is a limited set of knowns, but what we hope to gain is w.r.t. more broader set of knowns and unknowns.
C.8 MORE RESULTS
For the attribute word choice, we present the count of unique words in Table 2 and the entropy for the word distribution in Fig. 12 showing that model-generated reviews narrower vocabulary and is less diverse than the Human--ritter reviews.
We present results for various attributes and metrics in Fig. 13 and 14 where vary eral factors for detailed comparisons. Overall, the results suggest the prevalence and severity of generative monoculture, well the ineffectiveness of naive mitigations.
24

---

Published as a conference paper at ICLR 2025
[0,0.35]
(0.35,0.55]
[0.55,0.75]
(0.75.0.85)
[0.85,0.95]
[0.95,1.00]
T=1.2.pm100
==08.p=0.9 (3)
=T=12.p=1001
7 0.8.p ta)
-T=12p=1000)
T- 12.000..
T-12,p-0.88
T-08,p--
-7=122p=10(2)
T- 0.8.p
-T-12,p=1000
Varying sampling
Varying prompt
Varying model
1.0
1.0
1.0
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.0
7-12
T-
7-12
T-12
0.0
0.0
p=09
0.0.95
0.90
p=10
2=03
P-008
P=10
P.000
P=03
P-009
P=10
P.30
1.0
1.0
Percenlage
0.6
0.6
0.6
0.2
0.2
0.2
0.00
0.47
0.72
0.97
0.00
0.47 Entropy
0.72
0.97
0.00
0.47
0.72
0.95
Entropy
Einrropy
0.8
0.8
0.8
0.6
0.6
0.6
Density
0.4
0.4
0.4
0.0
0.0
Entropy
Entropy
Entropy
Figure 13: (Row 1): Stacked bar charts of the mean sentiment scores. each bar, darker hues (bottom) correspond to the lower scores while the lighter hues (upper) are the higher scores. See the legend for the detailed value range of each hue. (Row 2): Histograms of the entropy of the sentiment scores. Each bar group corresponds under the bar group. (Row 3): Kernel density estimate (KDE) plots of the entropy of the topic. (Left): generated by Llama -haht different generation kwargs, where we fixed T 1.2 and varied E {0.90, 0.95, 0.98, 1.00}. More variations can be found in Figure 6 in Appendix C The prompt used is (1); see later for the detail. (Middle): generated by ama-2 ama-2-chat ahhht under prompts (1) and (2). (Right): generated by two models (a) Llama- 1aina-2-aan 2-chat and (b) Vicuna-133 using the prompt (1).
range of entropy values in [ar, 2 +-22) where is the number
25

---

Published as a conference paper at ICLR 2025
src
0.5.5
T=
0.8
1=1.0
T=122
p
0.9, varying T
p
0.95, varying T
p= 0.98, varying
1.0, varying
1.0
1.0
1.0
1.0
Sentiment
Percentage
6.0
0.4
0.4
0.4
0.2
02
0.2
9-47
Eniroopy
72
0.97
0.0
0.00
6-47
Estropy
0.72
0.97
0.00
EEhtropy
0.72
0.97
0.00
0:67
Entropy
0.72
0.97
0.8
0.8
0.0
0.0
0.6
0.6
0.6
0.6
Topie
Density
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
Entropy
Entropy
Entropy
Entropy
src
p 0.90
p 0.95
p= 0.98
1.00
T
0.5, varying P
T' 0.8, varying P
T 1.0, varying P
T 1.2, varying P
1.0
1.0
1.0
1.0
Sentiment
Percentage
0.9
0.4
0.4
0.4
0.2
0.2
0.2
9,67
Entropy
0.72
0.97
0.00
Q.TT Enirupy
0.72
0.00
Entropy
0.72
0.M7
0.00
0.67
Enirupy
0.72
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
Topic
Density
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
0.0
Earropy
Entropy
Entroyy
Entrojy
Figure 14: (Rows 1-2): varying I' under fixed p: (upper) entropy of the sentiment for the conditional distribution and (lower) Entropy of the topic for the conditional distribution. (Rows 3-4): varying under fixed T: (upper) entropy of the sentiment for the conditional distribution and (lower) entropy of the topic for the conditional distribution. Results are obtained via Llama-2 under prompt (1).
26

---

Published as a conference paper at ICLR 2025
Table Count of unique words produced by different models (left: llama-2-13b-chat right: GPT- 3.5 (.5-instruct) under varying sampling parameters and prompt (1). As a reference, the count in the source dataset 85,334.
= 0.90
= 0.95
0.98
1.00
0.90
0.95
p 0.98
1.00
T 0.5
18,900
19,041
19,145
19,275
7 = 0.5
15,782
15,794
15,778
15,841
T 0.8
20,020
20,516
20,935
21,688
= 0.8
16,563
16,915
17,096
17,235
T 1.0
21,295
22,024
23.181
24,738
17,044
17,960
T 1.2
23,509
25,742
28,532
33,908
7 1.2
17,368
18,077
18.894
21,059
D ADDITIONAL DETAILS: CODING
D.I RESTRICTION TO LEVEL-A PROBLEMS
We limit our scope to Codeforces level-A problems only for the coding scenario, which are the easiest problems on the Codeforces competitive programming platform We note that even though these are the easiest problems on the platform, they still require non-trivial intellectual efforts to solve
There are two main reasons we restrict to level-A problems:
Most LLMs perform best level-A problems and much worse on more difficult ones. We started off evaluating set of 81 problems ranging from difficulty A to G, and obtained an average accuracy of 50% on 16 level-A problems, and less than 10% on others. Since we perform attribute extraction on the correct solutions only, low accuracy means that to ensure reaching a minimum number of correct generations (20 in our experiments), huge number solutions need to be generated (e.g.. over 200), which is prohibitively expensive in both time and money. Thus we settle with the level-A problems.
We manually verify the correctness of the attribute extraction (e.g., code summary, runtime com- plexity). Relatively simpler problems are easier to verify.
D.2 CORRECTNESS TESTING: AUTOJUDGE WITH TESTCASES
We simulated an Autojudge using the test cases provided the dataset. Concretely, for each prob- lem, obtain 10 test cases in the format of input and outpu We then measure each solution against the set of 10 test cases. We regard the solution that passes all 10 test cases as a correct solution.
D.3 MEASURING ACCURACY
We calculate accuracy as qo /nau per problem (where moooeec is the number of correct solutions and nall is the number of all solutions). For the source, both n nomeett and aall are available in the CodeContests dataset Li et al. (2022). For the generations, we test the correctness of the solutions via autojudge (see Appendix (D22) and measure the accuracy as n. comeet /k.
D.4 MEASURING RUNTIME EFFICIENCY
For runtime efficiency, the bash command fusr/bin/tine Kerrisk (0023) to measure elapsed real time (via &E) as well as the maximum resident set size of the process during its exccution (via %M). Concretely, for each solution, run it all 10 test cases with our autojudge
https::/codeforeee
com/ problemses
Interested readers can refer to this probleminte //ttttpoc//cooeooccss..... http://cw......oooooooo/////r//a
1198/ / get sense of the difficulty in problem understanding and solving
II Codeforces. each test case often consisted of multiple small tests; see an example at httpps:/ codeforcee wwtp///oo//ee.eeeeeeeeeeeeeeeeee werobtemset/// potpeee//rroplleoo proplem. hemm//40///.. As explained in the Input section on the page,
"The input consists of multiple test cases.". Thus, 10 test cases adequate in testing the correctness of solution
27

---

Published as a conference paper at ICLR 2025
and measure the runtime and memory. We take the max value on all 10 test cases a proxy of the runtime efficiency of the tested code.
D.5 PROMPTING GPT-3. GENERATE CODE SUMMARY (BOTH TEXT DESCRIPTIONS AND CATEGORICAL VALUES)
We present below the instruction we provide to GPT-3. 5. We bold keywords in the prompt simply for the of reading. When calling the GPT-3.5 API, used temperature T = 0 (i.e., greedy decoding) and max.tokens=500,
The list of tags (in the 1st prompt below) were collected from the CodeContests Li al. (2022)
dataset-we traverse all the problems in the dataset and union all the "tags" attribute.
The lists of algorithms and data structures (in the 2nd prompt below) were obtained from analyzing Wikipedia and querying GPT-6 turbo (0125) for a suggested list.
These categorical attributes serve as a complement to the text description described above and en- hance the reliability of the results.
Please provide description to the following code in natural language. Explain the functionality, algo- rithm, data structure, time complexity, space complexity of the code.
Finally, assign a few tags to the code. Here a list of tags you can choose from:
"binary search, math, special, trees, dp, greedy, games, df's and similar, expression parsing, number theory, chinese remainder thcorem, geometry. bitmasks, sortings, graph matchings, matrices, poet-in-the-midee graphs, combinatorics, probabilities, constructive algorithms, schedules, two pointers, brute force, dsu, shortest paths, hashing, interactive, data structures, strings, ternary search, fft, flows, implementation" Answer each a line example format of: 'Description: description FuFunciionaait::: functionality" (code}
Please read the following code and infer the algorithms and data structures used in it. For algorithms, select few) from following list:
"Sorting Algorithms, Searching Algorithms, String Algorithms, Divide and Conquer Algorithmss, Greedy Algorithms, Dynamic Programming, Recursion, Bit Manipulation, Backtracking, Graph Algorithms, Oth-
ers"
For data structures, select (a few) from the following list: "Arrays, Linked Lists, Stacks, Queues. Trees, Heaps. Hash Tables, Sets, Maps, Priority Queues, Others" Answer each a line following the format of: 'Algorithms: candidate 1, candidiate 2, Dnta structures: candidate 1, candidiate An. {code}
E ADDITIONAL RESULTS: CODING
E. EXAMPLES FOR PLAGIARISM SCORES
We present example model-generated code and human-writter code well the plagiarism scores associated with the pairs in Fig. 15 16 409A. The Great Game
17
All the code are correct solutions to the problem
From the demonstrations it is evident that model-generated code (Fig. 15 16] are highly similar in their style and structure; even the low score pair (Fig. [6) bear sigmificant level of similarity. In comparison. Human--ritten code (Fig. [7) are clearly distinctive. These results support the validity of using the plagiarism score evaluate code similarity.
E.2 ATTEMPTS ON VARYING PROMPTS
We experimented with four prompts GPT-4, one plain prompt, one instructing the model to gen- erate only the code solution, one employing role-playing Salewski et al. (2024) and one integrating -hain-of-thought prompting Wei et al. (002b)). We present the concrete prompts in Fig. 18
Link the httpa///////////e/eeee/e///////r
28

---

Published as a conference paper at ICLR 2025
Test file: train www__gen/traan-00000 0/traln-00000 otrali-00000soouttinnooooo colution_coddoon/ S. (78.00%)
Reference file: wrwinnnentrrinn-nee wwt.o//rr-nnooo 00000000000000000 corooossouotoonooooooo
htt.oooooooiioooooooooooee.....e wot.eooiy.....0rp.... (79.19%)
Token overlap:
View matched code
last_year (ingut() for In rangein]]
prev. year [input() for In range(n)] year [input() range(n)]
year
rangein]]
prun_
sefrent_yeer_ccom
current__ count
For size in last_years
for sice in prev_vears
lest year [sient
prev_ www_cconn[ssee +
lest_year caar__conn[[siee
prey. criv_counttsiize
för size current_yeart
for size in current_year?
current_year_coom
current_.ounti
current year, [size]
current, [size]
elsei
current_ (Size)
seconds
seconds 8
for current year_counti
size current counti if size in counts
if size last_year_countt
serrent__yar_ccuuee (size) last_ (ssie_
cerrent_ccun((dat prev_ccount [size]
seconds current_year carnnt__y______onSttt last_year_count|ee [si:
If diff B:
seconds (errent_yer__coun((sza))
else:
current_count(aa4
(scon(s)
print(seconds)
Figure 15: An example for a pair model generated code with high plagiarism score.
Test
http://000000oosottoooooooooomm www.ssooooooceeeeeeeeo 15taldoooccoeoooyy1115122212 15_19p (33.64%)
Reference file: train train_genn/raii-0000000 19/ra//-00//aai00000000 hww.://000ooooooonoooooooo www.toon_ codeeccy (23.00%)
Taken overlap: 72
def find wend_winnerrree teard actions, team?, teax2_acttinn actt))n)):
def determine, winner Laction,,cion,, teant, actions, actioos_ttear):
scores ("Be":s 0, "[l": "("":
score_ teand, scare_teanz tean2
systes
pairs
teas2_score
points, systen =
('Be', "[I"): 1, D'l', 'Betl: 1,
rangelo, ten(tean) acttionst, teant_actions [1:1=2]
('0), 18e1): D'cc, 'O''s CII,
actior2 conv__cctions(1:11 (1:1+2)
scores (actionll (scores team]_score
actions_teaal [actions_teantin actions_teamz persoons_eem2(iii122 for rangel8, tentacti
ranpel8,
etif scores aaction2) team2_scom
(scoresfactionl)
teant_score scores
for actiont, ziplactions_ seant, actions_team2):
return "TEAM KINS!
if (actionl, action2) in points, system:
allf tear?_score Score_
score_ sn points__ysteelll factionn, action2]]
elif tean? pystts (actionn, actionl]]
system:
elses
reture "TIE!
Deternine the based the return "TEM WINST
input sttrapp) tore2_actionn actions stesp()
el1f score_ 36a12 scone_teanii
print teent bettons tese2_actions)
TTEM WINS!
etsei
return "TIE!
actions, Input().stript)
actions, Leae2 Snput().stript3
result determine wirnerlactions. teasl, actions team?) print(result)
Figure 16: An example for a pair model generated code with relatively plagiarism score.
29

---

Published as a conference paper at ICLR 2025
men_amuteer 1mmn-00001 Ine--0000 Sipclltie
brain peclutions 7 trai-1000011 1twin000000, 10 onoklere Inem-00000_0
tren--00000_9 29,sclutio,, Bix
Inpute)
Made NestaTa_ bot
(eput()
Inpurcl
(rputt)
scone3
lan(at),
rangel Len(al)>
221cl
a111
scuret
in rangete, len(e)
7
affela]
614
Xf s2 1("
3
s11
and b111
scorez
lnc. inc.indexfaf3 113)
3
ifia
65' and b11l
18
st stfe] 1ie
18
18
sti+1s]
18"
and b11l
11
if s21cl 1C1
11
thel bisl 1kelle 0
Iffx
prant TTIE
iffx
smants TEAM WORS
s2lel 1811
prEst("TEAN sd HnVeSNEE txt
1rtx
prontt) wmss-
27
acoret
18
13
score2
20
21
storel an Scone2:
21
print (1T26")
DELNCETEEM () formact2
Figure 17: An example for three human-written code with zero plagiarism score for all the pairs.
Prompt 1: Please read the below problem description and generate python code solve the problem:
(problem.descciption)
Prompt 2: Please read the below problem description and generate a python code to solve the problem:
}
Please only generate code and nothing else.
Prompt Imagine you are grandmaster in solving competitive programming problems. Your skills in algorithms, data structures, and prohlem-solving are unparalleled. You have a deep understanding of various programming paradigms and can easily navigate through complex problems with efficiency and elegance.
Please read the below problem description and generate python code to solve the problem:
{problem_description
Please only generate code and nothing else.
Prompt 4: Imagine you are grandmaster in solving competitive programming problems. Your skills in algorithms, data structures, and prohlem-solving are unparalleled. You have deep understanding of various programming paradigms and can easily navigate through complex problems with efficiency and elegance.
Please read the below problem description and generate python code to solve the problem:
Please think through the problem step by step, and then provide your solution. Then, test your code against the provided test cases in the problem. your code fails to pass all the tests, please revise your code and try again until your code passes tests.
Figure 18: Four prompts for instructing the model GPT-4 to generate code solution to provided problem in {problem {problem.descripiine d.scriptionn. Prompt 1 is the most plain prompt; Prompt additionally in- structs the model to generate only the code without additional explanations; Prompt 3 employs the technique of role-playing Salewski et al. 2024), and Prompt 4 additionally integrates the technique of chain-of of-thought prompting Wei al. (202bb).
30

---

Published as a conference paper at ICLR 2025
1.0
#=16
#=17
#=18
#=17
Correct (ave over sources)
Incorrect (ave over 5 sources)
GPT-4 prompt (1 generation)
GPT-4 prompt 2 (1 generation)
Accuracy
GPT-4 prompt 3 (1 generation)
0.5
GPT-4 prompt 4 (1 generation)
0.0
A B
C
Increasing difficulty
Figure 19: Accuracy achieved by using the four prompts on Codeforces problems of varying difficulties. The "Correct" and "Incorrect" refer the results achieved by the human-written so- lutions in the source dataset CodeContests. Overall, we see that prompt 2 performs the best across different difficulty levels.
We evaluate the four prompts on Codeforces problems of varying difficulties and present the accu- racy achieved by each prompt Fig. Overall, prompt achieves the best results among the four.
E.3 HUMAN ANNOTATIONS FOR THE QUALITY LLM SUMMARY
We annotate the quality of the LLM--ennrated summary of code given by GFT-3.5.
Step 1: We randomly sample 20 solutions of different problems, and review the LLM-ggenrrate attributes for these solutions; the attributes include the textual ones (description, functionality, algo- rithm, data structure), the inferred asymptotic complexity (time complexity, space complexity), as well as the categorical ones (tags, algorithm. data structures).
Step 2:
We read through the problem description, the code solution, and the model-generated sum- mary to evaluate the quality model-generated summary.
Step 3: We follow the instruction "In each cell please give a score from 1-3 - meaning absolutely correct, 2 meaning with some minor errors but acceptable, meaning entirely incorrect" score each attribute.
Step 4: We calculate the average score received by each attribute across the 20 sample and present the results in Table B
The high scores indicate that GPT-3. can provide fairly accurate summary of the given code, supporting our choice of relying GPT-3. 5 to reliably extract the attributes.
Table 3: The average scores given by human annotators on each attribute. We adopted 3-point scale where means absolutely correct, 2 means with some minor errors but acceptable, means entirely incorrect. The high scores indicate that GPT-3. 5 can provide fairly accurate summary of the given code.
textual
complexity
categorical
Attribute
description
functionality
algorithm
data structure
complexity
space complexity
tags
algorithms
structures
Score
2.65
2.55
2.90
3.00
2.95
2.70
2.90
2.50
2.90
31

---

Published as a conference paper at ICLR 2025
E.4 FAILURE RESULTS WITH TEEN-SOURCE MODELS
We experimented with two open open-source code models on these competitive programming problems: CodeLlama Roziere et al. (2023 o and StarCoder Li et al. (202bb However, neither of the models were able produce code solutions in our trials. We present one example interaction in Fig. 20 where the model demonstrated the memorization of a problem dataset but failed to produce code solution to the given problem.
E.5 A COMPLETE, SET OF RESULTS FOR THE EXPERIMENTS ON GPT-4
Accuracy
The histogram of the accuracy can be found in Fig. left).
Plagiarism score
The histogram along with the KDE plot for the plagiarism scores can be found
in Fig. 21 rightt.
Efficiency The efficiency results in terms the asymptotic complexity (specifically, the histogram
of the time complexity and space complexity) can be found in Fig. [2] The efficiency results in terms of the runtime efficiency (specifically, the density plots of runtime and memory) can be found
in Fig. 23
Code summary The KDE plots of the mean pairwise embedding cosine similarity of the code summary (lexiual) be found in Fig. 24 The stacked bar plots of the mean pairwise Jaccard similarity of the code summary (categorical) can be found in Fig. 25
E.6 CLAUDE
Accuracy The histogram of the accuracy can be found in Fig. 26/left). In comparison to GPT-4 (see Fig. (e]ef))), the accuracy of C1 31aude-311 is much lower.
Plagiarism score
The histogram along with the KDE plot for the plagiarism scores can found in Fig. 26 right). The plagiarism scores achieved by Claude-3 remain high, similarly to that GPT-4.
The efficiency results in terms of the asymptotic complexity (specifically, the histogram of the time complexity and space complexity) can found in Fig. 27 The efficiency results in terms of the runtime efficiency (specifically, the density plots of runtime and memory) can be found in Fig. 28 Different from GPT-4, solutions generated by C1 cla are less efficient than source solutions. This result, together with the accuracy result, indicates that the code generation ability of Claude 3 is inferior than GPT-4.
Efficiency
Code summary The KDE plots of the mean pairwise embedding cosine similarity of the code summary (textual) can be found in Fig. 29 The stacked bar plots of the mean pairwise Jaccard similarity of the code summary (categorical) can be found in Fig, 30 Similar to GPT-4, Claude-3 also demonstrate narrower range of expressed ideas.
F COMPUTE RESOURCES AND DATA LICENSES
F.1
COMPUTE
The book review generation (N = 742, n 10, max new tokens=500) on open-source models took around 10 hours on one H100 card per run, i.e., per combination of sampling parameters (T and p) and prompts. The number increased to around 80 hours for GPT-4.
The code generation (N = 100, 60 hours per run. The rate of generation was similar on Claude- Sonnet; the total runtime doubled since we generated twice many solutions (π 200) due to its lower accuracy.
100, max www._new_tee www.weeeeese20009 on GPT-4 took around
at
1 Instruct-hf
13b-
Link at http:::///ggiingaaa.....///ii..
32

---

Published as a conference paper at ICLR 2025
Prompt
Please read the below problem description and generate python code to solve the problem:
Gregor is learning about RSA cryptography, and although he docsn't understand how RSA works, he is now fascinated with prime numbers and factoring them. Gregor's favorite prime number Gregor wants find bases of Formally, Gregor is looking for two integers a and b which satisfy both following properties.
P mod P mod b, where x mod denotes the remainder when is divided by y. and
#2 S b
Help Gregor find two bases of his favorite prime number!
Response Input Format
The first line contains integer the number of test T lines follow. Each line contains an integer P. Gregor's 's favorite prime number.
Output Format
For each test case, print two lines. The first line should contain the first base, and second line should contain second base.
Constraints
#2 < 1000 # P 0
Sample Input
Sample Output
Explanation
For the first test case, the two bases are 2 and 3. For the second test case, the bases and 5. (n)m)))nnn...........
Figure 20: Generation result of StarCoder. The model unable generate the requested code solution as instructed; instead, it generated some boilerplate content such as input format and output format, which often exists datasets of this kind of problems. After these boilerplate content, the model simply generated hundreds of i and then stopped.
33

---

Published as a conference paper at ICLR 2025
[0,0.2]
[0.2,0.4]
[0.4.0.6]
(0.6.0.8)
[0.8.1.0]
STC
T=0.5,p=0.9
==005,p=1.0
===10,0=003
===100p==0
Accuracy
Plagiarism score
1.0
1.0
0.8
Percentage
0.8
0.6
0.4
0.4
0.2
0.2
0.0
50
p=09
7-05
p=10
T-
p=
T-10
p=:10
T-100
0.0
0.00
0.00
0.25
1.00
Similarity
Figure 21: Model: GPT-4. (Left) Stacked bar charts of the accuracy values achieved by the source solutions well as GPT-4 generated solutions under various kwargs. (Right) Histogram plus kernel density estimation (KDE) plots of the plagiarism scores.
STC
T p- Time complexity (unconditional)
T 0.5.p 0.9
T= 1.0.p
0.9
- p p-00.pp...
Entropy of time complexity (conditional)
0.8
Ollegn)
0.6
O(y7)
Percentage
0.4
Otrlogn)
0.2
Oln?)
0.0
0.0
0.2
0.4
0.6
0.0
0.4
0.8
1.2
1.6
2.0
Percentage
Entropy
Space complexity (unconditional)
Entropy of space complexity (conditional)
Percentage
0.5
04n)
06n?)
0.0
0.0
0.2
0.4
0.6
0.0
0.4
0.8
1.2
Percentage
Entropy
Figure 22: Model: GPT-4. (Left) Histograms (or grouped bar charts) for the time and space complexity for the unconditional distribution. (Right) Histograms for the entropy of the time and space complexity for the conditional distribution. The figures suggest that the generated code is more efficient than the source code, while showing decrease in diversity.
0.5, 0.9 Runtime (mean)
T 1.0
-
T-1100
Memory (mean)
Runtime (std)
Memory (std)
0.8
0.8
Percentage
0.6
0.6
0.4
0.4
0.2
0.0
milliscconds (ms)
150
0.0
10000
kilohytes (KB)
20000
30000
0.0
milliseconds
100
10000 20000 (KB)
Figure 23: Model: GPT-4. Histograms of the mean and standard deviation (std) of runtime (in milliseconds) and memory usage (in kilobytes). We filtered out some datapoints for better visualization, specifically, runtime above 200 ms, and memory usage above 30,000 KB.
34

---

Published as a conference paper at ICLR 2025
STC
===0,,p==10 T=0
=0=
===10,p=10
Description
Functionality
Algorithm
Data structure
Density
0.4
0.6
0.8
1.0
0.6
0.8
1.0
0.6
0.8
0.4
0.6
0.8
Similarity
Similarity
Similurity
Similarity
Figure 24: Model: GPT-4. Kernel density estimation (KDE) plots of the mean pairwise cosine similarity for the four attributes (description, functionality, algorithm, and data structure) repre- sented as extracted embeddings of natural language descriptions.
[0,0.2]
(0.2,0.4] Tags
(0.4,0.6)
(0.6,0.8)
[0.8,1.0]
Algorithms
Data structures
1.0
1.0
1.0
0.8
0.8
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
S1C
T=0.5
0=0.5
7=10
7=10
src
0.5
T=10
7=100
S/C
7=0.5
7=0.5
T=100 1-000
T=10
p-0.9
1.0
0.9
1.0
p-
p-1.0
p-0.9
p=0.9
p
p-10
Figure 25: Model: GPT-4. Stacked bar charts of the mean pairwise Jaccard index for the three attributes (tags, algorithms, and data structures) represented sets of categorical variables.
[0,0.2]
[0.2,0.4]
[0.4.0.6]
(0.6,0.8
[0.8,1.0]
srC
T-0.5pp0.99
-T-0.5,
= -7-1.0pp-
Accuracy
Plagiarism score
0.4
Percentage
0.3
0.2
0.4
0.1
0.2
MC
0.0
0.00
0.25
0.50
0.75
1.00
Similarity
Figure 26: Model: Sllaude---Sonae (Left) Stacked bar charts of the accuracy values achieved by the source solutions as well as Claude generated solutions under various kwargs. (Right) His- togram plus kernel density estimation (KDE) plots of the plagiarism scores for source solutions and Claude generated solutions.
35

---

Published as a conference paper at ICLR 2025
src
= 0.9
-==10,p=0.9
-7=1.0p==10
Time complexity (unconditional)
Entropy of time complexity (conditional)
0(11
Ollogr)
0.3
Percentage
0(7))
0.2
0(n)
0(nlogn)
0.1
O(n?)
0.0
-
0.0
0.2
0.4
0.6
0.0
0.4
0.8
1.2
1.6
2.0
Percentage
Entropy
Space complexity (unconditional)
Entropy of space complexity (conditional)
0111
Percentage
0.2
001
0.0
0.0
0.2
0.4
0.6
0.0
0.4
0.8
1.2
Percentage
Entropy
Figure 27: Model: Claude-3-Sonnes (Left) Histograms (or grouped bar charts) for the time and space complexity for the unconditional distribution. (Right) Histograms for the entropy of the time and space complexity for the conditional distribution. The figures suggest that the generated code more efficient than the source code, while showing a decrease in diversity.
STC
0=0.5,
===1.0.p=0.9
-7=100p=1.0
Runtime (mean)
Memory (mean)
Runtime (std)
Memory (std)
1.00
0.8
0.6
0.8
Percentage
0.6
0.6
0.4
0.4
0.50
0.25
0.2
0.0
mill millissconds (ms
50
100
150
0.00
10000
kilobytes
20000
30000
0.0
20
40
(ms)
60
0.0
kilobyics (KB)
10000
20000
Histograms of the mean and standard deviation (std) of runtime (in milliseconds) and memory usage (in kilobytes). We filtered out some datapoints for better visualization, specifically, runtime above 200 ms, and memory usage above 30,000 KB.
Figure 28: Model: Claude-3-Sonmee
Src T= 0.9 Description
T==0
1T=100p=
Functionality
Algorithm
Data structure
Density
2
0.50
0.75
1.00
0.50
0.75
1.00
0.50
0.75
1.00
0.50
0.75
1.00
Similarity
Similarity
Similarity
Similarity
Figure 29: Model: Claude-3-Sonnet. Kernel density estimation (KDE) plots of the mean pairwise cosine similarity for the four attributes (description, functionality, algorithm, and data structure) represented as extracted embeddings of natural language descriptions.
36

---

Published as a conference paper at ICLR 2025
[0,0.2]
(.2.0044 Tags
(0.4,0.6]
[0.6,0.8]
[0.8,1.0]
Algorithms
Data structures
0.5
0.5
0.5
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0.0
SIC
T-0.5
T-0.5
7-1.0
7=1.0
7-0.5 T-0.5 T-1.0
T-1.0
SPC
T-0.5
7-B.5
T-100
T-1.0
p=0.9
1=00 0=9.9 p=
p=
p=1.0
p=
p=10
p=0.9
p=
p=0.9
p=10
Figure 30: Model: Claude-3- 3--dnnee... Stacked bar charts of the mean pairwise Jaccard index for the three attributes (tags, algorithms, and data structures) represented as sets categorical variables.
F.2 LICENSES
We used the Goodreads dataset Wan et al. (2019 X Apache License as provided in their repository.
for the book review scenario. Their license is
We used the CodeContests dataset Li et al. 2022 License as provided their repository.
for the coding scenario. Their license is Apache
G EXTENDED INVESTIGATIONS
G.1
INFLUENCE OF MODEL SIZE ON MONOCULTURE
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
50c
T=08
7=1.0
T=08
7=1.0
500
T=008
7=100
T=
T=100
13b.c/11
706- 13bc
706-(0
13b
13b-c.2
706-c12:
70b-2/2)
(a) Prompt (1)
(b) Prompt (2)
Figure 31: Monoculture persists despite model scaling.
We conducted experiments understand the pervasiveness of monoculture function of model size. To this end, we conducted the sentiment analysis study (on generated book-reviews) for Llama 70b (instruction fine-tuned aligned) denoted 70b-c(x) in the figure, where x denotes the prompt (refer $5.Ip we compare this with the 13b model from the same family. For the 70b model, we used a quantized version for faster inference times given the narrow window of the rebuttal period. Our results, in Figure 31 show that while this larger model has "greater diversity" than the smaller model (potentially due an increase the diversity the training dataset used train this model),
LLink the dataset website: httpah//ese..eeen.ee....c.hiiiia
LLink to the license: http:s/s//tthhcccoee/eeeeeaa///a
LI LICENSE
10 Link to dataset: http:///ss/.ttttooo/oooo//////oa
1'iink to the license:
blob/
37

---

Published as a conference paper at ICLR 2025
the phenomenon of monoculture (i.e., decreased diversity compared to the source reviews) remains. This is the case across both prompts, and despite variations in sampling.
G.2
INFLUENCE OF LENGTH ON SENTIMENT SCORES
0.8
0.7
Average sentiment of samples the interval
0.6
0.5
0.4
0.3
0.2
0.1
0.0
501
(50. 100]
150]
200]
250]
(250. 300]
(300. 350]
(350, 400]
(400. 450]
(450, 500]
(0.
(100.
(150,
(200.
Token length interval
Figure 32: Average sentiment score agnostic of length of reviews.
In Appendix B.I we state that our book-reviews curation process involved discarding those reviews less than 300 words. From Figure 32] we find that there is little correlation between sample's length and sentiment i.e., no biases are introduced by discarding shorter samples.
G.3 MORE TRAINING DATA CONTROL & CONNECTIONS TO MONOCULTURE
While one might be skeptical about the assumption that book-reviews are present in the training data, circumventing this skepticism would involve (a) full knowledge of the training dataset, and (b) picking downstream generative task that is influenced only one sub-component dataset (in some provable manner). However, satisfying both of these requirements is challenging because:
Should we have full control of the training dataset, training our own model from scratch remains prohibitively expensive.
Open source models, where the training set is known, demonstrate extremely low- performance on the tasks we perform in this paper: example OLMO is incapable of generating acceptable coding solutions, or reviews that are coherent in consistent/reliable manner.
We begin by obtaining a new, unseen dataset (denoted src): We filter out books published af- ter October 1, 2023 from the GoodReads datase In this dataset, majority of the book (72 out of 79) come with 5 reviews and the remaining books have fewer reviews. We thus retain the 72 books with 5 reviews, and compose a dataset of 360 samples. We then instruction-tune 1lama-13bb (both the pre-trained and chat versions) using the aforementioned dataset. The tem- plate we used was 'instructionn
Write a book review for the book
httpss: :wwp:////wwwwww........../..w.
http///www.kkgg.... kaggle. com h
38

---

Published as a conference paper at ICLR 2025
[0,0.35]
[0.35.0.55]
[0.55.0.75]
(0.75,0.85]
[0.85.0.95]
(0.95.1.00)
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
PT:SFT Chat-SFT (1) (1)
src
PT:SFT (2)
Chat-SFT
(2)
(a) Prompt
(b) Prompt
Figure 33: Even with full control of the fine-tuning dataset, monoculture persists.
titled (title)'', ''response'': "(review]'' We perform parameter effi cient fine-tuning (LoRA dimension=4) and fine-tuned for 3 epochs with learning rate=10-3 and batch size=2. We ensured that the perplexity (measured on "wikitext-2-raw-xx"") of the model after fine-tuning does not increase a lot, to avoid overfitting.
We then proceed to generate book-reviews for the aforementioned books, and measure the di- versity of sentiment has changed. Our results, in Figure 34 suggest that with either the pre-trained model (PT) or the aligned/chat (Chat) model as the initialization, fine-tuning on the curated small review dataset resulted in the average sentiment becoming more positive after fine-tuning the pre-trained model (but still being close to the data distribution), and the average sentiment becom- ing more negative after fine-tuning while fine-tuning may provide some reprieve, RLHF skews the sentiment significantly. Thus, mono- culture still persists this case.
the Chat model. The implication of this experiment that
G.4 EVIDENCE OF GOODREADS IN COMMONCRAWL
the ree'
D1 www. wntirtien CNNEL www.eootttteooooo.... www.eretttitteeooeooooooo
Figure 34: Screenshot of goodreads.com being present in the common crawl index
39

---

Published as a conference paper at ICLR 2025
Baack (2024) highlight that Llama vI is trained on CommonCraw Upon closer inspection (using the common crawl interface), one can see that the goodreads.com website and its associated URLs part of this dataset (see Figure 34.. This suggests that the training data of Llama v1 includes book reviews from the GoodReads dataset. While [Bacck (024) note that there is limited visibility into the training dataset for Llama 2, safely assume that was built atop of the data collected for Llama 1, including CommonCrawl. Thus, we would like to stress that the experiments in our paper valid.
13
hhttes::/ineexx.comoocearr
40


Reviews:
[{"review_id": "e3jwnF0zoe", "reviewer": "Reviewer_SPD6", "summary": "This paper addresses the critical issue of \"generative monoculture\" in large language models (LLMs), where output diversity narrows compared to training data. This is particularly concerning as LLMs are increasingly used in diverse applications like product reviews, sentiment analysis, and scholarly summarization. The study focuses on book reviews and code solutions to demonstrate this phenomenon and tests various mitigation strategies, such as temperature adjustment and prompt engineering, though with limited success.", "questions": "I would like authors to comment on feedback loop effects. How do they anticipate monoculture evolving as LLMs are increasingly used to generate their own training data in feedback loops ? Do they think this will further exacerbate the monoculture effect. What strategies and/or guardrails do the authors suggest to mitigate this very long term \"model collapse\" situation ?\n\nAuthors could also further comment on composition of training data. Would increased diversity play a role in mitigating feedback loops and how can LLM training incorporate such diversity. \n\nLastly, what role model size plays ? Have authors ablated along model size/complexity dimension ?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper highlights a critical issue in LLMs around narrowing of output diversity compared to the training data. The paper addresses an important problem esp when LLMs are being increasingly applied in diverse fields such as automated product reviews, sentiment analysis, scholarly paper summarization etc. The paper demonstrates the prevalence of narrowing of output diversity, which they refer to as 'generative monoculture'. They consider book reviews and code solutions as two primary use cases to study the narrowing phenomenon. \nThe paper tests various methods to mitigate 'monoculture', including temperature adjustment and prompting strategies. \n\nThe paper provides a well-structured methodology for measuring generative monoculture, including diverse metrics like entropy, mean pairwise similarity and other such metrics. \n\nThe paper discusses the impact of narrowing of output diversity on the societal and code security aspects. These discussions strengthens the paper.", "weaknesses": "While the paper tests various methods to mitigate 'monoculture', including temperature adjustment and prompting strategies, the attempted countermeasures showed limited efficacy in mitigating narrowing of output diversity. This warrants more experimentation and ideation. I would also think use cases/tasks other than book reviews and code generation should be investigated to test the generalizability of the method. Dialogue / chat bot as an application may be an important area to test these methods on. The inability to redefine the alignment process limits the degrees of freedom with which one could operate. I would have liked to see a bit broader scope that covers alignments and comparing the dispersions to alignment data rather than the training data of CPT. \n\nRich getting richer / Echo Chamber / Feedback loops are commonly studied phenomenon in recommender system literature. Methods like Bandits and Exploration/Exploitation are commonly used to mitigate the homogeneity of recommended items over time. Authors have an opportunity to connect to this rich work and draw inspiration from this field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XEycFGlueQ", "reviewer": "Reviewer_dgPh", "summary": "The main idea of this paper can be summarized in the following toy example: if 90% of humans say that chocolate is tasty, should language models *always* describe chocolate as tasty, or should they aim to somehow reflect the diversity in human opinions, and occasional refer to chocolate as bad-tasting?\n\nThe authors apply this thought experiment to two domains---book reviews and code implementation---and analyze the various ways in which language models enforce a monoculture by failing to model the diversity present in actual human-written content. Adding more entropy into the decoding pipeline helps a bit, but does not meaningfully bring diversity to human levels.\n\nThe authors include an interesting discussion of the scenarios in which diversity is desirable--for example, generated book reviews ought to reflect the range of opinions real reader mights have on said books---and the scenarious where diversity might be less important---for example, it is more important for generated code to be correct than to be diverse.", "questions": "1. What was `{person}` replaced with in the prompt?\n2. I do not understand the method used to generate correct solutions for the coding task. Specifically, I did not understand this sentence (line 309): We instantiated this by generating k samples (100 for GPT-4 and 200 for Claude-3), and verifying that at least 20 of them were correct.\" What did you do if it wasn't the case that 20 were correct? Does your sampling method here ensure then that \"correctness\" will never fall below 20%?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "The main idea of this paper is very interesting, and I am glad the authors have done this exploration. The authors have done a good job of discussing nuances around the merits of diversity, and I appreciate their selection of two complementary domains where the value of having diversity is quite different.", "weaknesses": "## Primary weakness - incomplete description of methodology\nUnfortunately, it is not possible to assess this paper as it was submitted because crucial information required to understand and reproduce the methodology is purported to be in the appendix, but no appendix was included in the submission. Since the paper is incomplete, there is no choice but to give a score of 1 (strong reject). Despite this, I have tried to leave some constructive feedback below for the authors.\n\n## LLMs for attribute extraction\nLine 163 states that \"care must be taken to use LLMs for attribute extraction, as they are known to be biased towards their own responses.\" It seems like this care was taken for the sentiment classifier (the paper notes the classifier's accuracy on SST-2 and it's widespread usage). However, the same care does not seem to have been taken for the \"efficiency\" attribute, which uses gpt-3.5 to assess runtime efficiency of generated code. It is unclear why a reader should trust that gpt-3.5 is efficient and unbiased at this task.\n\n## Takeaway #2\n\nLine 393 states that humans \"largely prefer text with positive sentiment.\" Is this actually true? In a 5 minute literature review, I found several papers such as the two listed below which suggest a much more complex story. Needless to say, statements such as this one should not be made without citation.\n\nSangwon Park, Juan L. Nicolau. Asymmetric effects of online consumer reviews, ISSN 0160-7383, (https://www.sciencedirect.com/science/article/pii/S0160738314001273).\n\nLotte M. Willemsen, Peter C. Neijens, Fred Bronner, Jan A. de Ridder. “Highly Recommended!” The Content Characteristics and Perceived Usefulness of Online Consumer Reviews, (https://academic.oup.com/jcmc/article/17/1/19/4067647).\n\n## Takeaway #3\n\nTakeaway #3 claims RLHF hurts diversity more than any of the other factors did. This claim would be much stronger if it had been backed by experiments with more than just one pair of models. I would like to see additional experiments with other model pairs (for example instruction-tuned and RLHF'ed OLMO). Otherwise, a caveat should be added to this claim about the limitation of the result.\n\n## Nitpicks\n\nHere are a few nitpicks (which wouldn't much affect my overall assessment).\n1. I am not a fan of the phrase \"human-generated.\" It is extremely atypical to talk about a human generating a book review or generating some code (unless the speaker wants to imply the human is using genAI tools); rather, in common parlance, humans **write** book reviews and **write** code. I suggest replacing instances of \"human-generated\" with \"human-written.\"\n2. I think the paper would be more engaging to read if the description of the attributes of interest was moved before the section on metric calculation (Section 3.3). Section 3.3 felt out of place to read when I didn't yet know what exactly the attributes were for each domain. \n3. Figure 4 has too much information in it, which impeded communication. It would be more effective to break this into multiple figures each with their own caption. In particular, the (a), (b) and (c) in the middle bar graph are especially confusing since (a), (b) and (c) are also used to refer to subfigures. Can you instead put shorthands for the actual names of the models? Also, for the topic model I am confused why the word 'novels' occurs in two groups. \n4. For Figure 5, an easier-to-understand x-axis label for the top-right plot would be \"Self-similarity\" rather than \"Similarity.\"\n5. One additional paper citation for the first paragraph of your Related Work section (\"Forcing Diffuse Distributions out of Language Models\" https://arxiv.org/abs/2404.10859).\n6. Many of the references are formatted incorrectly with missing capitalization in the paper titles.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QuyNxstx41", "reviewer": "Reviewer_rybD", "summary": "The paper defines and documents the problem of “generative monoculture” in LLMs – that is, the situation in which LLMs produce outputs that are significantly less diverse that what was present in their input training data. The authors focus on diversity over task-specific and intuitive metrics – specifically, the sentiment of book reviews or the algorithms employed in generated code. This is in contrast to measuring over some more systematic but arguably less informative metric such as lexical distributions. The authors find that, across models, the problem of generative monoculture is present and argue that it seems to get worse for models which are “aligned” to human preferences via RLHF.\n\nI really like this paper. It's a nice, intuitive idea that deserves to be highlighted. I think the study was executed well for the most part but would have preferred to see some human evaluations, rather than solely automatic ones. But I think despite this, it warrants publication in the current form.", "questions": "None beyond what was raised in weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper formalizes the idea of “monoculture”. This idea isn’t wildly novel–it’s intuitive and consistent with other similar ideas such as mode collapse–but to my knowledge there isn’t a clean documentation of it and thus the paper has value in being an official cite for this phenomenon\n\nThe authors focus on measuring monoculture using task-specific notions of salient attributes (e.g., sentiment in book reviews, algorithms in code) which differs meaningfully from measures that use e.g., vocabulary of generations. I think this distinction is meaningful as it's a better measure of the type of distribution shift that will matter in practice if GenAI is widely deployed.", "weaknesses": "My primary concern is that the evaluation focuses entirely on automatic metrics. Granted, there are many metrics that the authors use, and they are somewhat diverse. Still, many of the metrics rely on using LLMs themselves (mostly GPT 3.5) to evaluate LLM output. There is something circular (though hard to articulate) about doing this especially given the premise of the paper itself. That is: if we assume LLMs are not good at generating diverse outputs, might we also worry that they aren’t good at recognizing such diversity? Or, said differently, why are we confident that the collapse is due to actual differences in what the LLMs produce, and not differences in what the LLM evaluator can detect? \n\nTo address this, I think the paper should include some evaluations which are determined entirely by human judgments. I.e., ask humans to rate the sentiment rather than asking LLMs to do so, ask humans to evaluate the big-O complexity rather than having GPT do it, etc. I would be dramatically more convinced if the conclusions held up under human eval, rather than just automatic eval. But, admittedly, I do expect the result to hold even if all evaluations were switch to human eval.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0gDtKGS88M", "reviewer": "Reviewer_N2Ye", "summary": "This paper introduces the concept of “generative monoculture”, which refers to the phenomena that an LLM's output distribution is less diverse than the original input (data) distribution.\nGenerative monoculture is investigated on two tasks: generating book reviews, and generating coding solutions, across multiple LLMs.\nThe monoculture is measured with a number of metrics (depending on the task): distribution of the mean, entropy and standard deviation, and mean pairwise similarity.\nThe authors find that monoculture exists across the LLMs that are investigated. The authors propose several mitigations, such as changing the sampling parameters, or diversifying the prompts, but this has little impact.", "questions": "* Can the authors explain what they mean on line 309: “We instantiated this by generating k samples [...], and verifying that at least 20 of them were correct.” Does this mean 20 samples were checked, and 80 were not? \n* Can the authors provide a reference that shows that humans prefer positive reviews, to support take-away 2?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* **Interesting quantification of generic monoculture:** As the authors also mention in their related work section, the idea that current alignment practices hurt diversity is not necessarily new. However, this work presents a new way of measuring this, which gives additional insights in the output of (aligned) LLMs.\n* **Considerations (pros and cons) of approach are clearly presented, and largely make sense:** The authors clearly mention their considerations for adopting their chosen methodology, in a space where they sometimes had to make some shortcuts or assumptions. For example regarding their data selection and their used metrics.\n* **Authors present negative results, that are insightful:** The presented mitigation strategies do not really help, but this is an insightful finding for the community.", "weaknesses": "* **Investigated datasets are not very large.** This holds especially for the code solutions, where the data is limited to a subset of 100 easy solutions. Although this gives a first impression, I wonder how results hold over larger data samples, and especially when harder problems are included as well.\n* **Coding solutions are checked by GPT-3.5.** The authors give some details in the appendix of the autojudge, but it is not entirely clear to me how quality is ensured. After all, GPT-3.5 needs to check the results of a stronger model (GPT-4) and, as the authors also mention elsewhere in the paper, LLMs tend to be biased towards their own responses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "THaU79O4rX", "reviewer": "Reviewer_L9QX", "summary": "The paper investigates generative monoculture in LLMs, defined as the reduction in diversity from training data to the model’s outputs. It examines this phenomenon in two tasks: sentiment of book reviews and code generation. The paper found consistent monoculture in all experiments.", "questions": "Q1: Could the authors explain how the plagiarism score is computed in Figure 5 (b)? It seems is not mentioned in paper.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "Generative monoculture is a subtle phenomenon that may not be immediately noticeable to users but can have a lasting impact on society, making it an important area of study. The paper is well-written and clearly presented.", "weaknesses": "W1: One major risk in this study is that measuring generative monoculture with “a source dataset that is likely to have been used in training the LLM we wish to investigate” (line 128) could introduce substantial inaccuracies. This approach risks significant biases, undermining the validity of the measurements. I will elaborate below and hope the authors could discuss these risks and present evidence to support validity of their approach.\n\nW1-1: First, the selected source data may not actually be part of the LLM’s training data.\n\nW1-2: Even if the selected source dataset is indeed in the training data, it may not represent the full range of relevant training data. For instance, other sources or non-English reviews of the same books could dominate the distribution from training data, meaning that the measured distribution may not reflect the true distribution of the training data at all. This concern is especially relevant since the study uses relatively small datasets (742 books with 10 reviews per book for book reviews and 100 coding questions with 20 correct solutions each for code).\n\nW1-3: Additionally, the filtering of the source dataset introduces biases. The authors mention in Appendix B2 that book reviews are filtered to be between 300-700 words. Filtering by length could affect the sentiment distribution, potentially skewing the results. \n\nW2: Could the authors clarify why they filter out low-quality (generated) book reviews by examining perplexity? While low-quality reviews may seem less helpful to users, excluding them might introduce bias in the sentiment distribution of LLM-generated book reviews. I recommend testing whether this filtering step influences the measured sentiment distribution, although it might be preferable to retain these low-quality reviews to avoid introducing additional bias.\n\nW3: To strengthen the study’s validity, I suggest measuring generative monoculture in a more controlled environment, where the training data is known and the training distribution can be accurately measured. Without certainty that the source dataset is part of the training data, all findings are at risk of being unreliable.\n\nW4: In Line 317, the efficiency of LLM generated code is measured by ``prompt GPT-3.5 to infer the big O time and space complexity’’. Please establish the reliability of this LLM evaluation.\n\nW5: In line 328, Code Summary (categorical) is measured by “prompt GPT-3.5 to assign tags to a code segment by providing it a set of tags to choose from”. Please establish the reliability of this LLM evaluation.\n\nW6: In Figure 4 (d), decay T p = 1.0 seems to have higher entropy, therefore more diverse, than source distribution. Could the authors discuss this, as it disagrees with the statement in line 379 “there exists significant narrowing from the source to generation distribution in all attributes considered for both scenarios”?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)