Probing the “Creativity” of Large Language Models:  Can models produce divergent semantic association?
Authors: Honghua Chen
Nai Ding
Venue: EMNLP 2023 Findings (2023)
Keywords: Creativity
Semantic association
Text generation
TL;DR: Investigate the creative thinking of large language models using an objective measurement of semantic structure.

Abstract: Large language models possess remarkable capacity for processing language, but it remains unclear whether these models can further generate creative content. The present study aims to investigate the creative thinking of large language models through a cognitive perspective. We utilize the divergent association task (DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. We compare the results across different models and decoding strategies. Our findings indicate that: (1) When using the greedy search strategy, GPT-4 outperforms 96% of humans, while GPT-3.5-turbo exceeds the average human level. (2) Stochastic sampling and temperature scaling are effective to obtain higher DAT scores for models except GPT-4, but face a trade-off between creativity and stability. These results imply that advanced large language models have divergent semantic associations, which is a fundamental process underlying creativity.

Probing the "Creativity" of Large Language Models: Can Models Produce Divergent Semantic Association?
Honghua Chen and Nai Ding" Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engincering and Instrument Sciences, Zhejiang University Hangzhou, China (honghuachen, ding_ nai jēzju. edu. cn
Abstract
Novel
Large language models possess remarkable ca- pacity for processing language, but it remains unclear whether these models can further gen- erate creative content. The present study aims to investigate the creative thinking of large language models through a cognitive perspec- tive. We utilize the divergent association task (DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. We compare the results across differ- ent models and decoding strategies. Our find- ings indicate that: (1) When using the greedy search strategy, GPT-4 outperforms 96% of humans, while GPT-3.5-urbo exceeds the erage human level. (2) Stochastic sampling and temperature scaling are effective to obtain higher DAT scores for models except GPT-4 but face a trade-off' between creativity and sta- bility. These results imply that advanced large language models have divergent semantic asso- ciations, which is fundamental process under- lying creativity!!
Creativity
Cognitive control
Remote
C Common
Prrosae frr comnee)
association
C Cammin
Navel
Natoern
Valuable
Token
Figure 1: Creativity from the perspective of language distribution. Creative thoughts need be novel and valuable, which need cognitive control to inhibit com- mon tokens and remote association to find valuable tokens.
humor, is important during communication. Ope- nAl (2023) has reported GPT-4's ability to under- stand jokes, while subsequent works show limited capacity for LLMs to generate or explain humor (Jentzsch and Kersting, 2023; Hessel et al., 2023). As creativity is essential to the development of art, science, and everyday life for human (Gabora and Kaufman, 2010), non-trivial if models could produce creative content. Regarding to the curse recursion for LLMs that training on generated data makes models collapse, one promising solution might be the novel language distribution through creative generation (Shumailov et al., 2023). But since LLMs represent word meaning and predict the next word in context, it seems paradoxical that such models could create ideas not seen in training. Here, we empirically investigate the creativity LLMs by examining models' ability to generate divergent concepts.
1 Introduction
Large language models (LLMss) have exhibited un- paralleled mastery of natural language (Bubeck al., 2023). The primary capacity of producing the most probable next word broadly generalizable to many language tasks, suggesting underlying cog- nitive abilities beyond specialized linguistic rules and patterns. There is observation that LLMs may possess reasoning abilities which is a core aspect of intelligence, including decision soeision-making (Binz and Schulz, 2023) and theory of mind (Moghaddam and Honey, 2023). Meanwhile, there also increas- ing interest exploring LLMSs creativity, which is closely related to intelligence (Frith et al., 2021). Creative use of language, such as metaphor and
A general definition of creativity is the ability to create something both novel and valuable (Runco and Jacger, 2012). According to the dual-process theory of creativity (Beaty et al., 2014), creative thinking relies on remote association while inhibit- ing common ideas (Figure 1). Because of the intrin- sic complexity, creativity is universally accepted to be unique to human beings, while models are
*Conresponding author 'Wo relcase our code at https:///itthu com/ SinNlab//robinggcrrettww

---

regarded as great predictors to master the existing distribution but not qualified creators to generale new distribution. However, there have been evi- dences of models possessing creativity in differ- ent domains. For artistic creation, deep generative models reveals exquisite painting skills (Ramesh et al., 2022). For algorithms, models are able to generate original and superhuman strategies in board game (Silver al., 2016).
Prompi: Pleuse write 10 nouns English are as irrelvvant from each other os possible, in all meunings and uses of the words. Please note that the words you write should have only single word, only nowns ge.g. things, objects, concepts), no proper mouns (eg. na specilicc people or places). Your answer:
GPT-4
GPI-3,5-turbe:
Human:
Gimiffe
Chair
Objective
Microscope
Cloud
Birdcago
3. Volcano
3.
Freedom
3. Pasta
Tambourine
Guisar
Obittary
5. Philooophy
Filephamt
5.
Dance
6. Refrigetator
6.
Mathematics
6.
Romance
Asteroid
Ocean
Flagellate
8. Calligraphy
8. Justice
8.
Spectacles
9. Shoelace
Happiness
Bardage
As for language models, creativity an emerg- ing concern. Since GPT-2 (Radford et al., 2019), language models are able to naturally produce an- swers given a prompt, even for open-ended and cre- ative generation tasks (Kaddour et al., 2023). How- ever, when generating texts from these probabilistic models, decoding strategy has prominent effect on the quality of result. Decoding strategies that search for the highest-probability words tend to pro- duce texts that dull and repetitive (Zhang et al., 2021), while stochastic strategies, which randomly sample from models, generate texts with better hu- man preference (DeLucia et al., 2021; Holtzman et al., 2020). These texts have human-lik- informa- tion distribution, which are viewed as informative and interesting (Meister et al., 2022). Despite this, stochastic strategies distinet from human. The decoding process is probabilistic and independent, while humans produce language under elaborated cognitive control, especially for creative generation. Creative and nonsense contents are both infrequent during next word prediction that cannot be simply distinguished via sampling strategies (Figure 1). Thus, it is unclear whether LLMs genuinely have creativity during modeling, and whether decoding strategies help.
10. Marshhalllwww
10.
Television
10. Explorratoon
Figure 2: The DAT paradigm and example responses.
studies assessed LLMs' creativity on artistic and scientific creation (Crothers et al., 2023; Park et al., 2023). However, two elements of creativity, nov- eity and value, are both relative and ambiguous during evaluation. Human ratings are affected by subjective surprise and domain knowledge, and thus differ from each other.
There are other methods based on domain- general cognitive process of creativity. Divergent thinking, i.., generating a large varicty of solu- tions from an open-ended point, is an indicator for creative thinking (Beaty et al., 2014). One of the most widely used tasks on divergent thinking is the alternate use task (AUT), which asks par- ticipants to generate unusual uses of objects (e.g., brick) (Guilford, 1964). Previous studies used AUT to measure the creativity of LLMs (Summers- Stay et al., 2023; Haase and Hanel, 2023), but the evaluation is sample-dependens that the scores are various across the selected objects. AUT also relies on humans to rate the creativity of generated uses. Moreover, AUT has the the risk of data leakage that the answers confounded by the uses recorded in the training data.
To answer both of these questions, we evaluate the creativity of LLMs. Specifically, we use an ob- jective semantic measurement, the divergent asso- ciation task (DAT), which asks models to generate unrelated nouns and compute the pairwise seman- tic distance between them (Olson et al., 2021). In summary, we make the following contributions:
Creativity has long been linked to the flexibility of semantic retrieval (Zhang et al., 2023). An alter- native to probe creativity is through the structure of semantic memory, which can be automatically and reproducibly assessed (Beaty et al., 2021; Beaty and Johnson, 2021). The DAT, among these meth- ods, is valid and reliable that closely correlates with other metrics of creativity (Olson et al., 2021). Dif- ferent from measuring semantic similarity as usual, the DAT prompts participants to reject related as- sociations and produce unrelated nouns (Figuree 2). Formally, given n. words and their word embed-
Investigate the creativity of LLMs and com- pare the results with human.
Explore the effect of decoding strategies on the creative generation of LLMs.
2 Measuring Creativity
A direct measure of creativity is relying on experts to judge the creative quality of products. Several
"Other components of creativity, such as emotion and mo- tivation, are not considered in this study.

---

dings the DAT can be calculated as the average cosine distance as follows:
ing step. For stochastic algorithms, we use top-p sampling (Holtzman et al., 2020) that limit the sam- pling space to the top-p most likely tokens at each decoding step, truncating the undesirable tail of dis- tribution. We set p = 0.9 with temperature t = 0.7 for top-p sampling. Then we adjust different set- tings of to study the effect of temperature. For each model, we collect enough samples to ensure the results convergent (Appendix A).
100
Σ (1 cos (vi, 0ÿ]) DAT nnn 1)
In this study, we apply the DAT to assess the creativity of LLMs, but before that it is necessary to evidence the applicability of this method. Basi- cally, the validity of DAT for humans comes with the bias that humans retrieve semantics exploit- ing their semantic networks. The semantic net- works of humans reveal the semantic representa- tions about the world, which are also reflected in the language distribution of human corpus. Thus, LLMs pre-trained on the corpus should exhibit the similar bias. The semantic networks are also needed for LLMs accomplish general language tasks. Empirically, previous studies showed that language models have similar patterns of semantic activations with humans (Lake and Murphy, 2020; Digutsch and Kosinski, 2023). Additionally, con- sidering these studies assessing the semantic activa- tions differently from the present study, we provide another analysis to validate the DAT for LLMs. It is noteworthy that possessing similar semantic networks is not equivalent to being equally ere- ative. Although the semantic networks of humans are roughly consistent, the ability to produce re- mote associations is challenging and largely varies among humans.
3.3 DAT
In DAT, we ask models to generate 10 unrelated nouns. we constrain the models generate only nouns isolate the semantical distances from syn- tactic effects. We use the zero-shot prompt in Fig- ure that is consistent with the study for humans (Olson et al., 2021). We filter out the answers with invalid words, e.g., verbs. Then we select the first seven valid words that models provide, and com- pute the DAT score via Eq. (1).*
We use GLoVe (Pennington al., 2014) to cal- culate semantic distance (Figure 3a). In previous studies which also used semantic space to evaluate creativity, GLoVe was proved to be effective (Beaty and Johnson, 2020). We have also experimented Word2 (Mikolov et al., 2013) and Fasttext (Bo- janowski et al., 2016), and found results similar (with the correlation coefficient of 0.82 and 0.91 respectively).
The word vectors also encode word frequency that rare words have unstable semantic distance for the lack training (Schnabel et al., 2015). Thus, we also compute the average surprisal (negative log word frequency) to study this potential effect.
3 Experiment Setup
3.1 Models
To compare the result of models with humans, use the data collected on 8572 participants. We also randomly sample nouns from Wordnet (Miller, 1995) as a situation without the bias of semantic network.
We studied five recent LLMs with different sizes, including GPT-4 (OpenAI, 2023) and GPT-3.5- Turbo (OpenAT, 2022) from OpenAI, Dasst-Llama- 30B (Köpf et al., 2023) and Vicuna-13B (Chiang et al., 2023) fine-tuned from Llama (Touvron et al., 2023), and ChatGLM-6B based on GLM (Du et al., 2022).3 GPT-4 and GPT-3.5-Turbe have advanced performance through pre-train and RLHF (Ouyang et al., 2022), while other models trained by fine- tuning foundation models on collected instructions.
3,4 Validating DAT
As mentioned Section 2, we set two additional
prompts for comparison: (1) Base: write 10 nouns, and (2) Random: write 10 nouns randomly. We hypothesize that LLMs generate semantically as- sociated words if not instructed, but can also have divergent associations under the DAT prompt.
3.2 Decoding strategy
For deterministic algorithms,, that choose the most probable token at each decod-
greedy search
"The number seven is consistent with Olson 2021 because most answers have at least seven valid words, and results are stable using over seven words.
"Specifically, we use the following versions: gpt-4-0314, gpt-3.5-turbo--ut 7-a---tt----lla--300- Vicuna-13kh 13b-delta- v1.1 and chatglm-6b--v1.0.
'The data and code of DAT for human is available at https:///ol //sff io/vjazz/.

---

a)
b)
Models
100
26.1
got-4
Top-p Greedy
Drvosuiie
25.3
Phayssdfy
70.9
got- 10--3..an S-turb
Terteurre
301
Percentile
60
vicuna 13b
Uhar
human greedy
chatgim-GD 6b
Gaten-77
20
get-3 5.tuubbo
get-4
Eephani- P3
o0 300
Mathematics
human
vicuna-133
chotglm
Caler
Exphet-
50
60
80
90
100
Creativity (DAT)
Creativity (DAT)
Figure 3: The DAT for humans and models. (a) The distance matrix words generated by GPT-4 and turrbo.-turbo. The average distance is defined the DAT. (b) The DAT of models and human. (c) The percentile of models' DAT against human results.
4 Result
The DAT results are shown in Figure 3. Using greedy search, GPT-4 achieves the highest DAT of 89.1, surpassing 96.1 % of humans, and GPT- Turbo attains DAT of 80.8 that is above the average human-level (Figure 3c). Other models per- form less well with lower DAT, which are roughly proportional to the size of models. When using top-p sampling, models other than GPT-4 are ca- pable getting the DAT much higher than greedy search, but they also become unstable that probably generate answers with low DAT scores (Figure 3b).
100
90
Creativity (DAT)
80
70
random
60
human
gpt-3.a 3.5 turbo
50
0a-30l
vicuna-13b
We also report the relation between the DAT and surprisal in Figure 4.6 Theoretically, sur- prisal is corresponding to the novelty that is an element of creativity, and the original DAT met- ric including word frequency effect is valid for human well (Olson et al., 2021). But men- tioned in Section 3.3, word frequency might be confounding variable when calculating seman- tie distance. Indeed, we find two variables highly relevant for human and random baselines (also see Appendix B). For models, the results of top-p sampling have homogeneous slopes with two base- lines, but their intercepts and surprisal distributions are different. GPT-4 and GPT-3.5-Turhe exceed the average human DAT under the same surprisal, while other models fall short. Despite Vicuna- 13B
chatglm-6a Bb
40
8
9
10
16
Surprisal (-logtp))
Figure 4: Relationship between the DAT and surprisal. Rimmed points show the results of greedy search. The contour indicates the 95% confidence interval of hu- mans.
and Chatglm-6B have similar distributions of sur- prisal, the former generates words more divergently. Oasst-Llama-30B defeats Vicuna-133 13B on the DAT, but this might be explained by the capacity or ten- dency generate rarer words. To clarify this effect, control the surprisal for DAT in Appendix D. The results are similar that GPT-4 and GPT-3.5- Turbo outperform average human performance, but the superiority of GPT-4 is attenuated.
'The DAT and surprisal for more L.MSs are shown in Appendix C

---

22-4
(ooue chelgin da
Creaivity (DNT)
70-
23
25
0.4
1.0
0.4 Temperall atume
Tamparauur
Figure 5: Effect of temperature tuning. The bands indi- cate the standard deviations.
Figure 6: The DAT scores of conditions.
However, advanced LLMs show the implicit con- trol modulating the probability distribution and stably generating divergent answers. Stochastic decoding strategy may even degrade performances for introduced randomness.
We further investigate the effect of temperature (Figure 5). Temperature is a widely deployed pa- rameter for creative or original generation in prac- tice that high temperature skews the distribution shape towards low- (ow-probability tokens (Ackley et al., 1985; Fan et al., 2018). We vary the tempera- ture from 0.1 to 1 and found its positive effect for models except GPT-4. However, the effect is limited, and high temperature will also bring insta- bility and produce invalid answers. As for GPT-4, high-probability tokens are well aligned with high- quality answers.
Conclusion
In this work, provide creativity evaluation using divergent semantic lask. This task reveals distinguishable results across various LLMs and decoding strategies. We find GPT-4 demonstrates advanced human-level creativity stably, while GPT- 3.5- tur-o exceeds the average human level. For decoding methods, stochastic decoding strategy is effective but not enough for creative generation.
In the relationship between the DAT and sur- prisal, we find naive algorithm that samples nouns from Wordnet outperform the majority of hu- mans and models (Figure 4). It is because the algorithm has no constrains of the language distri- bution, which also means can barely accomplish general language tasks. Although LLMs have ex- hibited striking mastery of natural language, we wonder whether they process the semantics differ- ently with humans and the DAT test is accordingly invalid well. Thus, compare the DAT with Base and Random conditions (figure 6). We show that if not instructed, LLMs tend to produce more related words. When instructed with the prompts Random and the DAT, LLMs can modulate the language distributions to be more divergent.
Limitation
Creativity is a deeply debated concept. We selec- tively evaluate the "little-C" (creativity every- day life) that is a general capacity, instead the "big-C" (marvelous creative product) which is rare even for human. Mcasuring creativity is also con- troversial that requires evaluations from multiple perspectives, principles, and analysis. Thus, the re- sults of this study cannot be directly generalized to all language generation tasks. We also limited the range of this study within self-contained creativity, whereas another crucial aspect of Al's creativity is human-Al co-creation. We leave these for future work.
These results indicate that LLMs have the po- tential to generate divergent content with instruc- tion, but with the flaw of not inhibiting common words. Stochastic decoding strategy is helpful for promoting remote association, but since the ere- ative and nonsense content are both infrequent, cannot accurately produce high-quality content.?
Acknowledgement
We thank Mark Sun and the anonymous reviews for their thoughtful helps and suggestions. This work was supported by ST12030-Major Projects 2021ZD020410 105 and Fundamental Research Funds for the Central Universities
"Previous psychological researches reported similar result that mild imperfection of attention is related to higher creativ- ity (Abraham, 4).

---

References
Jan Digutsch and Michal Kosinski. 2023. Overlap in meaning is a stronger predictor of semantic activa- tion in GPT-3 than in humans. Scientific Reports, 13(1):5035.
Anna Abraham. 2014. Creative thinking as orchestrated by semantic processing vs. cognitive control brain networks. Frontiers in human neuroscience, 8:95.
David H. Ackley, Geoffrey B. Hinton, and Terrence J. Sejnowski. 1985. learning algorithm for boltz- mann machines. Cognitive Science, 91):147-169
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding. Jiezhong Qiu, Zhilin Yang. and Jie Tang. 2022. GL.M: General Language Model Pretraining with Autore- gressive Blank Infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volime 1: Long Papers), pages 320-335, Dublin, Ireland. Association for Computational Lin- guistics.
Roger E. Beaty and Dan R. Johnson. 2021. Automating creativity assessment with SemDis: An open plat- form for computing semantic distance. Behavior Research Methods, 53(2)7757778.
Roger E. Beaty and Dan Richard Johnson. 2020. Au- tomating creativity assessment with semdis: An open platform for computing semantic distance. Behavior Research Methods, 53:757 - 780.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi- erarchical Neural Story Generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898, Melbourne, Australia. Association for Computational Linguistics.
Roger E. Beaty, Paul J. Silvia, Emily C. Nushaum, Emanuel Jauk, and Mathias Benedek. 2014. The roles of associative and executive processes in cre- ative cognition. Memory & Cognition, 12(7):1186- 1197.
Emily Frith, Daniel B. Elbich, Alexander P. Christensen, Monica D. Rosenberg, Qunlin Chen, Michael Kane, Paul J. Silvia, Paul Scli, and Roger E. Beaty. 2021. Intelligence and creativity share common cogmitive and neural basis. Journal of Experimental Psychol- ogy: General, 150:609-632.
Roger E. Beaty, Daniel C. Zeitlen, Brendan Baker, and Yoed N. Kenett. 2021. Forward flow and creative thought: Assessing associative cognition and its role divergent thinking. Thinking Skills and Creativity, 41:100859.
Liane Gabora and Scott Barry Kaufman. 2010. Evolu- tionary approaches to creativity. In The Cambridge Handbook of Creativity, pages 279-300. Cambridge University Press, New York, NY. US
Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 120(6))2218825) 120.
J. Pv Guilford. 1964. Some new looks at the nature of creative processes. Contributions to mathematical psychology. New York: Holt, Rinehart & Winston.
Piotr Bojanowski, Edouard Grave, Armand Joulin, and
Tomas Mikolov. 2016. Enriching word vectors with subword information. Transactions of the Associa- tion for Computational Linguistics. 5:135-146.
Jennifer Haase and Paul H. P. Hanel. 2023. Artifi- cial muses: Generative artificial intelligence chatbots have risen to human-level creativity.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Fi|- dan, Johannes Gehrke, Fric Horvitz, Ece Kamar, Pe- ter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of Artificial General Intelligence: Early experiments with GPT-44 arXiv e-prints, page MtXiv:2303.12712
Jack Hessel, Ana Marasovic, Jena D. Hwang. Lillian Lee, Jeff Da. Rowan Zellers, Robert Mankoff, and Yejin Choi. 2023. Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 688-714, Toronto, Canada. Association for Computational Linguistics.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph F. Gonzalez, Ion Stoica, and Fric P. Xing. 2023. Vicuna: An Open- Source Chatbot Impressing GPT-4 with 90%* Chat- GPT Quality.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text de- generation. International Conference on Learning Representations.
Evan Crothers, Herna I. Viktor, and Nathalie Japkow- icz. 2023. In BLOOM: Creativity and Affinity in Artificial Lyrics and Art. In The AAAI-23 Workshop on Creative Al Across Modalities.
Sophie Jentzsch and Kristian Kersting. 2023. Chatgpt is fun, but it is not funny! humor is still challenging large language models.
Alexandra DeLucia, Aaron Mueller, Xiang Lisa Li, and João Sedoc. 2021. Decoding Methods for Neural Narrative Generation. In Proceedings of the 1st Work- shop Natural Language Generation, Evaluation, and Metrics ( (GEM 2021), pages 166-185, Online. Association for Computational Linguistics.
Jean Kaddour, Joshua Harris, Maximilian Mozes, Her- bie Bradley, Roberta Raileanu, and Robert McHardy. 2023. Challenges and Applications of Large Lan- guage Models. 1xXiv:2307.10166 [cs].

---

Andreas Köpf, Yannic Kilcher, Dimitri von Rütte. Sotiris Anagmostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richard Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexan- der Mattick. 2023. OpenAssistant Conversations Democratizing Large Language Model Alignment. ArXiv:2304.07327 [es].
Alee Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8):
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text- conditional image generation with clip latents.
Mark A. Runco and Garrett J. Jaeger. 2012. The Stan- dard Definition of Creativity. Creativity Research Journal, 24(1):92-90
Brenden M. Lake and Gregory L. Murphy. 2020. Word meaning in minds and machines. Psychological re- view.
Tobias Schnabel, Igor Labutov, David Mimno, and Thorsten Joachims. 2015. Evaluation methods for unsupervised word embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Nat- ural Language Processing, pages Portugal. Association for Computational Linguistics.
Clara Meister, Gian Wiher, Tiago Pimentel, and Ryan Cotterell. 2022. On the probability-uuaii paradox in language generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Papers), pages 36-45, Dublin, Ireland. Association for Computational Lin-
Lisbon,
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papemot, and Ross Anderson. 2023, The curse of recursion: Training generated data makes models forget.
guistics.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed repre- sentations of words and phrases and their composi- tionality. In Neural Information Processing Systems.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Ju- lian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Mare Lanctot, Sander Dieleman, Do- minik Grewe. John Nham, Nal Kalchbrenner. llya Sutskever, Timothy Lillicrap. Madeleine Leach, Ko- ray Kavukcuoglu, Thore Graepel, and Demis Hass- abis. 2016. Mastering the game of go with deep neu- ral networks and tree search. Nature, 529(7587):484- 489. Number: 7587 Publisher: Nature Publishing Group.
George A. Miller. 1995. WordNet: a lexical database for English. Communications of the ACM, 38(() I 1):39- 41.
Shima Rahimi Moghaddam and Christopher! Honey 2023. Boosting Theory-of-Minn Performance in Large Language Models via Prompting.
Jay A. Olson, Johnny Nahas, Denis Chmoulevitch, Si- mon Cropper. and Margaret E. Webb. 2021. Nam- ing unrelated words predicts creativity. Proceedings of the National Academy of Sciences, 118(25).
Douglas Summers-Stay, Clare R. Voss. and Stephanie M. Lukin. 2023. Brainstorm, then Select: A Generative Language Model Improves Its Creativity Score. In The AAA1-23 Workshop on Creative Al Across Modalities.
OpenAl. 2022. Introducing ChatGPT.
OpenAI. 2023. ArXiv:2303.08774 [cs].
GPT-4
Technical Report.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. ArXiv:2302. 13977 [cs].
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems,
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. 2021. Trading Off Diversity and Quality in Natural Language Generation. In Proceedings of the Workshop Human Evaluation of NLP Systems (HumEval), pages 25-33, Online. Association for Computational Linguisties.
Yang Jeong Park, Daniel Kaplan, Zhichu Ren, Chia- Wei Hsu, Changhao Li, Haowei Xu, Sipei Li, and Ju Li. 2023. Can chatgpt be used to generate scientific hypotheses?
Jingyi Zhang, Kaixiang Zhuang. Jiangzhou Sun, Cheng Liu. Li Fan, Xueyang Wang. Jing Gu, and Jiang Qiu. 2023. Retrieval flexibility links to creativity: evi- dence from computational linguistic measure. Cere- bral Cortex, 336447996.44976
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Confer- ence on Empirical Methods in Natural Language Pro- cessing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.

---

A Selecting sample size for each model
C
Result of the DAT and surprisal for more models using greedy search
Figure 7 shows the sample size to generate sta- ble results using top-p sampling for cach model. With confidence coefficient a = 0.05, standard deviation â and error e = 1, we choose N > (Aa x 0/e)² 1.96² x _². We find larger mod- cls (GPT-4 and HPPT-..5-Turbo generate answers
Figure 9 shows the results of the DAT and surprisal for more LLMs using greedy search.
90-
more stably.
Creativity (DAT)
85-
80
Upl- 3.5 turbo
200
70-
gph-1
chargleed8
gpn turbo
85
Recechan-1788
00-306
Text dovir 203
Creatiiity (DAI)
chatglm-Bh 86
55-
50
Surprisal (-log(p)
Figure 9: Results of the DAT and surprisal for more
LLMs. The contour indicate the 95% confidence inter- val.
10
10'
sample
Figure 7: Results of the DAT across different sample sizes. The bands indicate the standard deviations.
D Controlling surprisal as confounding variable
Considering the potential inffuence of word fre- quency on measuring semantic distance, we control surprisal (Figure 10 ). The results are similar as before that GPT-4 and GPT-3.5-Turbo outperform average human performance, while other models are below average human level. However, GPT-4 as well as Oasst-Llamaa30t lose their superiorities because their high DAT scores partially depend on generating rarer words.
B Results of the DAT and surprisal on human and random and baselines
Figure 8 shows the results of the DAT and surprisal on human and random baselines. We find positive relationship between surprisal and the DAT. Ran- dom baseline a strong baseline that has maximal remote association (uniform distribution) despite without inhibition. Even so, we find some people surpass random baseline and approach ceiling DAT at specific section of surprisal.
1004
80-
300
Percentle
human
50
Creativity (DAT)
54.
25 10 d5 10
30
70
20
Creativity (DAT)
Creat vity (DAT)
Figure 10: Results of the DAT when controlling sur-
60
prisal as a confounding variable.
12
15
Surprisal (-logip))
Figure 8: Results of the DAT surprisal on human and random baseline. Contours indicate the 95% confi- dence intervals.


Reviews:
[{"review_id": "OnNLQbwkY9", "reviewer": "Reviewer_9M5g", "summary": "", "questions": "A. What is the reason to use Glove to to calculate semantic distance in DAT?\n\nB. Have you thought about integrating surprisal score with DAT score for measuring creativity?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your helpful comments. As you said, creativity in language is related to the ability to put together two seemingly different words that are actually related and could mean something interesting. But measuring the combinations of multiple words directly is challenging that the interesting compositional meaning is hard to define and assess in a standard way. In DAT, we potentially require language models to recognize and reject these word pairs that are related in some aspects. As for the words other than nouns, we consider the syntactic factors might influence the semantic distances, which could be confusing. We note that using the semantic space of only nouns is sufficient to distinguish the creativity among models and humans. However, considering word combinations and words other than nouns is undoubtedly desirable. We would refine the method description and discuss these points in the limitations section.\n\nIn response to your more detailed questions, \n\n- Q1: What is the reason to use Glove to to calculate semantic distance in DAT?\n    \n    A1: Since the DAT task is not contextual, the semantic distance should be calculated using static word vectors, while Glove provides well-constructed static vectors for word representations. In the study that introduced DAT for humans [1], the authors have demonstrated that DAT scores calculated by Glove correlate strongly with other creativity metrics. In other studies which used semantic space to evaluate creativity, Glove was proved to be useful as well [2]. Thus, we kept the same experimental setup here. We have also experimented various word vectors, e.g., Word2Vec and Fasttext, and found similar results (with the correlation coefficient of 0.82 and 0.91 respectively). We would discuss the impact of choosing Glove in the appendix.\n    \n- Q2: Have you thought about integrating surprisal score with DAT score for measuring creativity?\n    \n    A2: Thank you for your suggestion. Surprisal score reveals the ability to generate low-frequency words, which is relevant to creativity. For humans, we do find surprisal score has a significant correlation with other creativity metrics. But as evidenced in the experiment, we find DAT score correlated with surprisal score closely, and integrating surprisal with DAT do not improve the predictive power for other metrics. In other words, the original DAT score has already integrates the effect of surprisal. \n    \n\n**Reference**:\n\n[1] Jay A. Olson, Johnny Nahas, Denis Chmoulevitch, Simon J. Cropper, and Margaret E. Webb. (2021). Naming unrelated words predicts creativity. Proceedings of the National Academy of Sciences, 118(25). https://doi.org/10.1073/pnas.2022340118\n\n[2] Beaty, R.E., Johnson, D.R. (2021). Automating creativity assessment with *SemDis*: An open platform for computing semantic distance. *Behav Res* **53**, 757–780. https://doi.org/10.3758/s13428-020-01453-w.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper discusses an interesting take on creativity of LLMs.", "reasons_to_reject": "The authors specifically looks at single nouns but do not adequately justify why combinations of multiple words (specifically creativity in language is tightly related to the ability to put together two seemingly different words that could mean something interesting, taken together) were not taken into consideration or why words other than nouns were not considered. These would be necessary points to be discussed in the limitations section.", "questions_to_authors": "A. What is the reason to use Glove to to calculate semantic distance in DAT?\n\nB. Have you thought about integrating surprisal score with DAT score for measuring creativity?", "paper_topic_and_main_contributions": "The authors try to quantify the creativity of LLMs through an  objective semantic measurement, the divergent association task (DAT). They further explore the effect of decoding strategies through decoding strategies.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The paper could use a complete revision to better structure the sentence and overall structure of the article. (e.g., Word frequency, however, is also encoded in the embedding vectors that rare words might have unstable semantic distances --> sentences like these are harder to understand)"}}, {"review_id": "Rg9aFkUE96", "reviewer": "Reviewer_hFka", "summary": "", "questions": "Figure 1 is rather unclear to me: what should it show?\n\npage 2, right column, mid: you mention AUT is not a good evaluation as it is sample-dependent, subjective, and risks data leakage: can you substantiate this criticism more?\n\npage 3, right column, top: how are valid words determined?\n\nyou operationalise surprisal as the inverse of token frequency, but the second word is generated in the context of the first, and so on, so why not also considering surprisal given the words already generated?\n\npage 4, left column, top: probably generate low quality answers: why probably? what do you mean?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive and thoughtful feedback. You raised an important concern, and below are some explanations about the remarks. \n\n- The main controversy is the applicability of the DAT paradigm on LLMs. We agree with your exact description about the biases of humans that retrieves associations by exploiting the semantic networks, and we would like to show the same biases for LLMs. Theoretically, the semantic networks of humans reveal the semantic representations about the world, which are also reflected in the corpus. Thus, LLMs pretrained on the corpus exhibit the similar bias. In other words, LLMs learn a language distribution from the corpus, and therefore derive similar word associations with humans. Empirically, Digutsch and Kosinski [1] revealed LLM’s similar patterns of semantic activations with humans that is mentioned in page2, line 146. More evidences can be found here [2-3]. These studies have shown the applicability and challenge of the DAT paradigm on LLMs that LLMs struggle to generate unrelated words under the constraint of language distribution. Our results also verified this restriction. In the final version, we would supply these arguments into Section Measuring Creativity.\n- The random baseline, as you said, does not have such a bias. That means an extreme flexibility of semantic retrieval [4], but does not mean all aspects of creativity. In the section of measuring creativity, we indicate that divergent thinking is an indicator for creative thinking, while creativity also need convergent thinking that is choosing the best answer to a problem based on divergent thinking. The DAT reflects an important aspect of creativity, and we appeal for more comprehensive evaluations.\n- Since the random baseline does not have such a bias, it seems meaningless to set such a random condition. But we would like to emphasize the importance of randomness towards creativity, which is also the reason we assess the impact of decoding strategies. For human, creativity roots in the incidental connections that seemly different things can be related. For machines, evidences from AI painting and NLG underline the effect of random seed and sampling strategies. Indeed, we found some models leverage sampling strategies to improve DAT scores. Even so, randomness is not perfect, for it cannot reject related words. The high DAT score of random baseline comes from the property of high dimensional vector space that two random words are generally distant. However, we can see many people and some models approach or exceed random baseline, in spite of the constraint on flexibility.\n- To make these arguments more convincible, we conduct two brief supplementary experiments:\n    1. **Random generation of LLMs**. We ask LLMs to generate nouns randomly and find models cannot literally do that like a random baseline. Instead, regardless of controlling surprisal or not, they get DAT scores worse than under the original DAT prompt, let alone the random baseline. The results indicate LLMs have the bias to generate related tokens, i.e., they have limited remote association.\n    2. **Choosing unrelated words from a set of given words**. We further control this bias that we sample 30 nouns words first and ask models to choose 10 nouns that are as unrelated as possible with each other. As the words are given, there are no difference on the retrieval flexibility. We find that when exposed to a set of words, LLMs can perform better than randomly choosing 10 nouns from 30 given nouns, especially for GPT-3.5-turbo and GPT-4. The results show that LLMs have the capacity of recognizing unrelated words, i.e., the ability of cognitive control in the theory framework of creativity.\n    \n    As creativity is a integrated process, we tend to use the DAT score as a global metric. However, these supplementary results detail the mechanisms of DAT task for LLMs. We would like to supply these results in the final version into the result section or appendix.\n    \nIn response to your more detailed questions, \n\n- **Q1**: Figure 1 is rather unclear to me: what should it show?\n    \n    **A1**: Figure 1 shows the theory framework of creative generation. Creative generation differs from the general language generation that the content should be novel and valuable. Considering language generation as decoding from a conditional probability distribution, in creative generation, we should inhibit the probability of common tokens and enhance the probability of creative tokens. Based on this framework, we investigate the creativity of LLMs that whether these models can reject related words and generate unrelated words. \n    \n- **Q2**: page 2, right column, mid: you mention AUT is not a good evaluation as it is sample-dependent, subjective, and risks data leakage: can you substantiate this criticism more?\n    \n    **A2**: AUT asks participants to generate unusual use of objects, and it has a standard questionnaire. The number of objects is generally 3-5, while the scores are various across the objects selected. What’s more, AUT relies on humans to rate the creativity of generated unusual uses, which is subjective. Lastly, since AUT is a classic test with standard scale, there are many answers on the web, which has the risk of data leakage.\n    \n- **Q3**: page 3, right column, top: how are valid words determined?\n    \n    **A3**: Valid words refers to the words that satisfy the requirement in the prompt, i.e., single words, only nouns, and no proper nouns.\n- **Q4**: you operationalise surprisal as the inverse of token frequency, but the second word is generated in the context of the first, and so on, so why not also considering surprisal given the words already generated?\n    \n    **A4**: We use the token frequency here in order to check the effect of independent word frequency, which might be confusing. We wonder if you want to confirm that, since DAT prompts models to generate unrelated words, the surprisal given the words already generated should be large. However, it is hard to calculate the surprisal because noun sequences are not natural sentences, and we cannot define a standard language distribution. Instead, we use the semantic distance to quantify the unrelatedness.\n- **Q5**: page 4, left column, top: probably generate low quality answers: why probably? what do you mean?\n    \n    **A5**: The low-quality answers refer to the answers with DAT scores lower than greedy search. We show that sampling strategies generate unstable results because these strategies would choose some related words with low probability that have been rejected by LLMs.\n\nWe would refine the article in the final version according to your useful suggestions. We hope that our responses and updates can be carefully considered.\n\n### **reference**:\n\n[1] Jan Digutsch and Michal Kosinski. (2023). Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans. Scientific Reports, 13(1):5035. https://doi.org/10.1038/s41598-023-32248-6\n\n[2] Lake, B. M. & Murphy, G. L. (2021). Word meaning in minds and machines. Psychol. Rev. https://doi.org/10.1037/rev0000297 \n\n[3] Lenci, A., Sahlgren, M., Jeuniaux, P. *et al.* A comparative evaluation and analysis of three generations of Distributional Semantic Models. *Lang Resources & Evaluation* **56**, 1269–1313 (2022). https://doi.org/10.1007/s10579-021-09575-z\n\n[4] Jingyi Zhang, Kaixiang Zhuang, Jiangzhou Sun, Cheng Liu, Li Fan, Xueyang Wang, Jing Gu, and Jiang Qiu. (2023). Retrieval flexibility links to creativity: evidence from computational linguistic measure. Cerebral Cortex, 33(8):4964–4976. https://doi.org/10.1093/cercor/bhac392", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The question is definitely timely and pressing, careful studies on the matter are needed and likely to get us new knowledge.", "reasons_to_reject": "See above: I think the paper does not show what it purports to show due to a conceptually flawed experimental design.\n\n-----------------------------------\nI think the analyses provided are now sufficiently informative. The rhetoric should probably be toned down a bit about this being creativity rather than a narrow aspect of it, but I do see the pros of having a catchy title. The presentations should also be improved.", "questions_to_authors": "Figure 1 is rather unclear to me: what should it show?\n\npage 2, right column, mid: you mention AUT is not a good evaluation as it is sample-dependent, subjective, and risks data leakage: can you substantiate this criticism more?\n\npage 3, right column, top: how are valid words determined?\n\nyou operationalise surprisal as the inverse of token frequency, but the second word is generated in the context of the first, and so on, so why not also considering surprisal given the words already generated?\n\npage 4, left column, top: probably generate low quality answers: why probably? what do you mean?", "paper_topic_and_main_contributions": "The study probes a host of LLMs for their ability to generate divergent associations using the DAT. The study further assesses the impact of the temperature parameter on the average semantic similarity of generated responses. In doing so, the study controls for surprisal, since low frequency words may have less stable semantic representations.\n\nIn general, I find the paper misses the mark substantially in probing LLMs for creative thinking, as evidenced by the performance of a random WordNet-based baseline, which succeeds at the task but can hardly be qualified as creative. Maybe I'm wrong here, but the DAT is validated with humans, who come with their biases about retrieving associations exploiting the semantic network and its structure. Showing divergent associations in a model on a task validated and calibrated with humans without first assessing whether that model replicates the same biases humans show when producing free associations does not show creativity at all. So, in order for these results to be taken seriously I think it is paramount to show that the same models exhibit foraging patterns when producing free associations (e.g., when ask to generate as many words starting with, say, the latter b, humans typically start from words from a same semantic field, deplete what they can think of then move to a different semantic field, deplete it, and so on, or when they are asked to generate words for animals they might begin from farm animals, then move to birds, then fish, and so on...). the DAT measures creativity given strong assumptions about the underlying semantic space and strategies to query it, but the paper does not do enough in showing that models match this. Without this bit, a simple random algorithm exploiting a taxonomy wins at being creative, which makes little sense.\n\n-----------------------------------------\nThe extra analyses provided in the rebuttal do improve the credibility of the paper but I still think a crucial aspect is missing, which I articulated in the comment to the rebuttal.\n\n-----------------------------------------\nThe second set of extra analyses now more convincingly show that if not explicitly instructed, LLMs will retrieve related concepts but some can overcome this bias if asked to retrieve unrelated items.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The paper is not very well written and would benefit from a thorough copy-editing.\n-----------------------\nThis likely still applies, but it's hard to judge whether it's been addressed from the rebuttals."}}, {"review_id": "hlqiLg6nUo", "reviewer": "Reviewer_qcAx", "summary": "", "questions": "- How can the (cosine) distance between words be larger than 100? (Fig. 3a, Philosophy-Tambourine).\n- How do you ensure/validate that the generated words are all nouns and not other types of words?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your helpful comments and careful corrections about the typos. \n\n- **The applicability of the DAT paradigm**. The first concern is that DAT might not transfer to LLMs without a validation. On the one hand, as there is no criterion about the creativity of LLMs, we cannot strictly validate it. However, we compared the results with a recent study [1], which uses AUT to evaluate the creativity of LLMs, and found similar results. It shows GPT-4 approaches the max human score, while GPT-3 achieves average human score, followed by Alpaca and other models. We appeal more related works for comprehensive validation and evaluation. On the other hand, previous works indicate that LLMs possess similar patterns of semantic activations with humans [3-4], which means LLMs has a similar semantic bias. Based on this assumption, we consider DAT as an appropriate method to transfer from humans to LLMs. In the final version, we would supply these arguments into the sections of *Measuring Creativity* and *Result*.\n- **The one-dimensional nature of cosine similarity**. As you said, a model could \"cheat\" by generating words that are not only semantically unrelated, but also syntactically. Thus, we constrain the models to generate only nouns to isolate the semantical unrelatedness. And you also concern about this requirement would restrict tokens to come from a particular “region”.  However, we note that using the semantic space of nouns is sufficient to distinguish the creativity among models and humans. It is effective, though considering words other than nouns is more comprehensive. We would refine the method description in the final version.\n- **DAT is computed only using the GLoVe cosine distances**. In previous studies which also used semantic space to evaluate creativity, GLoVe was proved to be useful [4-5]. We have also experimented various word vectors, e.g., Word2Vec and Fasttext, and found similar results (with the correlation coefficient of 0.82 and 0.91 respectively). In the final version, we would detail the comparing of word vectors to indicate this impact in the appendix.\n\nIn response to your more detailed questions, \n\n- **Q1**: How can the (cosine) distance between words be larger than 100? (Fig. 3a, Philosophy-Tambourine).\n    \n    **A1**: The cosine distance is calculated as 1-cos(word_1,word_2), and we multiply it by 100, thus the distance ranges from 0 to 200. In practice, scores commonly range from 65 to 90 and rarely exceed 100.\n    \n- **Q2**: How do you ensure/validate that the generated words are all nouns and not other types of words?\n    \n    **A2**: We add the requirement in the prompt, and the models generally follow our instruction. In the analysis, we also use wordnet to validate whether the answers are nouns.\n    \n- **Q3**: What does \"Using creativity for self-iteration\" imply? This is too vague to me.\n    \n    **A3**: A recent study shows that the use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear [6]. They refer to it as *model collapse*, i.e. the vanishment of the tails of the original content distribution would make LLMs converge to a limited set of outputs. In other words, LLMs at present always generate common tokens, while the lack of creative generation would pollute the corpus. Thus, if models can generate content creatively, this problem can be largely settled.\n    \n- **Q4**: I fail to see how Figure 1 exemplifies that \"inhibiting common ideas emphasizes top-down attention\".\n    \n    **A4**: We are sorry that we use figure 1 to show the remote association and inhibition (a process of cogtive control) in the creative generation, while the top-down control of attention is from the cited dual-process theory. We would like to compare this with the sampling strategies for LLMs. But as we have mentioned this point in the next paragraph, we will delete it in the current paragraph.\n    \n\nWe would refine the article according to your useful suggestions. We hope that our responses and updates can be carefully considered.\n\n**Reference**:\n\n[1] Haase, J., & Hanel, P.H. (2023). Artificial muses: Generative Artificial Intelligence Chatbots Have Risen to Human-Level Creativity. *ArXiv, abs/2303.12003*. *https://doi.org/10.48550/arXiv.2303.12003*\n\n[2] Jan Digutsch and Michal Kosinski. (2023). Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans. Scientific Reports, 13(1):5035. https://doi.org/10.1038/s41598-023-32248-6\n\n[3] Lake, B. M. & Murphy, G. L. (2021). Word meaning in minds and machines. Psychol. Rev. https://doi.org/10.1037/rev0000297 \n\n[4] Jay A. Olson, Johnny Nahas, Denis Chmoulevitch, Simon J. Cropper, and Margaret E. Webb. (2021). Naming unrelated words predicts creativity. Proceedings of the National Academy of Sciences, 118(25). https://doi.org/10.1073/pnas.2022340118\n\n[5] Beaty, R.E., Johnson, D.R. (2021). Automating creativity assessment with *SemDis*: An open platform for computing semantic distance. *Behav Res* **53**, 757–780. https://doi.org/10.3758/s13428-020-01453-w.\n\n[6] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. (2023). The curse of recursion: Training on generated data makes models forget. *ArXiv, abs/2307.10169*. https://doi.org/10.48550/arXiv.2307.10169", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper provides a focused analysis of creativity in LLMs, and addresses various interesting factors related to creativity (surprisal, temperature).\n- The authors propose a simple and computationally efficient way of assessing creativity in LLMs, leveraging findings from creativity assessment in human language processing.", "reasons_to_reject": "The paper makes various assumptions that are not adequately addressed:\n- The findings for creativity found in humans where the DAT task predicts human creativity does not necessarily transfer to LLMs; taking this as a given skips the step of validating the finding of Olson et al. (2021) for the domain of LLMs.\n- Cosine similarity has traditionally been shown to correlate with semantic relatedness, but its one-dimensional nature makes it suboptimal to measure semantic similarity in isolation. E.g., by requesting to only predict nouns we are already restricting token predictions to come from a particular \"region\" in the embedding space. Since cosine similarity does not differentiate between semantic relatedness and other linguistic dimensions (syntact/morphological/etc.), a model could \"cheat\" by generating words that are not only semantically unrelated, but also syntactically.\n- DAT is computed only using the GLoVe cosine distances, but the impact of this choice is not addressed. GLoVe vectors should not be taken as a gold truth for semantic similarity.", "questions_to_authors": "- How can the (cosine) distance between words be larger than 100? (Fig. 3a, Philosophy-Tambourine).\n- How do you ensure/validate that the generated words are all nouns and not other types of words?", "paper_topic_and_main_contributions": "The paper investigates the *creativity* of Language Models, in a quantifiable way. A common critique of LMs is that while they are highly adapt at imitating human language use, they still severely lack in creativity.\n\nCentral to the paper is the Divergent Association Task (DAT), a recently introduced task (Olson et al., 2021) which has been shown to correlate strongly with creativity in humans. The task asks to generate a list of 10 nouns that are as irrelevant as possible from each other. \n\nThe authors consider various recent LLMs of increasing sizes, and use the cosine similarity between the generated nouns using GLoVe embeddings to compute the divergence score of the DAT task. \n\nThe authors show that their largest model, GPT-4, scores better on the DAT task than human performance, and that creativity improves with model size. Other factors that are investigated is the connection between the DAT score and model surprisal (positive correlation), and the impact of the decoding strategy (greedy vs. stochastic) and the influence of temperature on decoding.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "-"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The title is ungrammatical, it should be \"Probing\" instead of \"Probe\" \n\n- L031: \"Reasoning as a core aspect of intelligence [..] has been found from LLMs\" reads odd\n- L045: What does \"Using creativity for self-iteration\" imply? This is too vague to me.\n- I fail to see how Figure 1 exemplifies that \"inhibiting common ideas emphasizes top-down attention\".\n- L068: \"program\" -> \"programming\"? \"Computer programs\"?\n- L115: \"process\" -> \"processes\"\n- In general the writing would benefit from another pass, in a lot of cases singular nouns are missing a determiner (or they should be pluralised).\n\n- The lines in Figure 4 are quite hard to discern with all the scatter info in there as well. Increasing the opacity of the scatter points could already help here, to make the lines stand out more."}}]

Author rebuttal:

Meta-review: This paper aims to probe the creativity of LLMs, operationalized in terms of divergent semantic association.
The reviewers agree that this is a timely topic, and provided a relatively positive assessment.

The scores were relatively consistent, in the medium to high range:  3,4,3 for both soundness and excitement. Some of the concerns raised in the original reviews were addressed in the discussion period, leading to increases in the scores. 

Distilling the reviewers, the following major strengths and weaknesses were noted.

Strengths:

- The paper was judged as studying an interesting and timely question, in an interesting and intuitive way.

Weaknesses:

- paper specifically considered single nouns (R1)
- R2 (hFka) originally considered the paper insufficient, but was convinced by additions made during the discussion period.
- R3 originally found various unaddressed assumptions (validating DAT task for LLMs, concerns about cosine distance, focusing on GLoVe). After the author response, they found some of their concerns to have been addressed.

In conclusion, this paper was judged to be good or strong in terms of soundness, and scored "ambivalent" or even "strong" in terms of excitement.
Decision: Accept-Findings