Venue: NAACL 2025

# Benchmarking Language Model Creativity:

A Case Study on Code Generation

Yining Lu\({}^{*,l}\)   Dixuan Wang\({}^{\alpha}\)   Tianjian Li\({}^{\alpha}\)    Dongwei Jiang\({}^{\alpha}\)

**Sanjeev Khudanpur\({}^{\alpha}\)  Meng Jiang\({}^{i}\)   Daniel Khashabi\({}^{\alpha}\)**

\({}^{1}\)University of Notre Dame   \({}^{\alpha}\)Johns Hopkins University

Work done at the Johns Hopkins University.

###### Abstract

As LLMs become increasingly prevalent, it is interesting to consider how "creative" these models can be. From cognitive science, creativity consists of at least two key characteristics: _convergent_ thinking (purposefulness to achieve a given goal) and _divergent_ thinking (adaptability to explore new environments or constraints) [26]. In this work, we introduce a framework for quantifying LLM creativity that incorporates the two design ingredients: (1) We introduce Denial Prompting which pushes LLMs to develop more creative solutions to a given problem by incrementally imposing new constraints on the previous solution, compelling LLMs to adopt new strategies. (2) We define NeoGauge, a metric that quantifies both convergent and divergent thinking in the generated creative responses by LLMs. We test the proposed framework on Codeforces problems, which serve as both a natural dataset for coding tasks and a collection of prior human solutions. We quantify NeoGauge for various proprietary and open-source models and find that even the most creative model, GPT-4, still falls short of demonstrating human-like creativity. We also experiment with advanced reasoning strategies (MCTS, self-correction, etc.) and observe no significant improvement in creativity. As a by-product of our analysis, we release NeoCoder dataset for reproducing our results on future models.1

Footnote 1: Our code and data: github.com/JHU-CLSP/NeCoCoder

_Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 2776–2794

April 29 - May 4, 2025 ©2025 Association for Computational Linguistics

## 1 Introduction

Most recent works on LLM creativity evaluation focus on open-ended generation tasks, such as storywriting [3, 8, 12], paper abstract generation [13], and role-play discussion [13]. However, the degree to which LLMs possess and utilize _creativity_ for problem-solving remains unclear. An automatic method for evaluating LLMs creativity could help developers better understand the emergence of model behaviors and serve as a design objective in solving complex real-world problems.

However, despite the importance of creativity evaluation in problem-solving, only a few works have touched upon it [1, 12] because of two major challenges: (1) eliciting diverse and creative solutions is difficult [1, 13, 14], and (2) there are no reliable and comprehensive quantitative measurements of LLM creativity. Below, we explain how we tackle these two challenges for evaluating LLM creativity in problem-solving settings.

LLM generations are often repetitive and regurgitating training data [15, 16, 17, 18, 19], making it hard to elicit creative generations. However, we argue that an effective creativity evaluation method should be based on the spectrum of maximal _creative solu

Figure 1: An overview of how Denial Prompting encourages creative solutions. A solution space is a collection of all possible solutions at a certain state. \(\wedge\), B indicate atomic techniques (e.g., for-loops, if-else, etc.) used in the solution.



tions_ attained from LLMs. Therefore, we introduce Denial Prompting (SS3.1), a prompting method that iteratively "denies" one of the basic tools, techniques, or strategies used in the previous solution (e.g., A: for loops and B: if-else in Figure 1), thereby pushing LLM to think out-of-the-box and elicit creative generations to its fullest extent.

Another challenge in creativity evaluation is to build a reliable and comprehensive quantitative measurement. We propose that such evaluation should be _state-aware_--_adaptive to different contexts_, and _human-grounded_--_comparing LLM-generated solutions to historical human solutions_. According to many cognitive studies, human creativity is viewed as taking place in the interaction with a person, environment, or another model Amabile [20]; Csikszentmihalyi [45, 67, 89, 12]. Similarly, the essence of LLM creativity should also be captured from its interaction with the current state (state-aware) and past human knowledge background (human-grounded). This understanding reveals that creativity evaluation should be dynamic, with an individual's creativity varying under different contexts. For example, in Figure 1, a solution at state \(t=0\) probably will not be judged at the same creative level as one at state \(t=2\), even if they solve the same problem. Because the latter solution is more likely to use novel techniques that humans hardly thought of, such as C: Recursion, to adapt to increasingly challenging constraints.

To address the second challenge, we propose NeoGauge score (SS4) which involves (1) verifying the correctness of the LLM-generated solution and whether it adheres to the specified constraints from Denial Prompting (convergent thinking), and (2) assessing solution novelty by contrasting it with techniques previously used in human solutions (divergent thinking). This aligns well with the arguments made by Runco [10] that creative achievement depends on both the number of alternative solutions and the generation of high-quality alternatives. By considering both convergent Lubart [20, 12, 13, 14] and divergent Guilford [11, 12, 13] creative thinking, NeoGauge not only offers a state-aware evaluation but grounds the evaluation in collective human knowledge through comparing the generated solutions with historical human solutions.

In our experiments, we apply Denial Prompting on Codeforces,2 a challenging Text-to-Code task where model solutions can be automatically verified and allows comparison to substantial historical human solutions.3 Specifically, we retrieve 199 latest problems from Codeforces along with 30 human solutions per problem that have successfully passed unit tests. We then run these problems on Denial Prompting to obtain our dataset NeoCoder which consists of original questions with sequences of temporally relevant and increasingly difficult constraints. Examples of NeoCoder are provided in Table 4. We benchmark a broad range of LLMs on NeoCoder and calculate their NeoGauge scores. Additionally, we evaluate four reasoning strategies, MCTS Zhang et al. [23], self-correction Shinn et al. [20], planning Jiang et al. [23], and sampling Chen et al. [20], on our dataset to study the correlation between augmented machine intelligence and creativity. In summary, our contributions are twofold:

Footnote 2: [https://codeforces.com/problemset](https://codeforces.com/problemset)

* We introduce Denial Prompting to elicit creative generations from LLMs and NeoGauge metric to evaluate LLM creativity in problem-solving that follows the two proposed protocols.
* We release a creativity benchmark NeoCoder and provide a thorough analysis of creativity on SOTA language models and reasoning strategies.

## 2 Background and Related Works

We discuss the existing works on machine creativity evaluation. Then, we explain the concepts of divergent and convergent creativity in cognitive science which our evaluation incorporates.

Machine Creativity Evaluation.While the extensive studies on human creativity from psychological and cognitive science Amabile [20, 12, 13, 14, 15, 16], LLM creativity has received little attention. Existing works studying LLM creativity in problem-solving settings DeLorenzo et al. [20, 12, 13], however, tend to overlook two challenges: (1) eliciting creative LLM solutions, and (2) ensuring evaluation metrics are grounded and comprehensive.

 

Tian et al. (2024) have released a challenging real-world problem dataset to push LLM to think out-of-the-box, but they do not provide an automatic creativity evaluation method built upon their dataset. Additionally, their problems are constructed from a single constraint. In contrast, our Denial Prompting is formulated for multiple iterations of constraint detection and problem refinement, making the generations more creative and providing more states for creativity evaluation. Another concurrent work Atmakuru et al. (2024) also employs multiple constraints to facilitate creative generation; however, their evaluation primarily targets linguistic creativity Lu et al. (2024b) and it is tested on open-ended story writing task. Zhu et al. (2024) and Xu et al. (2024a) design protocols to dynamically generate challenging problems with controllable constraints. However, their evaluation mainly focuses on accuracy rather than creativity.

Chakrabarty et al. (2024a), DeLorenzo et al. (2024), and Zhao et al. (2024) introduce automatic evaluation pipelines to quantify the four subcomponents of creativity proposed in the Torrance Tests of Creative Thinking Torrance (1966): fluency, flexibility, originality, and elaboration. However, the test is originally designed to study human divergent creative thinking (SS2) and is unclear whether it applies to machine creativity.

Divergent Creative Thinking.Diversent thinking is a cognitive process that involves exploring a multitude of potential applications for a given set of tools Holyoak and Morrison (2005). It typically occurs _spontaneously_ and _randomly_, leading to numerous possible solutions. Extensive research Amabile (1982); Guilford (1950) has been conducted to study divergent creativity, including popular psychometric approaches such as the Unusual Uses Test Guilford (1950). These are designed to let examinees think of as many uses for a (common or unusual) object as possible. The underlying idea of stimulating creative solutions from constrained and unusual settings is also adopted in our Denial Prompting.

Divergent thinking can also be viewed through the lens of \({\cal P}\)-creativity (Psychological) and \({\cal H}\)-creativity (Historical) defined by Boden et al. (1994). A valuable idea is \({\cal P}\)-creative if the person in whose mind it arises could not have come up with it before. Furthermore, a valuable idea is \({\cal H}\)-creative if it is \({\cal P}\)-creative, and no one else in human history has ever had it before. \({\cal P}\)-creativity measurement is embedded in the structure of Denial Prompting, where at each state, the LLM is prompted to come up with a brand new solution that it has never thought of before by imposing a new constraint. Therefore, we mainly consider \({\cal H}\)-creativity measurement in our NeoGauge score, where we compare the model-generated solution with a set of collected human solutions to examine if it has ever been proposed in human history (i.e., the ratio of the region out of human solution space in Figure 1). This makes our NeoGauge human-grounded and reflects the novelty from history.

Convergent Creative Thinking.Since the twenty-first century, more researchers have begun to accept the proposition that creative thought involves not merely the generation of many alternative solutions (divergent thinking) but also the identification of new _feasible_ solutions Baer (1994); Runco (2003). They frame this problem-solving process as convergent creative thinking and begin to examine how understanding human cognition and convergent thinking might be used to account for creative thought Finke et al. (1996); Mumford et al. (1991); Sternberg and Lubart (1991). Several famous cognitive approaches that study the mental representation and process underlying convergent creative thinking Lubart (2001) involve asking examineesch to predict future states from past states using incomplete information Sternberg (1981); 1982), or solving the problems as though the counterfactual premises are true Sternberg and Gastel (1989a, b). All these tests share certain characteristics, such as always having a single best answer and asking examinees to think in unconventional ways. In our work, besides computing \({\cal H}\)-creativity for evaluating divergent thinking, our work also measures convergent creativity by verifying the feasibility of the generated solution: whether they are correct and following the given constraints. Our NeoGauge metric delivers a more comprehensive evaluation of machine creativity.

## 3 Constructing the NeoCoder Dataset

We present Denial Prompting to stimulate creative responses from LLMs.

### Denial Prompting: Eliciting Creative Generations from LLMs

Our purpose is to construct a pipeline that iteratively imposes constraints on previous solutions (e.g., disallowing the use of hashmaps) to force 

more creative solutions. The setup is as follows: given an input problem, we use a highly capable _augmentation model_\({\bf P}_{\text{LM}}\) (e.g. GPT-4) to generate solutions and scrutinize "technique(s)" used in the generated solution, then update the problem by imposing the detected technique as a constraint. We repeat this process \(t\) times to obtain consecutive \(t\) problems with increasingly hard constraints (Figure 8 shows an example with \(t=2\)).

Specifically, as shown in Algorithm 1, given a reasoning problem \(x\) and an initial empty constraint list \(\mathcal{C}_{0}=\{\}\), we first let the augmentation model \({\bf P}_{\text{LM}}\) to generate an initial solution \(y_{1}\sim{\bf P}_{\text{LM}}(x)\) via a default problem-solving prompt and conversation history. We then use the same augmentation model \({\bf P}_{\text{LM}}\) to detect _atomic techniques_ (e.g., recursion, for loop, hashmaps, etc.), \(\mathcal{T}_{1}=\{\tau^{1},\tau^{2},\cdots,\tau^{i}\}\), used in \(y_{1}\) to solve \(x\) with a technique detection prompt. Then, one technique is randomly sampled \(\tau_{1}\sim\mathcal{T}_{1}\setminus\mathcal{C}_{0}\) to ensure it has never been used before as a constraint. Finally, we update the problem \(x\) to \(x\oplus\tau_{1}\) which explicitly **prohibits** the use of the technique \(\tau_{1}\) and update constraint list \(\mathcal{C}_{0}\) to \(\mathcal{C}_{1}=\{\tau_{1}\}\).4 This is the first iteration of Denial Prompting. We repeat the process to progressively obtain the overall constraint list \(C_{t}=\{\tau_{1},\tau_{2},\cdots,\tau_{t}\}\). The prompts for Denial Prompting (including technique detection; used across all experiments) are in Appendix D.

Footnote 4: We use \(\oplus\) to indicate text concatenation.

```
1:Input problem \(x\), augmentation model \(\mathcal{P}_{\text{LM}}\), max iterations \(T\)
2:Output: Constraint list \(\mathcal{C}_{T}\)
3:for\(t=1\)to\(T\)do
4:# Response generation
5:\(y_{t}\sim{\bf P}_{\text{LM}}(x\oplus\tau_{1}\oplus\cdots\oplus\tau_{t-1})\)
6:# Technique detection
7:\(\mathcal{T}_{t}\sim{\bf P}_{\text{LM}}(y_{t})\)
8:\(\tau_{t}\sim\mathcal{T}_{t}\setminus\mathcal{C}_{t-1}\)
9:\(\mathcal{C}_{t}=\{\tau_{1},\tau_{2},\cdots,\tau_{t}\}\)
10:endfor
```

**Algorithm 1**Denial Prompting

During Denial Prompting, we use a single conversation thread of \({\bf P}_{\text{LM}}\) to infer \(y_{t}\) such that the model can utilize the trace of previous interactions (including problem statements, constraints, and LLM solutions from each iteration). In practice, we observe adding prior interactions in the context improves model generations. Conversely, when detecting solution techniques \(\mathcal{T}_{t}\sim{\bf P}_{\text{LM}}(y_{t})\) (line 4 in Algorithm 1), we disregard the context from previous conversation rounds to focus the responses solely on the most recent round.

### NeoCoder Dataset to Support Benchmarking LLM Creativity

Challenging problems.To construct our creativity benchmark, we compile \(n=199\) latest Codeforces problems. We chose problems with a difficulty of 800 (easiest level) since, in our preliminary experiments, we observed near-random performance on more challenging problems when using well-known open-source models. Furthermore, we selected the recent data to prevent any memorization during pre-training Huang et al. (2023).

Human solutions.For each problem, we extract \(m=30\) correct human solutions per problem (total of \(5.9\)K human solutions).5 We use human solutions to measure \(\mathcal{H}\)-creativity of LLM responses.

Footnote 5: We consider 30 human-annotated solutions to construct a historical solution space for each problem to be sufficient given the high overlap rate among them.

Human annotated test examples.We also retrieve all test examples provided with each problem (4.5 test examples per problem on average, a total of 2.2K test examples). We then perform manual fixes to address any parsing or formatting issues in the collected test examples and ensure that follow a standardized input-output format. We use these test examples to measure \(\mathcal{P}\)-creativity or the functional correctness of LLM responses.

Augmentation with Denial Prompting.We use GPT-4 OpenAI (2024) as the augmentation model \({\bf P}_{\text{LM}}\) because we find that GPT-4 can achieve 94% technique detection recall compared to the human programmer in our pilot experiments.6 We feed the retrieved problems to Denial Prompting (SS3.1) with maximum iterations \(T=5\) to obtain our dataset NeoCoder. Our dataset consists of pairs \((x,\mathcal{C}_{t}=\{\tau_{1},\tau_{2},\ldots,\tau_{t}\})\), where \(x\) represents a problem (programming challenge), and \(\mathcal{C}_{t}\) represents the constraints that must be adhered to when solving the problem \(x\). This implies that a single programming problem may be associated with various sets of constraints, forming different pairs accordingly.

Footnote 6: We use gpt-4–1166-preview across all experiments, accessed from Dec 2023 through April 2024.

Statistics for NeoCoder.Table 1 shows the number of problems \(x\) and the number of the associated constraints \(|\mathcal{C}_{t}|\). Note that the number of 

problems decreases for a larger number of constraints. This is due to Denial Prompting potentially reaching a point where it can no longer generate new constraints after a certain number of iterations (i.e., \({\mathcal{T}_{t}}\setminus{\mathcal{C}_{t-1}}=\varnothing\) in Alg. 1). In such a case, we let \(\tau_{t}=\varnothing\) and jump to the next iteration \(t+1\) without updating the constraint list \({\mathcal{C}_{t}}={\mathcal{C}_{t-1}}\).

We also compare the distribution of the top 5 most common techniques from Denial Prompting in comparison to that of human solutions (Figure 2). It is evident that, without any constraints, models tend to use common techniques (e.g., forloops) similar to human solutions. However, as more constraints are imposed, the less common but more sophisticated techniques are employed.

## 4 State-Aware and Human-Grounded Evaluation of Machine Creativity

Augmentation model vs target model.So far, we have used \({\mathbf{P}_{\text{LM}}}(.)\) to denote the _augmentation model_, the language model used for dataset construction and extracting atomic techniques. Here, we introduce \({\mathbf{G}_{\text{LM}}}(\cdot)\) to represent the _target_ language model, whose creativity we evaluate using our dataset and the augmentation model \({\mathbf{P}_{\text{LM}}}(\cdot)\).

Setup.Here we introduce our metric of creativity NeoGauge for a given model \({\mathbf{G}_{\text{LM}}}\) and given NeoCoder. Denote instances of NeoCoder at state \(t\) (\(t\leq T\)) as:

\[{\mathcal{D}_{t}}=\Big{\{}(x^{i},{\mathcal{C}_{t}^{i}}=\{\tau_{1}^{i},\tau_{2 }^{i},\cdots,\tau_{t}^{i}\})\Big{\}}_{i=1}^{n},\]

where \(i\) is the problem index. To evaluate the creativity of the testing model \({\mathbf{G}_{\text{LM}}}\) at state \(t\), we feed \({\mathcal{D}_{t}}\) to \({\mathbf{G}_{\text{LM}}}\) to obtain its predictions:

\[{\mathcal{Y}_{t}}=\Big{\{}y_{t}^{i}\sim{\mathbf{G}_{\text{LM}}}( x^{i}\oplus{\mathcal{C}_{t}^{i}})~{}\Big{|}~{}|{\mathcal{C}_{t}^{i}}|=t,\\ \forall(x^{i},{\mathcal{C}_{t}^{i}})\in{\mathcal{D}_{t}}\Big{\}}. \tag{1}\]

Here \(|{\mathcal{C}_{t}^{i}}|\) denotes the cardinality of the constraints set. The constraint \(|{\mathcal{C}_{t}^{i}}|=t\) ensures that at a given state \(t\), the questions we evaluated always have \(t\) distinct constraints. Below, we present how we compute _convergent_ and _divergent_ creativity and introduce NeoGauge metric that unifies them.

Convergent creativity involves problem-solving and constraint following.To evaluate \({\mathbf{G}_{\text{LM}}}\)'s convergent thinking ability, we examine two characteristics of generated solutions: whether they are correct and whether they follow the given constraints. Therefore, given \({\mathcal{Y}_{t}}\) from Eq.1, we define its convergent creativity as follows:

\[\textbf{convergent}({\mathbf{G}_{\text{LM}}},t)=\\ \frac{1}{|{\mathcal{Y}_{t}}|}\sum_{y_{t}^{i}\in{\mathcal{Y}_{t}} }\mathbbm{1}^{{\mathcal{T}_{t}}\cap C_{t}^{i}=\varnothing}\times\mathbbm{1}^ {\text{Correct}(y_{t}^{i})}, \tag{2}\]

where atomic techniques \({\mathcal{T}_{t}^{i}}\sim{\mathbf{P}_{\text{LM}}}(y_{t}^{i})\). \(\mathbbm{1}^{\text{Correct}(y_{t}^{i})}\) is a measure of program correctness, set to 1 if the generated solution passes all the test examples. Otherwise it is 0. We use the augmentation model \({\mathbf{P}_{\text{LM}}}\) to detect all atomic techniques \({\mathcal{T}_{t}^{i}}\) used in solution \(y_{t}^{i}\), and compare them with the given constraint list \({\mathcal{C}_{t}^{i}}\) to check if the solution follows the given constraints. In Figure 3 examples, only the solution generated at \(t=0\) (which does not involve any constraint) exhibits convergent creativity.

Divergent creativity requires comparison to historical human solutions.As discussed earlier in SS2, a primary focus of our evaluation is on \({\mathcal{H}}\)-creativity, which requires a juxtaposition of model solutions with historical human solutions. Let's consider a finite set of correct human written solutions with size \(m\), denoted as \({\mathcal{H}^{i}}\), for problem \(x^{i}\). Rather than directly comparing solutions using

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline State (# of constraints) & 0 & 1 & 2 & 3 & 4 & 5 \\ \hline \# of problems & 199 & 199 & 198 & 194 & 176 & 97 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Number of instances at each state.

Figure 2: Proportion of the top 5 most common atomic techniques used by GPT-4 per state, compared to those in human solutions. **In absense of any constraints (the first column), the model default to common and accessible techniques, like humans (the last column).** This echoes our claim in §1 that eliciting creative solutions is crucial for creativity evaluation.

 

certain sentence-level similarity scores, as done by a few prior works such as DeLorenzo et al. (2024), we break down the comparison to the atomic technique level, which is more interpretable and generalizable across varying solutions. Our divergent creativity score is defined as:

\[{\bf divergent}({\bf G_{LM}},t)=\frac{1}{|{\cal Y}_{t}|}\sum_{y^{i}_{t}\in{\cal Y }_{t}}\frac{|{\cal T}^{i}_{t}\setminus\widehat{{\cal T}}^{i}_{t}|}{|{\cal T}^{ i}_{t}|}, \tag{3}\]

where \({\cal T}^{i}_{t}\sim{\bf P_{LM}}(y^{i}_{t})\) are the atomic techniques used in the model solutions, and \(\widehat{{\cal T}}^{i}\) indicate all the atomic techniques used by \(m\) human solutions, defined as: \(\widehat{{\cal T}}^{i}=\bigcup_{j=1}^{m}\big{\{}\widehat{{\cal T}}^{i}_{j} \sim{\bf P_{LM}}(\hat{y}^{i}_{j}),~{}\hat{y}^{i}_{j}\in{\cal H}^{i}\big{\}}\). We then compute the \({\cal H}\)-creativity as _the ratio of techniques used by \({\bf G_{LM}}\) that have never been used in the human solution set_. For example, as shown in Figure 3 at state \(t=1\), among the three techniques identified within the generated solution, only the recursion has never been used by humans, thereby resulting in a ratio of \(\frac{1}{3}\). Finally, we average ratios across different problems to obtain the final \({\cal H}\)-creativity at state \(t\).

**NeoGauge unifies convergent and divergent creativity.** Given the above definitions, NeoGauge of \({\bf G_{LM}}\) at state t can be formalized:

\[\begin{split}&\textbf{NeoGauge@t}=\\ &\frac{1}{|{\cal Y}_{t}|}\sum_{y^{i}_{t}\in{\cal Y}_{t}}\underbrace {\mathbbm{1}_{T^{i}_{t}\cap{\cal C}^{i}_{t}=\varnothing}^{*}\mathbbm{1}_{ \text{Convergent Creativity}}}_{\text{Convergent Creativity}}\times\underbrace{|{\cal T}^{i}_{t} \setminus\widehat{{\cal T}}^{i}_{t}|}_{\text{Divergent Creativity}},\end{split} \tag{4}\]

where \({\cal Y}_{t}=\{y^{i}_{t}\sim{\bf G_{LM}}(x^{i}\oplus{\cal C}^{i}_{t})~{}|~{}|{ \cal C}^{i}_{t}|=t,\forall(x^{i},{\cal C}^{i}_{t})\in{\cal D}_{t}\}\) (defined in Eq.1), \({\cal T}^{i}_{t}\sim{\bf P_{LM}}(y^{i}_{t})\) (defined in Eq.2), \(\widehat{{\cal T}}^{i}=\bigcup_{j=1}^{m}\big{\{}\widehat{{\cal T}}^{i}_{j} \sim{\bf P_{LM}}(\hat{y}^{i}_{j}),~{}\hat{y}^{i}_{j}\in{\cal H}^{i}\big{\}}\) (defined in Eq.3).

## 5 Experiments and Results

We report the creativity of current LLMs (SS5.2) and evaluate different reasoning strategies (SS5.3) for creativity.

### Experimental Setup

Models.We use GPT-4 as the augmentation model \({\bf P_{LM}}\). We benchmark the creativity performance of the following target models

Figure 3: Example of NeoGauge computation. The question comes from our NeoCoder dataset with ID 1829B and testing model \({\bf G_{LM}}\) here is GPT-4. For each state, we compute NeoGauge (Eq.4) as the probability of LM generating correct solutions that meet the given constraints (convergent creativity defined in Eq.2) and also exhibit \({\cal H}\)-creativity (divergent creativity defined in Eq.3). However, none of the above three solutions are considered to be “creative” since _convergent solutions may lack divergent creativity_ (e.g., state \(t=0\)). Alternatively, _LLMs’ hallucinated responses resulting in high \({\cal H}\)-creativity, but often lack correctness and constraint following_ (e.g., state \(t=1\)). Therefore, truly creative works should not only be innovative but also appropriately solve a problem.

 

\({\bf G}_{\text{LM}}\): GPT-4 (OpenAI, 2024), GPT-3.5 (Ouyang et al., 2022), Claude 3 Sonnet (Claude-3) (Anthropic, 2024), Llama3-70B (AI@Meta, 2024), Llama2-70B (Touvron et al., 2023), CodeLlama-34B-Python (CodeLama-34B) (Roziere et al., 2024), CodeGemma-7B (Google, 2024), and Mistral-7B (Jiang et al., 2023a). We access all non-proprietary models through Huggingface Transformers (Wolf et al., 2019). Following the parameter choice by Zhang et al. (2023), we apply a sampling temperature of 1 for code generation.

MetricsBeyond the three proposed metrics for evaluating convergent, divergent and overall creativity, we also compute pass@1 (Chen et al., 2021) and constraint following ratio for further comparison in Table 3. NeoGauge@T actually is a joint probability of \({\bf G}_{\text{LM}}\) being both convergent and divergent creative at state \(t\). Therefore, we also report the cumulative NeoGauge across states in Figure 4, which indicates the model's maximum creativity performance boundary. Additionally, we compute human convergent and divergent creativity in Figure 5 to compare LLM with human creativity performance (details in Appendix B). We summarize all used metrics in Table 2.

### Benchmarking Language Model Creativity

A number of psychological investigators have studied the link between creativity and intelligence Holyoak and Morrison (2005), agreeing on two key points: (1) creative individuals tend to have higher intelligence Renzulli (2005), and (2) people with extremely high intelligence not necessarily to be extremely creative Faris et al. (1962). We re-examine the two findings on LLMs and answer: _Are larger LLMs more creative? Do extremely large models of equal size exhibit comparable creativity?_ Our investigation is based on the widely accepted hypothesis that language model size correlates positively with intelligence Kaplan et al. (2020); Liu et al. (2023); Zhao et al. (2023).

GPT-4 is the most creative LLM thus far.We visualize NeoGauge and cumulative NeoGauge in Figure 4. GPT-4 consistently has the highest NeoGauge almost at every state \(t\). While others (e.g., Claude-3 and Llama3-70B) have a close NeoGauge@0 score to GPT-4, their NeoGauge quickly decreases to 0 within the next two states. According to cumulative NeoGauge, GPT-4 also has the highest creativity performance boundary, followed by Claude-3 and Llama3-70B, greatly outperforming smaller models such as GPT-3.5 and Llama2-70B. These observations could potentially answer the above two questions: _larger LLMs are generally more creative_, but _extremely large LLM is not necessarily exhibiting extremely creative performance_. In Figure 9, we provide example outputs from each model to show their different creativity abilities.

Which is more creative: machine or human?Figure 5 displays the creativity comparison between LLM and humans. LLM demonstrates minimally better performance in divergent creativity compared to humans at their lowest level (Eq.6). However, humans have significantly greater convergent creativity than LLMs in early states (prior to state 3). Thus, we reach a tentative conclusion that, in problem-solving settings, LLMs in Figure 5 barely exhibit human-like creativity. Future works could focus on measuring human divergent creativ

\begin{table}
\begin{tabular}{l l c c} \hline \hline
**Metric** & **Description** & **Definition** & **Place of Use** \\ \hline
**convergent\(({\bf G}_{\text{LM}},t)\)** & **Convergent creativity of \({\bf G}_{\text{LM}}\) at state \(t\)** & Eq.2 & Table 3, Figure 5, 7 \\
**divergent\(({\bf G}_{\text{LM}},t)\)** & **Divergent creativity of \({\bf G}_{\text{LM}}\) at state \(t\)** & Eq.3 & Table 3, Figure 5, 7 \\
**Neogauge@t** & **Creativity evaluation of \({\bf G}_{\text{LM}}\) at state \(t\)** & Eq.4 & Table 3, Figure 4 \\ \hline pass@1 Chen et al. (2021) & Probability of the first sample passes the unit tests & \(\underset{p\text{reforms}}{\mathbb{E}}[1-\frac{n-c}{n}]\) & Table 3 \\ constraint following & Average ratio of following the constraints at state \(t\) & \(\underset{p\text{reform}}{\mathbb{E}}[\frac{1+r^{c}\Gamma_{C}=c}{n}]\) & Table 3 \\ convergent(human, t) & convergent creativity of human at state \(t\) & Eq.5 & Figure 5 \\ divergent(human) & lowest divergent creativity of human at state 0 & Eq.6 & Figure 5 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Description of various metrics used across experiments.

Figure 4: NeoGauge (left) and cumulative NeoGauge (right) across states.

 

ity across states to enable a fairer creativity comparison. Moreover, we observe that both human and LLM convergent creativity declines drastically over the increase in state \(t\), which follows our expectation that there is a trade-off between solution quality and novelty. When stress-testing humans or LLMs to look for more creative solutions, they are very likely to make mistakes and may copy previous solutions during the process.

In-depth analysis of creativity evaluation.We provide evaluation results for GPT-4 in Table 3. It is evident that as the state increases (more hard constraints are imposed), the quality of solutions declines both in terms of correctness and constraint following. Even if the model may still generate new alternative solutions at state 5 (divergent(GPT-4, \(5)~{}=15.3\)), they fail at convergent evaluation (convergent(GPT-4, \(5)~{}=0\)). Therefore, at state 5, GPT-4 shows 0 creativity (NeoGauge@5 \(=0\)). Additionally, unlike the convergent score, which typically decreases as \(t\) increases, the divergent score of GPT-4 continually rises. This observation empirically proves the key assumption of Denial Prompting that LLMs tend to seek more creative solutions when facing an unconventional environment characterized by unusual hard constraints.

### Evaluating Reasoning Strategies for Creativity

We evaluate four reasoning strategies on our NeoCoder dataset to further study the correlation between augmented machine intelligence and creativity: _Whether such intelligence-enhancing techniques also improve creative thinking?_ We implement the following four works that are specifically designed for programming tasks:

* **MCTS**: Zhang et al. (2023) propose a novel decoding method that uses Monte-Carlo Tree Search (MCTS) to generate better programs using the pass rate as reward.
* **Self-Correction**: Shinn et al. (2023) use verbal feedback from a reflection agent to reinforce the performance of an agent in code generation.
* **Planning**: Jiang et al. (2023b) design a planning module to let LLM plan out concise solution steps from the intent, followed by an implementation module to generate code step by step.
* **Sampling**: Chen et al. (2021) generate \(k\) samples and compute the probability that at least one

Figure 5: A comparison of LLM and human creativity. denotes the performance difference of **convergent creativity**, and denotes the difference of **divergent creativity**. We observe that **Current LLMs still hardly demonstrate human-like creativity.**



of the \(k\)-generated code samples for a problem passes the unit tests. For creativity evaluation, we generate \(k=5\) samples for each problem and report the NeoGauge from samples that have the highest convergent and divergent creativity, \({\mathbbm{1}^{T_{i}^{r}\cap C_{i}^{t}=\varnothing}}\times{\mathbbm{1}^{\text{ Correct}(y_{i}^{t})}}\times\frac{|T_{i}^{s}\langle\tilde{T}^{r}|}{|T_{i}^{ r}|}\) in Eq.4, among \(k=5\) samples.

Note that these methods are originally applicable to different kinds of models. Considering the computation complexity and the cost, we re-evaluate MCTS on the open-source language model (CodeGamma-7BGoogle, 2024)) and re-evaluate others on the proprietary model ( GPT-3.5).

Most reasoning strategies fail to improve divergent thinking.According to Figure6, all reasoning strategies except sampling help to improve the model's convergent creativity thinking ability on multiple states, as they are fundamentally designed to improve the accuracy. Conversely, only MCTS successfully enhances divergent creativity, due to it rolling out numerous paths during the expansion. Strategies like self-correction, planning, and sampling, which operate on a single trial or path, fail to explore divergent solutions.

There is a tradeoff between divergent and convergent creativity.Noticeably, while MCTS consistently enhances divergent creative thinking in all 5 states, its improvement on NeoGauge is minimal and becomes 0 after \(t=2\). This suggests that divergent solutions generated by MCTS may not truly augment creativity, potentially due to incorrectness or failure to follow the given constraints. This also implies that MCTS might prioritize divergent thinking over convergent thinking. On the other hand, self-correction and planning sacrifice their divergent thinking ability in improving their convergent thinking because the divergent creativity difference even goes to negative at certain states (e.g., Divergent Diff \(=-1.2\) at \(t=0,3\) on sampling). None of the four reasoning strategies have been able to simultaneously improve both convergent and divergent creativity, resulting in limited improvement of NeoGauge. Thus, our findings indicate that _these intelligence-augmenting methods do not provide much benefit to LLM creativity_. We leave for future works to discover specialized strategies for better enhancing LLM's creative performance and NeoGauge.

## 6 Conclusion

We propose protocols for evaluating language model creativity in problem-solving and introduce the Denial Prompting framework and NeoGauge metric to provide a comprehensive creativity evaluation, measuring both convergent and divergent creativity, inspired by extensive research on human creativity. To facilitate future research, we release our NeoCoder dataset and shed light on the limitations of current reasoning strategies in improving LLM creativity.

Figure 6: Creativity performance difference before and after applying reasoning strategies. A larger difference value indicates that the strategy improves the testing model’s creativity. Detailed numeric changes are provided in Table5.

 



### Limitations

Application scope.While NeoGauge offers a general-purpose framework for evaluation of LLM creativity, our study is restricted to Text-to-Code, as it requires a historical human solution set. For most tasks in the literature, collecting a comprehensive set of distinct human responses is nontrivial.

Data leakage concern.Our proposed dataset NeoCoder is built using latest Codeforces problems. Despite their recency, future LLMs might get exposure to these problems during their pretraining. To alleviate such risks, future works can focus on more difficult problems or evaluate NeoGauge for higher states, besides incorporating a newer batch of problems.

## Acknowledgements

This work is in part supported by ONR grant N00014-241-2089, and generous gifts from Amazon and the Allen Institute for AI. We also greatly appreciate the help of the students at CLSP.

## References

* Al@Meta (2024) Al@Meta. 2024. Llama 3 model card.
* Amabile (1982) Teresa M Amabile. 1982. Social psychology of creativity: A consensual assessment technique. _Journal of personality and social psychology_, 43(5):997.
* Amabile (1996) T.M. Amabile. 1996. _Creativity In Context: Update To The Social Psychology Of Creativity_. Avalon Publishing.
* Anthropic (2024) Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku.
* Atmakuru et al. (2024) Anirudh Atmakuru, Jatin Nainani, Rohith Siddhartha Reddy Bheemreddy, Anirudh Lakkaraju, Zonghai Yao, Hamed Zamani, and Haw-Shiuan Chang. 2024. Cs4: Measuring the creativity of large language models automatically by controlling the number of story-writing constraints.
* Baer (1994) John Baer. 1994. Divergent thinking is not a general trait: A multidomain training experiment. _Creativity Research Journal_, 7(1):35-46.
* Boden et al. (1994) Margaret A Boden et al. 1994. Dimensions of creativity.
* De Bronnec et al. (2024) Florian Le Bronnec, Alexandre Verine, Benjamin Negreyrenge, Yann Chevaleyre, and Alexandre Allauzen. 2024. Exploring precision and recall to assess the quality and diversity of llms.
* Chakrabarty et al. (2024) Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. 2024a. Art or artifice? large language models and the false promise of creativity. In _Proceedings of the CHI Conference on Human Factors in Computing Systems_, CHI '24, New York, NY, USA. Association for Computing Machinery.
* Chakrabarty et al. (2024b) Tuhin Chakrabaryt, Vishakh Padmakumar, Faeze Brahman, and Smaranda Muresan. 2024b. Creativity support in the age of large language models: An empirical study involving emerging writers.
* Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, and Michael Petrov et al. 2021. Evaluating large language models trained on code.
* Csikszentmihalyi (1996) M. Csikszentmihalyi. 1996. _Creativity: Flow and the Psychology of Discovery and Invention_. Harper perennial Modern Classics. HarperCollinsPublishers.
* Csikszentmihalyi (1998) Mihaly Csikszentmihalyi. 1998. _Implications of a Systems Perspective for the Study of Creativity_, page 313-336. Cambridge University Press.
* DeLorenzo et al. (2024) Matthew DeLorenzo, Vasudev Gohil, and Jeyavijayan Rajendran. 2024. Creativeval: Evaluating creativity of llvm-based hardware code generation.
* Lee Faris et al. (1962) Robert E. Lee Faris, J. W. Getzels, and Philip W. Jackson. 1962. Creativity and intelligence: Explorations with gifted students. _American Sociological Review_, 27:558.
* Feldman (1998) David Henry Feldman. 1998. _The Development of Creativity_, page 169-186. Cambridge University Press.
* Feldman et al. (1994) David Henry Feldman, Mihaly Csikszentmihalyi, and Howard Gardner. 1994. _Changing the world: A framework for the study of creativity_. Praeger Publishers/Greenwood Publishing Group.
* Finke et al. (1996) Ronald A. Finke, Thomas B. Ward, and Steven M. Smith. 1996. _Creative Cognition: Theory, Research, and Applications_. The MIT Press.
* Google (2024) Google. 2024. Codegemma: Open code models based on gamma.
* Guilford (1950) J. P. Guilford. 1950. Creativity. _American Psychologist_, 5(9):444-454.
* Gomez-Rodriguez and Williams (2023) Carlos Gomez-Rodriguez and Paul Williams. 2023. A confederacy of models: a comprehensive evaluation of llms on creative writing.
* Holtzman et al. (2019) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. In _International Conference on Learning Representations_ (ICLR).
* Holyoak and Morrison (2005) K.J. Holyoak and R.G. Morrison. 2005. _The Cambridge Handbook of Thinking and Reasoning_. Cambridge Handbooks in Psychology. Cambridge University Press

Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, et al. 2023. Competition-level problems are effective llm evaluators. _arXiv preprint arXiv:2312.02143_.
* Jiang et al. (2023a) Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Le'ielo Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teresa Le Scao, Thibaut Lavril, Thomas Wang, Timothee Lacroix, and William El Sayed. 2023a. Mistral 7b.
* Jiang et al. (2023b) Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. 2023b. Self-planning code generation with large language models.
* Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. 2020. Scaling laws for neural language models. _ArXiv_, abs/2001.08361.
* Kirk et al. (2024) Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2024. Understanding the effects of RLHF on LLM generalisation and diversity. In _The Twelfth International Conference on Learning Representations_.
* Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pretrain, prompt, and predict: A systematic survey of prompting methods in natural language processing. _ACM Comput. Surv._, 55(9).
* Lu et al. (2024a) Li-Chun Lu, Suou-Jen Chen, Tsung-Min Pai, ChanHung Yu, Hung yi Lee, and Shao-Hua Sun. 2024a. Llm discussion: Enhancing the creativity of large language models via discussion framework and roleplay.
* Lu et al. (2024b) Ximing Lu, Melanie sclar, Skyler Hallinan, Niloofar Mireshghallah, Jiacheng Liu, Seungju Han, Allyson Ettinger, Liwei Jiang, Khyathi Chandu, Nouha Dziri, and Yejin Choi. 2024b. Ai as humanity's salieri: Quantifying linguistic creativity of language models via systematic attribution of machine text against web text.
* Lubart (2001) Todd I. Lubart. 2001. Models of the creative process: Past, present and future. _Creativity Research Journal_, 13(3-4):295-308.
* Mumford et al. (1991) Michael D. Mumford, Michele I. Mobley, Roni Reiter-Palmon, Charles E. Uhlman, and Lesli M. Doares. 1991. Process analytic models of creative capacities. _Creativity Research Journal_, 4(2):91-122.
* OpenAI (2024) OpenAI. 2024. Gpt-4 technical report.
* Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Alama, Alex Ray, et al. 2022. Training Language Models to Followstructions with Human Feedback. In _Advances in Neural Information Processing Systems_ (NeurIPS).
* Renzulli (2005) Joseph S. Renzulli. 2005. _The Three-Ring Conception of Giftedness: A Developmental Model for Promoting Creative Productivity_, page 246-279. Cambridge University Press.
* Roziere et al. (2024) Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Ittai Gat, Xiadojing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jeremy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Defossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialam, and Gabriel Synnaeve. 2024. Code llama: Open foundation models for code.
* Runco (2003) Mark A. Runco. 2003. _Critical creative processes_. Perspectives on creativity. Hampton Press, Cresskill, NJ.
* Shinn et al. (2023) Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunya Yao. 2023. Reflexion: Language agents with verbal reinforcement learning.
* Sternberg (1981) Robert J. Sternberg. 1981. Intelligence and nonentrenchment. In _Journal of Educational Psychology_.
* Sternberg (1982) Robert J Sternberg. 1982. Natural, unnatural, and supernatural concepts. _Cognitive Psychology_, 14(4):451-488.
* Sternberg and Gastel (1989a) Robert J. Sternberg and Joyce Gastel. 1989a. Coping with novelty in human intelligence: An empirical investigation. _Intelligence_, 13(2):187-197.
* Sternberg and Gastel (1989b) Robert J Sternberg and Joyce Gastel. 1989b. If dancersate their shoes: Inductive reasoning with factual and counterfactual premises. _Memory & Cognition_, 17:1-10.
* Sternberg and Lubart (1991) Robert J. Sternberg and Todd I. Lubart. 1991. An investment theory of creativity and its development. _Human Development_, 34(1):1-31.
* Tevet and Berant (2021) Guy Tevet and Jonathan Berant. 2021. Evaluating the evaluation of diversity in natural language generation. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 326-346, Online. Association for Computational Linguistics.
* Tian et al. (2024) Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marijieh, Nanyun Peng, Yejin Choi, Thomas L. Griffiths, and Faeze Brahman. 2024. Magyyer: Are large language models creative problem solvers?
* Torrance (1966) E Paul Torrance. 1966. Torrance tests of creative thinking. _Educational and Psychological Measurement_ 

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Ooyal, Eric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and efficient foundation language models. _arXiv preprint arXiv:2302.13971_.
* Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of-the-art natural language processing.
* Xu et al. (2024a) Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024a. WizardLM: Empowering large pre-trained language models to follow complex instructions. In _The Twelfth International Conference on Learning Representations_.
* Xu et al. (2024b) Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024b. Hallucination is inevitable: An innate limitation of large language models.
* Zhang et al. (2023) Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum, and Chuang Gan. 2023. Planning with large language models for code generation. In _International Conference on Learning Representations_ (ICLR).
* Zhang et al. (2024a) Tianhui Zhang, Bei Peng, and Danushka Bollegala. 2024a. Improving diversity of commonsense generation by large language models via in-context learning.
* Zhang et al. (2024b) Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, Zico Kolter, and Daphne Ippolito. 2024b. Forcing diffuse distributions out of language models.
* Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjiie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peijwu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models.
* Zhao et al. (2024) Yunpu Zhao, Rui Zhang, Wenyi Li, Di Huang, Jiaming Guo, Shaohui Peng, Yifan Hao, Yuanbo Wen, Xing Hu, Zidong Du, Qi Guo, Ling Li, and Yunji Chen. 2024. Assessing and understanding creativity in large language models.
* Zhu et al. (2024) Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, and Xing Xie. 2024. Dyval: Dynamic evaluation of large language models for reasoning tasks. In _International Conference on Learning Representations_ (ICLR).

 



## Appendix A Why Choose Codeforces for Creativity Evaluation?

In this study, we use competitive programming problems sourced from Codeforces for creativity evaluation. We provide our task choice motivation by answering the following three interrelated questions.

Why choose competitive programming problems?The general purpose of this paper is to benchmark the LLM's creativity performance in dealing with unconventional and challenging problems. Understandably, these problems usually do not have ground-truth answers (e.g., how to make coffee without a coffee maker). In such cases, we typically either evaluate the generated solution through human evaluation, similar to the approach taken by Tian et al. (2024), or through automated machine evaluation (ours). Real-world problems (Tian et al., 2024) naturally need human annotation. Collecting human annotations for measuring machine creativity is particularly challenging since the space is typically vast (because of the nature of creativity). Conversely, coding becomes an ideal source for problems that can be its functional correctness (as opposed to the choice of syntax) evaluated automatically with a minimal cost--based on whether they pass the test cases. Thus, we first chose coding problems to examine LM's creativity, as they provide an open-ended environment that could stimulate a model's creativity performance while making evaluation easy and cost-effective.

Low performance or low creativity?The low pass rate and constraint following ratio in Table 3 may raise a new question as to whether there are no reasonable solutions at all or no requisite creativity in finding solutions. Experimental evidence, however, suggests that LM simply lacks creativity. According to Figure 5, the huge gap between human and LLMs convergent creativity prior to State 3 (0-3 constraints) indicates there are valid human solutions for each problem, but the LLMs seem to be lacking creativity in finding it. Additionally, according to Figure 6, with suitable reasoning strategies, LLM still has room for improvement in both convergent and divergent creativity. Even though humans' convergent scores are nearing zero (Figure 5) at a large state (>3 hard constraints), the problems might not be fully infeasible.

Why not evaluate creativity based on problems but solutions?A motivational example for this question is that a creative student can always come up with innovative and insightful questions. However, in this work, we adopt a different standpoint on creativity used by many psychological and cognitive studies (discussed in section 2), which emphasizes problem-solving abilities. We evaluate a student to be creative if he/she can leverage all available tools and come up with novel solutions for challenging problems. Similarly, we study LLM creativity based on solutions they generated for challenging programming problems

Experiment Setup

### Human Creativity Evaluation

We compute human convergent creativity as follows:

\[{\bf convergent}({\rm human,}t)=\frac{1}{m|{\cal Y}_{l}|}\sum_{ \begin{subarray}{c}\iota\in\{i|{\cal C}^{t}_{l}=t,\\ i=1,2,\cdots,n\}\end{subarray}}\sum_{j=1}^{m}\mathbbm{\widehat{T}_{j}}^{\cap{ \cal C}^{t}_{l}=\varnothing},~{}{\rm where}~{}\widehat{{\cal T}_{j}}^{*}\sim{ \bf P}_{\text{LM}}(\hat{y}^{\iota}_{j}),~{}\hat{y}^{\iota}_{j}\in{\cal H}^{ \iota}. \tag{5}\]

Because the collected historical human solutions \(\hat{y}^{\iota}_{j}\) are always correct, for human convergent creativity evaluation, we focus on constraint following ratio by examining whether the atomic techniques \(\widehat{{\cal T}_{j}}^{i}\) used by each human solution follow the given constraints \({\cal C}^{t}_{t}\) at state \(t\). We use the same idea as Eq.3 to compute human divergent creativity.

\[{\bf divergent}({\rm human})=\frac{1}{mn}\sum_{i=1}^{n}\sum_{j=1}^{ m}\frac{|\widehat{{\cal T}_{j}}^{i}\setminus\widehat{{\cal L}_{j}}^{i}|}{| \widehat{{\cal T}_{j}}^{i}|},\] \[{\rm where}~{}\widehat{{\cal T}_{j}}^{i}\sim{\bf P}_{\text{LM}}( \hat{y}^{i}_{j}),~{}\widehat{{\cal L}_{j}}^{i}=\bigcup_{k=1,k\neq j}^{m} \widehat{{\cal T}_{k}}^{i}\sim{\bf P}_{\text{LM}}(\hat{y}^{i}_{k}),~{}\hat{y}^ {i}_{j},~{}\hat{y}^{i}_{k}\in{\cal H}^{i}. \tag{6}\]

Given total \(n\) problems, where each problem has \(m\) human solutions, we compute the average ratio of new techniques used by a single human solution \(\hat{y}^{i}_{j}\) (\(J^{\text{th}}\) human solution for \(i^{\text{th}}\) problem) that the remaining human solutions \(\{\hat{y}^{i}_{k}~{}|~{}k\neq j,k~{}=1,2,\cdots,m\}\) have never used. This is because collecting a human DP dataset would be quite costly and restrictive. We instead use a diverse collection of solutions from various human programmers as a proxy. Eq.6 is equivalent to \({\tt divergent}({\rm human},~{}t=0)\), representing the lowest level of human divergent creativity.

 



## Appendix C Experiment Results

It is crucial to consider both convergent and divergent thinking in creativity evaluation.We plot the stacked convergent and divergent creativity evaluation results in Figure 7. Among all models, GPT-4 generally exhibits the best performance on both convergent and divergent creative thinking across all states, followed by Claude-3 and Llama3-70B. It is noticeable that Llama3-70B even outperforms GPT-4 on convergent creative thinking when \(t=0~{}(\texttt{convergent}(\texttt{GPT-4},~{}0)~{}=16.16<\texttt{convergent}( \texttt{Llama3-70B},~{}0)~{}=19.19)\). We hypothesize that the latest Llama3 models are pre-trained on Codeforces problems and human solutions, so they have superior performance when there is no external constraint \(t=0\). However, as \(t\) increases, its convergent performance drops drastically. Moreover, divergent creative thinking never goes to 0 across all states and is sometimes even equally distributed on those less small models (e.g., CodeGemma-7B and Mistral-7B). Together with independent findings from Xu et al. (2024b), this observation indicates that LLMs with insufficient reasoning capabilities tend to make up new solutions regardless of the quality when facing unusual problems. Which, in turn, demonstrates the importance of the claim we made in section 1 that creative thinking involves not merely the generation of many diverse alternatives but also the verification of new valid alternatives.

Figure 7: Stacked results of convergent (Eq.2) and divergent (Eq.3) creativity evaluation across states.



Prompts for Denial Prompting and Benchmarking

We apply the same problem-solving prompt in both Denial Prompting and the benchmarking process.

[title=, topsep=0pt] Problans:Splicing Example on Gridboxes 



\begin{table}
\begin{tabular}{c l p{340.1pt}} \hline
* for loop You are given a sequence of integers a of length 2n. You have to split these 2n integers into n pairs; each pair will represent the coordinates of a point on a plane. Each number from the sequence a should become the x or y coordinate of exactly one point. Note that some points can be equal. ... \\  & & B.Points and Minimum Distance Programming constraints: DO NOT use the following techniques
- if statement
- for loop You are given a sequence of integers a of length 2n. You have to split these 2n integers into n pairs; each pair will represent the coordinates of a point on a plane. Each number from the sequence a should become the x or y coordinate of exactly one point. Note that some points can be equal. ... \\  & & B.Points and Minimum Distance Programming constraints: DO NOT use the following techniques
- while loop
- if statement
- for loop You are given a sequence of integers a of length 2n. You have to split these 2n integers into n pairs; each pair will represent the coordinates of a point on a plane. Each number from the sequence a should become the x or y coordinate of exactly one point. Note that some points can be equal. ... \\  & & B.Points and Minimum Distance Programming constraints: DO NOT use the following techniques
- while loop
- if statement
- for loop You are given a sequence of integers a of length 2n. You have to split these 2n integers into n pairs; each pair will represent the coordinates of a point on a plane. Each number from the sequence a should become the x or y coordinate of exactly one point. Note that some points can be equal. ... \\  & & B.Points and Minimum Distance Programming constraints: DO NOT use the following techniques
- while loop
- if statement
- for loop You are given a sequence of integers a of length 2n. You have to split these 2n integers into n pairs; each pair will represent the coordinates of a point on a plane. Each number from the sequence a should become the x or y coordinate of exactly one point. Note that some points can be equal. ... \\  & & B.Points and Minimum Distance Programming constraints: DO NOT use the following techniques
- sorting for loop If statement
- if statement
- if statement
- for loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop For loop loop For loop For loop For loop For loop For loop loop For loop For loop For loop For loop loop For loop For loop For loop loop For loop For loop For loop loop For loop For loop loop For loop For loop loop For loop For loop loop For loop For loop loop For loop For loop loop For loop For loop loop For loop For loop loop For loop loop For loop For loop loop For loop loop For loop For loop loop For loop loop For loop loop For loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop For loop loop loop For loop loop For loop loop For loop loop For loop loop loop For loop loop For loop loop loop For loop loop For loop loop loop For loop loop For loop loop loop For loop loop loop For loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop For loop loop loop loop For loop loop loop For loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop For loop loop loop loop loop For loop loop loop loop loop For loop loop loop loop loop For loop loop loop loop loop For loop loop loop loop loop loop For loop loop loop loop loop loop For loop loop loop loop loop loop loop For loop loop loop loop loop loop loop loop loop loop For


* [57] GPT-4 Closed-1 GPT-3.5 (Llama)-70B
* [69] Input data: A new state, output, failure and input data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output, failure and output data: A new state, output data: A new state, output, failure and output data: A new state, output data: A new state, output, failure and output data 



\begin{table}
\begin{tabular}{l c r l r}
**Strategy** & **State** & \(\Delta\)**Convergent\({}_{\text{(\bf old\to new)}}\)** & \(\Delta\)**Divergent\({}_{\text{(\bf old\to new)}}\)** & \(\Delta\)**NEOGAUGE\({}_{\text{(\bf old\to new)}}\)** \\ \hline  & 0 & 10.60\({}_{\text{(1.52$\rightarrow$12.12)}}\) & 7.79\({}_{\text{(5.18$\rightarrow$12.97)}}\) & 0.82\({}_{\text{(0.00$\rightarrow$0.82)}}\) \\  & 1 & 3.03\({}_{\text{(0.00$\rightarrow$3.03)}}\) & 8.62\({}_{\text{(6.31$\rightarrow$14.93)}}\) & 0.08\({}_{\text{(0.00$\rightarrow$0.08)}}\) \\  & 2 & 1.02\({}_{\text{(0.00$\rightarrow$1.02)}}\) & 8.61\({}_{\text{(5.55$\rightarrow$14.16)}}\) & 0.10\({}_{\text{(0.00$\rightarrow$0.10)}}\) \\
**MCTS** & 3 & 0.52\({}_{\text{(0.00$\rightarrow$0.52)}}\) & 9.11\({}_{\text{(5.35$\rightarrow$14.46)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 4 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 8.93\({}_{\text{(4.82$\rightarrow$13.75)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 5 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 9.69\({}_{\text{(4.23$\rightarrow$13.92)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\ \hline  & 0 & 12.12\({}_{\text{(2.53$\rightarrow$14.65)}}\) & -0.37\({}_{\text{(5.41$\rightarrow$5.04)}}\) & 0.29\({}_{\text{(0.25$\rightarrow$0.54)}}\) \\  & 1 & 2.53\({}_{\text{(0.51$\rightarrow$3.03)}}\) & 0.26\({}_{\text{(4.56$\rightarrow$4.82)}}\) & 0.29\({}_{\text{(0.00$\rightarrow$0.29)}}\) \\
**Self-Correction** & 2 & 1.02\({}_{\text{(0.00$\rightarrow$1.02)}}\) & 0.24\({}_{\text{(3.79$\rightarrow$4.03)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 3 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & -1.62\({}_{\text{(6.04$\rightarrow$4.42)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 4 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 1.77\({}_{\text{(3.70$\rightarrow$5.47)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 5 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 0.05\({}_{\text{(4.93$\rightarrow$4.98)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\ \hline  & 0 & 7.07\({}_{\text{(2.53$\rightarrow$9.60)}}\) & -1.16\({}_{\text{(5.41$\rightarrow$4.25)}}\) & 0.25\({}_{\text{(0.25$\rightarrow$0.50)}}\) \\  & 1 & 2.53\({}_{\text{(0.50$\rightarrow$3.03)}}\) & 1.28\({}_{\text{(4.56$\rightarrow$5.84)}}\) & 0.25\({}_{\text{(0.00$\rightarrow$0.25)}}\) \\
**Planning** & 2 & 1.02\({}_{\text{(0.00$\rightarrow$1.02)}}\) & 2.07\({}_{\text{(3.78$\rightarrow$5.85)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 3 & 0.52\({}_{\text{(0.00$\rightarrow$0.52)}}\) & -1.24\({}_{\text{(6.04$\rightarrow$4.80)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 4 & 0.57\({}_{\text{(0.00$\rightarrow$0.57)}}\) & 2.67\({}_{\text{(3.70$\rightarrow$6.37)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 5 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 1.94\({}_{\text{(4.93$\rightarrow$6.87)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\ \hline  & 0 & -0.50\({}_{\text{(2.52$\rightarrow$2.02)}}\) & -0.38\({}_{\text{(5.41$\rightarrow$5.03)}}\) & 0.00\({}_{\text{(0.25$\rightarrow$0.25)}}\) \\  & 1 & -0.51\({}_{\text{(0.51$\rightarrow$0.00)}}\) & -0.10\({}_{\text{(4.56$\rightarrow$4.46)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 2 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 1.05\({}_{\text{(3.78$\rightarrow$4.83)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\
**Sampling** & 3 & 0.52\({}_{\text{(0.00$\rightarrow$0.52)}}\) & -1.69\({}_{\text{(6.04$\rightarrow$4.35)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 4 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & 0.76\({}_{\text{(3.70$\rightarrow$4.46)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\  & 5 & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) & -1.86\({}_{\text{(4.93$\rightarrow$3.07)}}\) & 0.00\({}_{\text{(0.00$\rightarrow$0.00)}}\) \\ \hline \end{tabular}
\end{table}
Table 5: Creativity difference before and after applying reasoning strategies.