Title: What Shapes a Creative Machine Mind? Comprehensively Benchmarking Creativity in Foundation Models
Primary area: datasets and benchmarks
Keywords: Creativity Evaluation, Foundation Models, Convergent and Divergent thinking, Generative AI

Abstract: The meteoric rise of foundation models (FMs) has expanded their capabilities far beyond conventional tasks. Creativity, long regarded as a hallmark of human intelligence and a driver of innovation, is now increasingly recognized as a critical dimension of machine intelligence in the era of generative FMs, complementing traditional measures of accuracy. However, existing evaluation frameworks for creativity remain fragmented, relying on ad hoc metrics that are not firmly grounded in established theories of creativity. To address this gap, we introduce $\text{C}^2$-Eval, a holistic benchmark for the unified assessment of creativity in FMs. Specifically, $\text{C}^2$-Eval distinguishes between two complementary forms of Creativity ($\text{C}^2$): convergent creativity, where tasks admit constrained solutions (e.g., code generation), and divergent creativity, where tasks are open-ended (e.g., story telling). It evaluates both dimensions using fine-grained assessments derived from social science theories, focusing on Usefulness, Originality, and Surprise (U-O-S). Through extensive experiments on leading proprietary and open-source models, we provide a comprehensive analysis of trade-offs in their creative capabilities. 
Our results highlight the capabilities and challenges of current FMs in pursuing a creative machine mind, showing that $\text{C}^2$-Eval provides an effective lens for examining the evolving landscape of creative AI.