Towards an operative definition of creative writing: a preliminary assessment of creativeness in AI and human texts (2025)

Abstract: Nowadays, AI is present in all our activities. This pervasive presence is perceived as a threat by many category of users that might be substituted by their AI counterpart. While the potential of AI in handling repetitive tasks is clear, the potentials of its creativeness is still misunderstood. We believe that understanding this aspects of AI can transform a threat into an opportunity. This paper is a first attempt to provide a measurable definition of creativeness. We applied our definition to AI and human generated texts, proving the viability of the proposed approach. Our preliminary experiments show that human texts are more creative.

## introduction the oxford languages dictionary defines creative writing as "writing, typically fiction or poetry, which displays imagination or invention (often contrasted with academic or journalistic writing)," encompassing all forms of writing that engage the dynamics of thought, expressed through genuine artistry. in this context, the writer assumes the role of a "builder" of an alternative, often fictional, reality, intending to convey something meaningful to their readers. writers improve their creative skills through consistent practice, primarily by writing, refining ideas, reading the work of others and incorporating feedback. the spread of ai tools for creative writing. in the recent past, a number of ai-powered tools have emerged to support the writing activity. these range from the handling of technical aspects of the storytelling process, such as grammar and spelling checking (e.g. grammarly foot_0 ), translating text (e.g. lara foot_1 ) or even write a screenplay (e.g. sudowrite foot_2 ). these tools rely on modern ai techniques, such as recurrent neural networks (rnns) or transformers, like gpt (generative pretrained transformer). they are capable of examining context in sequence by learning linguistic patterns and how words logically follow one another, in order to: a) offering suggestions for terms, synonyms, and antonyms relevant to specific sentences or paragraphs; b) assisting with inspiration for character names or other narrative elements; c) proposing suitable titles for a book, considering the story, its themes, morals, and plot; d) functioning as an artificial beta reader, which performs the task of generating narrative suggestions for certain parts of the story based on patterns learned from similar works through machine learning. open question: how to evaluate the creative writing of an ai. despite the indisputable usefulness of ai tools to support the writing activity, a natural question is to assess to what extent ai tools can also generate creative content. it is often the case that artists-writers, in this study-may question how long their contributions will remain distinguishable from those of creative intelligences that are more efficient and faster, such as chatgpt, for example. humans are known for incorporating an emotional framework that enriches every creative process, making art, writing, and other forms of expression unique and deeply connected to their life experiences; and yet, it may really seem that chatgpt has a suite of tools to support writers that actually does not support writers at all, but rather that it reduces, trivializes and minimizes the effectiveness of a creative text, even potentially replacing the authors foot_3 . so, the question is: how can we measure the level of creativity of the machine and compare it with that of humans? contribution of the paper. we propose an initial operational definition of creativity based on the framework provided by @xcite , and we conduct preliminary experiments to quantitatively assess the creativity of ai compared to that of humans. to our knowledge, this is the first attempt to provide a practical definition of creativity in this context that can be fully automatized. ## related work there is an active line of research, as well online services foot_4 , aiming at using ai to classify aigenerated text to fight the risk connected to improper use of such technology such as misinformation, bias, intellectual property concerns and loss of human connection. in @xcite the authors study three versions of authorship attribution problem, among which the discrimination of texts written by a human from those written by machines. a recent research report @xcite pointed out that the available detection tools are often inaccurate and unreliable and have a main bias towards classifying the output as human-written rather than detecting ai-generated text. buz et al. @xcite discuss the creative quality in natural language generation. however, in their work creativity is entirely evaluated by humans, thus limiting the applicability of the proposed method. mark a. runco and garrett j. jaeger, in @xcite , offer a definition that runco revisits in his most recent paper @xcite , where he argues that "ai can only produce artificial creativity". given that the above definition of creativity does not inherently exclude ai-generated pseudocreativity, as it meets the standard requirements of originality and effectiveness, runco explores two possible approaches for humans to protect their genuinely creative potential: a) accept that ai is creative (since it adheres to the standard definition); b) revise the standard definition to distinguish human creativity from artificial creativity. if the second option were adopted, at least two additional parameters should be incorporated into the definition of creativity: authenticity and intentionality. starting with intentionality, we can refer to its definition: "character resulting from the active and conscious participation of the will in a given fact." (from the oxford languages dictionary) thus, as runco himself asserts, intentionality is a characteristic inherently tied to human beings, and by extension to human creativity, but not to artificial creativity. in the same way, authenticity is purely human and (at least for now) beyond the reach of ai to replicate (as runco himself mentions in his paper). authenticity arises from accepting one's own self, in a genuine way, without filters or limits (even ethical ones), and it's how humans express themselves when they "create." ai, however, lacks an experiential self to draw from or a history of experiences, which humans inherently possess. therefore, ai's creations, while elaborately crafted and impressive, remain an amalgamation of pre-existing content, not the result of personal lived experience. ## towards an operative definition of creative writing our proposal of a measurable definition of creativity relies on the quantitative evaluation of the two constituent ingredients of runco's definition @xcite , namely originality and effectiveness. more formally given a document d i , we denote by o(d i , d) a measure of the originality of d i with respect to a corpus d, and by e(d i , a) a measure of its effectiveness with respect to an audience a. the creativeness of d i is defined as follows: measuring originality. since originality in @xcite is strictly related to uniqueness, we propose to evaluate it by a similarity metric @xcite . specifically, given a document d i , ⃗ d i is its embedding (da costa et al., 2023), namely a vector representation of d i . we define the originality of d i respect to another document d j through their cosine similarity, namely: measuring effectiveness. effectiveness refers to the ability to convey a message or content to the audience a. being addressed to an audience implicitly requires humans in the loop and, consequently, a quantitative definition is difficult to scale and has to deal with the arbitrariness of human judgments. our initial proposal to evaluate the ability to convey a message is based on questionnaires administered to a. specifically, we assume that for each question q i in the questionnaire exists a set of answers ans i that proves the ability of the text (i.e. d i ) to convey the message or content. we measure the effectiveness as the fraction of the audience answering ans i . more precisely: ## preliminary experiment in this section we report on the results of a simple experiment to evaluate the creativity of aigenerated text compared to human-authored text. we do not aim here to drive concluding remarks on the outcome of the experiment, rather we simply show how our definition of creativity can be used in practice. the generated texts (ai 1 and ai 2 ), together with the texts produced by two human writers (h 1 and h 2 ), answering to the same request from the corpus d and are available in appendix a. first we removed the stopwords and then we embedded these texts into a 768 dimensional dense vector space using the all-mpnet-base-v2 sentence-transformer @xcite . the resulting vectors are ⃗ ai 1 and ⃗ ai 2 for the ai-generated stories, and ⃗ h 1 and ⃗ h 2 for the human-generated ones. figure 1 reports the pairwise originality between the texts. ai-generated texts show lower originality between themselves. a higher level of originality is shown when the two human-generated texts are compared between them and also when compared to ai-generated text. the originality of the documents with respect to the corpus according to eq. 3 is shown in to evaluate the effectiveness we showed the texts in the corpus to an audience of 15 readers unaware of both the creative source (i.e. ai and human) and the prompt. readers are described in appendix a. despite we still rely on humans to evaluate effectiveness, in section 5 we discuss how to automate also this process. they were then asked to answer the following questions, selecting a response among three options, one of which belongs to ans i as defined above: q1. what theme is discussed in the text? options: mistery, adventure, @xmath0. q2. which writer's narrative style do you recognize in the text? options: u. eco, c. doyle, j.k. @xmath1. ## conclusions and future works generative artificial intelligence is nowadays in all aspects of our lives, and a number of ai tools c(ai 1 ) c(ai 2 ) c(h 1 ) c(h 2 ) 0.42 0.40 0.54 0.47 the relationship between humans and machines must be seen as a productive and complementary alliance. there are effective and constructive approaches to achieving an optimal balance between the two, and it is essential to explore, refine, and continuously innovate in this regard. only in this way, the analytical power and learning capabilities of machines can be combined with the emotional depth and human experience, preserving and enhancing the creativity and innovation inherent in human intelligence. our work is a first attempt to provide a quantitative definition of creativeness and the preliminary experiment show the applicability of this definition to a simple but concrete use case. in the future, we plan to extend our experiments to a wider audience and to consider different and bigger corpus. a particularly interesting line of research, that will make out method fully automated, is the algorithmic classification of effectiveness. this goal needs the identification of high-quality corpus of homogeneous documents in terms of effectiveness, to train the classification algorithms.