Collective Critics for Creative Story Generation (2024)

Abstract: Generating a long story of several thousand words with narrative coherence using Large Language Models (LLMs) has been a challenging task. Previous research has addressed this challenge by proposing different frameworks that create a story plan and generate a long story based on that plan. However, these frameworks have been mainly focusing on maintaining narrative coherence in stories, often overlooking creativity in story planning and the expressiveness of the stories generated from those plans, which are desirable properties to captivate readers’ interest. In this paper, we propose Collective Critics for Creative Story Generation framework (CritiCS), which is composed of plan refining stage (CrPlan) and story generation stage (CrText), to integrate a collective revision mechanism that promotes those properties into long-form story generation process. Specifically, in each stage, a group of LLM critics and one leader collaborate to incrementally refine drafts of plan and story throughout multiple rounds. Extensive human evaluation shows that the CritiCS can significantly enhance story creativity and reader engagement, while also maintaining narrative coherence. Furthermore, the design of the framework allows active participation from human writers in any role within the critique process, enabling interactive human-machine collaboration in story writing.

## introduction recent progress in large language models (llms) has enabled the automatic generation of long-form stories containing several thousand words. this advancement has led to the proposal of various frameworks @xcite @xcite that are capable of generating stories maintaining narrative coherence. in pursuit 1 our code are publicly available at https://github.com/emnlp-2024-critics/ collective-critics-for-creative-story-generation of this capability, many frameworks adopt a twostep approach @xcite @xcite : (a) creating a story plan; and (b) generating a detailed story based on the plan. to explore further, some works have made effort to improve structure of story plan @xcite and refine the writing style @xcite . despite their ability to produce consistent and coherent text, these works frequently fall short in crafting creative long-range narratives that captivate human interest, such as stories with emotional flow, flashback structures, and other engaging elements @xcite . therefore, we propose collective critics for creative story generation (critics), a novel framework for long-form story generation that integrates collaborative critique into the plan-to-story framework. our framework focuses on creatively refining story plans (crplan) and enhancing the expressiveness of stories (crtext). inspired by studies demonstrating that collaborative work significantly enhances creativity through the sharing and integration of knowledge and ideas from various fields @xcite , critics adopts a collaborative critique approach. specifically, in each stage, multiple llm critics evaluate the draft and provide suggestions for improvement based on criteria (e.g., uniqueness of the narrative flow and vividness of the generated text) which is designed for assessing creativity @xcite . then, a leader selects a critique that best helps refine the draft. this revision process iterates through multiple rounds to produce a final plan and story (figure 1 ). in crplan, the critics focus on infusing originality into the story plan (e.g., unexpected shifts in the story's atmosphere, twisted endings). to do so, critics are endowed with personas that are adaptive to a given storyline, which helps make clear and diverse suggestions to enhance the story's creativity figure 1 : the framework comprises two stages: crplan and crtext. crplan involves five phases: creating a story plan from a premise, reviewing the plan with critics' persona-driven perspectives, selecting a critique for revision by a leader, storing the revised plan, and choosing a plan for further development by an evaluator. personas of critics are created based on the themes or content of the narratives, which helps in generating detailed and contextually relevant critiques. please refer to appendix a.1 critiques and refines long story text based on creative criteria, with a leader selecting and improving the best expressions. and coherence. crtext is designed to improve the expressiveness of stories. it refines the narrative to incorporate unique expressions and vivid descriptions. this includes comic book onomatopoeias (e.g., 'swooosh'), figurative phrases (e.g., 'silent twilight'), and detailed sensory experiences (e.g., a 'buzzing market' or the 'sharp tang of sea air'). extensive human evaluation demonstrates that our critics produces more creative and thus interesting plan, and generates expressive story. to be specific, critics outperforms the state-of-the-art method on the metrics of creativity and interestingness by a large margin. additionally, detailed analyses indicate that our design choices for crit-ics (e.g., applying multiple criteria, role of leader critics, and bestowing personas) in addition, critics facilitates interactive human-machine collaboration in writing by en-abling humans to participate as any of the players within the framework, refining stories according to their preferences. the main contributions of this study are: creative writing. the assessment of creativity in creative writing has been persistently discussed among researchers. traditional approaches to assessing creativity have relied on criteria like novelty and unexpectedness @xcite @xcite . however, these approaches are abstract, as they typically rely on broad standards for evaluation. the need for specific criteria to assess the writing has led to rubric-based methods using the traits of creative writing @xcite @xcite , generally assessing creativity in three aspects @xcite : image, voice, and originality. in this study, we adopt those criteria to give more detailed guideline for critcs to refine stories. ## collective critics for creative story generation critics is composed of two stages (crplan and crtext) that utilize the capabilities of llm to write creative stories. our framework enhances creativity at both the micro level (sentence expressiveness) and the macro level (story structure, themes, endings, etc. ## analysis ## conclusion we introduce critics, a novel framework that enhances story creativity and ensures narrative coherence through collaborative llm critiques. the critics systematically critique the story plans (crplan) and text expression (crtext), employing diverse creative criteria. their adaptive personas play a crucial role in maintaining narrative coherence, providing detailed critiques that help create a consistent, contradiction-free, and creative story. extensive human evaluation demonstrates that the critics significantly enhances the creative dimensions of stories while preserving narrative coherence. also, critics is designed to facilitate interactive collaboration between humans and machines, allowing active participation of human writers in the critique process. we hope critics opens new dimensions in computational creativity, enabling the generation of unique and captivating narratives.