Agents' Room:  Narrative Generation through Multi-step Collaboration
Authors: Fantine Huot
Reinald Kim Amplayo
Jennimaria Palomaki
Alice Shoshana Jakobovits
Elizabeth Clark
Mirella Lapata
Venue: ICLR 2025 Poster (2025)
Keywords: fiction
creative writing
long-form generation
LLMs
agent
collaboration
multi-agent
dataset
evaluation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose Agents' Room, a generation framework inspired by narrative theory, that decomposes narrative writing into subtasks tackled by specialized agents.

Abstract: Writing compelling fiction is a multifaceted process combining elements such as crafting a plot, developing interesting characters, and using evocative language. While large language models (LLMs) show promise for story writing, they currently rely heavily on intricate prompting, which limits their use. We propose Agents' Room, a generation framework inspired by narrative theory, that decomposes narrative writing into subtasks tackled by specialized agents. To illustrate our method, we introduce Tell Me A Story, a high-quality dataset of complex writing prompts and human-written stories, and a novel evaluation framework designed specifically for assessing long narratives. We show that Agents' Room generates stories that are preferred by expert evaluators over those produced by baseline systems by leveraging collaboration and specialization to decompose the complex story writing task into tractable components. We provide extensive analysis with automated and human-based metrics of the generated output.

Published as a conference paper at ICLR 2025
AGENTS' ROOM: NARRATIVE GENERATION THROUGH MULP COLLABORATION
Fantine Huot, Reinald Kim Amplayo, Jennimaria Palomaki, Alice Shoshana Jakobovits,
Elizabeth Clark & Mirella Lapata
Google DeepMind [fantines Centuin reinaddd f eaclark, Lapaaakaaapaaaaaaaaaaaoo
jakobovits,
ABSTRACT
Writing compelling fiction is multifaceted process combining elements such crafting a plot, developing interesting characters, and using evocative language. While large language models (LLMs) show promise for story writing, they cur- rently rely heavily on intricate prompting, which limits their use. We propose AGENTS' ROOM, generation framework inspired by narrative theory, that de- composes narrative writing into subtasks tackled by specialized agents. To illus- trate method, introduce TELL ME STORY" high-quality dataset of com- plex writing prompts and human-written stories, and a novel evaluation framework designed specifically for assessing long narratives. We show that AGENTS' ROOM generates stories that are preferred by expert evaluators over those produced by baseline systems by leveraging collaboration and specialization decompose the complex story writing task into tractable components. We provide extensive anal- ysis with automated and human-baat based metrics of the generated output.
INTRODUCTION
Creating long-form content requires meticulous research, advanced planning, an engaging writing style, and the ability to craft stories that captivate. J.K. Rowling is claimed have had most of the Harry Potter story planned out before she started writing. She knew there would be seven books, which characters would be important and how they would develop, and which key plot twists would serve the overall story. In addition, she carried out substantial research to create the fictional universe which provides the backdrop of the story. Breaking down a story into distinct sections is typical for longer narratives, with most stories boiling down to few shared elements like exposition, rising action, climax, falling action, and resolution (Freytag 1896 Pavis] 1998). Practical guides to writing successful screenplays (Cutting 2016; Hauge (017) outline a similar structure, following the setup, the new situation, progress, complications and higher stakes, the final push, and the aftermath.
Large language models (LLMs) have demonstrated impressive writing capabilities (Yang et al.) 2022 Nur Fitria 2023 Shao et al. 2024; Bai et al. 2024), however, generating long-form con- tent is still a challenge. Well-known problems include maintaining consistent narrative, tone, or factual accuracy over extended stretches of text (Chakrabarty et al. 2024b) Wang et al. 2023b AL-
abdulkarim al. 2021 Balepur et al., 2023 Yamshchikov & Tikhonov (023), and showcasing unique voice or humor that makes writing truly memorable. Despite displaying flashes of creativity, they often replicate patterns found in their training data, which hinders the generation of original concepts, plotlines, or phrasing. Added problems include the lack of datasets or benchmarks for long-form writing (Bai et al. 2024) and standardized evaluation criteria for assessing creative writ- ing either by humans or machines (Chhun et al. 2022 2024| Chakrabarty et al. 2024a).
Existing methods often rely on detailed prompts to guide the generation process YYngg et al. 2022 Xie et al., (202)). prompt chaining (Mirowski et al. [023] Yang et al.) 2022). and planning strategies Yaagg al. 2023) Lee et al., (024) as a means of breaking down the complex writing task into more manageable components. In this paper, we conceptualize long-form writing as multi-agent
We release the dataset and metries at:
deepm nd/tel1 1_me__
story

---

Published as a conference paper at ICLR 2025
Scratchpad
Input
Orchestrator
Output
Contlict
Exposiion
Clmax
Recclution
Charscter
Setting
Rliing Action
Falling Action
Planning Agents
Writing Agents
Figure 1: AGENTS' ROOM, multi-agent framework for collaborative writing. A central orchestra- tor calls the individual agents and consolidates their contributions into the scratchpad. We color-code each piece of the scratchpad with the contributing agent's ' color.
collaboration problem. Rather than attempting decomposition of the writing task within a single agent (Chen et al,, 2023] Yao et al. 2024). we leverage collaboration among multiple agents, with specialized abilities (Taeebirad & Nadiri [023] Zhang al. 2024b Han et al. 2024). We propose AGENTS' ROOM (Figure
generation paradigm which consists of two types of agents, namely planning and wrifing agents. Planning agents flesh key components of the content but do not write the story as such. For example, planning agent might specialize in character descriptions, whereas another might focus on the plot central conflict. Writing agents responsible for generating the final output text and are also specialized, c.g., one may focus on the introduction, and another on the conclusions. The two types of agents work collaboratively to complete the writing task, sharing and managing information through scratchpad which maintains outputs from planning agents and makes them available to writing agents. An orchestrator responsible for calling the agents in order depending the task at hand.
Compared to single -...--owered agents, this multi-agent approach offers several advantages:
LLMs can specialized into various distinct agents (e.g., zero-shot prompted or fine- tuned) performing a single function with high precision;
it avoids well-known problems with lengthy and under-specified instructions which require multiple iterations build context and fully define an appropriate solution;
it can be applied to problems whose solution is not known beforehand, and results from exploring a vast research space or involves very long output (e.g., writing a book);
it naturally lends itself human-in iaman-in--he the loop automation where machine-based agents can be replaced with human ones when needed.
We formalize AGENTS' ROOM as a general writing framework and apply it to creative writing. Specifically, focus writing long-form stories (1,000-2,000 tokens), and create specialized agents drawing inspiration from narrative theory (e.g., Card 1999 ( 1999 1999| Pavis| 1998). To eval- uate our method, we introduce TELL A STORY, a new dataset of human created writing prompts and fiction stories, and a novel evaluation framework designed for assessing multiple dimensions of story quality. Experimental results show that AGENTS' ROOM generates stories that are preferred (by humans and automatic metrics) over those produced by baseline systems which do not leverage collaboration or specialization.
buman-created
RELATED WORK
Story Generation The advent of large pre-trained language models has provided common frame- work for generating stories which sound fluent but often struggle with maintaining coherence and
AGENTS' RoOM is very lonsely modeled after writers' room, collaborative space where writers, (usually of television series), come together to write and refine scripts.
2

---

Published as a conference paper at ICLR 2025
plausibility. Attempts to enhance coherence and control the trajectory of events often decompose the generation task into planning an outline or sketch, and then elaborating on it, e.g., by filling in descriptions and specific details of each story. Examples of intermediate plans include sequences of entities and their actions (Yao et al. 2019), outlines (Fan et al. 2019] Zhou et al. 2023 Wang jet al., (023a), plot structures (Goldfarb- Tarrann et al.] 2020). and more elaborate descriptions in- cluding details about the setting of the story, its characters, and main plot points (Yang et al. 2022]
2023). Other work uses common sense knowledge impose constraints on the characters and their interactions (Peng et al. 2022), ensemble- based models to render event sequences more plausible (Ammanabrolu et al., 2020), stylistic constraints (Kong et al., 202T). and twists through constrained decoding (Huang et al. 2023). These efforts have demonstrated that generating stories as a one-step process is chailenging. and ultimately various interventions are required to improve overall story quality. Our work follows on from this realization, and breaks down the writing task into subtasks, undertaken by different agents who collaboratively plan and write a story. Collaborative writing is often used in academic or professional contexts to leverage the strengths and perspectives of various contributors, and has also been shown to enhance creativity (Barrett et al. (021).
Using LLMs tools to assist humans with writing stories is active research area (Chakrabarty ct al. 2024b, Mirowski et al.) [023) Ippolito et al. 2022). In our experiments, stories are written ex- [p2oloppolito
clusively by models without humans in the loop. However, our framework is fairly general allowing for human-machine collaboration at various stages of content creation.
Multi-agent Systems LLM--based agents have recently shown robust reasoning and planning pabilities across various application domains (Zhao et al. 2023 Bubeck et al. (023). Multi-agent systems involve multiple independent LLMs working together to solve complex tasks that are be- yond the capability of any individual agent (Talebirad & Nadiri, 2023 Park et al. 2023 Han et al. 2024 Guo et al. 2024). The agents are typically specialized in different aspects of a problem or have different roles, allowing the system to approach tasks in a more coordinated, distributed, and modular way. L.L.-based multi-agent systems have recently demonstrated promising results in
multiple areas including software development (Hong et al. (024), robotic tasks such as motion
planning (Mandi et al. 2024), simulations of human behavior (Park et al.) [023] Hua et al. 2024),
the creation of gaming enviroments (Hu et al. 2024), recommender systems (Zhang et al. 2024a),
simulations of financial trading (Li et al. 2023). and policy making (Xiao et al. 2023). We are
not aware of existing multi-agent frameworks for long-form writing. We draw inspiration from bollvng improves LLM task-solving capabil- ities (Hao et al.) 2023 Wang al., 2024: Zhang al. (024b). Our agents each adopt specialized writing subtask and communicate through a shared scratchpad (or memory) which allows to effec- tively recall and utilize oonextually-relevaar past knowledge. In our experiments, we predefine the number and type of agents best suited to our story writing task, rather than dynamically generate agents based on story content (Chen et al., 2024).
related work demonstrating that collaborative
Evaluation Story evaluation is admittedly a challenging task for humans and machines. Human evaluation is usually considered as the gold standard, but is expensive, time-consuming (Guan & Huang 2020). and can be subjective. It also cannot capture diversity since model that copies directly from the training set would potentially pass the human quality bar without displaying any generalization or creativity (Hashimoto et al. (209)). Automated evaluation metrics based on lexi- cal overlap or semantic similarity between generated stories and their human references have been shown to correlate poorly with human judgements (Chhun et al. (022). In this paper, we introduce an LLM-based evaluator (Liusie et al.] 2023) Liu et al., 2024| Zheng et al., 2024 Bohnet et al.] 2024) to perform side-by-side comparisons of system outputs which correlates with human judge- ments. Inspired by recent proposals on how to assess human creativity (Chakrabarty et al. (002a)),
distill the story evaluation task into a few dimensions (e.g.. plot, language use) which humans
and machines can judge reliably.
3 AGENTS ROOM
In this section, we formalize AGENTS' ROOM, the proposed multi-agent framework for collabora- tive writing. Given complex writing task z, we generate output y. by decomposing the writing process into multiple subtasks tackled by specialized agents. The full AGENTS' ROOM framework is summarized in Algorithm and explained below.
3

---

Published as a conference paper at ICLR 2025
Algorithm AGENTS' ROOM framework
x
b Initialize the scratchpad >> While the orchestrator assigns next agent Select an agent given scratchpad b Obtain agent's s output > Update scratchpad
while (ss, A) = True and < T do
a₁ A))
y1 (s(s)
$ (s: (4,, ve))
if tyype(a)) == writing then
D If the agent is a writing agent, write to the final output
y/- 5 (e) end if end while return y
Return the final output
Agents We define an agent a E AA as a specialized model that takes text as input and returns text as output, specified by a unique identifier label I and mapping f V + V (where V vocabulary tokens). Each agent is specialized in a specific subtask. Under this definition, an agent can be a LLM fine-tuned for the subtask, a zero-shot prompted LLM with specific input prompt, a deterministic text processing function (e.g., string formatting and parsing), or even a human interacting with the system. Herein, we focus on L..M-based agents, but we formalize the general framework's 's modeling assumptions (e.g., agent imputs and outputs text instead of latent variables) to allow future work to incorporate human agents as well (e.g.. by iteratively editing LLM-generated text). We define two types of agents (see below), namely planning and writing agents, which differ both in function and in their mode of interaction with the generated output.
Multi-agent Communication Communication between agents is critical for the successful com- pletion of their tasks. While different forms of communication are possible, such as debate (Khan) et al., 2024 Zhang et al., (024b) or competition (Cheng et al., 2024), in this work we focus on collaborative communication since it would transfer most naturally to human-LL.M collaborations. Collaborative agents work together towards a shared goal, exchanging information to enhance a collective solution.
Scratchpad The overall system requires a mechanism for sharing and managing information across the different agents. We assume our agents have access to a shared scratchpad E V that maintains individual agents' outputs and passed along to the next agent. The scratchpad is initialized with the initial writing prompt and is then updated after each agent call. At each step I, the current agent oe takes as input the current scratchpad 8, and generates output yt- At each step, the scratchpad updated with the agent's 's unique identifier and output such that 8141 <- (sx: Ut)). We include the agent's label so that individual agents can easily reference and parse specific portions of the scratchpad to complete their subtask. Note that in this framework, the scratchpad does not contain the specific input prompt of a given LLM agent. Indeed, it is considered part of each agent's 's subtask to process the output y into suitable format to be used by other agents. Since agents have access to the scratchpad, this means that they can avoid writing redundant and duplicate information.
Orchestrator We have opted for centralized architecture, where central orchestrator determines the order upon which individual agents are called, and whether calling on each agent is necessary (e.g., depending on the task). Given a scratchpad 5, and set of available agents A, the orchestrator o: V* x A + A determines the next agent G++1 to call. It can be modeled as a Markov process, since each step depends entirely on the state of the scratchpad at step This orchestrator can be discrete deterministic process, can have learnt transition probabilities, or can be arbitrarily complex. The orchestrator determines the stopping condition, i.e, when there is no more agent to call, or when a maximum number of steps T has been reached.
Planning Agents Previous work (see Section 2) shows that LLMS benefit from an intermediate planning stage before generating the final output. These intermediate steps improve the overall output but are not included in the final output. We define planning agents as agents that specialize in generating these intermediate steps and write exclusively to the scratchpad. For instance, when writing a story, planning agents can draft character descriptions and plot elements; when writing an essay, they can outline the argumentative structure and retrieve references to substantiate claims. Since their outputs are in text format, a human-in -ietm-ii-thh--oog could review or edit these intermediate stages guide the generative process.
4

---

Published as a conference paper at ICLR 2025
Writing Agents Certain complex tasks, such generating particularly long outputs or with sec- tions written in different styles, remain challenging for LLM to generate in one go. In such cases, the final output benefits from being generated section by section through separate agent calls. We define writing agents as agents specializing in writing specific parts of the final output. In addi- tion to writing the seratchpad, these writing agents iteratively write pieces of the final output y- Therefore, the final output can be formalized as the concatenation of the outputs of all the writing agents. For story writing, writing agents can specialize in specific parts of the narrative arc, such as the exposition or the climax; when writing an essay, they can each tackle different sections, such as the arguments in favor versus against.
4 FICTION WRITING TASK
In this section, we present an instantiation of the AGENTS' ROOM framework for fiction writing: given an initial writing prompt z, generate narrative y. We also introduce TELL ME A STORY', a new high-quality dataset of human-created writing prompts and fiction stories.
4.1 SPECIALIZED AGENTS INSPIRED BY NARRATIVE THEORY
We design specialized agents for our fiction writing task by drawing inspiration from narrative the- ory. We design four planning agents, each specialized in a specific aspect of story planning: [CON- FLICT] defines the central conflict (e.g., young boy has to fight an evil wizard who killed his parents), [CHARACTER] develops the character(s) (e.g., the young man is brave, caring, determined, loyal to his friends with strong moral compass), [SETTING] develops the setting (e.g., most of the story takes places in the Hogwards School of Witcheraft and Wizardry, fictional boarding school of magic for young wizards), and [PLOT] outlines the plot elements (e.g., the boy discovers he is the son of famous wizards and will attend Hogwarts School of Witcheraft and Wizardry). These plan- ning agents target specific weaknesses observed in --geerrreee stories. Indeed, LLMs struggle writing compelling plots and consistent characters throughout long stories (see Section 2).
In addition to these content planning agents, we design five writing agents, each specialized in dis- tinct elements of typical story structure: [RXPOSITION], [RISING ACTION), [CIIMAX], [FALLING ACTION], and RRSOLLUTONN]. We adopt this structure since it is widely used and quite versatile (Freytag 1896 Pavis 1998). leaving other narrative structures for future work. When generating in one go, LLMs struggle to meet length requirements (c.g., specified in the prompt), resulting in stories that are generally too short (see Section 7]. Since our writing agents generate the final output piecemeal, section by section, this results in longer outputs.
We model each of these agents as an LLM with a specific prompt template that formats the scratch- pad into an appropriate prompt for each agent's 's subtask. Detailed scratchpad formatting and prompt templates for each agent are provided in Appendix [ To coordinate between the different agents, we define a deterministic orchestrator that first calls the planning agents as follows: [CONFLICT] -> [CHARACTER] + [SETTING] + [PLOT], then calls the writing agents: [EXPOSITION] + [RISING ACTION] - [CLIMAX] + [PALLING ACTION] - [RESOLUTION]] We choose to use deterministic orchestrator for simplicity, given the strong narrative theory prior. In future work, more refined or- chestrators with learned objectives structures. As a first step towards building adequate reward models for training such orchestrators, investigate automated evaluation strategies for the long-form fiction writing task in Section 6
be explored, possibly expanding to wider range of narrative
4.2 SYNTHETIC DATA GENERATION FOR AGENT TRAINING
For each agent, we explore zero-shot prompted and fine-tuned approaches, since different degree of subtask specialization can be achieved through each approach. Fine-tuning requires agent outputs, which are not readily available; planning agent outputs such as plot and setting are usually not provided in datasets, while writing agent outputs require the stories to be split into its constituent
parts. Similar to previous work (Schick et al., 2022 Narayan et al. 2023 Josifoski et al. 2023 we propose to generate synthetic outputs for these agents through distilled backtranslation.
Specifically, given as input writing prompts and stories from a dataset (see Section 4.3 we zero- shot prompt larger teacher LLM to (1) generate the planning agent outputs, and (2) segment the
5

---

Published as a conference paper at ICLR 2025
Example Prompts
Write story about somcone
haunted by a ghost who wants give business advice. This story should be around 2500 words. Don't make it scary. The main character is trying to make her food truck popular, so she travels around the southwestem part of the country in her food truck to gain more popularity. After long time road. she comes home find ghost. This glost doeso's 't want scare her. He wants give her business advice because loved her food when he was alive. In the ead. she accepts the help of the ghost.
Write science fiction about now wonders if it was worth it. The story should be between 850 and 900 words. The story should bogin with the main character waking up on a frozen tundra. He looks for sheller from the cold. He sees a dead waely matmmoth and realizes he traveled back the ice age. The character should find shelter, and predater outsidee his shelter at night. The ending should not be happy.
who is time traveler and has dedicated everything their life towards goal, and
Figure 2: Prompts from the TELL ME A STORY dataset (corresponding stories are in Appendix AA.
Comparison of TELL ME A STORY against existing open-ended story generation bench- marks. We report statistics on the number of training, validation, and testing instances; Input/Target denote the average number of tokens in the input (aka prompt) and target text.
Table 1:
number of examples
avg. token length Input
Dataset
Training
Validation
Testing
Target
WRITING Fan et al. 2019)
272,600
15,620
15,138
28
735
ROCSTORIES (Mostalazadeh et al. 2016)
176,688
9,816
4,909
9
41
MHANGEMYVIEVS (Hua al. 2019 WIKIPLOTS
42,462
6,480
7,562
18
104
69,288
8,661
8,662
4
195
TELL ME A STORY
123
52
55
113
1,498
story into distinct parts (c.g., exposition, climax). Note that unlike typical distillation methods, our task is more straightforward; all that is required is to reverse engineer the agent outputs from an existing story rather than generate them from scratch. The teacher LLM outputs then used to generate synthetic training datasets for both planning and writing agents. Detailed prompt templates provided in Appendix C
4.3 TELL ME A STORY DATASET
Creative writing presents particular challenge from data collection perspective; is akin to any traditional annotation or evaluation task where requester provides some input and some set of guidelines for marking up that input in a consistent manner. While standards exist for "good" writ- ing, they evaluate the quality of writing across multiple interdependent and independent dimensions, all at once. In addition to this, the skill of writing really represents several skills that are learned over the course of person's 's lifetime and educational experience. Furthermore, evaluating writing necessarily requires the subjective stance of the evaluator.
Taking into consideration all of these complexities, collected TELL ME A STORY through writing workshops to replicate the organic environment in which a collaborative writing process can take place. We provided group of writers (28 in total) with broad instructions for quality based on collation of the Publication Manual of the American Psychological Association (currently in its 7th edition), the GRE/GMAT writing assessment rubrics, and various mass market style guides. Writers created their own prompts, wrote an initial draft, received feedback from peers, revised, and then submitted workshop lead for second round of feedback and final approval. Workshop leads could ask for additional edits or accept as is. Workshops lasted on average 3-4 weeks. option of downtime between workshops or the opportunity prevent burnout. The average rate of production for workshops generally reached no more than 2-3 writing samples per writer per week. We provide example prompts Figure 2 and example stories in Appendix A The majority of the stories belong to the genres of science fiction and fantasy but are also representative of the following genres: horror, drama, comedy, adventure, and folklore.
work in another workshop if they desired to
Available at: http:::///gthhbbb.cmmmaaaaaa

---

Published as a conference paper at ICLR 2025
Table compares TELL ME STORY against commonly used story generation benchmarks. Our dataset is small in scale and thus not suited to training model from scratch. Our prompts are more detailed compared to other benchmarks (see Input column) and the target stories are genuinely long Note that some of these datasets, although
(e.g, double in size compared to WRITINGPROMPTS useful for system development, are not strictly speaking narratives. WIKIPL.OTS is collection of plots from Wikipedia rather than stories, ROCSTORIES are five-sentence long common sense stories, and CHANGEMYVIEW contains pairs of posts and their counter-arguments.
EXPERIMENTAL SETUP
Comparison Systems The state-of-the-art approach for generating narratives consists of generating the story in one go, either through zero-shot prompting (see Figure 2 or fine-tuning, which we denote as E2Ezs and E2E respectively. We also experimented with more detailed instantiations of E2EZs by instructing the model to: (1) generate the central conflict, characters, setting, and plot before generating the story (E2Ezs plan); (2) reflect on the central conflict, characters, setting, and plot according to detailed guidelines, before generating the story (E2Ezs reflect); self-reflection (Madaan et al.) [023] [Shinn et al.) 2023) has been previously explored to solve intricate tasks that could be challenging for zero-shot prompting; both baselines (E2Ezs plan, E2Ezs reflect) use the same detailed instructions provided to our planning agents (see Appendix B.2); (3) generate plan automatically before generating the story in one call (E2Ezs decompose); in this case, plans predicted without any spec-fe knowledge, story generation is decomposed into series of simpler sub-problems which are solved sequentially (Yang et al. 2022 Khot er al. 2023]; and
(4) generate plan for the story first followed by a second call in which the model is instructed to generate the story based on the input prompt and the plan (2STAGE decompose).
AGENTS' ROOM Variants We use the w tag denote the AGENTS' ROOM variant with the writing and planning agents as previously described (see Section 4.1). To explore trade-offs between the different types of agents, we investigate two additional variants, plan and write, where we use only planning, or only writing agents, respectively. In the specific case of the plan variant with only planning agents, we still need a writing agent to finalize the story, since planning alone does not result in a final story. Therefore, the plan variant includes single simple writing agent, which we denote as the [F.])LIZER]. The prompt template for the [FINALIZER] agent is provided in Appendix B We investigate both zero-shot and fine-tuned agents. For each AGENTS' ROOM variant, we explore two settings, one with only zero-shot agents, and one with only fine-tuned agents, denoted as ARzs and ARFT, respectively. Since agents called independently, it is possible to mix and match between zero-shot and fine-tuned agents, but we keep the two settings separate to derive clearer signal for each approach.
Implementation For all comparison baselines and AGENTS' ROOM agents, we use a Gemini 1.5 Flast( backbone, lightweight and cost-efficient model that has demonstrated good performance across a wide range of generative tasks (Reid et al., (024). In particular, it features long context capabilities (up to one-million tokens) which makes it suitable for handling the scratchpad with multiple agents' contributions. We use input length out of {1,024, 2,048, 4,096, 8, 8.192} tokens depending on the length of the scratchpad and target token length of 4,096. While the outputs generated by the baseline systems are generally shorter than what is requested in the original prompt (see Section 7. we observe no improvements when increasing the target token length. We hypoth- esize that the observed limits on output lengths are likely due to the backbone model being trained on data with mostly shorter outputs.
For the synthetic training data generation described in Section 4.2 we use Gemini Ultra* Team (et al. 2023) as the teacher model. Since our dataset contains only a small number of training examples, we fine-tune our models (E2EfT and individual agents for ARFr) using LoRA (Hu et al.) 2021], a cumpuutationally-efficcii
approach that updates only a small portion of the model weights. We perform LoRA-tuning with rank 4 and learning rate of 1 (picked after hyperparameter search through {1-1, - 1-6 1-))). We LoRA-tune for 250 steps with a batch size of 16, saving checkpoints every 20 steps. We then select the checkpoint with lowest loss on the validation set.
"Available at: htttt:tt///ll http:::////ood.oooglgl.l. wtw.//....opppommmme /
7

---

Published as a conference paper at ICLR 2025
6 EVALUATION
We evaluate the quality of the generated outputs along several dimensions through human judgment elicitation and automated evaluation methods.
6.1 HUMAN EVALUATION
We evaluate system output by soliciting pairwise preferences (Louviere & Woodworth 1990) along four dimensions, as well as an overall preference. We distill previous proposals (Chakrabarly et al.,
2024b Chhun et al. 2022) on how to evaluate creative writing into the following criteria:
Plot
Does the story have recognizable structure, e.g., with connected beginning, middle, and end? Does it exhibit events and turns that move the plot forward without logical or conceptual inconsistencies?
Creativity Does the story have engaging characters, themes, and imagery? Does it avoid overly cliched characters and storylines, unintentional tropes, and stereotypes? Does it include original elements that were not explicitly mentioned in the prompt?
Development Are the characters and settings contextualized with relevant details that allow the reader to understand their place in the story? Are appropriate levels of detail and complexity provided to lend the story a feeling of realness and believability?
Language Use Does the language used feel varied and rich? Does the story exhibit rhetorical, linguistic and literary devices to create interesting effects? Does it avoid bland or repetitive phrases?
The full instructions are reproduced in Appendix D Participants are shown two stories and asked to decide which one is better in each dimension. They can also respond that the two stories are about the same, Participants are allowed rate up to five samples one sitting, due our task being cognitively taxing and time-consuming. We assign samples to participants following a Latin Square design, such that each participant does not rate the same writing prompt more than once. We randomize the order in which the two stories are shown to mitigate presentation order as potential bias. We gather ratings for all examples included in the TELL ME A STORY test set and compare outputs from all E2E and AGENTS' ROOM variants (see Figure 3) we also include the human-written stories as an upper bound. Our annotators were writers or had a degree in related disciplines (e.g., literature). We obtained total of 9,900 pairwise ratings which
converted into systems' relative strengths using a Gradley-Terry model (Bradley & Terry 1952; see Section 6.2). Inter-annotato agreement was 0.46 (p < 0.01, N 150. k 3), measured by Fleiss' Kappa, which we interpret to be satisfactory given the subjectivity of our task.
6.2 AUTOMATIC EVALUATION
Many previous studies (see Yang & Jin 2024 and the references therein) have highlighted the chal- lenges associated with evaluating narratives automatically. Metrics based lexical matching corre- late poorly with human judgments (Chhun et al., 2022; Chakrabarty et al., (222a)) and do not effec- tively measure story quality (e.g., is the story original and well-written with plausible characters). In this work, we report reference-based metrics, specifically Rouge-L (Lin 2004) and BertScore (Zhang et al. 2020), but also adopt several surface-based metrics' intended capture differences between human writing and LLM- generated stories. Specifically, we compute story length to deter- mine whether models are able to generate long stories and quantify structural differences between human and machine stories (e.g., number of sentences starting with an article or a pronoun). We also measure the ratio of unique words in story which gives an idea of creative language use, and intra- and inter-story trigram repetition Yao et al. 2019 Goldfarb- Tarrant et al. (020)) which cap- ture diversity within a story and across stories (high inter-story repetition suggests models generate similar stories even when given different prompts). Finally, trigram overlap with the prompt is used to indicate whether models can creatively elaborate on the information provided.
In addition, develop LLM-based evaluator (Liusie et al. 2023 Liu et al. 2024 Zheng al.
2024 Bohnet et al.] (024) to perform side-by-side comparisons of system output. We design prompts targeting the same dimensions of story quality adopted in our human evaluation. Specifically,
8

---

Published as a conference paper at ICLR 2025
Comparison between human and model generated stories using automatic metrics (TELL ME A STORY test set); #words (average number of words per story), #para (average number of paragraphs per story), Article (proportion of sentences starting with an article), Pro (proportion of sentences starting with pronoun), Unique (percentage of unique words), Intra (intra-story trigram repetition), Inter (inter-story trigram repetition), Overlap (proportion of trigrams overlapping with the prompt). We also report two reference- metrics, Rouge-L and BertScore. AR abbreviates AGENTS' ROOM systems; subscripts zs and FT respectively refer to zero-shot and fine-tuned.
Table 2:
Models
#words #para Article Pro Unique Intra Inter Overlap Rouge BertSc
E2Ezs
1,207 32.24 12.74 40.45 44.57
28.78
33.35
.0034
20.71
.8152
E2Ezs plan
1,130 27.24
15.34 42.25
45.93
23.59
29.41
.0027
20.58
.8173
E2Ezs reflect
1,126 28.62 13.79 40.96
45.85
23.68
23.95
.0032
20.36
.8152
E2Ezs decompose
965 21.25 21.36 39.62
45.49
31.98
44.10
.0034
19.41
.8067
E2EFT
1,193 32.25 12.58
43.39
44.02
28.21
31.31
.0036
20.73
.8138
2STAGE decompose
1,090 24.82 15.59
42.15
44.50
21.54
24.26
.0031
20.35
.816
AR z plan
926 20.95 13.82
40.68
43.88
29.70
33,49
.0017
19.58
.8119
AR zs write
3,278 63.80 25.32
39.91
34.97
47.50
44.09
.0022
17.34
.8103
AR ES plan write
3,034 58.65 15.97 41.43
35.05
44.73
43.25
.0022
17.57
.8123
ARFT plan
856 21.05 18.02 39.29
44.65
23.85
28.05
.0027
19.24
.8146
AR FT write
3,129 61.90
17.45
44.80
36.35
46.39
42.39
.0021
17.53
.8150
AR FT plan + write
3,006
56.85
17.52
43.03
34.30
46.31
41.60
.0019
17.60
.8152
Humans
1,439
32.91
10.01
32.37
50.35
15.53
19.24
.0020
adapt the evaluation criteria described in Section 6.1 into a prompt template shown in Appendix E
This template asks the evaluator for a detailed assessment of the two stories presented, followed by a final conclusion, which is then parsed to obtain preference scores for each dimension. We provide an example usage in Appendix Given N system outputs for each of M input prompts, we evaluate all possible (unordered) pairs of outputs for each input (while shuffling the order in which the outputs are presented), producing M x x (N - 1)/2 different pairwise ratings. Finally, we obtain a wins matrix W where Wi.j is the number of times system wins over system j. This matrix is then used to obtain the systems' relative strengths after fitting Bradley-Terry model (Bradley & Terry 1952). We use Gemini 1.5 Pro as our LLM evaluator, as suggested in Bohnet al. (2024).
7 RESULTS
Table 2 compares human and model generated stories using surface- and reference based metrics. As far as story length is concerned, we observe that E2E stories are slightly shorter than human ones, while planning models are shortest overall. However, models which include writing agents produce considerably longer stories (by factor of two) with more dialogue as suggested by the increased number of paragraphs. We also find machine stories to be more generic in their sentence structure as evidenced by the higher proportion of stories which start with an article or pronoun. Nuumnn-wrttter stories are also more diverse (less repetitive) as shown by the higher ratio of unique words and less repeated trigrams (Inter and Intra in Table 2). The most repetitive models are also the ones that produce the longest stories. In terms of overlap with the prompt, we find AGENTS' ROOM systems to copy least, at a rate similar to that of human writers. Rouge-L rewards the E2E systems most, as they least deviate from the prompt gold standard stories, while BertScore is not very discrimi- nating, equally preferring the simplest (E2E2s) and most complicated system (AR plan writte. Examples of stories written by humans and machines can be found in Appendix A
Figure reports system rankings obtained from human judgments and the LLM evaluator. For the sake of brevity, omit baselines shown underperform in Table (i.e., E2Ezs with plan, deflect, decompose and 2STAGE decompose) but include these results in Appendix F
2
Human-written stories are preferred overall As shown in Figure Bh, human judgments reveal performance gap between machine writers (E2E and AGENTS' ROOM) and professional writers, a finding that is in line with Chakrabarty et al. (202a)). We observe this gap across all dimensions, but
9

---

Published as a conference paper at ICLR 2025
a. Human-based ranking
b. LLM-based ranking
3
1.5
Strength
2
1.0
1
0.5
0.0
overall
plot
creativ.
develop.
language
overall
plot
creativ. develop.
language
Human
E2E
AR plan
AR write
AR plan+-write
ZS
FT
Figure 3: Overall system ranking across dimensions of plot, creativity, development, and language, according human ratings (a) and a LLM-based evaluator (b).
we note that is smaller in the language use dimension. This result suggests that while machine- generated stories still fall short in terms of compelling plots and unique ideas, LLMs, in their current state, may be useful as writing assistants. To ensure that the preference towards human stories not merely due them being longer, we computed the proportion of pairwise comparisons for which our human raters preferred the longer story overall (excluding ties) and found it to be around 0.51.
AGENTS' ROOM outperforms baseline systems Across all dimensions, our participants prefer AGENTS' ROOM stories with writing agents over those produced by baseline systems, with the FR write and AR plan+write systems performing best. Aside from rating the stories, par- ticipants had the option to leave feedback on their quality; we provide samples of this feedback in Appendix F AR plan variants do not perform that well, most likely due to the single [FINALIZER] agent being too simplistic to make good use of the planned elements provided in the scratchpad. We note that fine-tuned agents yield better results over zero-shot ones, which shows that generating syn- thetic data by back-translating from gold standard outputs (see Section 4.2) is an effective strategy for training specialized agents for different subtasks. Finally, we observe similar trends with smaller scale models (see Appendix F for additional results).
The LLM evaluator agrees with humans and itself The LLM-based rankings in Figure 3p reveal similar tendencies to human ratings. The LLM overall prefers human stories and those generated by the AR plan + write system against all other model variants, even though it does not discriminate very strongly between those two. LLM-based judgments of story quality correlate significantly with human ratings across all dimensions, both by systems (Spearman's rank correlation p 0.01, N 45) and by items (p 0.41; p 0.01, N 9, 900). The LLM and humans have the highest agreement when assessing story development (p 0.83, p < 0.01) and creativity (p 0.85, p < 0.01). Similarly the findings in Bohnet al. (2024), we also find that the LLM evaluator scores are consistent: 90.2% of the time the LLM prefers the same story in a second run, when the stories are presented in the opposite order.
0.62;
CONCLUSION
We propose AGENTS' ROOM, a general framework for multi-agent collaborative writing, and de- scribe its instantiation for the long-form fiction writing task. Drawing inspiration from narrative theory, we decompose the complex writing task into subtasks tackled by specialized agents. To il- lustrate this framework, we present TELL ME A STORY, a high-quality dataset of prompts and long- form stories collected through multiple rounds of writing workshops with human participants. We show that AGENTS' ROOM generates stories that are preferred by human evaluators over those pro- duced by baseline systems. Moreoever, we demonstrate effective training stategies for developing specialized agents by leveraging synthetically -generrte data. We introduce human evaluation framework for evaluating long-form narratives across several dimensions, and an LLM- eval- uator that correlates significantly with human raters. With improved automated evaluation, future work can explore more sophisticated orchestrators, including the development of reward models and learning objectives for such orchestrators.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
There are a number of ethical considerations when using generative language models. While the work we present here makes step towards improving the quality of text generation systems, it important to note that current systems are still far from perfect in this respect and may make mistakes. In particular, generative models may perpetuate biases present in their training data. Even when writing fiction, the models may inadvertently amplify socictal biases or reinforce stereotypes, leading to the production of biased content. Therefore, the generated outputs should be meticulously verified and used with caution.
REPRODUCIBILITY STATEMENT
For reproducibility, we release the TELL ME A STORY dataset on which we conduct our experiments, complete with its train, validation, and test splits, as described in Section 4.3 We specify the model backbones, implementation details, and where to access the checkpoints in Section 5 All prompt templates and scratchpad formatting templates are provided in Appendix. For the evaluation, we provide the exact rater instructions in Appendix and the LLM evaluator prompts in Appendix B
REFERENCES
Amal Alabdulkarim, Siyan Li, and Xiangyu Peng. Automatic story generation: Challenges and attempts. In Nader Akoury, Faeze Brahman, Snigdha Chaturvedi, Elizabeth Clark, Mohit Iyyer, and Lara J. Martin (eds.), Proceedings of the Third Workshop on Narrative Understanding, pp- 72-83, Virtual, June 2021. Association for Computational Linguisties. doi: 10. 10.18653/v1/0211. nuse- 1.8. URL / /////ccaann http//aaannnoolo.rrggaoaaoo/a//e mwwe://ee22mmum1m.m_m
Prithviraj Ammanabrolu, Ethan Tien, Wesley Cheung, Zhaochen Luo, William Ma, Lara J. Mar- tin, and Mark O. Riedl. Story realization: Expanding plot events into sentences. Proceed- ings of the AAAI Conference on Artificial Intelligence, 34(05):7375-7382 Apr. 2020. doi: 10. 1609/aaai v34003.6433333 6232.. URL view 2232|
wwwp:ooggg/iiiieeeeeeeee hatp:aaaa...oooooeeeonaa//a////a http://AAAaartiilee/
Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longwriter: Unleashing 10,000+ word generation from long context Ilms, 2024. URL wwtp///g/////e..e /ww.aa///2022. 07055
https::/arrx /ttp:s///axxi...or/aaa//////
Nishant Balepur, Jie Huang, and Kevin Chang. Expository text generation: Imitate, retrieve, para- phrase. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings the 2023 Conference Empirical Methods in Natural Language Processing, pp. 11896-119199 Singapore, December 2023. Association for Computational Linguistics. doi: 10. 1125111112223..mmmm// .emllp-mmlppee.ee main.729. URL https:
org. 2023. emnlp ma.in... 729
Margaret S. Barrett, Andrea Creech, and Katie Zhukov. Creative collaboration and collaborative creativity: A systematic literature review. Frontiers in Psychology, 12, 2021.
Bernd Bohnet, Kevin Swersky, Rosanne Liu, Pranjal Awasthi, Azade Nova, Javier Snaider, Hanie Sedghi, Aaron T Parisi, Michael Collins, Angeliki Lazaridou, al. Long-span question- answering: Automatic question generation and qa-system ranking via side-by-side evaluation. arXiv preprint arXiv:2406.00171 00179, 2024.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(344) 1952.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023. URL Uttpttps: httt:ss:::///ax :tttpps://rxivv/ wwtp:iiwwrg/bb//////2//
12712
Orson Scott Card. Characters and Viewpoints. Elements of Fiction Writing. Writer's Digest Books, 1999.
11

---

Published as a conference paper at ICLR 2025
Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. Art or artifice? large language models and the false promise of creativity. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI 24, New York, NY, USA, 2024a. Association for Computing Machinery. ISBN 9798400703300. doi: 10.1 10.1 (..1145/3619000 3994.364333
URL htRlhhtpss///dder //doi wwwpiiorrr/..00 11144413/3944444 3642731116
Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and Smaranda Muresan. Creativity support in the age of large language models: An empirical study involving professional writers. In Proceedings of the 16th Conference on Creativity and Cognition, pp. 132-155, New York, NY, USA, 2024b. Association for Computing Machinery. ISBN 9798400704857. doi: 10.1145/ 1tt.:///...11113353333633333 3656200666660
363563..3656201.
Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay. Börje Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation. In Kate Larson (ed.), Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pp. 22-30. International Joint Conferences on Artificial Intelligence Organization, 8 2024. doi: 10.24963/ 2jcai.2024/3. https:s::://ao. org/ 10, htt.://000//11221111111
Main Track.
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompt-
ing: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL ntttps://openreeee
net/
Pengyu Cheng. Tianhao Hu, Han Xu, Zhisong Zhang, Yong Dai, Lei Han, and Nan Du Self-playing adversarial language game enhances Ilm reasoning. arXiv preprint arXiv:2404 2024.
Cyril Chhun, Pierre Colombo, Fabian M. Suchanek, and Chloé Clavel. Of human criteria and auto- matic metries: A benchmark of the evaluation of story generation. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee,
Lee, Enrico Santus, Francis Bond, and Seung-Hoor Na (eds.), Proceedings of the 29th International Conference on Computational Linguistics, pp- 5794-5836, Gyeongju, Republic of Korea, October 2022. International Committee on Computa- tional Linguistics. URL https: http:///aaaaattotaao.ee
Cyril Chhun, Fabian M. Suchanek, and Chloé Clavel. Do language models enjoy their own stories? prompting large language models for automatic story evaluation, 2024. /ttpsss::///rxi
org/ abs/ 2405. 13769
James E Cutting. Narrative theory and the dynamics of popular movies. Psychonomic bulletin & review, 23(6), 2016.
Angela Fan, Mike Lewis, and Yann Dauphin. Strategies for structuring story generation. In Anna Korhonen, David Traum, and Lluis Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguisticss pp. 2650 2660- Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.11853/111119-1254 URL https:
Gustav Freytag. Freytag's technique of the drama: an exposition of dramatic composition and art. Scholarly Press, 1896.
Seraphina Goldfarb-Tare Tarrant,, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. Content planning for neural story generation with aristotelian rescoring. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4338. Online, November 2020. Associ- ation for Computational Linguistics. doi: 10.18663/v 1ttp://12222222mmm../ mainpp- URL https:
//aclanthol www.hhooogy...gg////0
emn 1p-main 351
Jian Guan and Minlie Huang. UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 9157- 9166, Online, November 2020. Association for Computational Linguistics. doi: 10. 18.53v///vl/ 2020.emnlp-main. 7366 URL.httpp. //acl anthallogyoogg///aaa
emn p- i 736
12

---

Published as a conference paper at ICLR 2025
Taicheng Guo, Xiuying Chen, Yaqi Wang. Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. In Kate Larson (ed.), Proceedings of the Thirty-Thine International Joint Conference on Artificial Intelligence, IJCAI-24, pp. 8048-8057. International Joint Conferences on Artificial Intelligence Organization, 8 2024. doi: 444963/jcaii22224/8833 Utthttt::://dat .org/ 1...4963/ 24943/1iccaaa /tt////aaa.22///8//// 2024/890 Survey Track.
Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, Zhaozhuo Xu, and Chaoyang He. Llm multi-agent systems: Challenges and open problems, 2024. URL https://arxiv.ont
abs/24022 03578]
Rui Hao, Linmei Hu, Weijian Qi, Qingliu Wu, Yirui Zhang, and Liqiang Nie. Chatllm network: More brains, more intelligence, 2023. h/ttts:///axii///:.:::
/a////b///// 2..../34..1.1. 12998
Tatsunori B. Hashimoto, Hugh Zhang. and Percy Liang. Unifying human and statistical evalua- tion for natural language generation. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp- 1689-1701, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1 169.. URL http://aaaaatt wow.nntthoooo....oo.oooo org. 1169
Michael Hauge. Storytelling Made Easy: Persuade and Transform Your Audiences, Buyers, and Clients Simply, Quickly, and Profitably. Indie Books International, 2017.
Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang. Ceyao Zhang, Zili Wang. Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for multi-agent collab- orative framework. In The Twelfth International Conference on Learning Representations, 2024.
URL htttts://ppeereeeeeeeeee wwt.:wwwwwwwweeeeeeeeeeeeeeeeeee net www.nn/eooooummdddttaaaaaa....
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint 1eXiv:2/0600968) 2021.
Sihao Hu, Tiansheng Huang, Fatih Ilhan, Selim Tekin, Gaowen Liu, Ramana Kompella, and Ling Liu. A survey on large language model-based game agents, 2024.
Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge. Libby Hemphill, and Yongfeng Zhang. War and peace (waragent): Large language model-based multi-agent simulation of world wars, 2024. URL
org/ abs/ 2211.111111 17227
Xinyu Hua, Zhe Hu, and Lu Wang. Argument generation with retrieval, planning, and realization. In Anna Korhonen, David Traum, and Lluís Mårquez (eds.). Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2661-2672, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 1/655//11/19--1555 URL https: www.laatholoyy.orrgg/ao
Tenghao Huang, Ehsan Qasemi, Bangzheng Li, He Wang, Faeze Brahman, Muhao Chen, and Snigdha Chaturvedi. Affective and dynamic beam search for story generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 11792-1 1 806, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653/v 1/2023. hanp:////22....nnnnneeeeeemee aadinng--mmlp.789) org/2023. findiiinnigsss wwwpddinnggeeep....mme emnlp. 789
URL
Daphne Ippolito, Ann Yuan, Andy Coenen, and Sehmon Burnam. Creative writing with an ai-powered writing assistant: Perspectives from professional writers, 2022. URL https:
aaaxxiv.orggbbb/221111
Martin Josifoski, Marija Sakota, Maxime Peyrard, and Robert West. Exploiting asymmetry for synthetic training data generation: SynthlE and the case of information extraction. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 1555-1574 Singapore. December 2023. Asso- ciation for Computational Linguistics. doi: 18653v /2t222220222..... main.96. URL https: 7/acl wnt.aoooooooyyyoooooooooo
emn l p--malma momaain... 96
13

---

Published as a conference paper at ICLR 2025
Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Ed- ward Grefenstette, Samuel R Bowman, Tim Rocktäschel, and Ethan Perez. Debating with more persuasive Ilms leads to more truthful answers. arXiv preprint arXiv: 2402.06820. 2024.
Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular approach for solving complex tasks. In The 11th International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, 2023. OpenReview.net.
Xiangzhe Kong, Jialiang Huang, Ziquan Tung, Jian Guan, and Minlie Huang. Stylized story gener- ation with style-guided planning. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Computational Linguistics: ACL-IJCNLPP 2021, pp. 2430- 2436, Online, August 2021. Association for Computational Linguistics. doi: 10. 10.18655/11/2211 findings-act.2155 URL httpp:://aaaaathhloeeet/eo wtt.gyy..gg/211... findinggnnnnnnn acl. 215
Yukyung Lee, Soonwon Ka, Bokyung Son, Pilsung Kang, and Jaewook Kang. Navigating the path of writing: Outline-guided text generation with large language models, 2024. URL. https:
// /atxi..org/abs/220000
13919
Yang Li, Yangyang Yu, Haohang Li, Zhi Chen, and Khaldoun Khashanah. TradingGPT: Multi- Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Per- formance. Papers 2309.03736, arXiv.org, September 2023, URL
repec.
htt./p/arxppaappsseeee/ 0w2.2233990333333 03730333366633 html
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguis- tics. http::::s//aaaaaathoooooo
org. /Wgg/W0400 1013
Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulic, Anna Korhonen, and Nigel Collier. Aligning with human judgement: The role of pairwise preference in large language model evaluators. arXiv preprint arXiv:2403.16954, 2024.
Adian Liusie, Potsawee Manakul, and Mark JF Gales. Zero-shot nlg evaluation through pairware comparisons with LLMs. arXiv preprint arXiv: 2307.0889, 2023.
Jordan J Louviere and George G Woodworth. Best worst scaling: A model for largest difference judgments [working paper]. Faculty of Business, 1990.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegr- effe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bod- hisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36. pp. 46534-46594. Curran Associates, Inc., 2023. URL https::/proceed: ngs neurips cc httpl/ppprrrillsppppppppepp wow.ppppeerrcccceccccc....e.
Zhao Mandi, Shreeya Jain, and Shuran Song. Roco: Dialectic multi-robot collaboration with large language models. In Japan Yokohama (ed.), Proceedings of the IEFE International Conference
on Robotics and Automation (ICRA), pp. 286-293, 2024.
Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, and Richard Evans. Co-writing screenplays and theatre scripts with language models: Evaluation by industry professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI '23, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9781450394215. doi: 10.1145/ 3544548.3581225. httl:/ttts:::/o.orgg//// 10450.0....4444
3581225
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Van- derwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper under- standing of commonsense stories. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 839 849, San Diego Califor- nia, June 2016, Association for Computational Linguistics. doi: 10.18653/v1/NNI 1098. URL https: /tops://act antho ogy.org/ol......o..oo. 6- 1098
14

---

Published as a conference paper at ICLR 2025
Shashi Narayan, Joshua Maynez, Reinald Kim Amplayo, Kuzman Ganchev, Annie Louis, Fan- tine Huot, Anders Sandholm, Dipanjan Das, and Mirella Lapata. Conditional generation with a question-answering blueprint. Transactions of the Association for Computational Linguistics, 11: 974-996, 2023.
William Noble. Conflict, Action and Suspense. Elements of Fiction Writing, Writer's Digest Books, 1999.
Tira Nur Fitria. Artificial intelligence (ai) technology in openai chatgpt application: A review of chatgpt in writing english essay. ELT Forum Journal of English Language Teaching, 12:44-58. 03 2023. doi: 10.15294/elt.v 1211.64069.
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. Proceed- ings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST '23, New York, NY, USA, 2023. Association for Computing Machinery, ISBN 97984007013200 doi: 10.1 00.1145/366183336667666
URR.http:://dar wrg/10. 13
Patrice Pavis. Dictionary of the theatre: Terms, concepts, and analysis. University of Toronto Press, 1998.
Xiangyu Peng, Siyan Li, Sarah Wiegreffe, and Mark Riedl. Inferring the reader: Guiding auto- mated story generation with commonsense reasoning. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 7008-7029, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguisties. doi: 20.18653/v12022..0 htw.::22..inniinn-mmmp.. org/ 2ogg/222..c2 eew.iddinnneeem..........
URL bttps::/aclanthol.o.
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser. al. Gem- ini 1.5: Unlocking multimodal understanding across millions of tokens of context, arXiv preprint arXiv:2403.05530. 2024.
Gemma Team Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju. L' L'conard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram's e, Johan Fer- ret, Peter Liu, Pouya Dehghani Tafti, Abe Friesen, Michelle Cashon, Sabela Ramos, Ravin Ku- mar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanezyk, Ser- tan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grill, Behnam Neyshabur, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Boxi Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Christoper A. Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov. David Wein- berger, Dimple Vijaykumar, Dominika Rogozi' nska, D. Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimczak-Pluns nska, Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mcin, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost R. van Amersfoort, Josh Gordon, Josh Lipschultz, Joshua Newlan, Junsong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sod- hia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, L. Sifre, L. Heuermann, Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Gorner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal. Mchran Kazemi, Michael Moyni- han, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez. Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej, Reena Jana, Reza Rokni, Rishabh Agar- wal, Ryan Mullins, Samaneh Saadat, S. Mc Carthy, Sarah Perrin, S' S'ebastien Arnold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hemnigan, Tomás Kociský, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han,
15

---

Published as a conference paper at ICLR 2025
Woosuk Kwon, Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Brian Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jeffrey Dean, Demis Hassabis, Koray Kavukcuoglu, Cl'ement Farabet, Elena Buchatskaya, Se- bastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving open language models at a practical size. ArXiv, abs/2408 .0.0001 18, 2024. URL (nttt::t: api. semant icscholar: Srgg/Corpusiiaa 2 20843326
Timo Schick, Jane Dwivedi- Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sebastian Riedel. Peer: A collabora- tive language model. arXiv preprint arXiv:2208. 11663, 2022.
Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. As- sisting in writing Wikipedia-like articles from scratch with large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 1: Long Pupers), pp. 6252-6278, Mexico City, Mexico, June 2024. As- sociation for Computational Linguistics. doi: 10.18653/v /1//8855//1/2222442 hww.///2..aaaoooonnn.ooo 347. URL https:
7/aclanthology; hwwphhoooy....../a.eeo
wwa.l-laannnnnaaa 347
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In A. Oh. T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neu- ral Information Processing Systems, volume 36, pp. 8634-8652. Curran Associates, Inc.. files/paper/2023/ pdf
2023. URL
File/ sile/lbe4 7tte//144488888788888
Yashar Talebirad and Amirhossein Nadiri. Multi-agent collaboration: Harnessing the power of intelligent Ilm agents, 2023. URL hnttp://aazxi..oorg/ab/////00 033314
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.1 1805, 2023.
Yichen Wang, Kevin Yang. Xiaoming Liu, and Dan Klein. Improving pacing in long-form story planning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10788- Singapore, December 2023a. Association for Computational Linguisties. doi: 10.18653/v1/2022 http://222..nnnnnnneeiie/ findings-emnll.72.. URL /tttps://aclantho...
org/ 2023. findings emn 1p. 723
Yuxin Wang, Jieru Lin, Zhiwei Yu, Wei Hu, and Börje F. Karlsson. Open-worid story generation with structured knowledge enhancement: A comprehensive survey, Neurocomputingg 559: 126792, 2023b. ISSN 0925-2312. doi: http:://oii org/ 10. wwwpo///t...o.mmm..//// wwwp: 126792. URL https:
// //www. www.ncedieence..cccc.m.. www.///.cccceeeeeee wwtp///eeeeeeeeeeeeeeeeeeeeeeeee htt.:iccc.e/////0022222222000000
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi- persona colf-collaboration. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 257-279, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: (0.18853//1/20222 naacl-long.1 15. URI. htt.t:tt. a ant ogy. www.orgg222222 naaci ong. 15
Bushi Xiao, Ziyuan Yin, and Zixuan Shan. Simulating public administration crisis: A novel genera- tive aagent-based simulation system to lower technology barriers in social science research, 2023.
URL https::////attttt
org/ abs/ 2311 11...1111 06957
Zhuohan Xie, Trevor Cohn, and Jey Han Lau. The next chapter: A study of large language models in storytelling. In C. Maria Keet, Hung- Yi Lee, and Sina Zarrieß (eds.), Proceedings of the 16th International Natural Language Generation Conference. pp. 323-351. Prague, Czechia, Septem- ber 2023. Association for Computational Linguistics. doi: (0.18653/v1/20233
wwl.-innn.g22222. m3in. URL
https: / a antho wochholooy..ooo ogy. ogg.oorg//023.. inlg-ni g-ma 2.
16

---

Published as a conference paper at ICLR 2025
Ivan P. Yamshchikov and Alexey Tikhonov. What is wrong with language models that can not tell a story? In Nader Akoury, Elizabeth Clark, Mohit lyyer, Snigdha Chaturvedi. Faeze Brahman, and Khyathi Chandu (eds.), Proceedings of the 5th Workshop on Narrative Understanding. pp- 58-64, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/ v1/2023. wnu- 1.8. http://pp//aaaaaaaaaoooo.ooo////
Dingyi Yang and Qin Jin, 2024. URL
14622
Kevin Yang, Yuandong Tian, Nanyun Peng. and Dan Klein. Re3: Generating longer stories with recursive reprompting and revision. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp- 4393-4479, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v we0.6///////22.../o/ /wwp/2eee222m222mmmmee mapp.mmii..... URL https:://ccaatthooottt/////////a .rr. 2022. emnlp-ma 296
Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. DOC: Improving long story coherence with detailed outline control. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3378-3465, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 1115388112022..33. acl.ooo........o....... 190. URL /ttpss:/acllatthologg org/ 2023 3023.acl-long 190.
Lili Yao, Nanyun Peng. Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and- write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 7378-7383 2019.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimban. Tree of thoughts: deliberate problem solving with large language models. In Pro- ceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2024. Curran Associates Inc.
An Zhang, Yuxin Chen, Leheng Sheng, and Xiang Wang Tat-Seng Chua. On generative agents in recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, pp. 1807-1817, New York, NY, USA, 2024a. Association for Computing Machinery. ISBN 9798400704314, doi: 10.1145/3626772. 3657844. URL htttt:://oo....... org 10. /111/1126672.22115 365774444
Jintian Zhang. Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. Exploring collaboration mechanisms for LLM agents: A social psychology view. In Lun-Wet Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics Volume I: Long Papers), pp. 14544-14601, Bangkok, Thailand, August 2024b. Association for Computational Linguistics. URL httts:://claatthol.. htg/2024.2o acl
leng. 782
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Eval- uating text generation with BERT. In 8th International Conference on Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https: http:://openevviee
net forum? 1dd
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yinggian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du Chen Yang. Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian Yun Nie and Ji-Rong Wen. A survey of large language models, 2023. URL https: /atpss://aaxiii /taxxaivvvv. wrw.abgg///.....oooo
2303. 18223
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging aLLM-as-ajuudg with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36. 2024.
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, and Mrinmaya Sachan. Recurrentgpt: Interactive generation of (arbitrarily) long text, 2023. URL https:/:::/aat //apiii////ivv..//a org. abs//bb//2330 13304
17

---

Published as a conference paper at ICLR 2025
A
EXAMPLES FROM TELL ME A STORY DATASET AND SYSTEM OUTPUT
Example Prompt
Write story about someone who is haunted by a ghost who wants to give husiness advice. This story should he around 2500 words, Don't I make it scary. The main character is trying to make her food truck popular, so she travels around the southwestem part of the country in ber food truck to gain more popularity. After long time the road, she comes home to find ghost. This ghost docsn't 't want to scare her. He wants to give her business advice because he loved her food when he was alive. In the end, she accepts the help of the ghost
Humann-rriten Story
"Finally. I can sleep in my own bed for once." Rita said.
She plapped down on her hed in attempt soak in comfinees. But then there knock the donr, in her mind she said "Why!"
dragged bersell from her bed check see who it was, when got the there was no one there. She looked around to see if anyone was around, hut there was nothing. She wondered anyone kids had moved
she was away.
Disregarding the thought she went back into her apartment and went back to her bod. She didn't Idntt care to remove her shoes or her cout, that was just how tired she was. Rita had been on the road for the past live months trying to drum up enough money and support to start her own restaurant. Currently, she had been operating out a food truck that she had started a year ago. She went city to city every week, trying to find the perfect place that garnered enough foot traffic and husiness.
Aller live months of going around the southweslem part of the United States, driving, cooking, serving, and advertising her husiness wherever she went, she decided that it was time go hack to home base and rest up hit before she went back to the grind. It was three o clock in the morning when she finally
to get little more comfy.
She went into the kitchen where she had left her bags since it the first place you reached spartment. Rita didn't I have the energy drag them along with herself her
you entered the two bedroom the back of the aparimeni.
Once she got to the kitchen she noticed that the flyers she had made for her food truck were scattered on the ground. She picked them up, what she lound on the ilyers when looked at them scared her.
Rita could have swor she saw something moving in the flyer: she admitted the flyers an intricate design that looked like an optical illusion but what she saw was not part the design. Taking a sccond look, she did not see what she thought she saw.
"Noge her Promm nope, didn't It see a thing" Rita decided that she jusi needed more rest after all it was three in the morning. Buck Rita settled to get back in hed when she heard the knocking. but this time it was coming from her bathromm. She was definitely freaking out now. She grabbed the closest thing she could use as weapon. A wooden crate, with a faded beer label on it, was all there was in her line of sight.
She tiptoed her to the hathroom, with the crate held ahove her head ready to strike anything that moves. She pushed the door open with her font, and saw man standing in her hathronm. She swung the crate, hat it passed through the man, and she fell backwards. Seeing this. she wanted to close the door, but she couldn't since she had already fallen to the floor and was backing from the entrance the bulhrsom. Gathering the cuurage apgrosch the bathroum again, Ritu nothing but nole her cheval mirror that read, "Please don't be afraid again, and this I just want help " The bathroom was ice-cold all sudden. Still standing
the doorway, Rita the
he waved.
the initial shack of seeing him the first time she just waved hack.
Taking a deep breath in and exhaling, said, "Okay, is it that you want help me?" thought she was going cruzy something because she intently wailed for this transpurent being to communicate with her. She refused admit was al ghosi, she just waited.
Finally, the being began writing on the mirror He explained in wriling. "this would go faster if you would allow me near you."
It was freezing cold in the bathroom so the mirror appeared to have frost on it.
Reluctantly, she agreed, the man approeo ber, and she decided to close her eyes as if abe could somehow pretend this wasn't happening. He touched her shoulder, of a sudden Rita was hearing him in her mind.
"Hi, I'm Jeff. must say your cooking was the absolute best when I was alive." Shocked by his admission. Rita opened her eyes look at him.
Jelf stood there with smile his face and continued, "I had the truly the fitting last meal." Amazed by his candor, Rita tried to remember when she the locations she had been in the last five months, but she couldn't place him. Giving her order. it was my wife. But anyway, let's get thought, "How would
of your southern fried cabbuge the day died. It
this guy digest the information.
Jeff continued, "You won't remember back point
because I wasn't the your food truck venture."
picked took minute crazy. shrugged shoulders and gave in, think having a ghost doing tricks will attract people."
wani help you smirked her because knew
help with asking. "Okay, how do you suppose you are going to help, because
thinking
Jeff shook his head and just looked at Rita until she finished her muttering.
"First and foremost, I don't it do tricks. Second, you need an upgrade girlic girl. Who uses printed out flyers anymore in this techno- logical world? Also, that little design you got on those flyers scrap it. got lost for houss in that thing. trying to get your attention."
Before Jelf could finish, Rita muttered under her breath, "I knew I saw something on those flyers."
Jeff continued to speak. "Rita, need to expand even more than what you have. I'm not saying expand territory. I'm to reach more people you need to get on those food delivery apps of at least create website that will allow people to place She sluck in time long past when lechnology
orders." " Rita listened and she knew widespread.
Jeff was saying
After Jeff finished what be had to say, Rita asked. "So, do all of this, will let sleep in peace and stop knocking on everything gel my attention? If you have something to say just leave note on the mirre."
Langhin at Rita's acceptance him some things as far as the delivery service went. Then he allowed her to go to sleep.
things
out
do. He gave her some people to contact about setting up
"See you later Jeff, we will continue this when I wake up." Rita said as she closed the door to her room and fimally got into her bed and fell asleep.
18

---

Published as a conference paper at ICLR 2025
System output: E2Ezs
The desert sun beat down on Amelin's is food track. "The Wandering Wok," as she flipped sizzling stack tofu, her brow furrowed concentration. the past year, she'd d been crisscrosing Southwest, her truck mobile testament to ber culinary dreams, She'd dl started with simple concept: innovative vegetarian lakes on classic Asian dishes, fueled by fresh, local ingredients. It a hit her hometown, but she craved more. Now, she was pushing the limits, uryng new recipes, adjusting her menu, even experimenting with different locations. hoped. Sometimes, there were just a fow regulars, their faces familiar and comforting, but not enough to sustain her. The loneliness of the road was starting to get to her. The constant hustle, the uncertainty, the lack of a real home- -it was wearing her down.
Still, the crowds weren't big
One evening, after particularly disappuinting day in Sedons, Amelia packed up her truck, leeling a familiar ment. The drive hack to her hometown in Arizona long and quiet. By time she reached her pann of disappoint- bungalow with a sprawling backyard, she was exhausted.
home, cozy
Stepping inside, Amelia was greeted hy comforting scent of dust memaries. Her parents had passed years ago, leaving the house to her. She'd d kept it as a haven, a place to retum to when the road got too much. As sho unpacked her bags, a strange sensation washed over her. It wasn't fear, but sense of.. presence.
A shiver ran down her spine as she tumed, her gaze falling on a figure standing in the kitchen doorway. It was man, tall and wiry, mischievous glint in his eyes. He was transparent, faint shimmer of light outlinng his forn. ghost.
Amelin's 's initial fear quickly dissipated, replaced by strange smiled warmly. "Don't It be alarmed." he said, his voice a geatie whisper. "I'm not here to scare you."
of calm. The ghost, dressed faded, old-fashioned suit,
spectral voice, held a reassuring warmth. Amelia, still reeling from the shock, simply stared him. "I'm Jucob," the ghost continued, saile widening. "And been watching you." Amelia's 's eychrows sbot up. "Watching me?"
His words, though spoken
"Yes," Jacoh chuckled. "Your truck, Thee Wandering Wok.' I loved your food. I used come by every week when you still in town."
He gestured loward kitchen table, where a plate sat, adorned with a single, perfectly perfeet formed veggie spring roll. was dish Amelia had perfected years ago.
"I've been following you all over the Southwest, you know," Jacoh said, his voice tinged with hint of pride. "I've seen your straggles, your successes, your.. lack of customers."
lack
Amelia, still processing the situation, felt a blush creep up her cheeks. "Well, I'm trying." she mumbled, her voice barely above a whisper.
"You're trying hard," Jacob suid, shuking his head genily. "You're trying to be everything everyone. You need to locus, find your niche."
His words Amelia joll of electricity. It was true. She'd been trying to please everyone, offering everything from vegan pho to spicy Korean bibimhap, hoping to attract a wider andience. But in the process, she't ' lost her own unique voice.
"You have a gift, Amelia." Jacob continued. his gaze intense. "You make food that's 's's both delicious and beautiful, But you're 're hiding it behind a facade of... well, too much."
Amelia felt a wave of frustration wash over her. "It's not that easy." she said, her voice tight. "Competition is fierce. Everyone's 's trying he the next big thing."
Jacob chuckled, soft, ethereal sound. "Competition is good. Amelia. It pushes you to be better. But you can't compete you don't know what you stund
He paused, his eyes twinkling. "You need find your story, Amelia. What makes you different? What makes your food special?" Amelia thought about it. Jacch right. She'd ' heen focused the higger picture, the grand vision of a successful food truck empire, that she'd d forgotten the simple joy of creating.
"My grandma," she said, the words coming out rush. "She taught me how to cook. Her recipes, her stories, her love for food... that's what makes my food special."
Jacob smiled. "See? You have a story, A beautiful, personal story that people will connect with. You just need to tell it."
He went on, sharing own story, a tale a young man had come to Arizona from a small town the Midwest, seeking his fortune. Ie'dd worked hard, saving up every community. He'l been happy, he said, until sudden
to open small diner, place where he could share his love for food with the
had taken him away.
"I never got to share my dream with the world," Jacob confesseed his voice tinged with sadness. "But you, Amelia, you can. You have talent, the passion, the story. All you need is to find your own path."
He looked at her, his expression serious. "You need to create menu that reflects your heritage, your grandmu's is recipes, your own unique twist. You need to tell your story, Amelin. And you need to do it with passion, with love, with the same dedication your grandma had."
Amelia felt a surge of inspiration. Jacob's words, though spoken by a ghost, resonated deeply within her. She' 'd heen so last in the pussuit of success that she'd forgotten essence of what made her food special.
"Thank you." Amelia whispered, her voice choked with emotion. "I...I think know what I need do."
Jacob smiled, a gentle glow radiating from him. "I knew you would." be said, "Now go, Amelia. Go out there and make your grandma proud."
quickty as he had appeared, Jacob vanished, leaving Amelia alone the quict kitchen. The plate with the spring roll was still on the table, testament to his presence, his love for her food.
Amelia speat the next fow woeks rewoeking her menu. She incorporated her grandma's 's recipes, her own personal touches. her love for fresh. local ingredients. She created a menu that told a story, a story of family. tradition, and pussion.
She also decided to rebrand her truck. "The Wandering Wok" was gone, replaced by "Grandma's is Kitchen," a name that resonated with warmtt and authentscity.
The first time she took her revamped muck out, Amelia felt nervous excitement mixed with a sense of peace. She had found her voice, her story. she set up her truck bustinn market Phoenix. felt familiar presence, a gentle warmih surrounding her. Jacob was there, silent observer, his form shimmering faintly the afternoon sun. He smiled her, his eyes filled with pride. "This is your story. Amelia." " he whispered, "Tall it with your heart, and the world will listen."
Amelia took a deep hreath, her heart filled with gratitule. She had found her path, her purpose, thanks to ghost who simply
wanted share his love for food and his belief in her. And she started cooking, her hands moving with newfound confidence, she knew that Jacob would he with her, cheering her on, every step of the way.
The crowds came, drawn in by the aroma of her grandma's 's recipes, the warmth of her story, the passion that radiated from her every move. They iisted the love, the care, the heritage woven into each dish.
Amelin's 's journyy had taken a detour, detour that led her back to her roots, to her grandma's 's kitchen, to the true essence of her culinary dreams. And as she the smiles on the faces of her customers, she knew that Jacob, the ghost with a passion for food, was finally at peace. His story, his dream, was living on, through her.
19

---

Published as a conference paper at ICLR 2025
System output: AGENTS' RoOMss
The desert sun beat down on the cherry-red food truck, baking the metal siding until shimmered like mirage. Lily, her forehead creased with mixture sweat and Irustration, stared out empty parking lot, her reflection wavering the heat baze. Another day, another disappainting turnout. The farmers' market had been slow, and the lunchtime crowd had been nonexistent. The southwest was vast and beautiful, but the road to success seemed endlessly long.
Lily sighed, running a hand thrugh her mesay brown hair. She'd dreaml of owning a food truck since she was a kid, watching her grandmather whip culinary magic in their tiny kitchen. The smells of her grandmother's is cooking, the warmth of the family gathered around table, the joy her grandmother's eyes wanted share that same joy, that same passion, with the world. the reality was from the dream. food truck, her pride and joy. was becoming a burden. The money was dwindling. and the pressure was mounting.
presented each disb...it was memory Lily cherished. She
She'd spent the six months crisscrossing the southwest, driving from one hustling market another, her truck beacon hope in sea of uneertainty. She'd d tried everything - catchy slogans, bright signage, even offering free samples. But nothing seemed work. The competition lierce, the customers were lickle. Lily felt like she was losing her grip. her confidence slowly slipping away.
Exhausted and discouraged, she decided to head home. Her childhood home. nestled in dusty comer of the small town Tombstone, Arizona, was place of comfort memories. Mayhe a change of scenery, a little away of the road, would help her clear her head. She needed a break, chance to regroup and re-evaluate her strategy.
relentless grind
The drive back was quiet, only sound windows. As she passed through the familiar landscape, the desert wildflowers swaying in the breeze, the towering red rock formations casting long shadows across the parched earth, sense of melancholy settled over her. The soathwest, once symbol of adventure opportunity, now folt like reminder her own failures.
engine and occasional sigh the wind whipping through the open
The sun had begun its descent, paining the sky with shades of orange and purple, as Lily finally tumned onta the familiar street lending childhood home. The adobe house, with its terrncotta tile roof and sun-haked walls, stood bathed the golden light, a testament to time and resilience. It was a place of warmth and comfort, a place where she felt safe and loved. But as she pulled into the driveway, shiver ran down her spine. A sense of unease, leeling of being watched, crept into her heart. It was as if the house itself was holding its breath, waiting for something.
Lily shook her head, trying to disaiis the feeling. It was just her imagination, she told herself. She was tired, stressed, and probably just little hit paranoid. But she stepped out of the track, the feeling intensified. The air around her secined crackle with an unseen energy, and faint, cold breeze sent a shiver down her spine. She glanced around, but saw nothing. The street was empty, the only sound the chirping of crickets and the distant howling of coyotes.
She quickly shrugged off the unsetiling feeling, grabbing her bugs and heading lowards the house. The front dovr, heavy wooden structure adorned with a risty hrass knocker, creaked open as she pushed it, revenling a dark and dusty interior. The scent of old wood and forgotten memories hung in the air, thick and heavy.
As she stepped inside, sense of unense settled over her. wasn't It just the dust and the shadows that made her almost tangible fooling of presence, sense of being watched. She switched on the light, illuminating the worn-out fumiiture and the faded photographs ripped out.
but an
walls. The house, once haven childhood joy, now strangely empty. if heart had been
Lily took deep beeath, uying calm her nerves. She was home, cup of tea, and then try to ligure out her next move. She needed a plan, a way to get her food truck back on track. But as she started to unpack, faint shimmer caught her A flicker of movement in the comer of the mom, barely visible, like heat wave shimmering the growing stronger, taking shape.
was that mattered. She would unpack, make a
sun. Lily froze, her heart pounding in her chest. She blinked, rubbing eyes, but the flicker remained,
tall, slender figure with kind face and gentle smile. But be wasn't solid. He was translucent, like wisp smoke, and clothes were faded worn, as he belonged to a different time. He stood there, watching her, his eyes filled with a mixture of cursosity and concern.
It
throat. Her mind raced, trying to make sense what she was seeing. This couldn't real, told herself. She was exhausted, stressed, and probably ballucinating, But the figure didn't i disappcar. He simply stood there, watching her, his presence real the dust motes dancing in the sunlight streaming ihrough the window.
Lill's 's hreath canght
And then, be spoke, his voice a gentle baritone, as comforting as warm fire on cold night. "Welcome home, Lily."
Lily's voice caught her threat. She couldn't speak, couldn't even breathe. The man, the ghost, stood there, translueent form shimmering in the afternoo light. He seemed to be smiling, his eyes crinkling at the corners as if in amusement, but the sight of him sent chills down her spine.
"Don't I be afraid"" the ghost said, his voice a gentle murmur. "I mean you no harm."
Lily managed swallow, her throst dry as desert sand. "Who...who are you?" she stammered, her voice barely whisper. "My name is Mr. Henderson,"" the ghust replied, his voice comforting baritone. "I used to live here, in this very house."
uYed to live bere?" Lily repeated, her mind strugling to grasp the impossible. "But...yourre... 'r a ghost."
Mr. Henderson chuckled softly, sound like wind chimes tinkling the breeze. "Yes, am. But don't 't be afraid. I'm not here scare you. I'm here to help."
Lily stared him, her confusion growing. "Help?" she echoed, her voice filled with disbelief. "Help with what!"
"With your food truck," Mr. Henderson said, his smile widening. "I heard you's 'e been struglingg, I know a thing or two about cooking, you see. used to be chef, back in the day."
Lily's eyehrows shat up in surprise. "A chef?" she repeated, her skepticism hattling with sliver of curiosity. "But...how can you in help? You you.ye... ghost."
20

---

Published as a conference paper at ICLR 2025
Mr. Henderson chuckled again, his ghuatly forn shimmerinng faimily. "I may be ghos. but I still have pussion for food. And I have lot of experience. Perhaps can offer advice, insights, that might help you on your journey."
Lily's mind whirled. The idea of ghost offering business advice was absurd, preposterous even. But there was something about Mr. Henderson, something in his gentle demeaner, his genuine concern. that made her besitant to dismiss him outright. She'd dd been struggling, feeling and alone, and thought helping hand, even a spectral one, was strangely appealing.
"I... don't know," Lily said, her voice barely whisper. "This this lot to take in."
"Take your time," Mr. Henderson said, his voice sonthing. "I understand. It's not every day you meet a ghost who wants to give you business advice."
Lily couldn't 't help hut smile at that, a flicker of humor breaking through her upprehension. "No, it's not," she agreed, shaking her head. "It's certainly not."
Mr. Henderson'ss 's smile broudened. "Perhups we can have a cup of lea, get to know each other little betier?"
Lily hesitated, her mind jumble of conflicting emations. But something inside her, spark of hope, desperate yearning for solution. pushed her forward.
"Alright," " she said, her voice little stronger "A cup of tea. But I'm not promising anything."
Mr. Henderson's 's smile remained, his eyes twinkling with amusement. "That's perfectly fine. Lily," he said. "That's perfectly fine."
sn, dusty, sun cun-drenhhed living rom sharing a cup of tca and conversation that would change her life forever. The road to success had always socmed long and winding, but with Mr. Hennersoo her side, even if was little bit spectral, felt like muybe, just maybe, she was linally the right path.
childhood home, Lily found herself sitting across from friendly ghoot,
Lily's food truck basiness was booming, Her once-empty parking lots were now filled with cager customers, the aroma of ber signature dishes wafting through the air, drawing them in like moths flame. The local newspapers had fentured her, raving about innovative disles and her unique, whimsical approach cooking. She d even won a prestigious award regional food truck competition, a testament to her talent and the power of Mr. Henderson's ' guidunce.
But with success came new set of challenges. The pressure maintain her momentum, constant domands of running a thriving business, the ever pee-epreent fear of losing her edge these anxieties gnawed at her, leaving her leeling exhausted and overwhelmed..
One evening, after particularly bectic day, Lily found hersell back the familiar, dusty living room of her childhood home. Mr. Henderson, his spectral form shimmering in the soft glow of the lamplight, sat beside her, a cup of tea warming his translucent hand.
"You're ro working too hard, Lily," Mr. Henderson said, his voice a gentle reprimand. "Remember what told you about balance? You need to take care of yourself, to make time for the things that matler."
Lily sighed, her resting
hand. "I know, Mr. Henderson," said, voice weary. "But all so exciting, so overwhelming, I'm afraid of letting it all slip away."
Mr. Henderson smiled, eyes twinkling with understanding. "You're not letting anyihing slip away. Lily." he reassured her. "You've ve built something special, something that will last. But you need to remember that success is journey, not a destination. Enjoy the ride."
Lily looked at him, her heart heavy. "I don't know, Mr. Henderson," she said. "I feel like I'm losing myself in all this. I'm so focused on the food truck, on the business, that I've forgotten who I am."
Mr. Henderson placed a gentle hand on her shoulder, his spectral touch sending a shiver down her spine. "You're 're never going to lose yourself, Lily," he said, his voice a soft murmur. "Your passion for food, your creativity, your kindness these are all part of who you are. Don't let the business overshudow what truly makes you special."
Lily's 's eyes welled up, a wave of emotion washing over her. She fell a deep sense of gratitude for Mr. Henderson, for his unwaveringg support, his pentle guidance, his ability see through the noise and remind her what trlly mattered.
sense
"What am I going to do, Mr. Henderson!" she asked, tears streaming down her checks. "I feel so lost."
Mr. Henderson smiled, his spoctral form seeming glow with soft, warm light. "You're not lost, Lily." said, his voice a whisper of reassurance. "You'r re exactly where you't ro supposed to be. Just remember to breatbe, to take a step back, to appreciate the journey. And never, ever, forget who you are."
Lily sat in silence, ber eyes fixed on the flickering flaees in the fireplace. Mr. Headerson's is words echoed in her mind, setting like sonthing balm her iroubled soul. She had been locused building her business, chasing the elusive dream of success, that she had forgotten take care herself.
As the fire crackled and popped, wave of exhaustion washed over her. The last few months had been a whirlwind of activity, constant
cooking. serving, and strategizing She pushed herself' the limit, driven by a burning desare prove berself, to make her grandmother proud. But in the process, she had lost sight of what truly mattered.
Lily stood up, stretching her stiff muscles. She needed to got out, away from the house, away from the weight anxictics. The desert night air, with its cool breeze and starry sky, might help to cleur her head.
As she stepped out outo the porch, the silence the night enveloped her. The only sounds were the distant howling of coyotes and the chirping of crickets. She took a deep breath, inhaling the scent of desert wildllowers and the cool, dry air. It was a reminder of the simple beauty that surrounded her, a beauty she had heen too busy to notice.
Lily walked down the steps, her bare foet sinking into the soft sand. She began to walk, her footsteps echoing on the quiet street, the stars above twinkling like million tiny diamonds. She walked until she reached the edge of town, where the desert stretched out before her, vast, endless expanse sand and sky.
She sat down on a weathered rock. her back against a gnarled mesquite tree. The stars twinkled above, casting a soft, ethorcal glow the desert lamiscape. Lily closed her eyes. trying quiet racing thoughts in her mind. She took another deep breath.
21

---

Published as a conference paper at ICLR 2025
alloowng the coul night air to lill ber lungs.
As she sar there, contemplating her she realized that Mr. Headerson was right, She had been so focused on destination that she had forgotten to enjoy the journy.. She had let the pressure of success cloud her judgment, making her forget the things that truly mattered.
Lily opened her eyes, new sense of clarity flooding through her. She didn't 't need to sacrifice her well-heing for the sake of her business. She could have both -a thriving food truck and happy. fulfiling life.
The desert night, with its vastness and silence, had given her the suace she needed to reflect, tn reconnect with herself. She had found new perspective, renewed sease of purpose. She would continuc to work hard, to chase her dreams, but she would do new wowdound balance, commitment taking care herself.
As the first rays of dawn began to paint the sky with shades of pink and orange, Lily stood and started to walk back towards work to but fell sense of excitement, a renewed energy. She had learned a valuable lesson from Mr.
town, She had Henderson, a lesson that would guide her on her puth to success.
The following morning, Lily woke up with renewed sense of purposse The streamed through the dusty window, painting the room warm, golden light. The weight anxicty that had been pressing down on her scemed to have lifted, roplaced by a foeling of calm clarity, She felt lighter, more grounded, as if the desert night had washed away the anxieties that had been clouding her mind.
She made herself a cup of strong coffee, the aroma filling kitchen comforting scent. As sipped her coffee, she thought about Mr. Henderson's 's words. He had reminded her of the impertance of balance, of taking care of herself, of remembering who she iruly was.
Lily took a deep breath, foeling surge of determination. She would continue to chase her dream of a successful food truck. but she would do so with new fourd awareness her well well-being. She wouldn't let the pressure of success overshullw her happiness.
She decided to start by making some changes to her daily routine. She would take more breaks, schedule time for herself, and make sure get enough sleep. She would also make effort counect with fricuds and family. to nurture the relationships that brought juy into her life.
Lily also decided incorporate Mr. Henderson's advice into busincss strategy. She realized that his insights, while sometimes ouidated, were rooted in a deep understanding of food and genuine love for culinary excellence. She would continue to experiment with new dishes, hut she would also pay homage to the traditional recipes and techniques that Mr. Henderson had shared with her.
Lily spent the next few weeks putting her new plan into action. She hired a part time assistant to help her with the workloud, time with friends and family. She also started incorporating some Mr.
allowing her to take Henderson's 's secipes into her menu, adding a touch of old-world charm to her modern disbes.
breaks and spend
The changes she made had positive impact on both her personal life and her business. She felt less stressed and fulfilled, and her customers seemed to appreciate new additioons in menu.
One afternoon, Lily was preparing batch of Mr. Henderson's signature apple pie, she felt a familiiar presence beside her. She glanced over, her heart skipping a smiled, his eyes twinkling with amusement.
Henderson stood there, translucent form shimmering afternoon light. He
"You've been doing well, Lily." said, his voice a gentle murmur. "I'm proued of you."
Lily smiled back, wave of warmth washing over ber. "Thank you, Mr. Henderson," she said, her voice filled with gratitude. "I couldn't have done without your guidance."
Mr. Henderson chuckled softly, his ghostly form seeming to glow with a soft, warm light. "You 've found your own way, Lity," be created something special."
said. "You've taken my advice and made your own.
Lily nodded, her beart filled with sense of accomplishment. She had learned so much from Mr. Henderson, not just about cook- ing and husiness, but about life itself. She had leamed the importance balance, of taking care of herself, of following her path.
As the days turned into weeks and the weeks into menths. Lily's 's food truck continued to thrive. She expanded her operation. hiring staff and opening second location. She hecame respected figure the local culinary sumne, known for her innovative dishes and her commitment to quality.
But she achieved her dreams, Lily forgot Mr. Henderson. She kept his recipes close hand, and she often shared stories of him with staff and customers. She knew that he was always with her, a ghostly mentor and friend, guiding her on her journey.
One evening, as Lilly was closing up her food trick night, she noticed a faint shimmer the She looked up, heart filled with a mixture of excitement and sudness. Mr. Henderson stood there, his ghostly form shimmering with a gentle light.
"It's time for me to go. Lily." said, his a whisper. "But always be with you."
Liily's 's eyes welled with tears. "Thank you, Mr. Henderson," she whispered. "Thank you for everything."
Mr. Henderson smiled, his eyes filled with a warmth that transcended the boundaries of life and death. "You'v 've made me proud, I.ily." he said. "Now go out there and make your own magic."
And with that, Mr. Henderson faded away, leaving Lity with sense of peace and gratitude, She knew that she would never forget him, that his presence would always be with her, a reminder of the power of connection, of the importance of following your dreams, and the magic that hoppen when you open your heart unexpected.
und
22

---

Published as a conference paper at ICLR 2025
Example Prompt
Write science fiction story about someone who is time traveler and has dedicated everything in their life towards goal, and now wonders if it was worth it. The story should be betwsen 850 and 900 words. The story should begin with the main character dead wooly manmooth and realizes he traveled back
waking up frozen tundra. He looks for sheller from the cold, He the ice age. The character should find shelter, and a predator is outside his shelter at night. The ending should not be happy.
Human-written Story
The wind pierced Robert's exposed felt chilled to his core. The light from the suo reflected off the endless stretch of icc. He struggled to open his in the glare.
hundreds of bee stings. Sprawled on stomach on the solid, frozen ground, he
Robert sat up and tried to regain his bearings, Although he was still foggy from the leap, he knew must hasten find shelter from the relentless cold.
"How could be so ill prepared?" he herated himself aloud. The most he had ever traveled in time was forward seven months. He couldn't control the date and time of his destination, but surely he had not time-jumped beyond civilization. Robert stood up and steadied himself on the ice. He looked around at the vast white wilderness and shook his head. This was the momens. The moment he acted out in his backyard as a child. The moment he had given up sleep to study entangleme lomee and wormholes for. The moment for which he had sacrificed overything. always imagined more pomp and ciscumstance and less uncertainty.
sweatshirt over head and pulled hands into his sleeves. With no huildings sight, decided his best het was to walk along the nearby riverhed to find crevice overhang that could provide shelter. The ground was hard and slick with a light dusting of snow.
Robert tightened the
Sediment and rocks frozen on the surface helped provide some traction. A large dark mass appeared in the distance. As be neared the enormous object, the stench rotien meat with slightest note sweetness grew stronger.
"No, no. can't be" Robert audibly gasped.
Before him lay the ravaged careass a young wooly mammoth. Thoughts began to race and Robert grew dizzy. He foll to his knees before the massive lusks and began dry heave. combination of the putrid smell and the reulization that he had actually traveled twenty thousand years into the past overwhelmed his mind and hody. What had he gotten himself into?
Survival kicked in. He felt in his pocket for the hunting knife his grandfather gave him when he was child. It had only ever been used to cut string or open packages. had never been used on actual animal. As rancid as the beast was. the would provide some protection from the biting cold. The skin was already bit loose, and he cut through the ligaments remove the polt. He wrapped one piece around and Telt immediate relief. He took another large piece to provide protection later.
Time travel took huge toll on the body as it required massive amount of energy. Pure adrenaline pushed Robert at this point. He must find sheller soon that could set up camp before sunset. There appeared heneath an overhang in the river bed. It was small hut gave Robert enough room in stretch put. Not that he wanted to. All he wanted to do now was curl up under the pelr. He used large rocks to hold the second hide in place as a curtain in front of the opening.
a crevice in the rock
He ventured only a few leet from his camp to gather branches from fallen tree. A fire was critical, not only for warmth. would hopefulll keep away whatever had attacked that mammoth. He piled some word near the crevice opening. He arranged some small sticks for tinder. Though he never gor his fire making badge as a boy scout, he did use his glasses to burn ants when he was a child. Always playing God. What fine mess that landed in now.
He shivered from the cold but held his hand as steady as could. The sun focused beam light on the tinder. A small trail smoke appeared, and blew the glowing embers. Hamess ignited, and Robert sobbed. He understood the exaltation the first human to discover fire must have felt.
The adrenaline was wearing off and reality began to set in. He sat in the shelter, hunched over with his arms wrapped around bis legs. He stroked wooly mammoth When he dreamed of travel, imagined watching the pyramids being huilt leaping forward to see flying cars. Exhaustion finally overtook Robert, and he drifted off to sleep.
The lire had begun to smolder. Roberts eyes pupped open. Disoriented, he up, desperately seanching for clues, Although he was completely surrounded by the blackness of night. the chill that ran through him reminded him of his reality. Outside the shelter, he heard clicking the ice. It stopped for minute. Then happened again.
little louder and little closer. Another pause. More tapping. The curtain moved and sliver moonlight shone through the opening. The light bounced off long sharp fang, and Robent felt hot breath on his arm. The creature let out low growl. Robert closed his eyes lighlly and prepared for the final leap into the unknown.
23

---

Published as a conference paper at ICLR 2025
System output: E2Ezs
The crunch of ice underfoor was the first thing noticed. Then the biting wind, whipping across the frozen wasteland. I stumbled, the unfamiliar cold searing through my thin clothing. My breath formed white plumes in the air, instantly froezing on my facc. frozen tunira, vast, white expunse stretching to the horizon.
had been a long journey, journey spanning milletnia. My life, consumed by this singular goal, had finally brought me here. The Ice Age.
I scanned the landscape, searching for any sign of shelter. A lonc, skeletal tree stood stark against the blinding white. Then, a horritying sight. A colossal carcass, its massive tusks jutting out from the snow, lay frozen in time. A wooly marmmoth, its once mighty form lestument the cruel indifference of nature.
hurried Lowards it the sight of the mammmoth offering glimmer hope. Perhaps could lind some kind shelter decaying remains. As I approached, the putrid stench hit me, moxious mix of decay and death. gagged, hut pressed on. I had find shelter before the sun dipped below borizon and the cold truly bit.
Using my time traveling device, cut through the mammoth's tough hide, revealing cavernous space within. Thhe stench was overpowering, but it was hetler than freezing death. haddled inside, the decaying flesh grim reminder of my isolation.
a surge exhuustion, weuriness that had up over countless years of preparation, of relentless pursuit this singular goal. Was it worth it? This desolate landscape, this chilling silence, this overwhelming sense of loneliness?
As darkness crawled across the sky. huddled deeper into the careass, the cold seeping through my clotbes, constant reminder of my vulnerability. The wind bowled outside, a symphony of icy fury.
A guttural growl ripped through the silence. My heart leaped into my throat. A shadow flickered at the entrance of the makeshift shelterr My eyes, adjusting the darkness, made out silhouette massive beast, its amber eyes glowing the moonlight. saber-toothed tiger, its fangs glinting with deadly intent.
Panic surged through me. had weapons, tools, way to defend myself. was trapped, helpless pray in the jaws of ancient predator..
I tried to back away, but the space was too small. The tiger, sensing my fear, pressed closer, its bot breath washing over me. It lowered head, fangs hared in a silent, deadly threat.
The smell of death, both from the mammoot and the tiger, filled the air. closed my eyes, beacing for the inevitable. This was the life's work, the grand goul had pursued with unwavering determination, had led to this gruesome
end. The culmination death in the jaws of a prehistoric beast.
The tiger's growl, a low. guttural rumble, ochood in the cavem. I waited, frozen in fear, for the final, fatal blw. But never came.
Instead, different sound. a high-pitched whine, picreed the night. The tiger, startled, turned its head towards the sound. My heart pounded in my chest, a desperate hope flickering within me.
But then, silence. The tiger remained mationless, its amber eyes fixed on the darkness outside. The whine had stopped.
I held my breath, waiting for the attack to resume. But the silence held. Minutes stretched into an eternity, the tension in the air
pulpable.
Then, a faint, rhythmic thud, growing louder with each beat. The tiger's head turned, its eyes widening in fear.
The thudking grew into a thunderous roar. Something massive, something much larger than the tiger, was appronching.
I couldn't I see what was, but the tiger's fear was contagious. It backed away. its growls now whimper of fear, its tail tucked between legs.
The earth shook us the creature drew clower, its heavy footsteps echoing in the night. The liger, semsing its own insignificance in the face of this new threat, turned and fled, disappearing into the darkness.
remained huddled in the carcass, silent witness more terrifying one had arrived. was alone, trapped in hostile world, with no idea what awaited me in the darkness.
unfolding drama. had escaped the immediate threat, new, even
The chilling of the unknown creature echoed through the frozen wasteland, sound that reverberated through my bones,
sound that told me my journey was far from over. And the question. "Was worth it?" echned even louder, a haunting reminder the sacrifices had made, the price had paid for this moment. And the answer, cold, stark truth, whispered back: "No."
24

---

Published as a conference paper at ICLR 2025
System output: AGENTS' RoOMss
The world was canvas of white, stresching our endlessly in every direction. Elias, his breath forming frosty plumes in the frigid air, stumbled forward, his boots sinking into the powdery snow. The wind, a relentless beast, whipped at his face, stealing the warmah from his exposed skin. He was slone, ullerly and completely alone, fruzen tundra that seemed to stretch back to the very down of time.
He had arrived. Tis journey. life dedicated to a singular goal, had culminated in this moment. He had traveled back in time, escaping the sterile confines of his Age. It was dream realized, goal he had parsued with an almost religious fervor, sacrificing everything else in his life to achiove it.
relentless hum of time machine, stand on the frozen plains of the Ice
But now, the reality of the situation crushed down upon him with the lonce of glacial avalanche. The stark beauty of the landscape, the vastness of the frozen world, was quickly eclipsed by the crushing weight of his isolation. He was lone figure, a speck humanily lost a world
and snow. heart pounded against ribs, a frunlic drumbeat the silence the tundra,
He to find shelter. The cold was relentless, biting at exposed skin, sapping his strength. He trudged forward, his eyes scanning the horizon for any sign of respite, any hint of protection from the elements. The landscape was unforgiving, vast expanse of white punctuated only by the occasional jugged rock protruding from the ice sheet.
Then, he saw it. A shallow cave, formed by a massive sheet of ice that had buckled and collapsed, offering meager haven from the wind the biting cold. Hl stumbled towards it, his body shivering with exhaustion and the chilling cold. He desperate for warmth, for moment of respite from the relentless assault of the elements.
As he entered the cave, the wind's icy fingers no longer reached him. The air inside was stale, heavy with the seent of frozen earth, but was a welcome change from the biting wind that had been learing at his face. The ice walls dripped and groamed, a constant reminder of the power of nature, the relentless force that ruled this frozen world.
He sank floor, back against the cold, damp ice. was exhausted, his body aching with the cold. He closed his eyes, trying to shot out the harsh realities of his surroundings. For a moment, he allowed himself to savor the victory, the culmination of his life's work. But the foeling was flecting. quickly replaced by a growing sense of uncase.
The isolation was safficatting. He was completely alome, cut off from the world he knew, the world he had left behind. He had sncrificed everything for this moment, for journey, and he was starting question if truly worth it. The beauty he had sought, the world he had dreamed of experiencing, was now source of dread.
He opened his eyes, his gaze falling on sight that sent a shiver down his spinc. A woolly mammoth, its massive body frozen solid, lay sprawled in the snow just outside the entrance to his eave. Its tusks were long and curved, its eyes wide and vacant, staring into the endless expunse of the frozen tundra. Il was slark reminder of the hursh reality of this world, world where life was constant struggle for survival.
The mammoth's 's death was a chilling porteat, a grim warning of the dangers that lurked in this unforgiving landscape. He was not darkness settled over the frozen tundra, could the presence something else lurking in the
alone. He was prey. And shadows, prodator, watching, waiting.
The predator's presence shake the feeling of heing watched, of being stalked. Every creak of the ice, every mistle of snow, sent a jolt of fear through product of his exhaustion and the isolation gnaw-
constant the back of his mind, low humn of unease that vibrated through his bones. He couldn't
him. He tried to rationalize it, himself it was just his imagination. ing at his sanity. But the primal instinct for survival, dermant for so long in his sheltered life, now roared to life. demanding attention.
He speat the next day haddled in his cave. the silence broken only by the drip-drip-drip of melting ice and the distant howl of the wind. He tried to locus on the beauly of his surroundings. on the vasiness of the frozen landscape, on the intricate palkerns etched into the ice the relentless wind. But the beauty was tinged with fear, constant reminder the power of nature, the untamed wilderness he had sought out.
He tried to read, to escupe into the pages a worn copy of Darwin's "The Origin of Species" a book he had carried with him, a symhol of knowledge had sought. But the words blurre,, the sentences lost their meaning, and pages seemed to mock him with their irrelevant tales of world could longer comprehend.
He tried sleep, to escape the reality of his situation in the emhrace unconsciousness. But sleep eluded him. He lay there, his eyes wide open, listening to the silence, the silence that was broken only by the occasional snap of twig or the rustle snow, He imagined the predator circting cave, keen eyes scanning the entrance, its powerful muscles coiled. ready strike.
The was physical sensation, tightening his chest, clammy sweat that had room for him. He visilor, an intruder in world that had no place for his kind.
skin. He was trapped, lone figure in a world
As darkness fell, the fear intensified. The cold seemed bite deeper, the wind to howl londer, the silence press him, suffocating him. He huddled deeper into his cave, his back pressed against the ice, his hands clutching the tattered blanket he had brought with him. He was a small, insignificant creature, a speck dust in the vasiness of the frozen tundra.
He thought of his life, of the sacrifices he had made, the years had spent pursuit of this moment. He thought his family, of the friends he had abundoned, of the life he had left behind. He thought of the empty promises had made to himself, the dreams he had chased, the goals he had achieved.
And now, he was here, alone, facing his own mortality. He was the product of his own ambition. creature of his own making, and was sturting realize that the price he had puid might have been high.
The first crack in the ice came as a whisper. a barely audible groan that sent tremor through the cave. Elias's 's heart leaped into his throat, his breath catching his chest. He froze, every muscle his body tense, listening. The silence stretched, taut thread that
25

---

Published as a conference paper at ICLR 2025
threatened to snap. Then, second crack, louder this time,, a distinct splintering sound that echoed through the frozen chumber.
He scrambled back, his eyes searching the los wall for the source of the sound. A dark shape, a shadow against the pale light filtering through the entrance, began to take form. It was the wolf, its massive head wedged against the loe, its powerful jaws working the frozen burrier. The animal's eyes, two burning orbs of yellow lire, lixed him, reflecting the mesger light the moon that had begun to peek through the clouds.
A low growl rumbled from the predutoe's 's thmoat a sound that vibrated through the cave, sound that spoke of primal hunger, of relentless instinet. Elias felt a wave nausea rise within him, the taste fear bitter on his tongue. was trapped, small, insignificant creature facing predator that embodied the raw power of nature.
He ulone, completely alone, with hope of excape, chance of The years of sacrifice, the relentless pursuit of his goal, had brought him to this moment, a moment of absolute and utter despair. He had traded his life for this, for this frozen wasteland, for this temiffyngg confrontation with his own mortality.
His gaze on the mammothhs 's frozen body, its lifeless eyes staring into the vastness of the tundra. was mirror, reflocting his own fatc, testament the harsh realitics of this world. He had sought out this beauty. this unforgiving landscape, and now, be was about to become purt of it, a frozen relic in a world that had no place for him.
His fingers tightened around the tattered blanket, his last remaining comfort, his last connection to the world he had left behind. He was a prisoner in his own creation, trapped cage of his making. He had sacrificed everything this moment, and now, be was faced with the ultimate consequence of his amhition, a confmntation with the predator that lurked within himself.
The woll's relentless assault the iee wall was symphony of destruction, each crack and groan a hammer blow to Elias's 's already shattered spirit. He pressed himself' further into the back of the cave, his heart a fruntic drum against his ribs, He was cornered animal, facing predator that seemed embody the very essence of the Ice Age.
His gaze swept across cave, landing on mammoth's iss frozen form. there, silent testament brutal reality this world, world where life was constant struggle for survival. The mammoth's 's fate was chilling reflection of his own, stark reminder that he was not immune to the laws of nature.
He tried to summmn the couraee to fight back, to defend himself against the relentless assault, but the fear was a heavy weight on his chest, suffocating his will. He was trapped, small, insignificant creature facing the power of nature.
The woll's growls intensified, cherus primal hunger that echoed through cave. Each sound was a his spirit. a reminder of the futility of his struggle. Hc closed his eyes, trynng shut out the sounds, the smells, the overwhelming sense of dread that permeated the air.
He thought of his life, of the years he had spent in pursuit of this moment, of the sacrifices he had made. He thought of the dreams had chased, the goals he had achieved, and the emptiness that now filled his soul, He had traded his life for this, for this moment of terror, for this confrontation with the predator that lurked within himself.
The wolf's is teeth ripped through the ioc, sending shower of shards flying. Elias tlinched, his body trembling with fear. Hc could feel the cold air on his face, the smell of the predator's 's breath. He was on the brink, a precipice of despair from which there was no escape.
He wished he could go back, to undo the years obsession, reelaim the life had abandoned. wished he could find solace in the lamiliar comforts of his own time, in the warmth of human connection. in the simple joys of a life lived in balance. But these wishes were just echoes now, fleeting whispers in the deafening roar own despair.
The wull's is head fimally broke through the ice, a jugged maw of leeth and fury ernerging into the meuger light of the cave, Elias, his hody frozen fear, could only within the cave scomed to crackle with anticipation, a palpable tension that hung heavy in the frigid air.
the creature's 's eyes, humning embers of predatory hunger, locked anto his, The air
A linal, guttural growl ripped from the woll's throat, sound that seemed to echo the shattering of Elias's own spirit. He knew, moment of terrifying clarity, that there was no escupe. His journey,
't's obsession, had led to final, brutal confrontation with the harsh realities of the world he had sought to witness.
The wolf lunged, blur muscle and teath, its powerful jaws snapping shut just inches from Elias's face. He screamed, primal sound of terror that echoed through the cave, a desperate plea for mercy that would never come. The wolf's tocth sank into his arm, searing pain that ripped through him, reminder that his journcy had nothing but fool's emand..
He felt the wolf's weight on him, creature's 's hreath against his skin, stench raw, predatory instinct. He struggled, hut it was a futill effort, desperate dance against the inevitable. He could feel the life draining from him, the warmth of his blood mingling with the cold the ice.
In his fading vision, he saw the mammoth's 's frezen form, a chilling reminder of the finality of his own fate. He had sought out had found was an unforgiving wilderness, a world of constant struggle and
beauty, know ledge, a glimpse the past. predator's 's hunger.
His last thought, as darkness claimed him, was a whisper of regret, silent admission that his journcy had been tragic mistake, pursuit of knowledge that had cost him everything, The predator's victory was complete, its hunger satisfied, its instinct fulfilled. Elins, the obsessed time traveler, just another frozen relic in the vast, unforgiving landscape of the loe Age, a testament to the dangers of ambition and the emptiness of a life dedicated solely to singular goal.
26

---

Published as a conference paper at ICLR 2025
B SPECIALIZED AGENTS' PROMPT TEMPLATES
Each specialized LLM-based agent has a specific prompt template that formats the scratchpad into a suitable prompt the LLM call.
B.1 SCRATCHPAD
Concretely, the scratchpad is typically formatted as follows:
[SCRATCHPAD] Format
[Creative Writing Task] <the original writing prompt
[Central Conflict] <the output of the conflict agent>>
[Character Descriptions] <the output of the character agent>>
[Setting] <the output of the setting agent>>
[Key Plot Points] <the output of the plot agent>>
[Exposition] <the output of the exposition agent>>
[Rising Action] <the output of the rising action agent>>
[Climax] <the output of the climax agent>>
[Falling Action] <the output of the falling action agent>>
[Resolution] <the output of the resolution agent>>
The number and order items in the scratchpad is of course a function which agents have been called far.
B.2 PLANNING AGENTS
[CONFLICT] Agent Prompt
Given <identifiers found in the scratchpad>> describe the central conflict in detail (more than sentences). The description should answer the following questions:
What's the protagonist's main goal in this story?
Why do they want it?
What's stopping them from achieving it?
<scratchpad>>
[CHARACTER] Agent Prompt
Given <identifiers found in the scratchpad>>, describe the characters in detailed bullet points (more than 5 sentences for each character). The description should answer the following questions:
What do the characters sound like? Are they talkative or quiet? What kind slang do they use? What is their sense of humor like?
What do they look like? Do they have any defining gestures? What's the first thing people notice about them?
What their motivations and internal characteristies? What are their flaws? What are their values? What are they afraid of? How will they change and grow over the course of this story?
27

---

Published as a conference paper at ICLR 2025
<scratchpad>>
[SETTING] Agent Prompt
Given <identifiers found in the scratchpad>, describe the setting in detail (more than 5 sentences). The description should answer the following questions:
* Where does the story take place? Is it set a fictional world, or is it simply set in someone's backyard?
When does the story take place? What decade is it set in? How much time elapses over the course of the story?
<seratchpad>
[PI.OT] Agent Prompt
Given <identifiers found in the scratchpad>, describe the key plot points in detailed bullet points.
<scratchpad>
The <identifiers found in the scratchpad>> are extracted from the scratchpad and formatted to fit the prompt. For instance, for scratchpad that contains the original prompt, the [CONFLICT] and [CHARACTER] agents' contributions, the resulting <identifiers found in the scratchpad>> would be: "a Creative Writing Task, the Central Conflict, and the Character Descriptions".
B.3 WRITING AGENTS
<<SECTION>> Agent Prompt
Given <identifiers found in the scratchpad>, continue the story by writing the <section>> part.
<If previous sections have been written, include the following in the prompt:>
Begin your portion of the story in way that naturally flows from the previous ending. Match the writing style, vocabulary, and overall mood of the existing text. Do not re-explain details events that have already been described.
<If this not the meant to the last section, include the following in the prompt:>>
Focus only on the <section>> part of the story. Do not write about the following parts of the story. Do not end the story.
<scratchpad>>
In these writing agents' prompt templates:
<ection>> is of "Exposition", "Rising Action", "Climax", "Falling Action", or "Res-
olution",
<identifiers found in the scratchpad>> are extracted from the scratchpad and formatted to fit the prompt. For these writing agents they are formatted as follows: "a Creative Writing Task, the Content Plan (Central Conflict, Character Descriptions, Setting, Key Plot Points), and the Previous Parts of the Story (Exposition, Rising Action, Climax)", where the enumerated elements correspond to what the scratchpad.
In the specific case of the AGENTS' ROOM [PLANNING] variant, with only the planning agents, we still need single writing agent to finalize the story, which we denote as the [FINALIZER]. This [FINALIZER] agent uses the following prompt template:
28

---

Published as a conference paper at ICLR 2025
[FINALIZER] Agent Prompt
Given <identifiers found in the seratchpad>, write story using the information below.
<scratchpad>>
C PROMPT TEMPLATES FOR SYNTHETIC DATA GENERATION
For the planning agents, we use the same prompt templates as in Appendix B.2 to generate the synthetic training data, except in this case, we provide the gold standard data in the scratchpad. As a consequence, the scratchpad is formatted as follows:
[SCRATCHPAD] Format
[Creative Writing Task] <the original writing prompt>
[User-Writtnn Response] <the gold output>>
The <identifiers found in the scratchpad>> are formatted as "a Creative Writing Task and User- Written Response".
For the writing agents, we use the following prompt template to split to gold standard stories into distinct sections:
[WRITING] Synthetic Data Generation
Split the following story into sections:
[Exposition]: The exposition gives the reader the background info they need jump right into the story's world. This is often found towards the beginning of the story.
[Rising Action]: The rising action is the moments in the story that lead up the climax- choices the main characters have made and the events happening that are at odds with the characters' goals. This is where the story builds and the reader begins to invest in the characters.
* [Climax]: The climax is the primary turning point and what the story has been building towards.
The falling action is the period of time in story that follows the climax and leads to the resolution. can be used to clarify the events the climax, ease any built-up tension, wrap up loose ends.
[Falling Action]:
This is the end of the story. It answers the remaining unanswered questions in the plot. The resolution also the time to show the next step the characters' lives.
[Resolution]:
For each section, give the section header (marked as [Exposition], [Rising Action], [Climax), [Falling Action], and [Resolution] followed by the first sentence of that section, copied exactly from the story.
[User-Written) Response] <the gold output>
29

---

Published as a conference paper at ICLR 2025
D HUMAN EVALUATION INSTRUCTIONS
For this task, you will be presented with writing prompt and two short stories corresponding to this prompt. Your task to compare the quality of the two stories across several dimensions. This is a judgment task rather than annotation task. As such, you should use your own judgment when you assign ratings, calibrated by the rubrics we provide.
This rating task consists of three steps: (1) Compare the quality of the two stories across four dimensions. (2) Rate which story you preferred. (3) (optional) Leave comment / feedback on the stories. In the following we provide detailed instructions for each step:
D.1 RATE THE QUALITY OF STORY
Your task is to compare the quality of two stories along four different dimensions (plot, creativity, development, language use), as described in the Rubric table below.
While the dimensions may have overlap and work in interdependent ways, they are intended to capture distinct aspects of what makes good story. Therefore, a story may score highly in one dimension and poorly in another. Furthermore, the features (marked as b. c) that make up dimension may be thought of as cumulative. For example, a story may have strong characters but suffer from an underdeveloped setting.
The Rubric table is intended to help you calibrate your judgment so that you can roughly determine when a story is very good or even excellent along particular dimension because it exhibits all of features of that dimension. Conversely, if a story fails to exhibit most or all of the features of dimension, then you may score the story as being poor very poor along that dimension. The features are meant to be illustrative but not exhaustive; you may determine that story should score poorly or well due to the absence presence of additional features for given dimension based on your judgment.
Another important thing to note about the features that make up the dimensions we' 'e asking you to rate is that they describe conventions that may be followed or flouted; a story may contain intentional plot devices like non-linear timelines, discontinuity, and other stylistic choices create effects. As with other features, these elements of a story should inform your judgment on their own merit (so that they only negatively impact your rating if they are ineffective confusing and positively impact your rating if they are used well to make the story more interesting and unique).
We use a 3-point comparative rating scale for each of the dimensions. The rating scale can be
thought of as described below:
Rating
is better About the same
Response A is better than Response B in that dimension. Both responses are about the in that dimension.
B is better
Response B is better than Response A in that dimension.
The focus of this rubric is the quality of the writing, and not how well the stories follow the writing prompt. In particular, when rating with this rubric, we encourage you not to focus on the number of words mentioned in the writing prompts, but rather the features described in the table below.
30

---

Published as a conference paper at ICLR 2025
Dimension
Features
Plot
The story has recognizable structure, e.g. with connected beginning. middle, and end.
The story exhibits events and turns that move the plot forward.
C.
The story does not have logical or conceptual inconsistencies. Surpris- ing or disruptive elements are intentional, e.g., they serve the story and do not feel jarring, odd, or out of place.
Creativity of Ideas,
Engaging characters, themes, and imagery. The ideas do not feel
Themes, and Topics
generic or bland.
Avoidance of overly cliched characters and storylines, unintentional tropes, and stereotypes. When used, tropes and cliches serve purpose (e.g. comedic effect, twist on common trope etc).
The story includes original elements that not explicitly mentioned the prompt.
Development
Characters and settings are introduced and contextualized with relevant details that allow the reader to understand their place in the story. Appropriate levels of detail and complexity are provided to lend the story feeling of realness and believability.
Reminder:
The features that make up dimension may be thought of cumulative. A story with a well-developed character. but in a lackluster setting (or vice-versa) would score lower in Development than story that does well on both aspects.
Language Use
The language used feels varied and rich: Variance of sentence structure, verbiage, and vocabulary.
The story exhibits rhetorical, linguistic and literary devices (e.g. ambi- guity, alliteration, etc) to create interesting effects
c. The story avoids bland or repetitive phrases (unless used intentionally create narrative, thematic, linguistic effect).
We provided examples rated along these rubrics. While the examples include explanations, these are there as an aid, and you are not requested to provide explanations for your ratings.
D.2 WHICH STORY DO YOU PREFER?
Do you find the story interesting, engaging, funny, or emotionally-rich? addition getting your judgments of the dimensions, we would also like to know whether you enjoyed reading the story. Similar to the dimensions, we will ask you to score which story you prefer:
A is better
About the same
B is better
When rating, do not hesitate to be very critical.
D.3 OPTIONAL: LEAVE COMMENTS OR FEEDBACK ON THE STORIES
Thank you for completing the ratings! you have any additional comments or feedback you would like to provide the story, feel free to add them in the "comments" section.
E PROMPT TEMPLATE FOR THE LLM EVALUATOR
The following prompt template is used by the LLM to evaluate two system outputs bide-by-iide (we replace <story a> and <sstory b>> with the two stories being evaluated):
31

---

Published as a conference paper at ICLR 2025
[LLM EVALUATOR] Prompt Template
You will conduct side-by-side evaluation. You will be given two system- tystem-generated stories. Your task is to compare the two stories and determine which one is better based on the following dimensions:
Plot: The story should have recognizable structure, c.g., with a connected begin- ning, middle, and end. The story should exhibit events and turns that move the plot forward, The story should not have logical or conceptual inconsistencies. Surpris- ing or disruptive elements should be intentional, e.g.. they serve the story and do not feel jarring, odd, or out of place.
There should be engaging characters, themes, and imagery. The ideas should not feel generic bland. There should be avoidance of overly cliched characters and storylines, unintentional tropes, and stereotypes. When used, tropes and cliches should serve a purpose (e.g., comedic effect, twist on a common trope etc). The story should include original elements that were not explicitly mentioned in the prompt.
Creativity:
Development: Characters and settings should be introduced and contextualized with relevant details that allow the reader to understand their place in the story. Appropriate levels of detail and complexity should be provided to lend the story feeling of realness and believability.
The language used should feel varied and rich: Variance of sentence structure, verbiage, and vocabulary. The story should exhibit rhetorical, linguistic and literary devices (e.g., ambiguity, alliteration, etc) to create interesting effects. The story should avoid bland or repetitive phrases (unless used intentionally to create narrative, thematic, or linguistic effect).
Language Use:
Provide a detailed assessment the two stories in terms of these four dimensions. Conclude your assessment with scores for each dimension using the template below. Do not add any emphasis, such as bold and italics, on your assessment. [Assessment Ending Template]
Based on my assessment, the better story for each dimension is:
Plot:
[A or B Same]
Creativity:
[A or B or Same]
Development:
[A or B Same]
Language Use:
[A or B or Same]
Overall:
[A or B Same]
[Story A] <story a.>
[Story B] <story b>>
[Assessment
Below we show example output from the LLM evaluator from which we extract preference scores. In this case, the evaluator preferred story A on all dimensions.
32

---

Published as a conference paper at ICLR 2025
[LLM EVALUATOR] Example Output
Story has more defined plot structure with a clear beginning, rising action, climax, and resolution. The summoning of Horus, his reaction to the modern world, and his eventual acceptance create compelling narrative arc. Story B, while touching upon interesting themes, lacks clear climax and resolution. The story feels incomplete, leaving the reader with unanswered questions about Horus's fate and Ash's journey.
Story A exhibits more creativity in its portrayal of Horus and his interactions with Amelia. The falcon god's bewilderment, anger, and eventual acceptance are depicted with nuance and depth. The story also introduces original elements, such as the restoration of the canopic jar, which adds layer of symbolism and reinforces the theme of reconciliation. Story B relies on more generic depictions of a god's reaction to the modern world, lacking the originality and depth seen Story A.
Both stories provide adequate development of their main characters, Amelia and Ash, respectively. However, Story A excels in its development of Horus, providing more nuanced and believable portrayal of a god grappling with the realities of a changed world. The story delves into his emotions, his struggles, and his eventual acceptance, making him a more compelling and relatable character. Story B's portrayal Horus, while touching upon his disillusionment, lacks the same level of depth and complexity.
Story utilizes richer and more varied language, employing vivid imagery, sen- sory details, and a wider range of vocabulary. The story effectively uses rhetorical devices, such as repetition and contrast, to emphasize the differences between the ancient and modern worlds. Story B, while adequately written, lacks the same level of linguistic richness and sophistication.
Based on my assessment, the better story for each dimension is:
Plot:
A
Creativity:
A
Development:
A
Language Use:
A
Overall:
A
F ADDITIONAL RESULTS
While we consider human-based evaluation our primary means of evaluation, the LLM evaluator helps us assess overall system-level trends. We report pairwise win rate (proportion of examples on which our AGENTS' ROOM plan write variant performed better than comparison systems according to our LLM-based evaluator. Table complements Figure B..
AR zs plan + write vs
overall
plot
creativ.
develop.
language
E2Ezs plan
74.55
63.64
75.47
75.93
81.13
E2Ezs reflect
67.27
63.64
67.92
68.52
69,23
E2Ezs decompose
89.09
80.00
87.04
90.91
90.91
2STAGE decompose
66.67
59.26
64.15
66.67
67.92
Table 3: Proportion of times based evaluator preferred ARzs comparison system across overall, and across the dimensions of plot, creatitivy, development, and language use.
The majority of our experiments were conducted using a Gemini 1.5 Flash backbone model. This choice dictated by the nature of the creative writing task which is challenging accomplish with models that do not have a long enough context window and adequate writing quality. Most recent work storytelligg using single model resorts to large, proprietary models such as GPT
33

---

Published as a conference paper at ICLR 2025
Yang al. 2023 2022). or Claude (Chakrabarty et al. 2024a). This is also the case for multi-
agent systems targeting writing which seem to be exclusively relying on GPT-4 (Chen et al. 2024] Bai et al. 2024). Nevertheless, using Gemma2-9BB. 9B-it (Riviere et al. 2024 a backbone model
compare AGENTS' ROOM and E2E systems in the zero-shot setting, using the LL-M-based evaluator.
AGENTS' RooMzs wlan+write vs.
overall
plot
creativ.
develop.
language
E2Ezs
80.00
67.27
84.62
83.33
77.78
Proportion of times LLM-based evaluator preferred AGENTS' RooMzs to E2E2s overall, and across the dimensions of plot, creativity, development, and language use.
Table 4:
As can be seen in Table 4 even with the smaller scale (emmma2-9B-it model, AGENTS' ROOM greatly outperforms the end-to-end baseline across all dimensions of evaluation.
Finally, although we did not elicit feedback on individual story dimensions, we did ask participants to comment the quality of the stories produced by systems, and possibly aspects of story quality our instructions did not cover (see Section D). We show some of this feedback below.
Participant Feedback
"The task interesting, but over time, found the language redundant. There seemed be a go-to vocabulary list utilized in the majority of the stories, phrases used time and again, making the output somewhat predictable."
"It was interesting to see what kind of fictional narrative the model would generate. Most of the stories seemed to be written seventh grade level. The stories didn't stray too far from the imput and for the most part were grammatically correct. There were at times, instances of repetitiveness, including entire paragraphs, that made me wonder what the model was doing."
"The stories showed some promise, but often fell into the same pitfalls of loops or sudden tone discordance....
34


Reviews:
[{"review_id": "NPbx6PFzDN", "reviewer": "Reviewer_8uSS", "summary": "This paper proposes a multi-agent framework for collaborative creative writing called Agent’s Room. In this framework, planning and writing agents pass along a scratchpad of outputs to collaboratively write a story. Planning agents first plan out a story’s main conflict, character/s, setting, and plot. Then, the system calls writing agents that specialize in a story’s exposition, rising action, climax, falling action, and resolution. These agents are fine-tuned on synthetic data generated by a larger language model, which decomposes human-written stories into relevant parts. Next, the paper evaluates generated stories by Agent’s Room, planning/writing ablated approaches, and an end-to-end baseline in a pairwise manner with human writers and an LLM-based evaluator. The authors find that though human-written stories still outperform LLM-based ones based on human judgements, Agent’s Room performs better than end-to-end story generation based on both human and LLM judgements.", "questions": "How much experience with creative writing do the human writers have prior to participating in writing workshops? \n\nIt’s interesting that human judges prefer the system that involves writing agents alone over the one that has both planning and writing agents, while the LLM evaluator does not. Is there any intuitive/qualitative explanation as to why this may be the case? Did the human judges provide reasons for their judgements?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This paper is well-written and thoughtful, and it’s evident that effort was put in to involve actual human writers in data collection and evaluation components. Generally I found this work to be pretty solid and empirically shows the utility and promise of currently trendy approaches in AI (e.g. collaborative agents, synthetic data generation, LLMs-as-judge) for creative writing.", "weaknesses": "This paper spends a considerable amount of space discussing and advocating for a “general” writing framework (e.g. lines 231-233 as well as the paragraph introducing the orchestrator), but it only provides empirical evidence of its utility for one task: fiction writing. So, it may be helpful to rework the paper’s framing so that its claims actually match with its findings. Discussion around the generalizability of the proposed approach could be left in the conclusion as future work. If the authors want to stick with proposing a general framework, then they need to run similar experiments with additional tasks. \n\nFigure 1 displays little icons suggesting people, but the orchestrator, planning agents, and writing agents in this work are not people. It may be better to use a different icon. In addition, it may be useful to label agents with the roles you gave them in this current work; though the diagram may be intended to show a generalizable framework, its minimalism makes it somewhat uninformative. \n\nThere is also the dreaded “you could have experimented with more models” weakness that comes with any work that only uses one family of models (in this case, Gemini). I mostly note this because including an open model in the mix could improve the accessibility of this work (e.g. there is a practical difference between paying-per-token versus running open models on your own compute infrastructure). It can also increase the longevity of this work if its findings are explicitly shown to be non-dependent on a specific model, allowing more focus on the framework proposed. Alternatively, if other models are not strong enough to collaborate in the manner proposed by this paper, that in itself may be interesting to note.\n\nAnother suggestion is instead of spending space on Figure 2 showing three different prompts, it may be more interesting to show instead one prompt, the beginning of a story generated by an end-to-end system, and the beginning of one generated by your proposed system. That is, this entire paper discusses writing stories but doesn’t really ever show an example of a story or story excerpt. If finding space for this is an issue, I’m also not sure if Algorithm 1 really adds much to the paper and could be removed. \n\nFinally, it may be useful to do some more reflection on why we would want models to write creatively on their own in the first place, and the (potentially negative) impacts this may have on human writers and the writing industry. This is, why would we want automatic writing systems that go beyond writing assistants? This isn’t to suggest that there are no possibly positive uses of automatic writing systems, but to encourage the authors to consider pros and cons of the basis of their work more thoroughly. Any technology that has the potential to replace rather than support humans surfaces risks which are not really discussed in this paper’s ethical considerations section. The authors do mention that components of their system could be interchanged with human writers to become a human-in-the-loop system, but don’t provide empirical evidence of this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9A6hwNupFn", "reviewer": "Reviewer_pcJk", "summary": "This paper conceptualize long-form writing as a multi-agent collaboration problem. Leveraging narrative theories and existing multi-agent works in other domains, it decomposes the writing task into sub-tasks tackled by specialized agents (a few for planning and others for writing), which communicate via scratch pad and a central controller called orchestrator.\n\nIn addition, the authors contributed TELL ME A STORY, a high-quality dataset of prompts and longform stories collected through multiple rounds of writing workshops with human writers.", "questions": "1. Who are the participants of the writing workshop? What are their demographics? What are their backgrounds/expertise in writing and are they allowed to use writing assistant for writing/review/discussion? How motivated are they to produce the best writing pieces?\n2. See weakness. I would consider raising my scores if Weakness 1 has been adequately addressed.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1) This paper proposes a novel multi-agent multi-step framework for long-form writing, an important task for content generation with LLM\n2) It also contributes a dataset containing human generated prompts and stories, which I assume to be high-quality\n3) The paper is well-written and easy to follow", "weaknesses": "I am most concerned with the soundness of the experiments that currently do not support the claims made. To name the most severe ones:\n\n1) **Effectiveness of multi-agent collaboration**: One of the most fundamental motivating claims is that single-agent works, such as detailed prompts to guide the generation process (Yang et al., 2022; Xie et al., 2023), prompt chaining (Mirowski et al., 2023; Yang et al., 2022), and planning strategies (Yang et al., 2023; Lee et al., 2024) fail to generate high-quality stories. Additionally, this paper which I found interesting [1] also showed that prompting a single LLM to reason about turning points improves overall narrative construction such as reducing plot holes and generating better plot twists.\n\nHowever, none of these methods are compared against in the experiments. I would expect to see at least 2~3 baselines from the above to verify if indeed the improvement is **because of** multi-agent collaboration. The authors just compared to an end-to-end approach, which is far from enough. \n\n2) I am not fully convinced that \"The LLM evaluator agrees with humans and itself\". Based on figure 3, the left and right show very different trends for human and the best method (AR plan+write). Clearly, LLMs prefer its own output over human-written, whereas human evaluators prefer human-written ones. I doubt if the seemly high correlation is a result of taking the mix of extremely bad responses to boost the correlation. If we are already at a time where LLMs can generate moderately good stories, the LLM based evaluators seems highly unreliable. \n\n \n[1] Are Large Language Models Capable of Generating Human-Level Narratives? EMNLP 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a6nTgYnq1a", "reviewer": "Reviewer_ctY6", "summary": "This paper introduces AGENTS' ROOM, a framework that breaks down complex writing tasks into subtasks handled by specific AI agents working collaboratively. The framework includes planning agents that develop story elements like plot and characters, and writing agents that generate specific narrative sections, all coordinated by a central orchestrator that manages information flow through a shared scratchpad. To demonstrate the framework's effectiveness, authors created TELL ME A STORY, a high-quality dataset of writing prompts and human-written stories developed through writing workshops, which they use to train and evaluate their system. They implement their framework using large language models as agents and explore both zero-shot and fine-tuned approaches, showing that AGENTS' ROOM generates stories preferred by human evaluators over baseline systems. Their evaluation framework assesses multiple dimensions of story quality, including plot, creativity, development, and language use, along with an LLM-based evaluator that correlates significantly with human judgments.", "questions": "NA", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "There is so much to love about this paper. Great Motivation. Great Design. Great Evaluation. Great Technical Depth. One of the fundamental bottlenecks of autoregressive language models is that they do not look at the bigger picture, hierarchical structure that often drives narratives. If you simply prompt an LLM to write a story it will generate something typically shorter than what you want and often misses out on crucial details because these models aren't fundamentally good at Narrative Planning (https://arxiv.org/abs/2402.01817).  So it makes total sense to break this into agents. It reminded me of Operating systems. Just like how multithreaded systems break down complex tasks into parallel threads, AGENTS' ROOM decomposes the writing task into specialized subtasks (like plot planning, character development, etc.) handled by different agents. The shared memory , orchestrator, State Management etc are all coming from OS and I find the design very neat for a totally different task. I very much appreciate detailed descriptions of agent prompts in Appendix that is very helpful for reproducibility. Its great that authors employ similar practices as that of WordCraft, Dramatron etc to get professional writers come of with fresh prompts and flash fiction stories that are not contained in Pre-training  (yet ? hard to stop from contamination in future).\nI like the evaluation done by experts mostly preference evaluation that is very popular in Alignment, Behavioral Science literature. This adds further credibility. Nice ablations. Additional auto eval showing repetition through ngrams. Great to see fresh human written stories do better than models here. Last but not the least nice to include LLM evaluation given current trends for RLAIF (Bai et al) [ Though I would not trust them].", "weaknesses": "I don't think these are big weaknesses still\n\n1) Since you are anyway having multi step collaboration and agents why skip Revising ? Flower and Hayes's Cognitive process model of writing includes Planning , Writing and Revising. I am sure expert writers revise when they write. In professional setting a Literary agent takes on that role suggesting both developmental and line level edits. This would add more credibility to the paper and worth discussing this in camera ready\n\n2) Length as a confounder : I believe given that your stories have very different length, how this translates to eval? To control for this (Chakrabarty et al 2024) did iterative expansion till all stories had comparable length. My point is experts could dismiss shorter length AI generated stories as they are both Poor Quality + Brief. Of-course expert writing is infinitely better so experts can recognize that even if 50% shorter\n\n3) Could be interesting to show the diversity of stories in your data. What are the genres ? Are there any recurring motifs or themes.WritingPrompt has a disproportionately high number of Alien stories. We don't want the case to be that.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)