SWAG : Storytelling With Action Guidance (2024)

Abstract: Automated long-form story generation typically employs long-context large language models (LLMs) for one-shot creation, which can produce cohesive but not necessarily engaging content. We introduce Storytelling With Action Guidance (SWAG), a novel approach to storytelling with LLMs. Our approach reduces story writing to a search problem through a two-model feedback loop: one LLM generates story content, and another auxiliary LLM is used to choose the next best “action” to steer the story’s future direction. Our results show that SWAG can substantially outperform previous end-to-end story generation techniques when evaluated by GPT-4 and through human evaluation. Our SWAG pipeline using only small open-source models surpasses GPT-3.5-Turbo.

## introduction large language models (llms) have recently changed the landscape of content generation. a number of works have proposed techniques for short story generation @xcite @xcite . however, it has been a major challenge for ai to generate long-form stories that are both coherent and interesting @xcite @xcite . this remains a challenge with sota llms such as gpt-4 (ope-nai, 2023), llama-2 @xcite , and mistral @xcite . we propose swag, an algorithm for iteratively generating engaging and captivating stories using llms. in our work, we structure storytelling as a search problem. this paradigm allows us to formulate the problem as finding the "optimal path" in a search space of possible stories given a story idea. by having another model guide the llm during the story writing process, we can improve control * equal contribution over the story direction and create more engaging content. at a high level, we train an action discriminator llm (ad llm) to determine the next best action to take given the current state of a story. we then prompt another llm to write the next part of the story based on the chosen action. this feedback loop generates long-form stories that are fascinating and amusing to read. the main component of our system is the ad llm, which helps pave the path for the story by selecting the next best "action" to continue the story. the ad llm can be paired with any open-source model (e.g. llama-2-7b) or closed models (e.g. openai's gpt-4) for generating the story. our algorithm allows fine-grain control over story content progression while providing the flexibility to integrate custom models for writing the story or using llm services offered by other companies through apis. all code is available at https://github.com/ jonnypei/swag-storytelling . ## related work prior works have attempted to improve the quality and/or diversity of story generations in a variety of ways. ## methods our creative storytelling method consists of two primary components: the story generation model and the action discriminator model (ad llm). swag enables the use of any open-source llm or llm service for story generation. we create an ad llm by collecting preference data for story actions, and aligning a pretrained llm on our preference dataset. we visualize our training pipeline in figure 1 . ## conclusion this paper proposes swag, a simple feedbackbased framework for creative story generation. the fine-tuned action discriminator llm enables more interesting and exciting plot development with little to no sacrifice in coherence or consistency. both machine and human evaluation exemplify our method's effectiveness compared to sota end-toend generation methods, even with the strongest closed-source models. we anticipate that our contribution will further advancements in content gen-eration, particularly through the lens of iterative feedback mechanisms.