Returning to the Start: Generating Narratives with Related Endpoints (2024)

Abstract: Human writers often *bookend* their writing with ending sentences that relate back to the beginning sentences in order to compose a satisfying narrative that “closes the loop.” Motivated by this observation, we propose RENarGen, a controllable story-generation paradigm that generates narratives by ensuring the first and last sentences are related and then infilling the middle sentences. Our contributions include an initial exploration of how various methods of bookending from Narratology affect language modeling for stories. Automatic and human evaluations indicate RENarGen produces better stories with more narrative closure than current autoregressive models.

## introduction narrative closure is an important feature of satisfying narratives. @xcite defines narrative closure as "the phenomenological feeling of finality that is generated when all the questions saliently posed by the narrative are answered." human writers often achieve closure through bookending @xcite (a.k.a circular construction or ring composition) whose minimum criteria is for the ending to relate back to the beginning @xcite . automatic story generation has advanced significantly recently @xcite @xcite @xcite @xcite @xcite @xcite @xcite . however, these approaches still struggle to generate satisfying and coherent stories with closure @xcite . to address this challenge, we propose related endpoint narrative generator (renargen) 1 to 1 code/resources: https://github.com/adbrei/renargen generate closed narratives via bookending with related first and last sentences. we refer to the first sentence as the start, the last sentence as the stop, and the start/stop sentence pair as endpoints. narrative closure can be achieved via related endpoints, which may be operationalized with various methods, the most common of which is semantic relatedness. endpoints are semantically related @xcite if they resemble each other w.r.t. elements like theme, character, action, place. figure 1 illustrates this idea with two stories: story 1 has related endpoints sharing semantic commonalities that complete themes introduced in the start (e.g., @xmath0, @xmath1, and @xmath2; story 2 has unrelated endpoints with fewer semantic similarities; the stop introduces new themes without satisfactorily fulfilling the initial narrative thought. to a reader, stories like story 1 are more "closed" than stories like story 2. renargen (figure 2 ) is a scheme that produces stories with closure using neural language models (lms) and large language models (llms) by (1) generating related endpoints given the start and (2) infilling middle sentences given left and right contexts. we approach these two challenges differently for lms versus for llms. for the first challenge for lms, we use semantic relatedness to encourage narrative closure. we generate a phrase list (salient words/phrases from the start) to emphasize narrative aspects that should be addressed in the stop. for llms, we experiment with different single/multi-prompting methods that address bookending with more sophisticated definitions of relatedness for narrative closure. for the second challenge for lms, we propose an interative infilling method, inspired by story-completing techniques described in narratology @xcite , that considers both left and right contexts and generates any number of sentences in a reasonable order. while adding sentences left-to-right is a common method of expanding a story, infilling is also a bonafide method: the basic intuition is to find two consecutive sentences between which additional story material is needed. infilling imitates human writers who add sentences to earlier locations where they determine additional information is necessary @xcite @xcite @xcite . our method is different from previous works using an automatic bidirectional attention strategy @xcite @xcite @xcite @xcite ) that require the infill to have fixed length, require knowledge of the infill location at the beginning of inference, and/or are not easily iterable. for llms, the story infiller generates all infills at one time. through piece-wise narrative generation, re-nargen offers user interactivity. for example, for lms the user can control the generated stop sentence by editing the phrase list. renargen uses both lms and llms because both have their strengths. lms are more accessible with predictable output format but are less coherent. llms produce higher quality generation but are less accessible and require more computing power. see appendix b.1 for further discussion. automatic and human evaluations indicate re-nargen outperforms baselines with stories that feel more complete. our contributions are: ## renargen given start sentence, s 1 , renargen generates a story @xmath3, s 2 , . . . , s n } where n is the total number of sentences and the endpoints s 1 and s n are related. renargen has two main components: the endpoint generator and story infiller. foot_0 ## conclusion we present renargen to automatically generate stories with related endpoints via various methods of bookending. renargen for lms uses semantic relatedness and renargen for llms uses several forms of semantic relatedness, erotetic closure, "matching ending," or entailment to guide the generation of related endpoints. we empirically demonstrate renargen produces more closed, satisfying, and coherent narratives than corresponding baselines. we also show that users find re-nargen-lm's element of controllability useful. through our experiments and corresponding human evaluations for pair-wise preference, we further demonstrate the applicability of narratology theory for improved automatic generations and the importance of narrative closure for satisfying narratives.