Coherent Story Generation with Structured Knowledge (2023)

Abstract: The emergence of pre-trained language models has taken story generation, which is the task of automatically generating a comprehensible story from limited information, to a new stage. Although generated stories from the language models are fluent and grammatically correct, the lack of coherence affects their quality. We propose a knowledge-based multi-stage model that incorporates the schema, a kind of structured knowledge, to guide coherent story generation. Our framework includes a schema acquisition module, a plot generation module, and a surface realization module. In the schema acquisition module, high-relevant structured knowledge pieces are selected as a schema. In the plot generation module, a coherent plot plan is navigated by the schema. In the surface realization module, conditioned by the generated plot, a story is generated. Evaluations show that our methods can generate more comprehensible stories than strong baselines, especially with higher global coherence and less repetition.

## introduction story generation is a challenging text generation task that uses a limited text, such as a first sentence, as a clue to form a story. the story must not only be grammatically correct but also have logic and consistency, which is known as coherence. illogical sentences, unrelated entities, and generic phrases are problems that affect the coherence of generated stories @xcite . incorporating knowledge into models has shown promising results in the story generation task; external information is provided to the models to capture more features not given in the context. for example, @xcite applied commonsense knowledge to their language model by using pre-training. as shown in ## experiments in this section, the details of the dataset, the experimental settings, and the baselines in our experiments are introduced. ## 4.2 experimental settings in our experiments, we use the first sentence as input, and the number of generated sentences was limited to four, following the dataset (@xmath0. we applied the spacy library for dependency parsing. we used the parameters of the large version of bart and the small version of gpt-2. 3 the number of knowledge pieces in a schema was tuned to 60 on the validation dataset. ## conclusion we presented a knowledge-based multi-stage model for coherent story generation. a structured knowledge, schema, was applied to navigate the story generation process, which makes the model able to readily absorb and integrate the knowledge not contained in the context to generate coherent content. the results of objective and subjective evaluations of the datasets showed that the proposed method outperforms strong baselines and often produces stories with more coherence and less repetition without harming grammatical correctness. furthermore, by exploring our different models, we found some limitations in the usage of knowledge in the multi-stage models. we hope our work can give good guidance to future work.