E tri CA : Event-Triggered Context-Aware Story Generation Augmented by Cross Attention (2022)

Abstract: One of the key challenges of automatic story generation is how to generate a long narrative that can maintain fluency, relevance, and coherence. Despite recent progress, current story generation systems still face the challenge of how to effectively capture contextual and event features, which has a profound impact on a model’s generation performance. To address these challenges, we present EtriCA, a novel neural generation model, which improves the relevance and coherence of the generated stories through residually mapping context features to event sequences with a cross-attention mechanism. Such a feature capturing mechanism allows our model to better exploit the logical relatedness between events when generating stories. Extensive experiments based on both automatic and human evaluations show that our model significantly outperforms state-of-the-art baselines, demonstrating the effectiveness of our model in leveraging context and event features.

## introduction story generation aims to generate fluent, relevant and coherent narratives conditioned on a given context. as the task is notoriously difficult, a common strategy is to employ storylines composed of events to support the generation process @xcite @xcite . this process imitates the behavior of human writers. firstly, a story will start from a sketch of key words containing events, and then human writers will unfold the story following the track of planned event sequences. despite recent progress, existing approaches are still ineffective in exploiting planned events when generating stories. usually, pre-trained generation models, e.g., bart @xcite @xcite are employed to generate stories after event planning. however, as shown by the conflicts in figure 1 , the separate sentences generated by bart look reasonable, figure 1 : conditioned on leading context and reference events (extracted from reference stories), existing generation models still suffer from problems of relevance and coherence. for instance, we fine-tune bart @xcite to generate stories. the leading context and reference text in this example are collected from roc stories @xcite . some conflicts among them are observed and coloured. but there are several issues observed considering the whole story: as a commonsense story, if the car needs to be "fixed and replaced" then it is too broken to "drive around"; "ken" should not drive the car "very fast" in the "snow"; if "ken" "got stuck in the ditch" or "lost traction", he cannot then be "driving long distances". we hypothesise that these problems come from the inadequacy of capturing contextual features when keeping track of event sequences, because (i) the planned events generally lack background information, e.g., ken (the character) and snow (the scene) and (ii) training stories may have the same events but different reference stories, which may lead to confusion during inference if not considering the story-specific scenario. therefore, to address these challenges we propose etrica -a novel event-triggered context-aware end-to-end framework for story generation. given both leading context and planned events, etrica can more effectively capture contextual and event features from inputs than state-of-the-art (abbr. sota) baseline models. traditional generation models struggle to learn contextual representations when implicitly keeping track of the state of events due to feature differences of events and contexts. as an abstract storyline, an event sequence only contains schematic information related to actions (e.g. the verb), while the context usually records story-specific details, e.g., the scene and characters in a story. to comprehensively leverage both features, we draw inspiration from prior work dealing with information fusion @xcite @xcite @xcite to encode heterogeneous features with a cross attention mechanism @xcite . we aim to inform our model of the context background when the neural module unfolds each event into a narrative. we propose a novel neural module that learns to implicitly map contextual features to event features through information fusion on their numeric vector spaces (we call this process contextualising events). the whole process is illustrated in figure . 2. with the contextualised event features, an autoregressive decoder is employed to dynamically generate stories by learning to unfold the contextualised events. we also introduce an auxiliary task of sentence similarity prediction @xcite to enhance the coherence between event-driven sentences. to support research on event-driven story generation, we propose a new task formulated by writing stories according to a given leading context and event sequence. we improve the event extraction framework of @xcite by exploiting dependency parsing to capture event related roles from sentences, instead of using heuristic rules. we also present two datasets where multi-sentence narratives from existing datasets are paired with event sequences using our automatic event extraction framework. importantly, our task formulation can also benefit the study of controllable story generation, considering there is increasing interest in storyline-based neural generative frameworks @xcite @xcite . according to our extensive experiments, etrica performs better than baseline models consid-ering the metrics of fluency, coherence, and relevance. our contributions foot_0 can be summarised as follows: ## conclusion we present a controllable story generation task conditioned with leading contexts and event sequences. we also provide two new datasets with extra annotated events, and introduce a new set of automatic metrics to measure coherence and fluency. we propose etrica, a novel generation model, which better exploits context and event features with a cross attention based contextualising network. through extensive experiments and analyses, we demonstrate etrica can generate stories with better fluency, coherence, and relevance compared to competitive baselines.