Generating Narrative Text in a Switching Dynamical System (2020)

Abstract: Early work on narrative modeling used explicit plans and goals to generate stories, but the language generation itself was restricted and inflexible. Modern methods use language models for more robust generation, but often lack an explicit representation of the scaffolding and dynamics that guide a coherent narrative. This paper introduces a new model that integrates explicit narrative structure with neural language models, formalizing narrative modeling as a Switching Linear Dynamical System (SLDS). A SLDS is a dynamical system in which the latent dynamics of the system (i.e. how the state vector transforms over time) is controlled by top-level discrete switching variables. The switching variables represent narrative structure (e.g., sentiment or discourse states), while the latent state vector encodes information on the current state of the narrative. This probabilistic formulation allows us to control generation, and can be learned in a semi-supervised fashion using both labeled and unlabeled data. Additionally, we derive a Gibbs sampler for our model that can “fill in” arbitrary parts of the narrative, guided by the switching variables. Our filled-in (English language) narratives outperform several baselines on both automatic and human evaluations

## introduction a narrative is a textualized sequence of events that serves as a coherent outline for an actual story @xcite . effective narratives are typically built on top of higher level narrative scaffolds 1 which specify at an abstract level how the story should evolve along different dimensions. example scaffolds include descriptions of the emotional trajectory of a story @xcite , the goals of characters throughout the story @xcite , or the abstract types of events that may occur @xcite . the parts of a scaffold are generic, and like propp's originally proposed narrative functions @xcite , can be reused across stories. to be fully reusable, one needs to go beyond just identifying what the elements of the narrative scaffold are, and also indicate how each scaffold element changes the properties of the current story state. we refer to the explication of these transformation/transitions as the narrative dynamics. prior work on automatic narrative generation has a rich history of modeling both narrative scaffolds and narrative dynamics @xcite @xcite @xcite . the modeling of both narrative scaffold and dynamics often imbued these systems with a greater degree of control for the user in generating stories, allowing users to flexibly specify desired outcomes or plot points (or more generally, the state of the narrative) that should be achieved at certain sections of the story. constrained generation (for example, constraining the story to start and end with particular sentences, such as that given in in this work, we show how the insight of modeling the structure of a narrative along with general purpose dynamics can be combined with modern neural network based language models. we do this by explicitly modeling the narrative state with a latent vector, and modeling how this state transforms over time as a switching linear dynamical system (slds). we show how this formulation captures the concepts of narrative dynamics and scaffolds in a way compatible with current neural generation systems. finally we show that, by explicitly modeling these dynamics, our models obtain the ability to "fill in" narratives; all without being explicitly trained to do so and with no further training required. we evaluate our model with both human evaluation and several automatic measures 2 and show that our model outperforms several strong baselines. ## a switching dynamical system for narrative generation in this section, we give a brief overview of switching dynamical systems and how they can be used to capture both a scaffold of the narrative as well as the narrative dynamics. we then describe in detail the components of our model and its relation to existing models. ## learning and posterior inference due to the conditionals parameterized by neural networks we use amortized variational inference in a manner similar to variational autoencoders (kingma and welling, 2013), both to learn an approximate posterior q(s, z|x) and to learn the generative model parameters by maximizing a lower bound on the data likelihood (elbo). we assume that the approximate posterior factorizes as follows: like in vaes, computing these individual factors is done through a parameterized function called the inference or recognition network whose parameters are trained jointly with the generative model. in our case there are two forms for the factors in our posterior: (1) the first form, q(s i |x) = q s i is parameterized by a classifier that takes in the set of sentences x and outputs a categorical distribution over the switching variables. (2) the second form, q(z borrowing terminology from vaes, the approximate posterior (the factors given above) act as an 'encoder', while the generative model from the previous section can be seen as the 'decoder'. this type of training has been previously used in @xcite @xcite @xcite . ## interpolations via gibbs sampling one of the benefits of probabilistic formulation is the possibility (if an inference procedure can be found) of generating narratives with specific constraints, where the constraints may be specified as clamped variables in the model. in this section, we show how narratives may be generated conditioned on arbitrary bits and pieces of the narrative already filled in, using approximate gibbs sampling. this allows one to, for example, interpolate a narrative given the first and the last sentence (similar to how earlier story generation systems were able to generate with a given end goal in mind). some examples of these interpolations generated by our system can be found in ## evaluations as described above, our model is able to perform narrative interpolations via an approximate gibbs sampling procedure. at the core of our evaluations is thus a fill-in-the-sentences task. we provide 1 or 2 sentences, and require the model to generate the rest of the narrative . we evaluate this via automatic evaluations as well as with crowd-sourced human evaluations. we also report perplexity to evaluate the models' ability to fit the data. lastly, we look at whether the transitions learned by the slds models capture what they are intended to capture: does using the transition matrix associated with a sentiment tag (positive/negative/neutral) lead to a generated sentence with that sentiment? ## related work story/narrative generation has a rich history in the field of ai. many early systems were based on structured formalisms for describing common narrative structures @xcite @xcite , many being inspired by the initial work of @xcite . there has been a swath of recent work that has looked to add some semblance of a 'narrative scaffold' back into generation methods @xcite @xcite . many of these methods work as conditional lms (conditioned directly on the scaffold). this line of work may be combined with our formalization as well, by conditioning the generation on the switching state as well, as done in the model of @xcite . recent work by @xcite has similar goals to ours in permitting more controlability in generation systems, developing a rl-based system that allows users to specify an end goal for a story (by specifying the event class that is desired to appear at the end). their work differs from ours in that it does not deal with text directly, modeling only the sequences of events in the narrative. it may be possible to utilize this model as the scaffolding component in our model (utilizing their rl policy for the scaffold planner, rather than the simple in this paper, we formulated the problem of narrative generation as a switching dynamical system. we showed how this formulation captures notions important in narrative generation, such as narrative dynamics and scaffolds. we developed an approximate gibbs sampling algorithm for the model that permits the system to generate interpolations conditioned on arbitrary parts of the narrative, and evaluated these interpolations using both human and automatic evaluations. though in this work we used sentiment tags for our scaffolds/switching variables, future work may look at utilizing different kinds of information to guide the generation of narratives. utilizing the main predicate of a sentence as a scaffold would be a logical next step, and may prove more informative than the sentiment trajectory. a scaffold such as this can take on many more possible values than a sentiment tag, and as such, it may prove difficult to assign a set of dynamics to each value. another avenue for future work would deal with this possible problem. one potential solution could be to associate each switching variable value with a (learned) vector in a probability simplex, and use this vector to combine a small set of "primitive" dynamics matrices in order to get that value's associated set of dynamics. of some z i given everything else as follows: the rest can be derived by taking the pdfs of the two gaussian densities above, getting rid of constants that don't depend on z i , multiplying them together, and completing the square to obtain the numerator of a gaussian over z i (such that z i appears nowhere else in the equation). this numerator can then be multiplied by the normalizing constant (that does not depend on z i ) to obtain exactly a gaussian pdf with the mean and variance as given below: ## manual error analysis of generations we evaluate the quality of sentences of 20 generated narratives are not considered coherent by turkers. we find that, in a broader context, 16 stories out of 20 are not good enough in terms of connecting the ending with the previous sentences. also, in 14 out of 20 stories, mild off-topic sentences are introduced, which are aligned with the main topic of the story but not along with local coherency (i.e. the previous and the next sentences). when considering a narrower context, or sentence level, we confirm that only 9 out of 60 generated sentences are ungrammatical, so they fail to deliver their meaning. ## initializing the gibbs sampler initializing z : we first initialize the z variables in the sampler. this is done as follows: the z variables are initialized in increasing order. if sentence x i is provided as input, then we sample from the approximate posterior q z in order to initialize z i . if x i is missing, then we sample using the dynamics distribution, p (z i |z i-1 , s i ). since we initialize in increasing order, we are guaranteed to have z i-1 . initializing x : we next initialize the missing text. initializing the missing text x i is simply done by greedy decoding from the language model conditioned on z i , and previous sentences. ## automatic evaluation results with gold labels below in