Towards Coherent and Consistent Use of Entities in Narrative Generation

Large pre-trained language models ( LMs ) have demonstrated impressive capabilities in generating long , fluent text ; however , there is little to no analysis on their ability to maintain entity coherence and consistency . In this work , we focus on the end task of narrative generation and systematically analyse the long-range entity coherence and consistency in generated stories . First , we propose a set of automatic metrics for measuring model performance in terms of entity usage . Given these metrics , we quantify the limitations of current LMs . Next , we propose augmenting a pre-trained LM with a dynamic entity memory in an end-to-end manner by using an auxiliary entity-related loss for guiding the reads and writes to the memory . We demonstrate that the dynamic entity memory increases entity coherence according to both automatic and human judgment and helps preserving entity-related information especially in settings with a limited context window . Finally , we also validate that our automatic metrics are correlated with human ratings and serve as a good indicator of the quality of generated stories . 1 INTRODUCTION . Large pre-trained language models ( LMs ) ( such as GPT-2 ( Radford et al. , 2019 ) , GPT-3 ( Brown et al. , 2020 ) , and models based on the Transformer-XL architecture ( Dai et al. , 2019 ) ) have radically improved text generation , producing seemingly fluent text – Clark et al . ( 2021 ) even showed that non-expert human judges can not distinguish between machine-written and human-authored texts , based on surface cues . Assuming the quality of generated text as given , most recent efforts have then focused on trying to control generation with a desired topic , factual information , or specific style ( Keskar et al. , 2019 ; Dathathri et al. , 2019 ; Shin et al. , 2020 ; Li & Liang , 2021 ) . However , anecdotally , there are still common failure cases of machine generated text in terms of entity coherence and consistency , which are fundamental properties of language . In this work , we specifically focus on the task of narrative generation in order to analyse and improve entity coherence and consistency . Entities play a central role in narratives , since they guide the plot , and all events revolve around them ( Fludernik , 2002 ; Jannidis , 2009 ; Frow , 2014 ; Bamman et al. , 2013 ) . Despite the importance of entities , recent work has mainly emphasised on controlling the topic of the generated stories using outlines , keywords or other relevant knowledge ( Xu et al. , 2020 ; Rashkin et al. , 2020 ; Fan et al. , 2019 ; Goldfarb-Tarrant et al. , 2020 ) . At the same time , entity-related structure in narrative generation has been largely understudied for large-scale pre-trained LMs . First , we propose a set of metrics for automatically measuring entity coherence and consistency . Based on these metrics , we observe that the current LMs fail to follow the patterns of entity usage we find in human-written narratives . Overall , the generated stories present significantly lower coherence and consistency , and this is especially evident for stories with complex events and many named entities . We further validate these observations by performing a human evaluation study , showing that our automatic metrics correlate with human judgment of entity coherence . Next , in order to improve these properties in narrative generation , we propose augmenting a pretrained LM with a dynamic entity memory . Motivated by prior work on language modeling ( Clark et al. , 2018 ; Ji et al. , 2017 ) , which uses dynamic entity representations for improving generation on smaller RNN-based models , we augment the LM with an entity memory and cross-attention blocks at each layer of the model for attending to entities that participate in the narrative . In contrast with prior work , we introduce an end-to-end trainable network with soft attention for performing reads and writes to the memory instead of separately training models to predict entity detection and reference . We also relax the hard constraints of Clark et al . ( 2018 ) and Ji et al . ( 2017 ) , who only condition on one entity per step and update an entity representation only when encountering one of its mentions . Instead , in order to increase both efficiency in the context of transformerbased networks and flexibility of the entity-token mapping , we instead perform soft reads from the entity memory based on a cross-attention mechanism . Thus , our model can condition on multiple relevant entities , and update all slots depending on the cross-attention scores after regular intervals within the narrative . Moreover , we exploit token-level entity annotations in order to regularize the cross-attention scores and better guide the reads and writes to the entity memory . We perform experiments on two narrative datasets , WritingPrompts ( Fan et al. , 2018 ) and WikiPlots,1 and find that utilizing an entity memory especially increases entity coherence according to both automatic metrics and human judges . Moreover , we experiment with different scenarios , where the LM has access to a limited narrative context ( i.e. , varying smaller context windows ) , in order to simulate model behavior in settings with much longer narratives , such as books or screenplays . Since narratives of this length can not fit into the LM ’ s short-term memory , we investigate the loss of entity-related information as we move to later narrative sections . By measuring perplexity and uncertainty on entity mentions on the original stories , we find that the dynamic entity memory is able to preserve significantly more entity-related information in limited context settings . 2 TASK FORMULATION . This work aims at the exploration of entity coherence and consistency in the context of narrative generation . Entities play a central role in narratives and are crucial for the development and quality of the story ( Jannidis , 2009 ; Frow , 2014 ; Bamman et al. , 2013 ) . According to Fludernik ( 2002 ) , there can even be narratives without plot , but not without a human experiencer in their center . Narrative theories have also studied character archetypes with specific attributes and actions as a means for analysing them ( Fludernik , 2002 ; Jung , 2014 ) . We formulate the task of entity-driven generation as conditional text generation on a set of given entities . Specifically , we identify and provide the gold entities that participate in a narrative via an entity prompt . Each entity may consist of more than one token and different entities are separated with a special separator token . Examples of entity prompts are presented in Figure 1a and details about their construction are given in Section 4.3 . Our objective is to investigate the patterns of entity usage in generated stories in comparison with human-written ones . More formally , we consider a LM that is conditioned on an entity prompt P and learns the distribution p ( x|P ) for generating narratives . The LM is trained on sequences of raw narrative text prepended with the corresponding entity prompts . The LM operates autoregressively ; that is , given P and the context generated so far x≤t = { x0 , x1 , ... , xt } , the LM computes a distribution for the next word in the narrative . Next , we define metrics for automatically measuring entity coherence and consistency in both human-written and generated stories . We evaluate the proposed metrics against human ratings in Section 5.3 . 1https : //github.com/markriedl/WikiPlots Entity coherence Various local entity coherence metrics have been suggested in literature , such as distance-based clustering and linkage coefficients ( Lioma et al. , 2016 ) and local entity coherence ( Barzilay & Lapata , 2008 ; Mesgar & Strube , 2014 ; Guinaudeau & Strube , 2013 ) . However , current LMs present high local coherence when compared with human-written stories , giving the impression that coherence has been achieved . In contrast , during preliminary analysis of longer narratives , we observed that LMs still struggle with maintaining long-range entity coherence ( see Figure 1b for a short incoherent example and Tables 7 , 8 , and 9 of the Appendix for longer examples of real generated text ) . Our main observation from generated stories is that LMs tend to drop the initial protagonists after a while and instead introduce new , irrelevant entities ( details in Section 5 ) . For quantifying this observation , we propose a new metric . We consider the protagonists of the narrative ( i.e . the entities with the most mentions ) and divide the narrative into L equal sections . Next , we compute the maximum span of mentions for each protagonist i as the maximum interval of sections where i appears in : Ci = sli − sfi . Here , sfi and sli are the indices of the sections containing the first and last mentions respectively of entity i . Entity consistency Another important aspect that we evaluate in the context of entity usage is the attributes that are given to each entity throughout the narrative ( see Figure 1b for an inconsistent example ) . Traditionally , narratives use archetypes for the protagonists ( e.g. , the “ hero ” and the “ trickster ” ; Fludernik 2002 ; Jung 2014 ) with rich and diverse features , personalities and consistent actions . As a measure of how well-developed and consistent each entity is within the narrative , we measure attribute consistency . Specifically , given all mentions per entity in a story , we consider as the attributes of the entity all verbs and adjectives that appear in the same sentence as each of its mentions . Next , we compute the percentage of unique attributes Ui for the ith entity as follows : Ui = | ⋃N j=1 , i∈Ej Aj | − ∣∣⋃N j=1 , i∈Ej Aj ⋂ ⋃N j=1 , i/∈Ej Aj ∣∣ | ⋃N j=1 , i∈Ej Aj | ( 1 ) where N is the number of sentences in the story , Ej are the entities that are mentioned in the jth sentence , Aj is the set of all attributes that appear in the jth sentence , and | · | is the size of the set . 3 METHOD . Our base LM is a pre-trained Transformer-XL ( T-XL ) model ( Dai et al. , 2019 ) conditioned on P . The T-XL LM allows us to consider an extended context window within the narrative when computing token representations in self-attention by using a cache memory , where all intermediate representations of the M tokens prior to the current context are stored and used for as context . In this work , we propose augmenting the pre-trained base LM with an entity memory ( MNEMELM ) . For attending to the entity memory , we add new , randomly initialized cross-attention blocks in parallel with self-attention per layer resembling the architecture of adapters2 ( Houlsby et al. , 2019 ) . We propose using the entity memory together with the prompt for richer entity representations and to better preserve entity-related information over a long time horizon . This addresses two limitations of prompts : 1 . They do not allow for more meaningful entity representations . For example , given a named entity such as “ Sarah King ” , the tokens from the prompt do not provide any information related to who Sarah King is , or which the attributes of Sarah King are within the context of the narrative . In contrast , our dynamic entity memory can store attributes of the entity as they appear in the text , which offers more information beyond the surface form . 2 . LMs eventually forget about the prompt when given long enough narratives ( i.e . the prompt will fall out of the short-term memory of the LM ) . In contrast , our method can efficiently store entity-related information in a fixed-size memory and independently of the current context window . We demonstrate this empirically in Section 5.1 . Memory initialization We first initialize the entity memory based on the information given in the prompt P . Specifically , each memory slot Mj , j ∈ [ 1 , Z ] represents one of the Z − 1 entities that participate in the narrative or corresponds to non-entity information ( i.e . the Zth slot is reserved for entity-irrelevant information ) . Each of the entity-related slots is initialized based on the prompt 2In contrast to adapters , we find that just training the new parameters is insufficient for narrative generation . tokens that correspond to this entity ( i.e . tokens allocated within two special separator tokens ) . For contextualizing the entity tokens before the memory initialization , we process the prompt via the LM and consider the output token-level embeddings . Next , the final representation for the jth slot is : Mj = 1K ∑K k=1 yk , where K is the number of tokens that correspond to the j th entity and yk is the output embedding of the kth token . Conditioning on a dynamic entity memory ( D-MNEMELM ) Each slot Mj = [ Kj , Vj ] of the entity memory contains a static key Kj ( i.e . a fixed surface entity representation ) and a dynamic value Vj ( i.e . a frequently updated representation based on narrative context ) , initialised as described above . To update the memory , we divide the narrative into equal-length chunks , update the entity memory after processing each chunk , and use the T-XL memory to store the previous chunks . At each layer of the pre-trained LM , we add a new , randomly initialized cross-attention block that operates in parallel with the pre-trained self-attention one . The cross-attention block takes as input the representation xi of the ith token ( either an embedding or intermediate representation ) and all memory slots M = [ K , V ] , and computes an entity-aware representation ei as follows : ait = softmax ( W tQxiW t KM T √ dM ) , t ∈ [ 1 , H ] ( 2 ) Mattit =W t MaitM ei =WE [ M att i1 ; ... ; M att iH ] ( 3 ) where H is the number of attention heads in cross-attention , [ · ; · ] denotes the concatenation operation , ait ∈ RZ , and ei ∈ Rdh . Next , we combine the entity-aware hidden representations ei with the self-attended hidden representations hi via a gating mechanism : gi = σ ( WR [ hi ; ei ] ) h ′ i = ( 1− gi ) hi + giei ( 4 ) We use the final representation h′ as the output of the modified attention block . After processing each chunk in the narrative , we compute a weighted average representation of the current chunk per memory slot given the cross-attention weights of the final layer aijt for token i , slot j and head t , and update the memory value Vj accordingly via a gating mechanism : hj = softmax ( max H t=1aijt/τ ) h ( 5 ) wj = max T i=1max H t=1aijt gj = sigmoid ( WU [ hj , Vj ] ) ( 6 ) V ′j = ( 1− wjgj ) Vj + wjgjhj , ( 7 ) where τ is a temperature hyperparameter , wj is the maximum contribution of the jth memory slot to the current chunk across all tokens T and heads H for the last layer , gj is a gate vector for updating the slot , and M ′j is the new updated value of the memory slot . Note that in addition to the gate value gj that the model computes , we also include an extra weight wj for updating the memory slots . This is used to discourage the model from updating all slots at each step and reflects which entities were used the most during reading from the memory . We also consider a variation of our model ( S-MNEMELM ) with a static entity memory . For this variation , we only consider the static keys per memory slot and do not perform any updates . Regularization of cross-attention scores Finally , although the soft attention during reading and writing to the memory allows the model to explore all entity slots , we still guide the reads and writes via an auxiliary regularization loss in the objective function . Specifically , we want to encourage the model to attend to the correct entities per token during reading from the memory , and update those slots when writing to the memory . We label every token in the context ( i.e . in the same sentence ) of an entity mention with that entity ; if a context contains multiple entities , we allow multiple labels . Given the entity labels per token i , we construct a few-hot distribution qi over all entities that participate in the narrative by attributing equal probabilities to all entities assigned to token i . Next , we minimize the per-token KL divergence loss DKL between the computed cross-attention weights aitl , where t ∈ [ 1 , H ] , l ∈ [ 1 , L ] , H the number of attention heads , and L the number of layers , and the ground-truth distribution qi for the ith token . Hence , our extra regularization loss is : R = DKL ( aitl||qi ) , and our final objective is the weighted sum of the individual losses : L = 1 T T∑ i=1 ( − log p ( xi|x < i ; P ) + λ 1 LH L∑ l=1 H∑ t=1 DKL ( aitl||qi ) ) ( 8 )

Summary: This work focuses on improving the long-range entity coherence and consistency when generating long narrative stories. The authors proposed two metrics to measure the entity coherence and consistency in terms of entity usage in generated narratives. They also propose to augment a pre-trained LM with a dynamic entity memory and cross-attention to improve the entity usage.