Towards Coherent and Consistent Use of Entities in Narrative Generation

Large pre-trained language models ( LMs ) have demonstrated impressive capabilities in generating long , fluent text ; however , there is little to no analysis on their ability to maintain entity coherence and consistency . In this work , we focus on the end task of narrative generation and systematically analyse the long-range entity coherence and consistency in generated stories . First , we propose a set of automatic metrics for measuring model performance in terms of entity usage . Given these metrics , we quantify the limitations of current LMs . Next , we propose augmenting a pre-trained LM with a dynamic entity memory in an end-to-end manner by using an auxiliary entity-related loss for guiding the reads and writes to the memory . We demonstrate that the dynamic entity memory increases entity coherence according to both automatic and human judgment and helps preserving entity-related information especially in settings with a limited context window . Finally , we also validate that our automatic metrics are correlated with human ratings and serve as a good indicator of the quality of generated stories . 1 INTRODUCTION . Large pre-trained language models ( LMs ) ( such as GPT-2 ( Radford et al. , 2019 ) , GPT-3 ( Brown et al. , 2020 ) , and models based on the Transformer-XL architecture ( Dai et al. , 2019 ) ) have radically improved text generation , producing seemingly fluent text – Clark et al . ( 2021 ) even showed that non-expert human judges can not distinguish between machine-written and human-authored texts , based on surface cues . Assuming the quality of generated text as given , most recent efforts have then focused on trying to control generation with a desired topic , factual information , or specific style ( Keskar et al. , 2019 ; Dathathri et al. , 2019 ; Shin et al. , 2020 ; Li & Liang , 2021 ) . However , anecdotally , there are still common failure cases of machine generated text in terms of entity coherence and consistency , which are fundamental properties of language . In this work , we specifically focus on the task of narrative generation in order to analyse and improve entity coherence and consistency . Entities play a central role in narratives , since they guide the plot , and all events revolve around them ( Fludernik , 2002 ; Jannidis , 2009 ; Frow , 2014 ; Bamman et al. , 2013 ) . Despite the importance of entities , recent work has mainly emphasised on controlling the topic of the generated stories using outlines , keywords or other relevant knowledge ( Xu et al. , 2020 ; Rashkin et al. , 2020 ; Fan et al. , 2019 ; Goldfarb-Tarrant et al. , 2020 ) . At the same time , entity-related structure in narrative generation has been largely understudied for large-scale pre-trained LMs . First , we propose a set of metrics for automatically measuring entity coherence and consistency . Based on these metrics , we observe that the current LMs fail to follow the patterns of entity usage we find in human-written narratives . Overall , the generated stories present significantly lower coherence and consistency , and this is especially evident for stories with complex events and many named entities . We further validate these observations by performing a human evaluation study , showing that our automatic metrics correlate with human judgment of entity coherence . Next , in order to improve these properties in narrative generation , we propose augmenting a pretrained LM with a dynamic entity memory . Motivated by prior work on language modeling ( Clark et al. , 2018 ; Ji et al. , 2017 ) , which uses dynamic entity representations for improving generation on smaller RNN-based models , we augment the LM with an entity memory and cross-attention blocks at each layer of the model for attending to entities that participate in the narrative . In contrast with prior work , we introduce an end-to-end trainable network with soft attention for performing reads and writes to the memory instead of separately training models to predict entity detection and reference . We also relax the hard constraints of Clark et al . ( 2018 ) and Ji et al . ( 2017 ) , who only condition on one entity per step and update an entity representation only when encountering one of its mentions . Instead , in order to increase both efficiency in the context of transformerbased networks and flexibility of the entity-token mapping , we instead perform soft reads from the entity memory based on a cross-attention mechanism . Thus , our model can condition on multiple relevant entities , and update all slots depending on the cross-attention scores after regular intervals within the narrative . Moreover , we exploit token-level entity annotations in order to regularize the cross-attention scores and better guide the reads and writes to the entity memory . We perform experiments on two narrative datasets , WritingPrompts ( Fan et al. , 2018 ) and WikiPlots,1 and find that utilizing an entity memory especially increases entity coherence according to both automatic metrics and human judges . Moreover , we experiment with different scenarios , where the LM has access to a limited narrative context ( i.e. , varying smaller context windows ) , in order to simulate model behavior in settings with much longer narratives , such as books or screenplays . Since narratives of this length can not fit into the LM ’ s short-term memory , we investigate the loss of entity-related information as we move to later narrative sections . By measuring perplexity and uncertainty on entity mentions on the original stories , we find that the dynamic entity memory is able to preserve significantly more entity-related information in limited context settings . 2 TASK FORMULATION . This work aims at the exploration of entity coherence and consistency in the context of narrative generation . Entities play a central role in narratives and are crucial for the development and quality of the story ( Jannidis , 2009 ; Frow , 2014 ; Bamman et al. , 2013 ) . According to Fludernik ( 2002 ) , there can even be narratives without plot , but not without a human experiencer in their center . Narrative theories have also studied character archetypes with specific attributes and actions as a means for analysing them ( Fludernik , 2002 ; Jung , 2014 ) . We formulate the task of entity-driven generation as conditional text generation on a set of given entities . Specifically , we identify and provide the gold entities that participate in a narrative via an entity prompt . Each entity may consist of more than one token and different entities are separated with a special separator token . Examples of entity prompts are presented in Figure 1a and details about their construction are given in Section 4.3 . Our objective is to investigate the patterns of entity usage in generated stories in comparison with human-written ones . More formally , we consider a LM that is conditioned on an entity prompt P and learns the distribution p ( x|P ) for generating narratives . The LM is trained on sequences of raw narrative text prepended with the corresponding entity prompts . The LM operates autoregressively ; that is , given P and the context generated so far x≤t = { x0 , x1 , ... , xt } , the LM computes a distribution for the next word in the narrative . Next , we define metrics for automatically measuring entity coherence and consistency in both human-written and generated stories . We evaluate the proposed metrics against human ratings in Section 5.3 . 1https : //github.com/markriedl/WikiPlots Entity coherence Various local entity coherence metrics have been suggested in literature , such as distance-based clustering and linkage coefficients ( Lioma et al. , 2016 ) and local entity coherence ( Barzilay & Lapata , 2008 ; Mesgar & Strube , 2014 ; Guinaudeau & Strube , 2013 ) . However , current LMs present high local coherence when compared with human-written stories , giving the impression that coherence has been achieved . In contrast , during preliminary analysis of longer narratives , we observed that LMs still struggle with maintaining long-range entity coherence ( see Figure 1b for a short incoherent example and Tables 7 , 8 , and 9 of the Appendix for longer examples of real generated text ) . Our main observation from generated stories is that LMs tend to drop the initial protagonists after a while and instead introduce new , irrelevant entities ( details in Section 5 ) . For quantifying this observation , we propose a new metric . We consider the protagonists of the narrative ( i.e . the entities with the most mentions ) and divide the narrative into L equal sections . Next , we compute the maximum span of mentions for each protagonist i as the maximum interval of sections where i appears in : Ci = sli − sfi . Here , sfi and sli are the indices of the sections containing the first and last mentions respectively of entity i . Entity consistency Another important aspect that we evaluate in the context of entity usage is the attributes that are given to each entity throughout the narrative ( see Figure 1b for an inconsistent example ) . Traditionally , narratives use archetypes for the protagonists ( e.g. , the “ hero ” and the “ trickster ” ; Fludernik 2002 ; Jung 2014 ) with rich and diverse features , personalities and consistent actions . As a measure of how well-developed and consistent each entity is within the narrative , we measure attribute consistency . Specifically , given all mentions per entity in a story , we consider as the attributes of the entity all verbs and adjectives that appear in the same sentence as each of its mentions . Next , we compute the percentage of unique attributes Ui for the ith entity as follows : Ui = | ⋃N j=1 , i∈Ej Aj | − ∣∣⋃N j=1 , i∈Ej Aj ⋂ ⋃N j=1 , i/∈Ej Aj ∣∣ | ⋃N j=1 , i∈Ej Aj | ( 1 ) where N is the number of sentences in the story , Ej are the entities that are mentioned in the jth sentence , Aj is the set of all attributes that appear in the jth sentence , and | · | is the size of the set . 3 METHOD . Our base LM is a pre-trained Transformer-XL ( T-XL ) model ( Dai et al. , 2019 ) conditioned on P . The T-XL LM allows us to consider an extended context window within the narrative when computing token representations in self-attention by using a cache memory , where all intermediate representations of the M tokens prior to the current context are stored and used for as context . In this work , we propose augmenting the pre-trained base LM with an entity memory ( MNEMELM ) . For attending to the entity memory , we add new , randomly initialized cross-attention blocks in parallel with self-attention per layer resembling the architecture of adapters2 ( Houlsby et al. , 2019 ) . We propose using the entity memory together with the prompt for richer entity representations and to better preserve entity-related information over a long time horizon . This addresses two limitations of prompts : 1 . They do not allow for more meaningful entity representations . For example , given a named entity such as “ Sarah King ” , the tokens from the prompt do not provide any information related to who Sarah King is , or which the attributes of Sarah King are within the context of the narrative . In contrast , our dynamic entity memory can store attributes of the entity as they appear in the text , which offers more information beyond the surface form . 2 . LMs eventually forget about the prompt when given long enough narratives ( i.e . the prompt will fall out of the short-term memory of the LM ) . In contrast , our method can efficiently store entity-related information in a fixed-size memory and independently of the current context window . We demonstrate this empirically in Section 5.1 . Memory initialization We first initialize the entity memory based on the information given in the prompt P . Specifically , each memory slot Mj , j ∈ [ 1 , Z ] represents one of the Z − 1 entities that participate in the narrative or corresponds to non-entity information ( i.e . the Zth slot is reserved for entity-irrelevant information ) . Each of the entity-related slots is initialized based on the prompt 2In contrast to adapters , we find that just training the new parameters is insufficient for narrative generation . tokens that correspond to this entity ( i.e . tokens allocated within two special separator tokens ) . For contextualizing the entity tokens before the memory initialization , we process the prompt via the LM and consider the output token-level embeddings . Next , the final representation for the jth slot is : Mj = 1K ∑K k=1 yk , where K is the number of tokens that correspond to the j th entity and yk is the output embedding of the kth token . Conditioning on a dynamic entity memory ( D-MNEMELM ) Each slot Mj = [ Kj , Vj ] of the entity memory contains a static key Kj ( i.e . a fixed surface entity representation ) and a dynamic value Vj ( i.e . a frequently updated representation based on narrative context ) , initialised as described above . To update the memory , we divide the narrative into equal-length chunks , update the entity memory after processing each chunk , and use the T-XL memory to store the previous chunks . At each layer of the pre-trained LM , we add a new , randomly initialized cross-attention block that operates in parallel with the pre-trained self-attention one . The cross-attention block takes as input the representation xi of the ith token ( either an embedding or intermediate representation ) and all memory slots M = [ K , V ] , and computes an entity-aware representation ei as follows : ait = softmax ( W tQxiW t KM T √ dM ) , t ∈ [ 1 , H ] ( 2 ) Mattit =W t MaitM ei =WE [ M att i1 ; ... ; M att iH ] ( 3 ) where H is the number of attention heads in cross-attention , [ · ; · ] denotes the concatenation operation , ait ∈ RZ , and ei ∈ Rdh . Next , we combine the entity-aware hidden representations ei with the self-attended hidden representations hi via a gating mechanism : gi = σ ( WR [ hi ; ei ] ) h ′ i = ( 1− gi ) hi + giei ( 4 ) We use the final representation h′ as the output of the modified attention block . After processing each chunk in the narrative , we compute a weighted average representation of the current chunk per memory slot given the cross-attention weights of the final layer aijt for token i , slot j and head t , and update the memory value Vj accordingly via a gating mechanism : hj = softmax ( max H t=1aijt/τ ) h ( 5 ) wj = max T i=1max H t=1aijt gj = sigmoid ( WU [ hj , Vj ] ) ( 6 ) V ′j = ( 1− wjgj ) Vj + wjgjhj , ( 7 ) where τ is a temperature hyperparameter , wj is the maximum contribution of the jth memory slot to the current chunk across all tokens T and heads H for the last layer , gj is a gate vector for updating the slot , and M ′j is the new updated value of the memory slot . Note that in addition to the gate value gj that the model computes , we also include an extra weight wj for updating the memory slots . This is used to discourage the model from updating all slots at each step and reflects which entities were used the most during reading from the memory . We also consider a variation of our model ( S-MNEMELM ) with a static entity memory . For this variation , we only consider the static keys per memory slot and do not perform any updates . Regularization of cross-attention scores Finally , although the soft attention during reading and writing to the memory allows the model to explore all entity slots , we still guide the reads and writes via an auxiliary regularization loss in the objective function . Specifically , we want to encourage the model to attend to the correct entities per token during reading from the memory , and update those slots when writing to the memory . We label every token in the context ( i.e . in the same sentence ) of an entity mention with that entity ; if a context contains multiple entities , we allow multiple labels . Given the entity labels per token i , we construct a few-hot distribution qi over all entities that participate in the narrative by attributing equal probabilities to all entities assigned to token i . Next , we minimize the per-token KL divergence loss DKL between the computed cross-attention weights aitl , where t ∈ [ 1 , H ] , l ∈ [ 1 , L ] , H the number of attention heads , and L the number of layers , and the ground-truth distribution qi for the ith token . Hence , our extra regularization loss is : R = DKL ( aitl||qi ) , and our final objective is the weighted sum of the individual losses : L = 1 T T∑ i=1 ( − log p ( xi|x < i ; P ) + λ 1 LH L∑ l=1 H∑ t=1 DKL ( aitl||qi ) ) ( 8 )

Summary: In this paper, the main contributions are 1)analyze the entity coherence of  pretrained LMs by leveraging some existing metrics and also four newly proposed metrics 2) propose a new generative model to have a higher quality generated narratives from entity coherence and consistency perspectives. Authors develop a model by adding a dynamic entity memory to the existing pretrained LMs. The cross attention between input text and entity memory alongside the self-attention coming from pretrained-LM helps to not forget the previous entities and their attributes during generating narratives.