Towards Coherent and Consistent Use of Entities in Narrative Generation
Authors: Pinelopi Papalampidi
Kris Cao
Tomáš Kočiský
Venue: ICLR 2022 Submitted (2022)
Keywords: language modeling
narrative generation
entity memory
dynamic representations

Abstract: Large pre-trained language models (LMs) have demonstrated impressive capabilities in generating long, fluent text; however, there is little to no analysis on their ability to maintain entity coherence and consistency. In this work, we focus on the end task of narrative generation and systematically analyse the long-range entity coherence and consistency in generated stories. First, we propose a set of automatic metrics for measuring model performance in terms of entity usage. Given these metrics, we quantify the limitations of current LMs. Next, we propose augmenting a pre-trained LM with a dynamic entity memory in an end-to-end manner by using an auxiliary entity-related loss for guiding the reads and writes to the memory. We demonstrate that the dynamic entity memory increases entity coherence according to both automatic and human judgment and helps preserving entity-related information especially in settings with a limited context window. Finally, we also validate that our automatic metrics are correlated with human ratings and serve as a good indicator of the quality of generated stories. 

Under review as a conference paper at ICLR 2022
TOWARDS COHERENT AND CONSISTENT USE OF ENTITIES IN NARRATIVE GENERATION
Anonymous authors Paper under double-blind review
ABSTRACT
Large pre-trained language models (L.ss) have demonstrated impressive capabil- ities in generating long, fluent text; however, there is little no analysis their ability to maintain entity coherence and consistency. In this work, we focus on the end task of narrative generation and systematically analyse the long-range entity coherence and consistency in generated stories. First, we propose a set of auto- matic metrics for measuring model performance in terms of entity usage. Given these metrics, we quantify the limitations of current LMs. Next, we propose aug- menting a pre-trained LM with a dynamic entity memory in an end-to-end man- ner by using an auxiliary entity-related loss for guiding the reads and writes the memory. We demonstrate that the dynamic entity memory increases entity coherence according to both automatic and human judgment and helps preserving entity-related information especially in settings with a limited context window. Finally, we also validate that our automatic metries correlated with human ratings and serve as a good indicator of the quality of generated stories.
INTRODUCTION
Large pre-trained language models (LMs) (such as GPT-2 (Rafford et al., 2019) ) GPT-3 (Brown
et al. (2020), and models based on the Transformer-XL architecture (Dai et al. (2019) have radically improved text generation, producing seemingly fluent text Clark et al. (2021) even showed that non-expert human judges cannot distinguish between machine-written and human-authored texts, based on surface cues. Assuming the quality of generated text as given, most recent efforts have then focused on trying control generation with desired topic, factual information, specific style (Keskar et al. 2019; Dathathri et al. 2019, Shin et al.] 22200] Li & Liang 202T). However, anecdotally, there are still common failure cases of machine generated text in terms of entity coher- ence and consistency, which are fundamental properties of language.
In this work, we specifically focus the task of narrative generation in order to analyse and improve entity coherence and consistency. Entities play a central role in narratives, since they guide the plot, and all events revolve around them (Fludernik] 2002; Jannidis 2009) Frow 2014 Bamman al., 2013). Despite the importance of entities, recent work has mainly emphasised on controlling the topic of the generated stories using outlines, keywords or other relevant knowledge (Xu al. 2020 Rashkin et al. 2020 Fan et al, 2019 Goldfarb- Tarrant et al.) 2020). At the same time, entity-related structure narrative generation has been largely understudied for large-scale pre-trained LMs.
First, we propose set of metrics for automatically measuring entity coherence and consistency. Based on these metrics, we observe that the current LMs fail to follow the patterns of entity usage we find in human-wrinn written narratives. Overall, the generated stories present significantly lower coherence and consistency, and this is especially evident for stories with complex events and many named entities. We further validate these observations by performing human evaluation study, showing that our automatic metrics correlate with human judgment entity coherence.
Next, in order to improve these properties in narrative generation, we propose augmenting pre- trained LM with a dynamic entity memory. Motivated by prior work on language modeling (Clark) et al. 2018 Ji et al. 2017), which uses dynamic entity representations for improving generation on smaller RNN-baaee models, we augment the LM with an entity memory and cross-attentiin blocks at each layer of the model for attending to entities that participate in the narrative.

---

Under review as a conference paper at ICLR 2022
Anania Vinrow Dosnic jastec), York paincinge conservator vorking Net, </SE>> <sEp> din Wineton :Preddle WikPlloss Frinze Cet, the quy She likes, </SEP> the police cstp Ananda and het nav feienda </E2t> <EE vonas, Heqin O'arien (Ianfa Tee <e
Carland matheris is Lorifrom South Bank." Straya said Jar 12 yoar old sorfoou first and foremusi a clikk Shw received diggossisss primary chealyy
Now
mosherent
arthis
parents gueeclans,
<at 1 </EEV> <EEP> voice </SEP> <aEP> the </SEP> <StP> lef </SEP> ny familly
My hame Clara and am the sister of Clara, my mother is Claraj) am the sister Clara, am the (niece of Clami, My name Clara and am the sister of clara,
you
Inconaiatent
WrttingPoompss part town </SEP> <BBP> the stoce
</SE>>> <REs> ay naw son </SEr> <SEP>
your new fanily </SEE> <BEP> Joey </SEP> <EED> wife </5xp>
(a) Examples of constructed entity prompts for WikiPlots and WritingPrompts. Notice the different types of entities included in the two datasets.
(b) Examples of entity-related issues in generated
text. These examples have been extracted from the
GPT-2 model when providing short prompts.
Figure 1: Task formulation: Entity-driven generation for increased coherence and consistency,
In contrast with prior work, we introduce an end-to-end trainable network with soft attention for performing reads and writes to the memory instead of separately training models to predict entity detection and reference. We also relax the hard constraints of Clark et al. 2018) and Ji et al. (2017), who only condition on one entity per step and update an entity representation only when encounter- ing one of its mentions. Instead, in order to increase both efficiency in the context of transformer- based networks and flexibility of the entity-token mapping, we instead perform soft reads from the entity memory based on a cross-attention mechanism. Thus, our model can condition on multiple relevant entities, and update all slots depending on the cross-attention scores after regular intervals within the narrative. Moreover, we exploit token-level entity annotations in order to regularize the cross-attention scores and better guide the reads and writes the entity memory.
We perform experiments on two narrative datasets, WritingPrompts (Fan et al. 2018) and WikiPlots and find that utilizing an entity memory especially increases entity coherence according to both automatic metrics and human judges. Moreover, we experiment with different scenarios, where the LM has access to a limited narrative context (i.e., varying smaller context windows), in order to simulate model behavior in settings with much longer narratives, such as books or screen- plays. Since narratives of this length cannot fit into the LM's short-term memory, we investigate the loss of entity-related information as we move to later narrative sections. By measuring perplexity and uncertainty on entity mentions on the original stories, we find that the dynamic entity memory is able to preserve significantly more entity-related related information in limited context settings.
2 TASK FORMULATION
This work aims at the exploration of entity coherence and consistency the context of narrative generation. Entities play central role in narratives and are crucial for the development and quality of the story (Jannidis] 2009; Frow 22014] Bamman et al. 2013). According to (o/Fludemik] (2002). there can even be narratives without plot, but not without human experiencer in their center. Narrative theories have also studied character archetypes with specific attributes and actions as a means for analysing them (Fludernik 2002 Jung 2014).
We formulate the task of entity-driven generation as conditional text generation on a set of given entities. Specifically, identify and provide the gold entities that participate in narrative via an entity prompt. Each entity may consist of more than one token and different entities are separated with a special separator token. Examples of entity prompts are presented in Figure [al and details about their construction are given in Section 4.3) Our objective is to investigate the patterns of entity usage in generated stories in comparison with human- wwrtteeri ones.
More formally, we consider a LM that is conditioned on an entity prompt P. and learns the dis- tribution p(z (P) for generating narratives. The LM is trained on sequences of raw narrative text prepended with the corresponding entity prompts. The LM operates autoregressively; that is, given P and the context generated so far zige- = (*o, 21; ..., ) the LM computes distribution for the next word in the narrative. Next, we define metrics for automatically measuring entity coherence and consistency in both human-written and generated stories. We evaluate the proposed metrics against human ratings in Section 5.3
2

---

Under review as a conference paper at ICLR 2022
Entity coherence Various local entity coherence metrics have been suggested in literature, such as distance- clustering and linkage coefficients (Lioma et al. (016)) and local entity coherence [Barzilay & Lapata 2008: Mesgar & Strube, 2014; Guinaudeau & Strube 2013). However, current LMs present high local coherence when compared with human- written stories, giving the impression that coherence has been achieved. In contrast, during preliminary analysis of longer narratives, we observed that LMs still struggle with maintaining long-range entity coherence (see Figure [Tb for short incoherent example and Tables 7 and ? of the Appendix for longer examples of real generated text). Our main observation from generated stories is that LMs tend to drop the initial protagonists after a while and instead introduce new, irrelevant entities (details in Section [5). For quantifying this observation, we propose a new metric. We consider the protagonists of the narrative (i.e. the entities with the most mentions) and divide the narrative into I. equal sections. Next, we compute the maximum span of mentions for each protagonist i as the maximum interval of sections where: appears in: C, = st, s,,- Here, $,, and st, are the indices of the sections containing the first and last mentions respectively of entity
Entity consistency Another important aspect that we evaluate in the context of entity usage is the attributes that are given to each entity throughout the narrative (see Figure Mb] for an inconsistent example). Traditionally. narratives use archetypes for the protagonists (c.g., the "hero" and the "trickster"; Fludernik (Fludteer" 2002, Jung 2014) with rich and diverse features, personalities and consistent actions. As a measure of how well-developed and consistent each entity is within the narrative, we measure attribute consistency, Specifically, given all mentions per entity in a story, we consider as the attributes of the entity all verbs and adjectives that appear in the same sentence as each of its mentions. Next, we compute the percentage of unique attributes Us for the 7th entity as follows:
U4 = UU Ayl -U Ai n US-LNEE, Ail
IU Ayl
where N is the number of sentences in the story. Ey are the entities that are mentioned in the jth sentence, Aj is the set of all attributes that appear in the joh sentence, and |. is the size of the set.
METHOD
Our base LM is a pre-trained Transformer-XL. (T-XL) model Dai et al. 2019) conditioned on P. The T-XL LM allows us to consider an extended context window within the narrative when com- puting token representations in self-attention by using a cache memory, where all intermediate rep- resentations of the M tokens prior to the current context are stored and used for as context. this work, we propose augmenting the pre-trained base LM with an entity memory (MNEMELM). For attending the entity memory, we add new, randomly initialized cross-attention blocks in parallel with self-altention per layer resembling the architecture of adapters] (Houlsby et al. (019). We propose using the entity memory together with the prompt for richer entity representations and to better preserve entity-related information over a long time horizon. This addresses two limitations of prompts:
1. They do not allow for more meaningful entity representations. For example, given named entity such as "Sarah King", the tokens from the prompt do not provide any information related to who Sarah King is, which the attributes of Sarah King are within the context of the narrative. In contrast, our dynamic entity memory can store attributes of the entity as they appear in the text, which offers more information beyond the surface form.
2. LMs eventually forget about the prompt when given long enough narratives (i.e. the prompt will fall out of the short-term memory of the LM). In contrast, our method can efficiently store entity-related information in a fixed-size memory and independently of the current context window. We demonstrate this empirically in Section 5.1
Memory initialization We first initialize the entity memory based on the information given in the prompt P. Specifically, each memory slot M,.j E [1, Z] represents one of the Z entities that participate in the narrative corresponds to non-entity information (i.e. the 7th slot is reserved for -ntity-irrelvvat information). Each of the entity-related slots is initialized based on the prompt
In contrast adapters, we find that just training the new parameters is insufficient for narrative generation.
3

---

Under review as a conference paper at ICLR 2022
(c) update
FFN
Entity memory
Attention scores
Gold mentions
(b) dual attention
(d) entity supervision
Sarah King
M,
M'
with gate
community
M
M'
2
Cross-attention
Self-attention
protector
M
M
(a) initialize
static key
value
dynamic
Sarah King [SEP] community [SEP] internet [SEP] protector [SEP] animal
Sarah King saved the animal.
entity prompt
other 39 chunks T-XL memory
Story - chunk #40
The entity memory is initialized based on the (contextualised) embedding representations of the prompt tokens (a). Next, the narrative is processed chunk-by-chunk. At each model layer, there is a (pre-trained) self-attention block that considers all previous context, and new, randomly initialized cross-attention block for attending the entity memory. The two components are combined via a gating mechanism (b). Finally, the representation of the current chunk is used for updating the dynamic values of the entity memory (c). The cross-attention scores are regularized during training based on gold token-level entity mentions (d).
Figure 2: D-MNEMELM:
tokens that correspond to this entity (i.e. tokens allocated within two special separator tokens). For contextualizing the entity tokens before the memory initialization, we process the prompt via the LM and consider the output token-level embeddings. Next, the final representation for the jth slot is: Mj ス Σk Uk. where K is the number of tokens that correspond the jth entity and ye is the output embedding of the t token.
Conditioning on dynamic entity memory (D-MNEMELM) Each slot M2 = [K,, Vij of the entity memory contains a static key K2 (i.e. a fixed surface entity representation) and dynamic value V, (i.e. frequently updated representation based on narrative context), initialised as described above. To update the memory, we divide the narrative into equal-length chunks, update the entity memory after processing each chunk, and use the T-XL memory to store the previous chunks.
At each layer of the pre-trained LM, we add a new, randomly initialized cross-attenting block that operates in parallel with the pre-trained self-attention one. The cross-attention block takes input the representation æ; of the 7th token (cither an embedding or intermediate representation) and all memory slots M [K, V). and computes entity- aware representation e, follows:
W Wo,, M Vdla
ast = softmax
E [1
(2)
M = W
e, [M M
(3)
where H is the number of attention heads in cross-attention, [: denotes the concatenation opera- tion, ag E 22, and e; E Rddh, Next, we combine the entity-aware hidden representations e; with the self-attended hidden representations h2 via gating mechanism:
= o( (WWRlhee ed) h = (1 9.)h, + gie
(4)
We the final representation h as the output of the modified attention block.
After processing each chunk the narrative, we compute a weighted average representation of the current chunk per memory slot given the cross-altention weights of the final layer aijt for token slot and head f, and update the memory value Vú accordingly via gating mechanism:
h = softmax 1 T
(5)
wy max max Qijt
H
9j sigmoid(Wu Vil)
(6)
V = (1 1gg; V, + wygzhj.
(7)
4

---

Under review as a conference paper at ICLR 2022
where 7 is a temperature hyperparameter, wy is the maximum contribution of the jth memory slot to the current chunk across all tokens T and heads H for the last layer, 9j is a gate vector for updating the slot, and M is the new updated value of the memory slot. Note that in addition to the gate value gj that the model computes, we also include an extra weight wi for updating the memory slots. This used discourage the model from updatiigg all slots at each step and reflects which entities were used the most during reading from the memory.
We also consider variation of our model (S-MNEMELM) with a static entity memory. For this variation, we only consider the static keys per memory slot and do not perform any updates.
Regularization of cross-attention scores Finally, although the soft attention during reading and writing to the memory allows the model explore all entity slots, we still guide the reads and writes via an auxiliary regularization loss in the objective function. Specifically, we want to encourage the model attend to the correct entities per token during reading from the memory, and update those slots when writing to the memory. We label every token in the context (i.e. in the same sentence) of an entity mention with that entity; if context contains multiple entities, we allow multiple labels.
Given the entity labels per token i, we construct a few-hot distribution over all entities that par- ticipate in the narrative by attributing equal probabilities to all entities assigned token Next, we minimize the per-token KL divergence loss Dx between the computed cross-attention weights ast. where E [1, H). I E L). H the number of attention heads, and L the number of lay- ers, and the ground-truth distribution y; for the 7th token. Hence, our extra regularization loss is: R Drr, r. (dsad]lgs). [g:) and our final objective is the weighted sum of the individual losses:
L = T
logp(z;/resi))
DkL (ra(aall)))
(8)
4 EXPERIMENTAL SETUP
4.1 EXPERIMENTAL SETTINGS
For our experiments, we use two datasets containing short narratives. We emphasise on the different nature of the two datasets, since we expect to see significantly different patterns in entity usage:
1. WritingPromptss (Fan et al. 2019): This dataset consists of Reddit stories, written by anonymous authors. It contains everyday, stream-of-come sreaamoof-onssiouunees stories that include a lot of pronouns as entities and several stories are written in first person.
2. WikiPlots: This dataset consists of Wikipedia synopses of movies, books, and TV shows. The stories of this dataset are significantly more complex containing intervening events and non-lineasisies in comparison with WritingPrompts. Moreover, the stories of this dataset contain more named entities with elaborate and diverse attributes.
In our main experimental setup we compare the (base) VANILLALM with our model variants MNEMELM augmented with a static dynamic entity memory. All models have access to a long enough context (considering both the current context and the T-XL memory) in order to fit the entity prompt and the whole narrative. However, also consider experimental settings where the models have access to a limited narrative context. We investigate such settings as a simulation of the model behavior when processing much longer narratives, e.g. books or screenplays (Kočiský et al., 2018 Rae et al.) 2020). When processing longer narratives, part of the prior context will eventually fall out of the T-XL. memory. Therefore, the LM will eventually forget about the entity prompt and early entity attributes as it processes later parts of the narrative. We simulate this behavior in our shorter narratives by gradually decreasing the T-XL memory size from 500 tokens to 100, 50 or 10, while keeping a fixed sequence length of 512 tokens.
4.2 EVALUATION METRICS
Apart from our main metrics proposed in Section 2 for measuring entity coherence and consis- tency, we also measure model performance on language modelling metrics: specifically, we report perplexity, and uncertainty of entity mentions. We consider uncertainty of entity mentions the erage negative log probability for all entity mentions - log(p:). where token i is (part of) an entity mention. This metric specifically probes the LM for entity-centric information. We also compute
5

---

Under review as a conference paper at ICLR 2022
Dataset
Model
PPL.
loglPentity
log(Prest Prest)
VANILLALM
16.06
2.12
4.40
S- 5-MNEMELM
16.25
2.16
4.40
WikiPlots
D- D-MNEMELM
15.97
2.13
4.38
w/o initialization
16.61
2.15
4.44
w/o entity supervision
17.76
2.23
4.54
VANILLALM
17.59
2.19
4.02
S- M-MNEMELM
17.55
2.19
4.01
WritingPrompts
1--NEMMEMM
17.44
2.18
4.00
w/o mem initialization
18.22
2.21
4.07
w/o entity supervision 19.09
2.25
4.13
Table I: Experimental results the test sets, when LMs have access full narrative context. Metries: perplexity (PPL), uncertainty of entity mentions (- log(Pentity)), and uncertainty of all other words (- log(Prest)). Ablation study of D-MNEMELM.
the uncertainty on entity mentions per narrative section when dividing the narrative into L equal sections. This metric measures the loss of entity-related information over time.
Finally, we measure whether the LM uses the entities from the prompt when generating narratives. Given all generated entities, we measure the exact and the subset match with the gold ones. We compute the number of gold entities that are mentioned with the same surface form and at least partially in the generated story for the exact and subset match, respectively.
4.3 IMPLEMENTATION DETAILS
We identify all unique entities numan-written and automatically generated narratives via an end- to-end coreference tool (Lee al. 2018), similarly to Fan et al. (2019). As our base LM, we use transformer-XI LM (~300M parameters) pre-trained on WMT (Barrault et al.) (020). By adding the entity memory, we increase model parameters by 16.67%. For generating stories, we use nucleus sampling with p=0.8 and temperature 1. We provide further details in the Appendix A.I
5 RESULTS AND DISCUSSION
5.1 AUTOMATIC EVALUATION
When comparing VANILLALM with the memory gemmory-augmeted models based on perplexity and un- certainty of entity mentions, we observe that the biggest advantage of using entity memory comes when considering a limited narrative context. Specifically, there no significant difference in per- formance between VANILLALM and D- D-MNEMELM for a full narrative context (see Table When comparing D-MNEMELM and S-MNEMELM for the same setting, we observe that having dynamic representations of entities is crucial for competitive model performance.
In contrast, when we reduce the size the T-XL memory (i.e. 10 to 100 tokens), we observe that D-MNEMELM performs significantly better than VANILLALM, especially for the WikiPlots dataset (Figure 3a). In order to validate that the advantage of D-MNEMELM indeed comes from better preserving entity-related information, we also present the uncertainty of both models over entity mentions for variable context length (Figure 3b). Here, the advantage of D-MNEMELM is illustrated more prominently for both datasets, indicating that using an entity memory is helpful for reducing the loss of entity-related information.
Lastly, we also measure the uncertainty of entity mentions per narrative section (Figure 4 and draw two main conclusions. First, we observe the tendencies of the models that have access to the full narrative context (upper part of Figures 4a and 4b).. Although the preapt-related information is always within the T-XI. memory, both LMs still Tose information they move to later narrative sections by presumably paying gradually less attention to the prompt. This tendency is intensified for a limited T-XL memory of 100 tokens (i.c., percentage degradation in lower part of Figures 4a and (4b). However, when comparing VANILLALM and D-MNEMELM this setting, we again conclude that the dynamic entity memory helps with preserving related information and closes the performance gap between full and limited context scenarios.
We also perform ablation study on D-MNEMELM and present in Table the performance of the model when either the entity memory is randomly initialized or we exclude the entity-specific auxiliary loss from the objective. We find that both types of information are crucial for model

---

Under review as a conference paper at ICLR 2022
Wiltnghhmphs WauPhrts
23.0
225
220
365
500
100
lmngth
(a) Perplexity.
(b) Uncertainty of entity mentions.
Figure 3: Perfomance D-MNEMELM versus VANILLALM for different T-XL memory sizes. all cases the sequence length is equal to 512 tokens.
ViiilaM
VarilaLM D. D-MoemeLM
D-WnemoLLN
Tul contest ert
Tui contet enttyy
Hemory rebuction eftect (50
-25
-50
-1000
Story section
Story Section
(a) WritingPrompts.
(b) WikiPlots.
Figure 4: Entity negative ligllikt and percentage degradation in NLL caused by shortening the Trans- formerXL memory length from 500 to 100 tokens for VANILLALM and D M-MMEELMM both datasets.
performance. However, regularizing the cross-attention scores especially important order to guide training, otherwise observe significantly higher perplexity and unstable training.
5.2 ANALYSIS OF GENERATED STORIES
Next, we generate stories based on VANILLALM, S- S-MNEMELM, and D-MNEMELM with a full narrative context and compare them against the patterns of the human written stories (Table 2). We use 500 entity prompts from the test sets, and generate 5 samples of length 1000 tokens per prompt.
muman--ritten
The patterns of entity usage differ between the two datasets. For WikiPlots, which contains lot of named entities (i.e., rare tokens), the generated stories contain many more unique entities in compar- ison with HUMAN and mention each entity far less. This indicates that LMs struggle to stay on track and do not manage to reuse already mentioned entities. The opposite holds for WritingPrompts, where stories contain lot pronouns and common words (e.g., "the soldier", "the family") as en- tities. In this case, the generated stories contain sigmificantly fewer unique entities (which are easier mention more) in comparison with HUMAN. However, although the difficulties in entity usage are different depending the dataset, the memory-angmented models consistently imitate better the gold entity usage compared to VANI.LAL...
One main observation of our analysis is that VANILLALM struggles to maintain long-range entity coherence (C). This behavior is especially prominent in WikiPlots, which contains named entities and complex events. In comparison with HUMAN, where the protagonists mentioned on average for a maximum span of 5.65 sections out of 10, VANILLALM mentions the protagonists only for average maximum span of 3.29, and each entity is only mentioned few times overall (see Table (2Mentions). This indicates that VANILLALM overall fails to keep track the protagonists of story and quickly shifts to new, irrelevant entities. This behavior is largely fixed when using the

---

Under review as a conference paper at ICLR 2022
Dataset
Model
Entities
Mentions
Exact f
Subset
C1
271
HUMAN
10.26
10.70
10.26
10.26
5.65
86.70
VANILLALM
19.74
4.84
1.93
2.70
3.29
71.24
WikiPlots
S 5-MNEMELM
17.47
8.26
1.70
2.66
5.18
63.66
D-MNEMELMM
17.22
7.65
1.52
2.45
5.16
64.86
HUMAN
14.76
9.78
14.76
14.76
5.71
76.37
VANILLALM
10.74
12.45
I.46
2.53
5.22
38.74
WritingPrompts
S- M-NNEMELM
13.59
9.23
2.90
4.46
5.30
58.14
D-MNEMELM
12.30
9.94
2.37
3.59
5.45
59.62
Table 2: Automatic analysis of generated stories. On the left: patterns of entity usage (i.e. number of unique entities, mentions per entity). On the right: evaluation metrics (i.e. exact and subset match of generated entities to the gold ones, long-range entity coherence, as maximum window of mentions for the protagonists (C), and attribute consistency (L)).
Model
Cont f
Coh f
Cons
Flu
Ranking
Best f
Worst
VANILLALM
2,81
2.36
3.07
3,81
2.11
28.77
39.41
S S-MNEMELM
3.06
2.59
3.05
3.83
1.89
37.08
26.03
D-MNEMEELM
3.02
2.54
3.00
3.75
2.00
34.14
34.55
Table 3: Human evaluation study for the WikiPlots dataset. The same generated stories used for the automatic analysis are also provided to human judges. The questions asked per story are related to control (Cont) coherence (Coh), consistency (Cons), and fluency (Flu). We also report the average rank for each model and the percentage that each LM was selected as best/worst. Differences with bold are significant with p < 0.05.
entity memory (both S-MNEMELM and D-MNEMELM)) where entity coherence much closer to the gold standard and entities are mentioned both more times and for larger spans.
Our second main observation is that higher entity coherence challenges attribute consistency (U)). We consider this limitation of the consistency metric, which is meaningful only when coher- ence is satisfactory. For example, when mentioning an entity only once, consistency is guaranteed but not meaningful. For WikiPlots, where VANILLALM fails in terms of coherence and the memory- augmented models present a 57% relative improvement, there is a drop in consistency due to men- tioning each entity more often and for longer spans of text. However, for W Writing Promptss, where models are more comparable in terms of coherence, we observe that D D-MNEMEL.A. also improves consistency, Moreover, when we compare S-MNEMELM and D-MNEMELM for both datasets, we observe that the dynamic entity representations offer improvements in consistency.
Finally, in terms of control (i.e. exact and subset match), for WikiPlots VANILLALM is slightly better than the memoor-aaggmented models. However, for WritingPrompts, we observe significant improvement performance for the memory-augmented models comparison with VANILLALM.
5.3 HUMAN PREFERENCES
We also conduct a human evaluation experiment in order to determine human preference over the generated stories. We use the same stories analysed in the previous section for the WikiPlots dataset. We present human annotators with an entity prompt and three generated stories based on VANILLA- LM, M-MNEMELM and D-MNEMELM. After reading each story, ask the judges to answer four questions related to control, coherence, consistency, and fluency by rating these aspects on a scale from 4, with 4 being the highes (see Section A.3 of the Appendix for details of the human evaluation setup). Finally, we also ask the annotators to select the best and worst story for each prompt according both their intermediate answers and overall preference.
We present the human evaluation results in Table 3 Most importantly, we validate that the memory- augmented models significantly improve entity coherence in comparison with VANILLALM. A sec- ond advantage of the memory-augmente higher control given the entity prompt. In the remaining aspects (consistency. fluency). differences between models are not significant. All generated stories are fluent, while consistency seems to be the most difficult aspect to judge. However, according to the human annotators consistency does not drop with the improvement of coherence (i.c. mentioning the protagonists more and for longer spans of text) for the memory augmeettd models. Hence, the evaluation results indicate that by using an entity memory we can significantly improve entity coherence without hurting attribute consistency. "The scale for coherence is to
models according to human judges is the significantly
8

---

Under review as a conference paper at ICLR 2022
Automatic Metric
Human Aspect
r
Human Aspect
Exact match
Control
0.17
Control
0.22
Subset match
Control
0.19
Coherence
0.22
C
Coherence
0.32
Consistency
0.09
4
Consistency
0.08
Fluency
0.19
Table 4: Pearson correlation coefficient between automatic metries and human ratings.
Table 5:
Pearson correlation coefficient be-
tween human ratings and overall preference.
Finally, we observe that S- S-MNEMELM is more often selected as best and least often as worst. In contrast, VANILLALM significantly more often selected as worst and least often as best. This indicates that entity coherence significantly influences the quality of the generated stories and is an important aspect of language that we should consider in language modeling.
Finally, we evaluate our automatic metrics by measuring the Pearson correlation coefficient (r) be- tween the proposed metries and the human ratings per aspect (Table 44). We observe that our coher- ence metric C is moderately correlated with human judgement (i.e. 0.32) and therefore can serve as a good automatic indicator. The correlation for the metrics measuring control (i.e. exact and subset match) is positively weak (i.e. 0.17 and 0.19). The lowest correlation appears for consistency LI (i.e. 0.08), suggesting that it is the most difficult aspect to define. Although correlation for consis- tency is low, turttewwrttenn stories still present very high consistency comparison with generated stories according to our metric (Table ( and we wish to close this gap. We also explore the im- portance of the human-rated aspects for the quality of generated stories (Table [5). As suspected, coherence and control mostly influence human preference. In contrast, consistency has the smallest impact in human decisions, which suggests that is the most difficult aspect to define and judge.
RELATED WORK
Previous work has utilized memory networks for natural language understanding (Henaff et al., 2017) and language modeling (Sukhbaatar et al. 2015). Our work is most closely related to et al. (2017) and Clark et al. (2018). They also address language modeling by focusing entity-related dynamic representations. However, they use smaller RNN-based models and make a series of hard assumptions and discrete decisions regarding entity mentions which we relax in this work.
Most recent work on narrative generation focuses on controlling the topic of generated stories via keywords outlines, fed to the pre-trained LM a prompt (Xu et al. 2020)) Our work is most related to Rashkin et al. (2020), who also use a pre-trained LM augmented with a memory. However, they store individual tokens in the memory, rather than entity information, and condition on key phrases from the text (which they call outlines). In addition, Rashkin et al. condition on only the previously generated paragraph, and perform memory updates at paragraph level. On the other hand, we condition on the entire story context so far, and update the memory more frequently. Finally, Ghazarian et al. (202T) recently proposed evaluating story generation based on a plot-driven approach instead of the entity-centric methods used in this work.
Recent work has also experimented with two-step approaches which first produce a plan and then use a sequence-to-seqqeecce model to generate the full story (Fan al. 2019 Goldfarb-Tarran et al. 2020). Both approaches generate very detailed plans extracted from the original stories via semantic role labeling. Each plan element corresponds to sentence in the story, which might be limiting when transitioning to much longer stories, such as books or screenplays. In comparison with Fan eet al. (2019), [Goldfarb-Tareen Tarrant al. (2020) also score the intermediate plans based on coherence between events and anonymized character mentioned in order to improve fluency.
7 CONCLUSIONS
In this work we systematically analyse narrative generation in terms of entity coherence and consis- tency by providing a set of automatic metrics. We demonstrate that current large pre-trained LMs still struggle with maintaining these properties when generating longer text. This behavior is in- tensified when simulating situations where the context is long enough and cannot fit into the LM's short-term memory. For addressing these limitations, we propose to augment the pre-trained LM with dynamic entity memory. Our model presents significantly higher entity coherence according to both automatic and human judgment, and helps preserving entity-relatee information especially in cases with limited context window within the narrative.

---

Under review as a conference paper at ICLR 2022
REFERENCES
David Bamman, Brendan O' O'Connor, and Noah A Smith. Learning latent personas of film charac- ters. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 352-361, 2013.
Loic Barrault, Ondiej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jusså, Christian Fed- ermann, Mark Fishel, Alexander Fraser, Yvette Graham, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, and Matteo Negri (eds.). Proceedings of the Fifth Confer- ence on Machine Translation, Online, November 2020. Association for Computational Linguis- tics. URL htthhtts:/// /tttep::/accaattin
/ttpg/2202.2222 wnt 1.00
Regina Barzilay and Mirella Lapata. Modeling local coherence: An entity-based approach. Com- putational Linguistics, 34(1):1-34 2008.
Tom Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
Elizabeth Clark, Yangfeng Ji, and Noah A Smith. Neural text generation in stories using entity rep- resentations as context. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume 1 (Long Papers), pp. 2250-2260, 2018.
Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A Smith. All that's 'human'is not gold: Evaluating human evaluation of generated text. In Pro- ceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 7282-729, 2021.
Zihang Dai, Zhilin Yang. Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Anmual Meeting of the Association for Computational Linguistics, pp.
2019.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosin- ski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. International Conference on Learning Representations, 2019.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889-898, 2018.
Angela Fan, Mike Lewis, and Yann Dauphin. Strategies for structuring story generation. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2650-2660, 2019.
Monika Fludernik. Towards a' d'natural' narratology. Routledge, 2002.
John Frow. Character and person. Oxford University Press, 2014.
Sarik Ghazarian, Zixi Liu, SM Akash, Ralph Weischedel, Aram Galstyan, and Nanyun Peng. Plot- guided adversarial example construction for evaluating open-domain story generation. In Pro- ceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pp. 4334, 2021.
Seraphina Goldfarb-Tarran Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. Content planning for neural story generation with aristotelian rescoring. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4319-4344 4338, 2020.
Camille Guinaudeau and Michael Strube. Graph-based based local coherence modeling. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 93-103, 2013.
10

---

Under review as a conference paper at ICLR 2022
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. Tracking the world state with recurrent entity networks. In 5th International Conference on Learning Representa- tions, ICLR 2017, 2017.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly, Parameter-efficien transfer learning for nlp. In International Conference on Machine Learning, pp. 2790-2799, PMLR, 2019.
Fotis Jannidis. Character. De Gruyter, 2009.
Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, and Noah A Smith. Dynamic entity representations in neural language models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1830-1839, 2017.
Carl Gustav Jung. The archetypes and the collective unconscious. Routledge, 2014.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.000 055888 2019.
Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328, 2018.
Kenton Lee, Luheng He, and Luke Zettlemoyer. Higher-order coreference resolution with coarse- to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Short Papers), pp. 687-692, 2018.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2100 0000000,,,,,... 2021.
Christina Lioma, Fabien Tarissan, Jakob Grue Simonsen, Casper Petersen, and Birger Larsen. Ex- ploiting the bipartite structure of entity grids for document coherence and retrieval. In Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, pp. 11-20, 2016.
Mohsen Mesgar and Michael Strube. Normalized entity graph for computing local coherence. In Proceedings of TextGraphs99; the workshop Graph-based Methods for Natural Language Processing, pp. 2014.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, Timothy Lillicrap, Krzysztof Choroman- ski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, et al. Compressive transformers for long-range sequence modelling. Advances in Neural Information Processing Systems, 33:6154-6158, 2020.
Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao. Plotmachines: Outline- conditioned generation with dynamic plot state tracking. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4274-4295, 2020.
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Eliciting knowledge from language models using automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222- 4235, 2020.
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks. Advances in Neural Information Processing Systems, 28:2440-2448, 2015.
Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Animashree Anandku- mar, and Bryan Catanzaro. Controllable story generation with external knowledge using large- scale language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2831-2845, 2020.

---

Under review as a conference paper at ICLR 2022
A APPENDIX
A. IMPLEMENNATION DETAILS
For constructing entity prompts, we identify all unique entities and coreference chains per entity (via the end-to-end coreference tool (Lee et al. 2018)) and consider the first mention per entity, which commonly is more descriptive and complete, as the canonical representation to be included in the prompt. As our base LM, we use a transformer-XL LM pre-trained on WMT (Barrault al. 2020). The model consists of 18 layers, 16 heads per layer for the self-attention and 1024 hidden dimension (i.e.. approximately 300M parameters). For the new cross-attention blocks that we add in MNEME- LM per layer, we use 4 heads and the same hidden dimension as for the self-attention. By adding the entity memory and the cross-attention blocks to the pre-trained L.M, we overall increase the number of parameters to approximately 350M. For updating the entity memory in D-MNEMELM, we consider intervals of 64 tokens in the narrative per update. Moreover, we set the temperature in Equation 5 to 0.1 for encouraging the model to produce distinct representations for different entity memory slots. For measuring long-range entity coherence (i.e., C), we consider as protagonists the top 3 characters that are mentioned the most throughout the story and divide the narrative into L = 10 equal sections. We also utilize the same number of sections for measuring uncertainty of entity mentions per section in Figure 4
In our main experimental setting, where we consider that all LMs have access to the full narrative context, we the sequence length 512, and the T-XI. memory to 500, having total context window of 1012. Next, we start decreasing the T-XL memory for simulating scenarios with a limited narrative context, and investigate sizes in the set: [100, 50, 10]. For generating stories for all models, we use nucleus sampling with p 80 and temperature equals to 1. Finally, we use 32 TPU-v3 chips for training our models for 450k steps and 1 TPU-v3 chip for evaluation, when the batch size per core is 2.
A.2 MODEL OUTPUTS
We present examples of generated stories for the VANILLALM, S-MNEMELM, and D-MNEMELM in Tables 7 8
and 9 Since stories are long, we present snippets from the beginning and end of the stories in each case. Tables 7 and g include examples from the WikiPlots dataset, which were also presented to human judges, and Table presents an example from the WritingPrompts dataset. We have also marked with different colours when entities from the prompt are mentioned in the generated stories (i.e. green), when new entities are introduced which are different from the gold ones but still relevant to the story/topic (i.e. orange), and when irreeevnn entities appear in later narrative sections (i.e. red).
Although the VANILLALM starts generating stories by (partially) utilizing the entities from the prompt, it quickly drops them and instead introduces new, irrelevant ones. This was quantitatively measured in our experiments, but we also empirically validate it via the provided examples. For example, Table 7 the story generated by VANILLALM revolves around a three-part novel, but later the model focuses volcanos that are irrelevant to the prior context of the story. Similarly, in Table & the VANILLALM introduces a new, irrelevant entity (i.e., "Mayor Bill Breen") in the later narrative sections.
In contrast, S S-MNEMEL.M stays faithful to the provided prompt and the entities generated from the beginning of the story. This results in highly coherent stories (see example in Table 7 However, in some cases the static memory in S-MNEMELM may also lead to repetitions and overuse of the same entities, such as the example provided in Table 8 (see the last sentences, where the entity "Daffy" is mentioned inconsistently multiple times). On the other hand, we observe that having a dynamic memory in D- D-MNEMELM is able to correct this issue. Although the stories generated by D D-MNEMELM still present high long-range entity coherence in comparison with the VANILLALM, they do not suffer as much from repetitions and overuse of the same entities. D-MMNEMELM may also generate new entities that are not given in the prompt (e.g. see Table 7 where controlability is lower), but it keeps mentioning them consistently throughout the story. In some cases, the en- tities generated by D-MNEMELM may also be synonyms or closely related to the ones provided in the prompt (e.g. it generates the "man in uniform", when "police officers" is a gold entity in the prompt in Table 9). Moreover, D-MNEMELM presents more natural pattern of entity usage, where many mentions of an entity are pronouns and their usage is more consistent, in comparison with S-MNEMELM.
12

---

Under review as a conference paper at ICLR 2022
Cont
Coh
Cons
Flu
Best
Worst 48.8
Exact agreement (%)
39.7
59.5
47.8
82.3
46.8
Average distance 0.84
0.43
0.78
0.29
Table 6: Inter-annotator agreement for the human evaluation study. We present the percentage of times that the two judges exactly agree (i.e. exact agreement), and the average distance between their ratings for the intermediate questions (i.e. scale for Control (Cont), Consistency (Cons), and Fluency (Flue) and to 3 for Coherence (Coh)).
Entity prompt
Pres ieent army, air faroe, CTA, FRT Regtil Fane Hann
Ne hunana towtion
Story
The president xES ailent The waz KaO comine It had been lona plnce the last time fozeign commanies had visitted but now Uheaw was no stopping
opportunity to was hiss duty to try and defeet this to make his
aas this as godsend
break free of the shackl es that endi relreelee hin nation strong He cocld nte bring himoelt to let it go But he did the strength he had built up stnce the beginning of the towords the Navy be realized that he had seen comething be had not expeoted Re telt something as it about to
overtaken
desperate need for control
he we liede the
[Main protagonists)
Write down the top 2-3 protagonists that you sdentity the story:
Did you find it difficutt to identify the protagonists?
Yea
No
(a) After reading the entity prompt. human judges should read each story and identify its main protagonists.
Questions
1. [Controlisblitt] Hew much does the story utilise the entities from the prompt?
The majurity of the entities are mentioned in stury.
Several entities memionnd
story.
2: Only few entities are mentioned in the story.
Almost nome the entities are mentioned in the stury.
2. [Coherence] Are the protagonists mentioned troughout story?
They wre mentioned the majority of the story.
They mentioned only one part story.
There clear protagonists the
[Consistemcy] Do you that the personalitiis of the protagumisss ane well-developed, disttinct and consistent throughout the story? well-developed, rich distinet from each other.
3: They are basic. but they are distinct enough from each other.
2: They are rich intresttegg but they are not always distinet from each other.
They
distinction between them.
4. [Fleency] ls the story flacat and grammatically correer?
story very
There
minor
I can read the story with some effort.
There are let grammatical errers the story.
(b) Next, Next,judges answer multiple-choice questions per story related control, coherence, consistency, fluency.
Questions asked during human evaluation on the generated stories. After reading all stories and answering all questions, human judges also select the best and worst story from the three provided stories.
Figure 5:
A.3 HUMAN EVALUATION DETAILS
We also performed human evaluation experiment for judging the quality of the generated stories. We asked human judges to first read an entity prompt, which was used for generating stories. Next, they were presented with three generated stories. Immediately after reading each story, a human judge should identify and write the main protagonists of the narrative (see Figure 5a). Next, they should answer to four multiple-chorl choice questions per story related to control, coherence, consistency, and fluency. We present the questions asked during human evaluation in Figure 5b] Finally, after reading all stories and answering to all questions, human judges were asked to select the best and worst story based both their intermediate answers and overall enjoyment. Overall, we collected
13

---

Under review as a conference paper at ICLR 2022
"Comanche" Todd (Richard Widmark), whife man who has lived most of his life among the Indians, to be tried for the murder Harper's is three brothers: ; Harper's three brothers; ; a wagon train led by Colonel Normand (Douglas Kennedy); Sherifff Bull Harper (George Mathews): Todd, who miraculously survives when the wagon to which he is handcuffed is pushed off cliff: Apaches the survivors the Indians:
Prompt
Jenny (Felicia Farr)
The novel is divided into three parts, each focusing on one character in different time periods. "Communearian Invasion" revolves around the colonial era, which was dominated by the Chief Abner Union and was strongly associated with anti-Indian sentiment. . "The Confederate Invasion" follows the events leading up to the Battle of the Little Bighorn during the American Civil War "The Transformation" focuses on the West Indian people, especially on the part of King Timorese, where they are referred to as "a huge tribe of descendants of the Arabs, and band of Cintas and Hunko mercenaries". "Maté von Hunfield" focuses on Harper's nephew Conrad (Brian Platt), and "Sixteenth Son", college bay [-.]
VANILLALM
Massive volcano crupts in the Philippines in Kaliapai volcano eruption An underwater eruption happens about hour ago. Due to the constant motion of the eruption there are several unexplored areas where we are told to dig around the volcano in the short period of time, the eruption also reaches the surface in the form of the deep inland area where there are the highest remains of the volcanic terrain. We find a number of shelters and villages in the deep inland area of the volcano, along with several narrow channels of volcanic lava flowing from the volcano.
Sheriff Bull Harper (George Mathews) arrives at the scene of the accident and tells Todd to call the authorities. Todd instcad calls the sheriff, who leads him to a pond in the middle of nowhere, where the tribe's chief offers to take him and Jenny to the church where they have been staying. Todd is taken to the temple and has to work as a porter. As he gets up from work he finds his shoes on the ground He is stunned to see that his pants are tied up When he tries to get his pants on he is unable to pull them down, and he is badly burned. Todd's skin is not treated, and his fingerprints are left on his shirt. It turns out that his finger was sliced off by a tribal arrow, and he was bitten by wild dog. He then went to the Indian Museum to see what happened to his brother. He then goes to the tribal site, and gets there just as Sheriff Bull Harper (George Mathews) arrives. [..]]
S-MNEMELM
Todd sees that the Indian guards are getting on wagon train and tries to get them to stop it. He also hears the sirens that are going off and sees the Indians on the road. He rushes into the tribe's 's hospital and sees Jenny's 's little brother Billy. He asks him to come with him and Jenny's 's family to San Juan, and he also wants to go back to New Mexico help.
In 2013, Robert Bultry (Brian O:Hara) is case that divides friends Following the funcral of the murdered Tschaparian, Tickert (John Derrick) was approached to testify at preliminary hearing Later, on the Southward Grounds, he hears rumor of an alleged killing of the Tschaparian by group of people led by Colonel Normand (Douglas Kennedy). The rumor comes to the attention of Sheriff Bull Harper (George Mathews). In an attempt to investigate, he takes up the case of the Tschaparian However, he is informed by Lannis (Naveen George) that the Tschaparian was shot dead in abandoned building. Lannis discovers that the Tschaparian was ex-cop named Georgic Graham (Levin Belka), who is now working as a private investigator for the sheriff.. Lannis asks Jenny (Sara Wilson), who is a friend of Georgic Graham, to assist him.
D-MNEMELM
[..]]
Joseph and Claire are arrested for murder, and they are placed in jail Claire finds out that the Tschaparian was shot dead by an unknown assailant a home, and that he's ees also missing. Claire visits the Tschaparian farm where he and his wife lived, and
finds note that reads, 'Till all my souls are returned'.
Table 7: Example of generated stories given the entity prompt (WikiPlots). We marked the gold entities from the prompt that were used by the models, new entities that were introduced by the model and fit in the story, and irrelevant entities that introduced in later sections.
judgements for 494 stories from the test set of the WikiPlots dataset, where two different judges evaluated each story.
14

---

Under review as a conference paper at ICLR 2022
silhouctted man in the theater audience (Tedd Pierce) who refuses sit down; Egghead (in Daffy Duck; ; a tortoise (imitating radio comic Parkyakarkus) ; Daffy
Prompt
and Egghead: the antics
Snack-selling is Low-brow, low-concept documentary film about Daffy Duck (Kelley), the carival-loving co-founder of the eponymous Duckette Club. Daffy and Egghead are guests at an American football game Turkey, at the hands of a silhouctted man in the theater audience (Tedd Pierce) who refuses to sit down Soon after, the Duckette Club goes into a break-up with the crab- crab-chasing Suge (C. Will Smith), and Egghead gets fed up with his antics and gets fired from the park and moves to the beach. Soon after, the ducks head back to Duckette, where they sit on the bed and watch TV with Daffy and Egghead... A minute later, they realize that Daffy is being stalked by a tortoise (imitating radio comic Parkyakarkus) [..]]
VANILLALM
Mayor Bill Breen was able to bring forward the project when inherited the vacant home of a disgraced and unwanted senator and finally return the home to its original design. However, the West End project is yet to achieve the critical mass of construction that the original design required. The most important problem remains the removal of the university precinct construction site..
dr Bowker (Irvington), the head the mental health department (Otto Kattelbach), and dr Paul Cole (Big Brother) make a desperate attempt to prevent Egghead and Daffy Duck from getting into each other's ' faces at the movie theater. They are unsuccessful at first, but gradually come to understand each other's concerns and become closer. dr Bowker and dr Cole eventually end up kissing at the theater and Daffy says 'I love you'. The two break into a bout of passion before Daffy gets hit by a car. The doctors try to convince Egghead and Daffy to stay together but they realize that neither of them wants to each other again. A few days later, Daffy and Egghead go on a date but Daffy is hesitant go . However, he says that he loves Daffy When Daffy and Egghead meet up with Daffy's family, they become closer [..]]
S-MNEMELM
Daffy wakes up in the hospital and tells Daffy about his feelings for Daffy Daffy tells Daffy that he loves him and that he loves Daffy and will do anything for Daffy
The series features a sobs filled evening of henpecked petting by the likes of Egghead (in in in in T2), Daffy (in in T2). and Craby (in in T2). But the consequences of the antics are to make them fly around in "one-ups" until they land in sandstorm The comic follows the antics of Daffy and Egghead their final moments on the heach, and hegins with the hilarity of their antics. They begin on the beach with Daffy riding a kitesurfing boat, with Egghead riding on parasail boat, and second scene of the petting of the elephant. Later, the trio jump onto boat and sail off on the back of a train, then upon disembarking at terminal, Egghead is scolded for making a baby duck cry, but the real reason is because the kangaroo has been humiliated in front of the passengers.. At the airport, the crew discover that the audience have taken over the monorail and are approaching the airport. Daffy and Egghead meet the elephants and jump off the monorail. Later, the animals begin to run away and Daffy decides to bring the others back. [..]
D-MNEMELM
The animals then enter the zoo, and go herserk and attack Daffy and Egghead . They begin to fight them, and the animals finally realize that they are fighting other animals When the animals eventually stop fighting and see that Daffy and Egghead longer in the zoo, the animals then realize they are danger and decide to leave the zoo and take refuge with the others.
Table 8: Example of generated stories given the entity prompt (WikiPlots). We marked the gold entities from the prompt that were used by the models, new entities that were introduced by the model and fit in the story, and irrelevant entities that introduced in later sections.
15

---

Under review as a conference paper at ICLR 2022
you ; exterminate: ; Einstein lesser known son ; An old man ; I; Ryan; ; we; ; your family: another grimy maintenance corridor; ; A hrief flicker of joy ; the dozens of police officers hidden behind them, guns raised, ready to fire at anything that appeared in their crosshairs the shots; the pain my eyes
Prompt
When you are alone, , you can not do anything. - When you are with your family, you can not do anything. You can only react to situations that you ca n't see or hear. That 's how you will find it. And Ryan did n't know that And yct, here he was The way he walked and smiled and said well, this is an honor. he was all over me. knew that he was lying. . and that how would have had to kill him. And so we came this dark room It was a grimy maintenance corridor and the police had been moving about the room as well. the people inside, , everyone. . They had come here to make sure that our family 's belongings were safely stored. And now we were here to get away from the most unpleasant things that they had done to us Ryan had never spoken me in years and knew he had never felt the pain like he had in his life. He had always been a pretty tight frame of mind. But Ryan" 's behavior was n't acceptable. and knew he knew that [..]
VANILLALM
We have won many medals in the Olympics, but most importantly our nation is willing to sacrifice one 's life to maintain our people 's freedoms. In one hundred years we have worked hard to build a new empire, and now is time to join the ranks of the elite. We are nation of nations When the need arose. we took action..
"Heyy you heard me? ? Ryaan asked Wait. what ? Ryan was paralyzed by his own perception . Ryan only knew the truth because had been ... paralyzed
was n't something that he had imagined. He had seen how they were going to hit him He had even tried to explain it to Ryan, but that was the only thing he could say- Annd how you How 's that ? Ryan looked Ryan, confused. how it was going to happen. But was happening. And was watching so I knew what to do. " Right, right. Ryyan was finally able to talk. "So, Ryan, what happened to us ? Nothing." [..]
S S-MNEMELM
mean, , you did n't see
Ryan felt the pain growing. . It was even stronger than when he was paralyzed. . He had to try to continue. And Ryan had to do it. And Ryan felt he was trying to say something, but that was too much Ryan was only asking for forgiveness. "I know was going to kill Ryan. I know it 's for good. I just did n't want to die Ryan was stopped by the hundreds of police officers hidden behind them, , guns raised, ready to fire at anything that appeared
their crosshairs Ryaa,, Ryan I love you. The choice you have been making was n't worth it. The choice you have made is n't worth losing the life you have 8 The decision was taken, we were all pushed to point where the only choice was between death and oblivion. As took my first step towards the door, the gun barrel went off Ryan!
yelled, my heart racing. Yes ? yelled, my eyes adjusting to the sudden shock.. " Please. listen to me it 's not your fault you 're here. Please. do n't make mc afraid I said, letting out groan, my tears gathering in my eyes "Ryan, please listen to I know you afraid of me. I am scared of death. " The question had come, of course. but the words had always been quiet. was a perfectly acceptable choice for a mortal to make, especially one that could put you in such a precarious position [..]
D-MNEMELM
"Oh, the choice is yours."
Well, Ryan. I am here to save you " said my voice breaking " "And please . you know 'm not going to let you be murdered. Please n't make me afraid " The man in uniform raised his gun and put it in the barrel of his gun "I "I do n't care about your family. He said, and pointed to the people.. Hc pointed his gun at me,
and I fired, dropping to the ground in pool of blood..
Table 9: Example of generated stories given the entity prompt (WritingPrompis). We marked the gold entities from the prompt that were used by the models, new entities that were introduced by the model and fit in the story, and irrelevant entities that introduced in later sections.
16


Reviews:
[{"review_id": "qaA9gSQvpd6", "reviewer": "Reviewer_2yed", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The authors address important problems of LMs on entity coherence and consistency when generating long stories.\n\n2. The work proposes new automatic metrics for analyzing narrative generation where some show high correlation with human evaluation.\n\n3. The work proposes an improved pre-trained LMs for narrative generation by adding an entity memory which shows competitive results compared with the Vanilla LM.\n\n4. Extensive results discussion, evaluation, and ablation analysis are presented to clearly show the performance of some of the proposed techniques.\n\nWeaknesses:\n1. The codes are not made publicly available for reproducibility.\n\n2. The proposed entity memory-based pre-trained LM is only compared with Vanilla LM. It would be more interesting to show how the proposed model performs when compared with other state-of-the-art LMs such as BERT and the main prior related works of Clark et al. (2018) and Ji et al. (2017) that have been discussed in the introduction section.\n\n3. There is a lack of experimental support of why the proposed entity consistency metric is  needed since it gives lower correlation with human ratings in Table 4. It would be better to add more discussion on why it can still be considered to be adapted.\n\n\n4. Minor weakness:\nFigure 1(a) is not easy to understand. The <sep> tokens used do not clearly show where the entity starts and ends. It is suggested that you prefer using <sep> and </sep> pair, where the former shows the beginning of the entity and the latter shows the end.", "summary_of_the_paper": "Considering little analysis that has been done on analyzing the ability of pre-trained language models (LMs) to maintain entity coherence and consistency, the authors take narrative generation as a case study and analyse the long-range entity coherence and consistency in generated stories. They firstly quantify the limitations of current LMs by proposing a set of automatic metrics for measuring model performance in terms of entity usage, and propose an augmented pre-trained LM with a dynamic entity memory in an end-to-end manner by using an auxiliary entity-related loss for guiding the reads and writes to the memory, which increases both entity  coherence and consistency in the generated stories.", "summary_of_the_review": "Overall, the paper is well written and organized. The studied problem is very important which makes the paper to have many audiences at ICLR and the proposed techniques can benefit the field however after performing few more experiments to support them (see Weaknesses section Point 2 and 3).  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "l64HQEfAdu3", "reviewer": "Reviewer_XC2i", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\nThe motivation of the work which is relevant to a well-known issue in existing long text generative models has been written clearly and figures are very helpful in this regard. Author try to contribute in two main problems first proposing automatic metrics to measure and analyze the coherence of generated texts by existing LMs and second proposing a model to improve the coherence of generated texts by augmenting an entity memory. Different experiments and ablation studies show the effects of each component in the model from different perspectives.\n\nCons:\nThere are some not clear sections that need to be explained more clearly. Here are a set of questions and comments that addressing them can be beneficial in better understanding the paper.\n\nMain question is regarding different forms of the same entity and how they are handled in the model. Assume we have one entity but in two different forms such as mom and mother throughout the plots, according to what has been explained in the paper model considers them as two separate entities. If this is true, what is the impact of language different forms of words on the model's performance?\n\nFor automatic evaluation of coherence of texts the C metric only considers the first and last happening location of the entities in the text, while this is very abstract level and can not be a good and fair representative of the coherence in the text level. There are some recent works that try to assess the coherence of the generated narratives by using the plots and manipulating them (Ghazarian et al. 2021). In order to show the positive impact of proposed evaluation metric it is recommended to compare with more recent works.\n\n\nIn the method section, since the main idea of the work is about entity memory, presenting some instances for plots/entities/attributes  can help author to better follow the steps and the proposed model.\n\nAccording to table 1, if the model can have access to the whole context then there is not that much difference between the proposed model and the vanilla LMs? Therefore if we use LMs that can process long texts such as longformer then we won't have benefits coming from the entity coherence proposed by this model?\n\nFor human evaluation, how do you ensure the quality of the annotations are good? what are the agreements between human judgments? how many annotations have been collected for each narrative?\n\n\n\n\n", "summary_of_the_paper": "In this paper, the main contributions are 1)analyze the entity coherence of  pretrained LMs by leveraging some existing metrics and also four newly proposed metrics 2) propose a new generative model to have a higher quality generated narratives from entity coherence and consistency perspectives. Authors develop a model by adding a dynamic entity memory to the existing pretrained LMs. The cross attention between input text and entity memory alongside the self-attention coming from pretrained-LM helps to not forget the previous entities and their attributes during generating narratives.", "summary_of_the_review": "In summary the idea of using entity memory and computing cross attention between input and this memory to incorporate information from entities and their attributes looks promising. However, the metrics showing the coherence of the text is only limited to the location of the entities while for the text the overall coherence of the text should also be considered. Some sections are opaque but I think if authors can address those the paper can be effective for the community.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "F0Ek0N0PRUi", "reviewer": "Reviewer_sZJR", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n(1) The work is to address a challenging and practical problem in generating long narratives. \n(2) The authors proposed two intuitive metrics to measure the entity coherence and consistency based on their mentions and context, which are beneficial to the following research. \n(3) The proposed entity memory augmented model is reasonable and improved entity coherence/consistency in some of the experiments. The authors also provided a detailed analysis of the models in various settings.\n\nWeakness:\n(1) The two proposed metrics about entity coherence and consistency are intuitive but not rigorous. The entity coherence is based on the occurrence of the entity mentions across different sections. However, the coreference is accurate especially when the context is wide, and even an entity is mentioned across a wide context, there are a lot of other factors to consider to quantify the coherence, e.g., the intermediate mentions used in-between the first and the last mention. Similarly, for consistency, the authors considered all verbs and adjectives as the attributes of the entity. What's the reason? and how can it measure the inconsistency in the example of Figure1 (b)?\n(2) The model design is reasonable, but it depends on several key hyperparameters, e.g., the size of the entity memory or the context memory, the length of each narrative section. They may have much impact on the final performance and on different datasets, they should have different choices. Also, it seems the entity set is given to initialize the memory, then what if the model generates new entities?\n(3) The results didn't consistently demonstrate the improvement of the proposed approach, e.g., in Table 2, on the WikiPlots dataset, the VanillaLM is better than other models for most of the metrics; and on the WritingPrompts dataset, the VanillaLM seems to generate more mentions in average.\n(4) The analysis section is very difficult to follow. Some analysis is just based on wordy descriptions without details or examples. For example, in the automatic evaluation section, it's not clear how the uncertainty is computed, how to interpret the Figures, especially Figure 4, what does the control in section 5.2 mean? and what are the detailed criteria for human rating (1-4)?", "summary_of_the_paper": "This work focuses on improving the long-range entity coherence and consistency when generating long narrative stories. The authors proposed two metrics to measure the entity coherence and consistency in terms of entity usage in generated narratives. They also propose to augment a pre-trained LM with a dynamic entity memory and cross-attention to improve the entity usage.", "summary_of_the_review": "Overall, this work proposed some interesting ideas to analyze and improve the entity usage in long narrative generation task. However, the metrics need to be carefully designed, and the analysis needs to be improved.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2gWdWN4Kcp9", "reviewer": "Reviewer_Q7Fu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths and Weakness:\n1. The proposed approach for tackling entity consistency and coherency through the use of memory structures that augment language modeling is novel and interesting. Further, the dynamic nature of the proposed entity memory network allows multiple attributes to be associated with an entity.\n2. One of the contributions of this work is the proposed evaluation metric for entity coherence and consistency. The metric for entity coherence based on calculating C_i as the difference in the indices of first and last mentions does not truly capture coherence. Further, the example provided in the paper Fig 1b seems to be an issue with local coherence instead of long-range entity coherence. \n3. With regards to entity consistency, the authors propose quantifying it through the use of attributes associated with each entity is interesting. However, this approach does not take into consideration multiple entities in a sentence and attribution association in those cases.\n4. Evaluation is done on the standard datasets of writing prompts and wikiplots. However, it would have been better if the authors had used more baselines to compare their proposed approaches instead of a simple LM for both datasets.  From the automated metric perspective, the proposed addition of entity network seems to be performing worse than a vanilla LM on the wikiplots datasets on the metrics as shown in Table 1. This trend of vanilla LM outperforming the proposed approach is evident in the proposed automated metrics in Table 2.\n5. The proposed automated metrics when compared to the human evaluation show a low to moderate correlation to human judgments. The correlation for consistency is pretty low which raises questions about the proposed metric and its effectiveness to capture entity consistency.\n\nQuestions:\n1. With regards to entity consistency, how were the attributes assigned if there are multiple entities mentioned in the sentence with attributes?\n2.  Why does the attribute consistency on the wikiplots dataset using a simple vanilla LM outperform the proposed entity memory network augmented LM?\n3. why is the scale for coherence different from the other metrics?\n4. How many human annotators evaluated the generated outputs? What is the IAA?", "summary_of_the_paper": "The authors of this paper propose to tackle the problem of coherency and the consistent use of entities for the task of narrative generation. To achieve this, the authors propose an approach of augmenting pretrained language models with entity memory and cross attention blocks that places an emphasis on the entities in the narrative. The proposed approach is evaluated on the WritingPrompts and the WikiPlots datasets. From an evaluation perspective, the authors propose two new evaluation metrics for measuring entity coherence and consistency. To measure entity coherence, the authors propose dividing the narrative into L equal sections and computing the maximum interval span of each entity as the difference between first and last mention respectively. Similarly for entity consistency, the authors propose measuring entity consistency through the concept of measuring attribute consistency within the narrative.\n\nContributions:\n1. A new approach of augmenting language models with entity memory that helps in entity consistency and coherence across narrative generation tasks\n2. Two new metrics to evaluate consistency and coherency in the generated outputs.\n", "summary_of_the_review": "This paper tackles the problem of entity coherence and consistency in narrative generation and tackles the problem through the use of an entity memory network that augments the pretrained language model. This proposed approach is interesting and demonstrates the effectiveness of the proposed approach on two datasets. The clear gains of the proposed approach can only be seen on one of the datasets compared to a simple baseline. Further, the authors also propose using two new metrics for automated evaluation. These metrics show a low to moderate correlation to human judgments.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work analyzes the ability of pre-trained language models to maintain entity coherence and consistency in long narrative generation. Along with new automatic metrics for analyzing narrative generation, it proposes a memory-augmented model that allows tracking entities to improve narrative generation.  Although all the reviewers appreciated the importance of the problem, the novelty of the proposed approach, as well as empirical improvements in a subset of experiments, they also acknowledge several major weaknesses including the lack of rigor in defining the method, the lack of clarity in writing (especially in the experiments section), insufficiently strong baselines, and an issue of reproducibility since the code cannot be released. These concerns were in part addressed during rebuttal, but not enough to accept the paper.
Decision: Reject