Venue: NAACL 2022

# Persona-Guided Planning for Controlling the Protagonist's Persona in Story Generation

Zhexin Zhang, Jiaxin Wen1, Jian Guan, Minlie Huang2

The CoAI group, DCST; Institute for Artificial Intelligence; State Key Lab of Intelligent Technology and Systems; Beijing National Research Center for Information Science and Technology; Tsinghua University, Beijing 100084, China.

{zx-zhang18, wenjx17, j-guan19}@mails.tsinghua.edu.cn aihuang@tsinghua.edu.cn

Footnote 1: Equal contribution.

###### Abstract

Endowing the protagonist with a specific personality is essential for writing an engaging story. In this paper, we aim to control the protagonist's persona in story generation, i.e., generating a story from a leading context and a persona description, where the protagonist should exhibit the specified personality through a coherent event sequence. Considering that personas are usually embodied implicitly and sparsely in stories, we propose a planning-based generation model named ConPer to explicitly model the relationship between personas and events. ConPer first plans events of the protagonist's behavior which are motivated by the specified persona through predicting one target sentence, then plans the plot as a sequence of keywords with the guidance of the predicted persona-related events and commonsense knowledge, and finally generates the whole story. Both automatic and manual evaluation results demonstrate that ConPer outperforms state-of-the-art baselines for generating more coherent and persona-controllable stories. Our code is available at [https://github.com/thu-coai/ConPer](https://github.com/thu-coai/ConPer).

+
Footnote †: Corresponding author.

## 1 Introduction

Stories are important for entertainment. They are made engaging often by portraying animated and believable characters since a story plot unfolds as the characters interact with the object world created in the story Young (2000). Cognitive psychologists determined that the ability of an audience to comprehend a story is strongly correlated with the characters' believability Graesser et al. (1991). And the believability mostly depends on whether the characters' reaction to what has happened and their deliberate behavior accord with their personales (e.g., weakness, abilities, occupations) Madsen and Nielsen (2009); Riedl and Young (2010). Furthermore, previous studies have also stressed the importance of personas in stories to maintain the interest of audience and instigate their sense of empathy and relatedness Cavazza et al. (2009); Chandu et al. (2019). However, despite the broad recognition of its importance, it has not yet been widely explored to endow characters with specified personalities in story generation.

In this paper, we present the first study to impose free-form controllable persona on story generation. Specifically, we require generation models to generate a coherent story, where the protagonist should exhibit the desired personality. We focus on controlling the persona of only the protagonist of a story in this paper and leave the modeling of personas of multiple characters for future work. As exemplified in Table 1, given a context to present the story settings including characters, location, problems (e.g., "Boruc" was suffering

\begin{table}
\begin{tabular}{p{365.9pt}} \hline \hline
**Context:** ... Artur Borue, a Polish national pilot, was going to get the group infiltrated into the area and prepared for the attack. \\ Mental began to tear through the winings outside the small windows and flipped heavily off the untoride of the plane, a quiet arrival in Poland wasn’t going to be an option anymore. \\ \hline
**Persona A:** [Burch] a skilled pilot, trained in operating fight controls on the most common planes. \\ \hline
**Generated Story by ConPer Conditioned on Persona A:** ... He had been doing much before, almost a week ago. He took a long pull of the article, checked his controls, and made a quick mental note of the exact sequence of instructions. He knew that he couldn’t be sure if this would be safe for much longer.. \\ \hline
**Persona B:** [Burch] a skilled pilot, and never trained in operating light controls. \\ \hline
**General Story by ConPerConditioned on Persons B:** ... He caused as the plate suffered a complete tablure and if a way had caused it to come to a stop. & He noted fewer room before so she didn’t know how to plot in this situation and his experience of the controls had not been good either ... \\ \hline \hline \end{tabular}
\end{table}
Table 1: An example for controlling the protagonist’s persona in story generation. The **Context** and **Persona A** are sampled from the storium dataset Akoury et al. (2020). The protagonist’s name is shown in the square bracket. And we manually write **Persona B** based on **Persona A**. We highlight the sentences which embody the given persons in red.



from a plane crash) and a persona description, the model should generate a coherent story to exhibit the persona (e.g., what happened when "Boruc" was "skilled" or "unskilled"). In particular, we require the model to embody the personality of the protagonist implicitly through his actions (e.g., "checked his controls" for the personality "skilled pilot"). Therefore, the modeling of relations between persona and events is the **first challenge** of this problem. Then, we observe that only a small amount of events in a human-written story relate to personas directly and the rest serve for explaining the cause and effect of these events to maintain the coherence of the whole story. Accordingly, the **second challenge** is learning to plan a coherent event sequence (e.g., first "finding the plane shaking", then "checking controls", and finally "landing safely") to embody personas naturally.

In this paper, we propose a generation model named ConPer to deal with _Cont_rolling _Persona_ of the protagonist in story generation. Due to the persona-sparsity issue that most events in a story do not embody the persona, directly fine-tuning on real-world stories may mislead the model to focus on persona-unrelated events and regard the persona-related events as noise Zheng et al. (2020). Therefore, before generating the whole story, ConPer first plans persona-related events through predicting one target sentence, which should be motivated by the given personality following the leading context. To this end, we extract persona-related events that have a high semantic similarity with the persona description in the training stage. Then, ConPer plans the plot as a sequence of keywords to complete the cause and effect of the predicted persona-related events with the guidance of commonsense knowledge. Finally, ConPer generates the whole story conditioned on the planned plot. The stories are shown to have better coherence and persona-consistency than state-of-the-art baselines.

We summarize our contributions as follows:

**I.** We propose a new task of controlling the personality of the protagonist in story generation.

**II.** We propose a generation model named ConPer to impose specified persona into story generation by planning persona-related events and a keyword sequence as intermediate representations.

**III.** We empirically show that ConPer can achieve better controllability of persona and generate more coherent stories than strong baselines.

## 2 Related Work

Story GenerationThere have been wide explorations for various story generation tasks, such as story ending generation Guan et al. (2019), story completion Wang and Wan (2019) and story generation from short prompts Fan et al. (2018), titles Yao et al. (2019) or beginnings Guan et al. (2020). To improve the coherence of story generation, prior studies usually first predicted intermediate representations as plans and then generated stories conditioned on the plans. The plans could be a series of keywords Yao et al. (2019), an action sequence Fan et al. (2019); Goldfarb-Tarrant et al. (2020) or a keyword distribution Kang and Hovy (2020). In terms of character modeling in stories, some studies focused on learning characters' persona as latent variables Bamman et al. (2013, 2014) or represented characters as learnable embeddings Ji et al. (2017); Clark et al. (2018); Liu et al. (2020). Chandu et al. (2019) proposed five types of specific personas for visual story generation. Brahman et al. (2021) formulated two new tasks including character description generation and character identification. In contrast, we focus on story generation conditioned on personas in a free form of text to describe one's strengths, weaknesses, abilities, occupations and goals.

Controllable Generation Controllable text generation aims to generate texts with specified attributes. For example, Keskar et al. (2019) pretrained a language model conditioned on control codes of different attributes (e.g., domains, links). Dathathri et al. (2020) proposed to combine a pre-trained language model with trainable attribute classifiers to increase the likelihood of the target attributes. Recent studies in dialogue models focused on controlling through sentence functions Ke et al. (2018), politeness Niu and Bansal (2018) and conversation targets Tang et al. (2019). For storytelling, Brahman et al. (2020) incorporated additional phrases to guide the story generation. Brahman and Chaturvedi (2020) proposed to control the emotional trajectory in a story by regularizing the generation process with reinforcement learning. Rashkin et al. (2020) generated stories from outlines of characters and events by tracking the dynamic plot states with a memory network.

A similar research to ours is Zhang et al. (2018), which introduced the PersonaChat dataset for endowing the chit-chat dialogue agents with a consis 

tent persona. However, dialogues in PersonaChat tend to exhibit the given personas explicitly (e.g., the agent says "I am terrified of dogs" for the persona "I am afraid of dogs"). For quantitative analysis, we compute the ROUGE score Lin (2004) between the persona description and the dialogue or story. We find that the rouge-2 score is 0.1584 for PersonaChat and 0.018 for our dataset (i.e., storium). The results indicate that exhibiting personas in stories requires a stronger ability to associate the action of a character and his implicit traits compared with exhibiting personas in dialogues.

Commonsense KnowledgeRecent studies have demonstrated that incorporating external commonsense knowledge significantly improved the coherence and informativeness for dialog generation Zhou et al. (2018); Zhong et al. (2020), story ending generation Guan et al. (2019), essay generation Yang et al. (2019), story generation Guan et al. (2020); Xu et al. (2020); Mao et al. (2019) and story completion Ammanabrolu et al. (2021). These studies usually retrieved a static local knowledge graph which contains entities mentioned in the input, and their related entities. We propose to incorporate the knowledge dynamically during generation to better model the keyword transition in a long-from story.

## 3 Methodology

We define our task as follows: given a context \(X=(x_{1},x_{2},\cdots,x_{|X|})\) with \(|X|\) tokens, and a persona description for the protagonist \(P=(p_{1},p_{2},\cdots,p_{l})\) of length \(l\), the model should generate a coherent story \(Y=(y_{1},y_{2},\cdots,y_{|Y|})\) of length \(|Y|\) to exhibit the persona. To tackle the problem, the popular generation model such as GPT2 commonly employ a left-to-right decoder to minimize the negative log-likelihood \(\mathcal{L}_{ST}\) of human-written stories:

\[\mathcal{L}_{ST} =-\sum_{t=1}^{|Y|}\text{log}P(y_{t}|y_{<t},S), \tag{1}\] \[P(y_{t}|y_{<t},S) =\text{softmax}(\textbf{s}_{t}\boldsymbol{W}+\boldsymbol{b}),\] (2) \[\textbf{s}_{t} =\texttt{Decoder}(y_{<t},S), \tag{3}\]

where \(S\) is the concatenation of \(X\) and \(P\), \(\textbf{s}_{t}\) is the decoder's hidden state at the \(t\)-th position of the story, \(\boldsymbol{W}\) and \(\boldsymbol{b}\) are trainable parameters. Based on this framework, we divide the training process of ConPer into three stages as shown in Figure 1.

### Target Planning

We observe that most sentences in a human-written story do not aim to exhibit any personas, but serve to maintain the coherence of the story. Fine-tuning on these stories directly may mislead the model to regard input personas as noise and focus on modeling the persona-unrelated events which are in the majority. Therefore, we propose to first predict persona-related events (i.e., the target) before generating the whole story.

We use an automatic approach to extract the target from a story since there is no available manual annotation. Specifically, we regard the sentence as the target which has the highest semantic similarity with the persona description. We consider only one sentence as the target in this work due to the persona-sparsity issue, and we also present the

Figure 1: Model overview of ConPer. The training process is divided into the following three stages: (a) Target Planning: planning persona-related events (called “target” for short); (b) Plot Planning: planning a keyword sequence as an intermediate representation of the story with the guidance of the target and a dynamically growing local knowledge graph; And (c) Story Generation: generating the whole story conditioned on the input and plans.

 

result of experimenting with two sentences as the target in the appendix B.1. More explorations of using multiple target sentences are left as future work. We adopt NLTK Bird et al. (2009) for sentence tokenization. And we measure the similarity between sentences using BERTScore\({}_{\text{Recall}}\)Zhang et al. (2019) with RoBERTa\({}_{\text{Large}}\)Liu et al. (2019) as the backbone model. Let \(T=(\tau_{1},\tau_{2},\cdots,\tau_{t})\) denote the target sentence of length \(\iota\), which should be a sub-sequence of \(Y\). Formally, the loss function \(\mathcal{L}_{TP}\) for this stage can be derived as follows:

\[\mathcal{L}_{TP}=-\sum_{t=1}^{\iota}\text{log}P(\tau_{t}|\tau_{<t},S). \tag{4}\]

In this way, we exert explicit supervision to encourage the model to condition on the input personas.

### Plot Planning

At this stage, ConPer learns to plan a keyword sequence for subsequent story generation Yao et al. (2019). Plot planning requires a strong ability to model the causal and temporal relationship in the context for expanding a reasonable story plot (e.g., associating _"unskilled"_ with _"failure"_ for the example in Table 1), which is extremely challenging without any external guidance, for instance, commonsense knowledge. In order to plan a coherent event sequence, we introduce a dynamically growing local knowledge graph, a subset of the external commonsense knowledge base ConceptNet Speer et al. (2017), which is initialized to contain triples related to the keywords mentioned in the input and target. When planning the next keyword, ConPer combines the knowledge information from the local graph and the contextualized features captured by the language model with learnable weights. Then ConPer grows the local graph by adding the knowledge triples neighboring the predicted keyword. Formally, we denote the keyword sequence as \(W=(w_{1},w_{2},\cdots,w_{k})\) of length \(k\) and the local graph as \(\mathcal{G}_{t}\) for predicting the keyword \(w_{t}\). The loss function \(\mathcal{L}_{KW}\) for generating the keyword sequence is as follows:

\[\mathcal{L}_{KW}=-\sum_{t=1}^{k}\text{log}P(w_{t}|w_{<t},S,T,\mathcal{G}_{t}). \tag{5}\]

Keyword ExtractionWe extract words that relate to emotions and events from each sentence of a story as keywords for training, since they are important for modeling characters' evolving psychological states and their behavior. We measure the emotional tendency of each word using the sentiment analyzer in NLTK, which predicts a distribution over four basic emotions, i.e., _negative_, _neutral_, _positive_, and _compound_. We regard those words as related to emotions whose scores for _negative_ or _positive_ are larger than \(0.5\). Secondly, we extract and lemmatize the nouns and verbs (excluding stopwords) from a story as event-related keywords with NLTK for POS-tagging and lemmatization. Then we combine the two types of keywords in the original order as the keyword sequence for planning. We limit the number of keywords extracted from each sentence in stories up to 5, and we ensure that there is at least one keyword for a sentence by randomly choosing one word if no keywords are extracted. We don't keep this limitation when extracting keywords from the leading context and the persona description, since these keywords are only used to initialize the local knowledge graph.

Incorporating KnowledgeWe introduce a dynamically growing local knowledge graph for plot planning. For each example, we initialize the graph \(\mathcal{G}_{1}\) as a set of knowledge triples where the keywords in \(S\) and \(T\) are the head or tail entities, and then update \(\mathcal{G}_{t}\) to \(\mathcal{G}_{t+1}\) by adding triples related with the generated keyword \(w_{t}\) at \(t\)-th step. Then, the key problem at this stage is representing and utilizing the local graph for next keyword prediction.

The local graph consists of multiple sub-graphs, each of which contains all the triples related with a keyword denoted as \(\varepsilon_{i}=\{(h_{n}^{i},r_{n}^{i},t_{n}^{i})|h_{n}^{i}\in\mathcal{V},r_{n}^ {i}\in\mathcal{R},t_{n}^{i}\in\mathcal{V}\}|_{n=1}^{N}\), where \(\mathcal{R}\) and \(\mathcal{V}\) are the relation set and entity set of ConceptNet, respectively. We derive the representation \(\textbf{g}_{i}\) for \(\varepsilon_{i}\) using graph attention Zhou et al. (2018b) as follows:

\[\textbf{g}_{i} =\sum_{n=1}^{N}\alpha_{n}[\textbf{h}_{n}^{i};\textbf{t}_{n}^{i}] \tag{6}\] \[\alpha_{n} =\frac{\text{exp}(\beta_{n})}{\sum_{j=1}^{N}\text{exp}(\beta_{j})},\] (7) \[\beta_{n} =(\boldsymbol{W}_{r}\mathbf{r}_{n}^{i})^{T}\text{tanh}( \boldsymbol{W}_{h}\textbf{h}_{n}^{i}+\boldsymbol{W}_{t}\mathbf{t}_{n}^{i}), \tag{8}\]

where \(\boldsymbol{W}_{h},\boldsymbol{W}_{r}\) and \(\boldsymbol{W}_{t}\) are trainable parameters, \(\textbf{h}_{n}^{i},\textbf{r}_{n}^{i}\) and \(\textbf{t}_{n}^{i}\) are learnable embedding representations for \(h_{n}^{i},r_{n}^{i}\) and \(t_{n}^{i}\), respectively. We use the same BPE tokenizer Radford et al. (2019) with the language model to tokenize the head and tail entities, which may lead to multiple sub-words for an entity.

Therefore, we derive \(\textbf{h}_{n}^{i}\) and \(\textbf{t}_{n}^{i}\) by adding the 

embeddings of all the sub-words. And we initialize the relation embeddings randomly.

After obtaining the graph representation, we predict the distribution of the next keyword by dynamically deciding whether to select the keyword from the local graph as follows:

\[P(w_{t}|w_{<t},S,T,\mathcal{G}_{t})=\gamma_{t}P^{t}_{k}+(1-\gamma_{t})P^{t}_{l}, \tag{9}\]

where \(\gamma_{t}\in\{0,1\}\) is a binary learnable weight, and \(P^{t}_{l}\) is a distribution over the whole vocabulary while \(P^{t}_{k}\) is a distribution over the entities in \(\mathcal{G}_{t}\). We incorporate the knowledge information implicitly for computing both distributions:

\[P^{t}_{k} =\text{softmax}(\mathbf{W}_{k}[\textbf{s}_{t};\textbf{c}_{t}]+\mathbf{b} _{k}), \tag{10}\] \[P^{t}_{l} =\text{softmax}(\mathbf{W}_{l}[\textbf{s}_{t};\textbf{c}_{t}]+\mathbf{ b}_{l}), \tag{11}\]

where \(\mathbf{W}_{k},\mathbf{b}_{k},\mathbf{W}_{p}\) and \(\mathbf{b}_{p}\) are trainable parameters, and \(\textbf{c}_{t}\) is a summary vector of the knowledge information by attending on the representations of all the sub-graphs in \(\mathcal{G}_{t}\), formally as follows:

\[\textbf{c}_{t} =\sum_{n=1}^{N}\alpha_{n}\textbf{g}_{n}, \tag{12}\] \[\alpha_{n} =\text{softmax}(\textbf{s}_{t}^{T}\mathbf{W}_{g}\textbf{g}_{n}). \tag{13}\]

where \(\mathbf{W}_{g}\) is a trainable parameter. During training process, we set \(\gamma_{t}\) to the ground-truth label \(\hat{\gamma}_{t}\). During generation process, we decide \(\gamma_{t}\) by deriving the probability \(p_{t}\) of selecting an entity from the local graph as the next keyword. And we set \(\gamma_{t}\) to \(1\) if \(p_{t}<0.5\) otherwise 0. We compute \(p_{t}\) as follows:

\[p_{t}=\text{sfingoid}(\mathbf{W}_{p}[\textbf{s}_{t};\textbf{c}_{t}]+\mathbf{b}_{p}), \tag{14}\]

where \(\mathbf{W}_{p}\) and \(\mathbf{b}_{p}\) are trainable parameters. We train the classifier with the standard cross entropy loss \(\mathcal{L}_{C}\) derived as follows:

\[\mathcal{L}_{C}=-\big{(}\hat{\gamma}_{t}\text{log}p_{t}+(1-\hat{\gamma}_{t}) \text{log}(1-p_{t})\big{)}, \tag{15}\]

where \(\hat{\gamma}_{t}\) is the ground-truth label. In summary, the overall loss function \(\mathcal{L}_{PP}\) for the plot planning stage is computed as follows:

\[\mathcal{L}_{PP}=\mathcal{L}_{KW}+\mathcal{L}_{C}. \tag{16}\]

By incorporating commonsense knowledge for planning, and dynamically updating the local graph, ConPer can better model the causal and temporal relationship between events in the context.

Target GuidanceIn order to further improve the coherence and the persona-consistency, we propose to exert explicit guidance of the predicted target on plot planning. Specifically, we expect ConPer to predict keywords close to the target in semantics. Therefore, we add a bias term \(\textbf{d}^{t}_{k}\) and \(\textbf{d}^{t}_{l}\) into Equation 10 and 11, respectively, formally as follows:

\[P^{t}_{k} =\text{softmax}(\mathbf{W}_{k}[\textbf{s}_{t};\textbf{c}_{t}]+\mathbf{b} _{k}+\textbf{d}^{t}_{k}), \tag{17}\] \[\textbf{d}^{t}_{k} =[\textbf{s}_{\text{tar}};\textbf{c}_{t}]^{T}\mathbf{W}_{d}\textbf{E} _{k}+\mathbf{b}_{d},\] (18) \[\textbf{s}_{\text{tar}} =\frac{1}{\iota}\sum_{t=1}^{\iota}\textbf{s}_{|\tau_{t}|}, \tag{19}\]

where \(\mathbf{W}_{d}\) and \(\mathbf{b}_{d}\) are trainable parameters, \(\textbf{s}_{\text{tar}}\) is the target representation computed by averaging the hidden states at each position of the predicted target, and \(\textbf{E}_{k}\) is an embedding matrix, each row of which is the embedding for an entity in \(\mathcal{G}_{t}\). The modification for Equation 11 is similar except that we compute the bias term \(\textbf{d}^{t}_{l}\) with an embedding matrix \(\textbf{E}_{l}\) for the whole vocabulary.

### Story Generation

After planning the target \(T\) and the keyword sequence \(W\), we train ConPer to generate the whole story conditioned on the input and plans with the standard language model loss \(\mathcal{L}_{ST}\). Since we extract one sentence from a story as the target, we do not train ConPer to regenerate the sentence in the story generation stage. And we insert a special token Target in the story to specify the position of the target during training. In the inference time, ConPer first plans the target and plot, then generates the whole story, and finally places the target into the position of Target.

## 4 Experiments

### Dataset

We conduct the experiments on the storium dataset (Akoury et al., 2020). storium contains nearly 6k long-form stories and each story unfolds through a series of scenes with several shared characters. A scene consists of multiple short scene entries, each of which is written to either portray one character with annotation for his personality (i.e., the _"card"_ in storium), or introduce new story settings (e.g., problems, locations) from the perspective of the narrator. In this paper, we concatenate all entries from the same scene since a scene can be seen as an independent story. And we regard a scene entry written for a certain character as the 

target output, the personality of the character as the persona description, and the previous entries written for this character or from the perspective of the narrator in the same scene as the leading context. We split the processed examples for training, validation and testing based on the official split of storium. We retain about 1,000 words (with the correct sentence boundary) for each example due to the length limit of the pretrained language model.

At the plot planning stage, we retrieve a set of triples from ConceptNet Speer et al. (2017) for each keyword extracted from the input or generated by the model. We only retain those triples of which both the head and tail entity contain one word and occur in our dataset, and the confidence score of the relation (annotated by ConceptNet) is more than 1.0. The average number of triples for each keyword is 33. We show more statistics in Table 2.

### Baselines

We compare ConPer with following baselines. **(1) ConvS2S:** It directly uses a convolutional seq2seq model to generate a story conditioned on the input Gehring et al. (2017). **(2) Fusion:** It generates a story by first training a convolutional seq2seq model, and then fixing the model and initializing another trainable convolutional seq2seq model with its parameters. Then the two models are trained together by a fusion mechanism Fan et al. (2018). **(3) Plan&Write:** It first plans a keyword sequence conditioned on the input, and then generates a story based on the keywords Yao et al. (2019). **(4) GPT\({}_{\rm Scr}\):** It has the same network architecture with GPT2 but is trained on our dataset from scratch without any pretrained parameters. **(5) GPT\({}_{\rm Ft}\):** It is initialized using pretrained parameters, and then fine-tuned on our dataset with the standard language modeling objective. **(6) PlanAhead:** It first predicts a keyword distribution conditioned upon the input, and then generates a story by combining the language model prediction and the keyword distribution with a gate mechanism Kang and Hovy (2020). We remove the sentence position embedding and the auxiliary training objective (next sentence prediction) used in the original paper for fair comparison.

Furthermore, we evaluate the following ablated models to investigate the influence of each component: **(1) ConPer w/o KG:** removing the guidance of the commonsense knowledge in the plot planning stage. **(2) ConPer w/o TG:** removing target guidance in the plot planning stage. **(3) ConPer w/o PP:** removing the plot planning stage, which means the model first plans a target sentence and then directly generates the whole story. **(4) ConPer w/o TP:** removing the target planning stage, which also leads to the removal of target guidance in the plot planning stage.

### Experiment Settings

We build ConPer based on GPT2 Radford et al. (2019), which is widely used for story generation Guan et al. (2020). We concatenate the context and the persona description with a special token as input for each example. For fair comparison, we also add special tokens at both ends of the target sentence in a training example for all baselines. We implement the non-pretrained models based on the scripts provided by the original papers, and the pretrained models based on the public checkpoints and codes of HuggingFace's Transformers*. And we set all the pretrained models to the base version due to limited computational resources. We set the batch size to 8, the initial learning rate of the AdamW optimizer to 5e-5, and the maximum training epoch to 5 with an early stopping mechanism. And we generate stories using top-\(p\) sampling with \(p=0.9\)Holtzman et al. (2019). We apply these settings to all the GPT-based models, including GPT\({}_{\rm Scr}\), GPT\({}_{\rm Ft}\), PlanAhead, ConPer and its ablated models. As for ConvS2S, Fusion and Plan&Write, we used the settings from their respective papers and codebases.

Footnote *: [https://github.com/huggingface/transformers](https://github.com/huggingface/transformers)

### Automatic Evaluation

MetricsWe adopt the following automatic metrics for evaluation on the test set. **(1) BLEU (B-n):** We use \(n=1,2\) to evaluate \(n\)-gram overlap

\begin{table}
\begin{tabular}{l c c c} \hline \hline  & **Train** & **Valid** & **Test** \\ \hline
**\# Examples** & 47,910 & 6,477 & 6,063 \\
**Avg. Context Length** & 332.7 & 324.8 & 325.7 \\
**Avg. Description Length** & 23.8 & 22.7 & 24.6 \\
**Avg. Story Length** & 230.5 & 225.7 & 234.3 \\ \hline
**Avg. Target Length** & 21.8 & 21.7 & 22.2 \\
**Avg. \# Keywords Input** & 101.1 & 99.2 & 99.6 \\
**Avg. \# Keywords (Story)** & 31.2 & 30.5 & 31.5 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Dataset statistics. We compute the length by counting tokens using the BPE tokenizer of GPT2. Keywords are extracted either from the input to initialize the local graph, or from the story to train the model for plot planning.

 

between generated and ground-truth stories (Papineni et al., 2002). **(2) BERTScore-target (BS-t):** We use BERTScoreRecall (Zhang et al., 2019) to measure the semantic similarity between the generated target sentence and the persona description. A higher result indicates the target embodies the persona better. **(3) BERTScore-max (BS-m)**: It computes the maximum value of BERTScore between each sentence in the generated story and the persona description. **(4) Persona-Consistency (PC):** It is a learnable automatic metric (Guan and Huang, 2020). We fine-tune RoBERTaR BASE on the training set as a classifier to distinguish whether a story exhibits a consistent persona with a persona description. We regard the ground-truth stories as positive examples where the stories and the descriptions are consistent, and construct negative examples by replacing the story with a randomly sampled one. After fine-tuning, the classifier achieves an 83.63% accuracy on the auto-constructed test set. Then we calculate the consistency score as the average classifier score of all the generated texts regarding the corresponding input.

ResultTable 4 shows the automatic evaluation results. ConPer can generate more word overlaps with ground-truth stories as shown by higher BLEU scores. And ConPer can better embody the specified persona in the target sentence and the whole story as shown by the higher BS-t and BS-m score. The higher PC score of ConPer also further demonstrate the better exhibition of given personas in the generated stories. As for ablation tests, all the ablated models have lower scores in terms of all metrics than ConPer, indicating the effectiveness of each component. Both ConPer w/o PP and ConPer w/o TP drop significantly in BLEU scores, suggesting that planning is important for generating long-form stories. ConPer w/o TP also performs substantially worse in all metrics than ConPer w/o TG, indicating the necessity of explicitly modeling the relations between persona descriptions and story plots. We also show analysis of target guidance in Appendix C.

### Manual Evaluation

We conduct a pairwise comparison between our model and four strong baselines including PlanAhead, GPT\({}_{2\text{Ft}}\), Fusion and ConvS2S. We randomly sample 100 stories from the test set, and obtain 500 stories generated by ConPer and four baseline models. For each pair of stories (one by ConPer, and the other by a baseline, along with the input), we hire three annotators to give a preference (_win_, _lose_ or _tie_) in terms of _coherence_ (interference relatedness, causal and temporal dependencies) and _persona-consistency_ with the input (exhibiting consistent personas). We adopt majority voting to make the final decisions among three annotators. Note that the two aspects are independently evaluated. We resort to Amazon Mechanical Turk (AMT) for the annotation. As shown in Table 3, ConPer outperforms baselines significantly in coherence and persona consistency.

Furthermore, we used human annotation to eval

\begin{table}
\begin{tabular}{p{71.1pt} p{22.8pt} p{22.8pt} p{22.8pt} p{22.8pt} p{22.8pt}} \hline \hline
**Models** & **B-1** & **B-2** & **BS-t** & **BS-m** & **PC** \\ \hline
**ConvS2S** & 12.5 & 4.7 & 22.2 & 32.8 & 17.1 \\
**Fusion** & 13.3 & 5.0 & 22.7 & 33.3 & 30.8 \\
**Plan\&Write** & 7.2 & 2.8 & 6.2 & 29.7 & 23.6 \\ \hline
**GPT2\({}_{\textsc{Ser}}\)** & 13.3 & 4.8 & 24.7 & 38.0 & 26.6 \\
**GTP2\({}_{\textsc{Fit}}\)** & 13.5 & 4.7 & 26.7 & 37.8 & 39.5 \\
**PlanAhead** & 15.4 & 5.3 & 26.1 & 37.8 & 50.2 \\ \hline
**ConPer** & **19.1** & **6.9** & **32.1** & **41.4** & **59.7** \\
**w/o KG** & 17.4 & 6.3 & 31.6 & 39.7 & 53.4 \\
**w/o TG** & 17.7 & 6.3 & 31.9 & 40.2 & 56.3 \\
**w/o PP** & 14.9 & 5.3 & 32.0 & 40.0 & 46.9 \\
**w/o TP** & 16.4 & 5.8 & 27.8 & 37.7 & 44.9 \\ \hline
**_Growth Truth_** & _N/A_ & _N/A_ & _42.6_ & _42.6_ & _75.2_ \\ \hline \hline \end{tabular}
\end{table}
Table 4: Automatic evaluation results. The best performance is highlighted in **bold**, and the second best is underlined. All results are multiplied by 100.

\begin{table}
\begin{tabular}{c|c c c c|c c c} \hline \hline  & \multicolumn{3}{c|}{**Coherence**} & \multicolumn{3}{c}{**Persona consistency**} \\
**Models** & **Win(\%)** & **Lose(\%)** & **Tie(\%)** & \(\kappa\) & **Win(\%)** & **Lose(\%)** & **Tie(\%)** & \(\kappa\) \\ \hline
**ConPer vs. ConvS2S** & 89.0* & 5.0 & 6.0 & 0.625 & 82.0* & 8.0 & 10.0 & 0.564 \\
**ConPer vs. Fusion** & 71.0* & 23.0 & 6.0 & 0.213 & 61.0* & 22.0 & 17.0 & 0.279 \\
**ConPer vs. GPT2\({}_{\textsc{Pt}}\)** & 54.0* & 18.0 & 28.0 & 0.275 & 53.0* & 11.0 & 36.0 & 0.215 \\
* & 53.0
* & 25.0 & 22.0 & 0.311 & 59.0
* & 28.0 & 13.0 & 0.280 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Manual evaluation results. The scores indicate the percentage of _win_, _lose_ or _tie_ when comparing our model with a baseline. \(\kappa\) denotes Randolph’s kappa to measure the inter-annotator agreement.
* means ConPer outperforms the baseline model significantly with p-value\(<0.01\) (Wilcoxon signed-rank test).
 

uate whether the identified target sentence embodies the given persona. We randomly sampled 100 examples from the test set, and identified the target for each example as the sentence with the maximum BERTScore with the persona description. And we used a random policy as a baseline which randomly samples a sentence from the original story as the target. We hired three annotators on AMT to annotate each example ("Yes" if the sentence embodies the given persona, and "No" otherwise). We adopted majority voting to make the final decision among three annotators. Table 5 shows our method significantly outperforms the random policy in identifying the persona-related sentences.

### Controllability Analysis

To further investigate whether the models can be generalized to generate specific stories to exhibit different personas conditioned on the same context, we perform a quantitative study to observe how many generated stories are successfully controlled as the input persona descriptions change.

Automatic EvaluationFor each example in the test set, we use a model to generate ten stories conditioned on the context of this example and ten persona descriptions randomly sampled from other examples, respectively. We regard a generated story as successfully controlled if the pair of the story and its corresponding persona description (along with the context) has the maximum persona-consistency score among all the ten descriptions. We regard the average percentages of the stories which are successfully controlled in all the ten generated stories for each example in the whole test set as the controllability score of the model. We show the results for ConPer and strong baselines in Table 6. Furthermore, we also compute the superiority (denoted as \(\Delta\)) of the persona-consistency score computed between a generated story and its corresponding description compared to that computed between the story and one of the other nine descriptions Sinha et al. (2020). A larger \(\Delta\) means the model can generate more specific stories adhering to the personas.

As shown in Table 6, there are more stories successfully controlled for ConPer than baselines. And the larger \(\Delta\) of ConPer suggests that it can generate more specific stories to the input personas. The results show the better generalization ability of ConPer to generate persona-controllable stories.

Manual EvaluationFor manual evaluation, we randomly sampled 50 examples from the test set, and manually revised the persona descriptions to exhibit an opposite persona (e.g., from "skilled pilot" to "unskilled pilot"). We required a model to generate two stories conditioned on the original and its opposite persona description, respectively. Finally we obtained 300 stories from three models including GPT2Ft, PlanAhead and ConPer. Then, we hired three graduates to judge whether each story accords with the input persona. All annotators have good English language proficiency and are well trained for this evaluation task. Table 7 shows the evaluation results. We can see that 66% of the stories generated by ConPer are acordant with the input persona, suggesting the better controllability of ConPer.

### Case Study

We present some cases in Table 8. We can see that the story generated by ConPer exhibits the specified persona with a coherent event sequence. The planned keywords by ConPer provide an effective discourse-level guidance for the subsequent story generation, such as tablet, which has a commonsense connection with computer skills and

\begin{table}
\begin{tabular}{l c c} \hline \hline
**Models** & **Controllability Score** & \(\Delta\) \\ \hline
**Plan\&Write** & 10.6 & 0.01 \\
**GPT2Fv** & 24.2 & 11.2 \\
**PlanAhead** & 23.1 & 11.2 \\ \hline
**ConPer** & **29.5** & **15.1** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Automatic evaluation results for the controllability. All results are mulitplied by 100.

\begin{table}
\begin{tabular}{l|c c c} \hline \hline
**Policies** & **Yes (\%)** & **No (\%)** & \(\kappa\) \\ \hline
**Random** & 22.0 & 78.0 & 0.25 \\
**Ours** & 75.0 & 25.0 & 0.36 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Percentages of examples labeled with “Yes” or “No” for whether the identified sentence reflects the given persona. \(\kappa\) denotes Randolph’s kappa to measure the inter-annotator agreement.

 

Internet in the input. In contrast, the baselines tend to not generate any persona-related events. For example, the given persona description emphasizes the strong computer skills of the protagonist while the stories generated by PlanAhead and GPT2 have nothing to do with the computer skills. We further analyze some error cases generated by our model in Appendix G.

## 5 Conclusion

We present ConPer, a planning-based model for a new task aiming at controlling the protagonist's persona in story generation. We propose target planning to explicitly model the relations between persona-related events and input personas, and plot planning to learn the keyword transition in a story with the guidance of predicted personala related events and external commonsense knowledge. Extensive experiments show that ConPer can generate more coherent stories with better consistency with the input personas than strong baselines. Further analysis also indicates the better persona-controllability of ConPer.

### Acknowledgement

This work was supported by the National Science Foundation for Distinguished Young Scholars (with No. 62125604) and the NSFC projects (Key project with No. 61936010 and regular project with No. 61876096). This work was also supported by the Guoqiang Institute of Tsinghua University, with Grant No. 2019GQG1 and 2020GQG0005. This work was also sponsored by Tsinghua-Toyota Joint Research Fund. We would also like to thank the anonymous reviewers for their invaluable suggestions and feedback.

### Ethics Statements

We conduct the experiments by adapting a public story generation dataset storium to our task. Automatic and manual evaluation results show that our model ConPer outperforms existing state-of-the-art models in terms of coherence, consistency and controllability, suggesting the generalization ability of ConPer to different input personas. And our approach can be easily extended to different syntactic levels (e.g., phrase-level and paragraph-level events), different model architectures (e.g., BART Lewis et al., 2020)) and different generation tasks (e.g., stylized long text generation).

In both storium and ConceptNet, we find some potentially offensive words. Therefore, our model may suffer from risks of generating offensive content, although we have not observed such content in the generated results. Furthermore, ConceptNet consists of commonsense triples of concepts, which may not be enough for modeling inter-event relations in long-form stories. We resort to Amazon Mechanical Turk (AMT) for manual evaluation. We do not ask about personal privacy or collect personal information of annotators in the annotation process. We hire three annotators and pay each annotator $0.1 for comparing each pair of stories. The payment is reasonable considering that it

\begin{table}
\begin{tabular}{l l} \hline \hline
**Context:** & \(\cdots\) the group has gathered on the rooftop garden of Miyamoto Mansion \(\cdots\) the TV set out near the long table on \\ the patio is talking about some spree of thefts at low volume \(\cdots\) the issue of Chloe’s disappearance and the missing statue \\ still hang over their heads. \\ \end{tabular} \\
**Persona Description:**[Aito] You are above average in your computer skills. If information is power, then your ability to \\ use the internet makes you one of the most powerful people on the planet. \\ \hline \multicolumn{2}{l}{**GPT2\({}_{\textbf{F1}}\):**_Atio looked at the others, still trying to help find a way out of the hotel. He wasn’t sure what the rest of the group wanted to see if they were going to survive and all knew if he needed to be needed \(\cdots\)_} \\ \hline
**PlanAhead:**Miyamoto Mansion \(\cdots\) perhaps it’s just a bit farther away. The music sounds bright enough but the line of visitors does not. _Aito was once a pretty girl, he had always been quite witty when talking to people but she always found it annoying that a group of tourists looked like trash just to her \(\cdots\)_} \\ \hline
**ConPer:** & \(\cdots\) “Oh, wait \(\cdots\) wait \(\cdots\) people are talking about Chloe?” \(\cdots\) ‘**Ihave a feeling the internet is probably over best chance to get through this**’ \(\cdots\) Aito looked around the table a moment before pulling out her tablet and starting typing \\ furiously into her computer. She looked up at the tablet that had appeared, and she could see that it was working on a number of things\(\cdots\) \\ \hline
**Planned keywords:** & people \(\rightarrow\) look \(\rightarrow\) around \(\rightarrow\) tablet \(\rightarrow\) see \(\cdots\) \\ \hline \hline \end{tabular}
\end{table}
Table 8: Generated stories by different models. _Italic_ words indicate the improper entities or events in terms of the consistency with the input. The **bold** sentence indicate the generated target by ConPer. Red words denote the consistent events adhering to the input. And the extracted keywords are underlined.

 

would cost average one minute for an annotator to finish a comparison.

## References

* Akoury et al. (2020) Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020. Storium: A dataset and evaluation platform for machine-in-the-loop story generation. _arXiv preprint arXiv:2010.01717_.
* Ammanabrolu et al. (2021) Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, and Mark O Riedl. 2021. Automated storytelling via causal, commonsense plot ordering. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 35, pages 5859-5867.
* Bamman et al. (2013) David Bamman, Brendan O'Connor, and Noah A Smith. 2013. Learning latent personas of film characters. In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 352-361.
* Bamman et al. (2014) David Bamman, Ted Underwood, and Noah A Smith. 2014. A bayesian mixed effects model of literary character. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 370-379.
* Bird et al. (2009) Steven Bird, Ewan Klein, and Edward Loper. 2009. _Natural language processing with Python: analyzing text with the natural language toolkit_. " O'Reilly Media, Inc..".
* Brahman and Chaturvedi (2020) Faeze Brahman and Snigdha Chaturvedi. 2020. Modeling protagonist emotions for emotion-aware storytelling. _arXiv preprint arXiv:2010.06822_.
* Brahman et al. (2021) Faeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao, Mrinmaya Sachan, and Snigdha Chaturvedi. 2021. " let your characters tell their story": A dataset for character-centric narrative understanding. _arXiv preprint arXiv:2109.05438_.
* Brahman et al. (2020) Faeze Brahman, Alexandru Petusca, and Snigdha Chaturvedi. 2020. Cue me in: Content-inducing approaches to interactive story generation. In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_, pages 588-597.
* Cavazza et al. (2009) Marc Cavazza, David Pizzi, Fred Charles, Thurid Vogt, and Elisabeth Andre. 2009. Emotional input for character-based interactive storytelling. In _Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 1_, pages 313-320.
* Chandu et al. (2019) Khyathi Chandu, Shrimai Prabhumoye, Ruslan Salakhutdinov, and Alan W Black. 2019. "my way of telling a story": Persona based grounded story generation. In _Proceedings of the Second Workshop on Storytelling_, pages 11-21.
* Clark et al. (2018) Elizabeth Clark, Yangfeng Ji, and Noah A Smith. 2018. Neural text generation in stories using entity representations as context. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 2250-2260.
* Dathathri et al. (2020) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* Fan et al. (2018) Angela Fan, Mike Lewis, and Yann Daquphin. 2018. Hierarchical neural story generation. _arXiv preprint arXiv:1805.04833_.
* Fan et al. (2019) Angela Fan, Mike Lewis, and Yann Daquhin. 2019. Strategies for structuring story generation. _arXiv preprint arXiv:1902.01109_.
* Gehring et al. (2017) Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In _International Conference on Machine Learning_, pages 1243-1252. PMLR.
* Goldfarb-Tarrant et al. (2020) Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph M. Weischedel, and Nanyun Peng. 2020. Content planning for neural story generation with aristotelian rescoring. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_, pages 4319-4338. Association for Computational Linguistics.
* Graesser et al. (1991) Arthur C Graesser, Kathy L Lang, and Richard M Roberts. 1991. Question answering in the context of stories. _Journal of Experimental Psychology: General_, 120(3):254.
* Guan et al. (2020) Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, and Minlie Huang. 2020. A knowledge-enhanced pretraining model for commonsense story generation. _Transactions of the Association for Computational Linguistics_, 8:93-108.
* Guan and Huang (2020) Jian Guan and Minlie Huang. 2020. UNION: an unreferenced metric for evaluating open-ended story generation. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_, pages 9157-9166. Association for Computational Linguistics.
* Guan et al. (2019) Jian Guan, Yansen Wang, and Minlie Huang. 2019. Story ending generation with incremental encoding and commonsense knowledge. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 33, pages 6473-6480.
* Holtzman et al. (2019) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. _arXiv preprint arXiv:1904.09751_ 

Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, and Noah A Smith. 2017. Dynamic entity representations in neural language models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1830-1839. Cited by: SS4.
* D. Kang and E. Hovy (2020)Plan ahead: self-supervised text planning for paragraph completion task. arXiv preprint arXiv:2010.05141. Cited by: SS1.
* P. Ke, J. Guan, M. Huang, and X. Zhu (2018)Generating informative responses with controlled sentence function. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1499-1508. Cited by: SS1.
* N. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher (2019)CTRL: a conditional transformer language model for controllable generation. CoRRabs/1909.05858. External Links: Link, 1909.05858 Cited by: SS1.
* M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer (2020)BART: denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 7871-7880. External Links: Link, Document Cited by: SS1.
* J. Li, M. Galley, C. Brockett, J. Gao, and W. B. Dolan (2016)A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 110-119. Cited by: SS1.
* C. Lin (2004)ROUGE: a package for automatic evaluation of summaries. In Text Summarization Branches Out, Barcelona, Spain, pp. 74-81. Cited by: SS1.
* Z. Lin, D. Cai, Y. Wang, X. Liu, H. Zheng, and S. Shi (2020)The world is not binary: learning to rank with grayscale data for dialogue response selection. arXiv preprint arXiv:2004.02421. Cited by: SS1.
* D. Liu, J. Li, M. Yu, Z. Huang, G. Liu, D. Zhao, and R. Yan (2020)A character-centric neural model for automated story generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, pp. 1725-1732. Cited by: SS1.
* Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019)Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. Cited by: SS1.
* S. Madsen and L. Nielsen (2009)Exploring persona-scenarios-using storytelling to create design ideas. In IFIP Working Conference on Human Work Interaction Design, pp. 57-66. Cited by: SS1.
* H. Mao, B. Majumder, I. McAuley, and G. W. Cottrell (2019)Improving neural story generation by targeted common sense grounding. arXiv preprint arXiv:1908.09451. Cited by: SS1.
* T. Niu and M. Bansal (2018)Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics6, pp. 373-389. Cited by: SS1.
* K. Papineni, S. Roukos, T. Ward, and W. Zhu (2002)BLEu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318. Cited by: SS1.
* A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever (2019)Language models are unsupervised multitask learners. OpenAI blog1 (8), pp. 9. Cited by: SS1.
* H. Rashkin, A. Celikyilmaz, Y. Choi, and J. Gao (2020)Plotmachines: outline-conditioned generation with dynamic plot state tracking. arXiv preprint arXiv:2004.14967. Cited by: SS1.
* M. O. Riedl and R. Michael Young (2010)Narrative planning: balancing plot and character. Journal of Artificial Intelligence Research39, pp. 217-268. Cited by: SS1.
* K. Sinha, P. Parthasarathi, J. Wang, R. Lowe, W. L. Hamilton, and J. Pineau (2020)Learning an unreferenced metric for online dialogue evaluation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 2430-2441. External Links: Link, Document Cited by: SS1.
* R. Speer, J. Chin, and C. Havasi (2017)Conceptnet 5.5: an open multilingual graph of general knowledge. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31. Cited by: SS1.
* J. Tang, T. Zhao, C. Xiong, X. Liang, E. P. Xing, and Z. Hu (2019)Target-guided open-domain conversation. arXiv preprint arXiv:1905.11553. Cited by: SS1.
* T. Wang and X. Wan (2019)T-CVAE: transformer-based conditioned variational autoencoder for story completion. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pp. 5233-5239. External Links: Link, Document Cited by: SS1.
* P. Xu, M. Patwary, M. Shoeybi, R. Puri, P. Fung, Anima Anandkumar, and Bryan Catanzaro (2020)MEGATRON-CNTRL: controllable story generation with external knowledge using large-scale language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, 

November 16-20, 2020_, pages 2831-2845. Association for Computational Linguistics.
* Yang et al. (2019) Pengcheng Yang, Lei Li, Fuli Luo, Tianyu Liu, and Xu Sun. 2019. Enhancing topic-to-essay generation with external commonsense knowledge. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_, pages 2002-2012. Association for Computational Linguistics.
* Yao et al. (2019) Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 33, pages 7378-7385.
* Young (2000) R Michael Young. 2000. Creating interactive narrative structures: The potential for ai approaches. _Psychology_, 13:1-26.
* Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? _arXiv preprint arXiv:1801.07243_.
* Zhang et al. (2019) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. _arXiv preprint arXiv:1904.09675_.
* Zheng et al. (2020) Yinhe Zheng, Ringsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020. A pre-training based personalized dialogue generation model with persona-sparse data. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 34, pages 9693-9700.
* Zhong et al. (2020) Peixiang Zhong, Yong Liu, Hao Wang, and Chunyan Miao. 2020. Keyword-guided neural conversational model. _CoRR_, abs/2012.08383.
* Zhou et al. (2018a) Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018a. Commonsense knowledge aware conversation generation with graph attention. In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden_, pages 4623-4629. ijcai.org.
* Zhou et al. (2018b) Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018b. Commonsense knowledge aware conversation generation with graph attention. In _Proceedings of the 27th International Joint Conference on Artificial Intelligence_, pages 4623-4629.

## Appendix A Implementation Details

We train our model on one Quadro RTX 6000 GPU. It costs about 25 hours to train our model, and 4 hours to generate stories using our model.

## Appendix B Analysis of Extraction Strategy

### Target Extraction

We regard one sentence which has the maximum BERTScore with the persona description as the target in our model. We conducted two experiments to further investigate the influence of target extraction strategy: **(1)****ConPer (Rand):** It regards a sentence randomly sampled from the story as the target for training in the target planning stage. **(2)****ConPer (Multi):** It regards two sentences which have the maximum BERTScore with the persona description as the target.

As shown in Table 9, when using a random sentence as the target, all the metrics drop significantly. And Table 5 in the main paper shows that it is hard for the random policy to select persona-related sentences. The results indicate the benefit of our methods for modeling relations between personas and events. Moreover, using multiple sentences as the target is inferior to using only one in terms of most metrics. It is possibly because stories in storium tend to embody personas sparsely, and modeling the relations between personas and multiple persona-unrelated events directly may hurt the performance. The BS-t score is higher when using multiple sentences because more words can easily lead to a higher recall score.

### Keyword Extraction

We extracted at most 5 keywords from each sentence for the plot planning stage. We also experimented with a more sparse plan by extracting only one keyword from each sentence (called **ConvPer (Sparse)**). Table 9 shows that using a more sparse plan performs worse in all metrics. It is possibly because the limited planning keywords could not make the best of the external knowledge to form coherent and persona-related plots.

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Models** & **B-1** & **B-2** & **BS-t** & **BS-m** & **PC** \\ \hline
**ConPer** & **19.1** & **6.9** & 32.1 & **41.4** & **59.7** \\ \hline
**ConPer (Rand)** & 17.4 & 6.2 & 26.0 & 38.9 & 52.1 \\
**ConPer (Multi)** & 17.9 & 6.6 & **32.6** & 40.0 & 55.1 \\
**ConPer (Sparse)** & 18.0 & 6.6 & 31.6 & 40.2 & 57.0 \\ \hline
**_Growth Truth_** & _N/A_ & _N/A_ & _42.6_ & _42.6_ & _75.2_ \\ \hline \hline \end{tabular}
\end{table}
Table 9: Automatic evaluation results for several variants of ConPer. The best performance is highlighted in **bold**. All results are multiplied by 100.

 

Analysis of Target Guidance

We visualize how target guidance affects word prediction in the plot planning stage in Figure 2. The original word distribution is weighted to those words irrelevant to the target sentence, while the bias term (Equation 18) is weighted to those words related to the target sentence in semantics such as _bar_. After combining the original word distribution with the bias term, the final distribution can balance the trade-off between target guidance and language model prediction. This validates our hypothesis that target guidance can draw the planned plots closer to the target, which helps improve the story coherence and persona-consistency.

## Appendix D Diversity

We compare the diversity of ConPer with baselines using distinct-\(n\) (D-\(n\)) Li et al. (2016), the ratio of distinct \(n\)-grams to all \(n\)-grams in generated stories. The results in Table 10 show that ConPer has better coherence and persona consistency without sacrificing the diversity.

## Appendix E Manual Evaluation

We conduct manual evaluation on Amazon Mechanical Turk. To improve the annotation quality, we provide a detailed instruction for annotators, which contains: (1) a summary of our task; (2) a formal definition for coherence and persona consistency; and (3) good and bad examples for coherence and persona consistency. The detailed evaluation guideline is shown in Figure 3.

## Appendix F Model Parameters

We compute the number of parameters for some models used in our experiments. The result is shown in Table 11.

## Appendix G Error Analysis

Although the proposed model outperforms the strong baselines, Table 7 in the main paper shows that there are still many generated stories that exhibit opposite or irrelevant with the given persona. Therefore, we presented some typical error cases generated by our model for each error type in Figure 4. These cases show our model still does not completely control personas in story generation. When there is a slight conflict between the generated target sentence and the given persona (e.g., you're here for fun is slightly conflict with slow to action), the generated plan would further deviate from the input under the guidance of the target sentence (e.g., excit, like), and finally the generated story exhibits an opposite persona. Similarly, when the generated target sentence is irrelevant with the given persona (e.g., That was the hardest thing too see), the final generated story doesn't have any persona-related event. These errors also indicate the target sentence plays an important role in controlling the protagonist's persona in story generation.

Figure 2: A case showing the effect of target guidance. The planning keywords are brought closer to the target in semantics under the target guidance.

\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Models** & **D-1** & **D-2** & **D-3** & **D-4** \\ \hline
**GPT2\({}_{\text{scr}}\)** & 0.021 & 0.134 & 0.381 & 0.653 \\
**GPT2\({}_{\text{pt}}\)** & 0.022 & 0.184 & 0.501 & 0.777 \\
**PlanAhead** & 0.032 & 0.256 & 0.618 & 0.863 \\ CoNPer & 0.016 & 0.148 & 0.439 & 0.730 \\ \hline
**Growth Truth** & _0.062_ & _0.368_ & _0.739_ & _0.927_ \\ \hline \hline  & & & & \\ \end{tabular}
\end{table}
Table 10: Automatic evaluation results. ConPer is comparable with fine-tuned GPT2 in diversity performance.

 



## Appendix H Discussion of the Persona-Consistency Metric

To measure whether the generated story is consistent with the given persona, we propose the Persona-Consistency(PC) metric. In our experiments, we replace the ground-truth story with a randomly sampled one to construct a inconsistent story-persona pair as a negative sample. The fine-tuned classifier achieves an 83.63% accuracy on the auto-constructed test set. However, it is possible that the PC metric depends on the word overlap to make predictions because of the simple random sampling of the negative samples Lin et al. (2020). We thus conduct a case study to investigate whether our PC metric depends on word overlap to make judgments. As shown in Table 12, the first example gets a high PC score since the story embodies a consistent persona with the given persona description, in spite of a low rouge score. In contrast, the second example has an overlapped phrase "in command of" with the persona description but does not embody the corresponding persona description, and thus gets a high rouge score and low PC score. The results show that PC may not depend on shallow features like word overlap to make judgments.

What's more, we have taken into account the shortcomings of the automatic metrics for NLG and thus additionally added the human evaluation to further prove the effectiveness of our method.

\begin{table}
\begin{tabular}{p{79.5pt} p{142.3pt} c c} \hline \hline
**Persona** & **Generated Stories** & **PC** & **Rouge-2** \\ \hline You have a loose grasp on your emotion and are quick to lash out. Often hurting the ones you care about most. stood, just standing, and was staring at the class like an idiot, her head down... & 0.90 & 0.0 \\ \hline In command of some measure of magical power & ...Wilhelm is in command of all forces in the north. After the war ended, his mind was full of food and drink, and he was ready for a quick trip to the bar. He had been to the bar on many occasions. Only in there that he could fully relax and forget about all those mess things... & 0.16 & 0.14 \\ \hline \hline \end{tabular}
\end{table}
Table 12: Typical cases by PC (Persona-Consistency) score. **Bold** words denote the consistent events adhering to the given persona. The overlapping words are underlined.





**Human Evaluation Guideline**

This study aims to evaluate automatic story generation systems. Specifically, for each story, we will put the context and the persona description into a generative system, and the following sentences will be generated by the system. In the process of evaluation, you will be given two generated stories by two difference systems, denoted as A, B. You need to compare A with B in terms of the coherence and persona consistency.

**Notes:**

* Please annotate the stories carefully based on comprehensive comparison and technically following the instruction.
* Please make a more fine-grained comparison when annotating persona consistency, as it is very common that neither of the two systems can clearly reflect the given persona. Be extra careful when you choose tie.

\begin{tabular}{l|p{341.2pt}} \hline \(\mathbf{\succ}\) & **Coherence**: context relatedness and inter-sentence causal and temporal dependencies. \\ \hline Options & 1. System A is better & 2. System B is better & 3. System A and System B are good or bad equally. \\ \hline e.g. & Persona: [Pintoos] Go where you've never been before and share your experiences through song and story. \\  & Context: You wake up from an uneasy rest in the Inn. The rain batters the window and occasional lightning briefly illuminates the tops of low-rising huts and novels outside. As your level of consciousness increases you become vaguely aware of a commotion of some sort coming from outside your door... \\  & Generated Story 1: Pintoos descends the wooden sativray from his shabby _in-room confident_ that this day will be the beginning of his journey to new lands, interesting characters, and forgotten knowledge. Approaching the _imkeeper he sits and surveys the room. There is a steady dripping of rainwater falling_ from a leak in the inn's roof, a few inches from his head... "Say, what brings you here all the way from.... well where you from stranger?" Pintonos nods, "Yes, my name is Pintoos Ive just come here from the Southern rains. Dwelese is what we call our homeland. There once a boy is of age he is given leave of the land to explore the world and learn... well anything. **Good coherence, some premise related parts are underlined)**

Generated Story 2: Something about this commotion is... unsettling, and it leaves his mind racing. Well, I should probably stop talking to that commotion, it seems to have been bothering him for a while. He looks around, and a loud clutter, then shakes his head, clearly confused. "Hey, no... _Ive been out of town, so I don't see much of this, but I think we could get there. You should see if anyone's coming over there, I don't know if I ve seen anyone lately... but I think it’s best if you go talk to that commotion and see if there's anyone out there. **Bad coherence, some conflicting parts are underlined)**

Figure 3: The guideline of story quality evaluation.



* [15] M. C. Bader, A. K. Goyal, and R. J. Brox. The role of the word-level representation for a word-level representation. In _Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 349-368, 2007.
* [16] S. D. Manning, P. L. McAllester, and N. J. Russell. The word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 133-146, 2000.
* [17] T. H. Lee, Y. Li, and Z. Zhang. Word-level representation for word-level representations in words: A review. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-146, 2000.
* [18] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [19] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [20] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [21] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [22] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [23] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [24] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [25] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [26] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [27] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [28] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [29] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [30] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [31] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [32] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [33] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [34] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [35] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [36] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [37] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [38] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [39] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [40] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [41] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [42] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [43] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [44] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [45] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [46] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [47] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [48] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [49] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [50] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [51] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [52] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [53] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [54] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [55] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [56] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [57] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [58] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [59] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [60] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [61] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2000 ACM SIGKDD international conference on knowledge discovery and data mining_, pages 133-147, 2000.
* [62] W. Liu, X. Wang, and Z. Zhang. Word-level representation for word-level representations. In _Proceedings of the 2