SlowGuess/NAACL_2022_OCR huggingface dataset:
Table 1: Evaluation results for ROCStories. All values in the bottom block are averaged over three runs. MEGA-TRON's Ref. PPL is missing as it is not reported in the paper and the model is not publicly available. MEGA-TRON and VANILLA-GEN do not use any temporal prompts, so accuracy is not applied. MEGATRON's temporal diversity* is high due to large amount of  $\langle vague\rangle$  annotations in its generated stories.
Input prompt	[s1] my favorite class was english literature.
langle before\ranglelangle after\ranglelangle before\rangle
baseline	[s2] i was in english class for the first time.
[s3] i had to take a class with my local author this semester.
[s4] i decided to ask him to speak to me at a local book festival.
[s5] he said that he was really good and i liked him.
VANILLA	[s2] one day, i decided to go to the library.
[s3] i found a book that i really liked.
[s4] i read the whole book in one day.
[s5] i was so happy that i went back to school.
RL	[s2] i would go to the library to read all the books.
[s3] i had never been to a library before.
[s4] i was so excited to go.
[s5] i ended up spending a lot of time in the library.
Input prompt	[s1] tom was excited to go on a date.
langle after\ranglelangle before\ranglelangle before\rangle
baseline	[s2] he went to his favorite restaurant to pick up his date.
[s3] the date went well and then tom got drunk.
[s4] he told his date they were going to a comedy show.
[s5] the date went on and they had a great time at the movie.
VANILLA	[s2] he was going to take his girlfriend to the movies.
[s3] they went to the movie theater.
[s4] they had a great time.
[s5] tom couldn't wait to go back again.
RL	[s2] he had been looking forward to it for months.
[s3] he went to the mall and bought a new dress.
[s4] he wore it to his date's house.
[s5] his date loved it and they had a great time.
Table 2: Generated stories by our compared models. baseline: MEGATRON, VANILLA-GEN and RL-based. end-to-end. Input is the first sentence. Highlighted sentences contain an event that started earlier than the preceding event.
adopts the Plan-and-Write workflow as well as structured event representations. Appendix D describes more details of baseline systems.
We describe our own model variants below,
1. VANILLA-GEN uses the parameters of a pretrained language model (LM), specifically BART-base (Lewis et al., 2020), to initialize both the storyline and story models. Its workflow is illustrated in the upper block of Figure 2. Since no information other than the prefix (first sentence, prompt, etc.) is used to generate the story, we denote this model as vanilla LM generation or VANILLA-GEN.
2. STRUCTURED-PROMPT enhances VANILLA-GEN by using a structured storyline of events to encode temporal prompts, which is associated with the workflow of the bottom block of Figure 2.
3. PRETRAINED. For ROCStories data only,
we initialize the storyline model of STRUCTURED-PROMPT with the pretrained parameters.
4. RL uses the same inputs as STRUCTURED-PROMPT. The difference is that reinforcement learning is used to train storyline and story models jointly. As Algorithm 1 shows, RL-based model is trained following the same forward workflow as STRUCTURED-PROMPT, but during backpropagation, the storyline models' parameters are updated.
The main results for ROCStories and WritingPrompts are shown in Table 1 and Table 3 respectively. Examples of generated stories can be found in Table 2 and Table 6 for ROCStories and Table 11 in the appendix for WritingPrompts. We organize our discussions and analysis in the following sections by answering the four research questions. Q1) Can our proposed models (with temporal prompts) produce stories with good textual quality? Q2) Are our proposed models effective at generating flashbacks? Q3) Can our proposed models maintain event temporal coherence in stories? Q4) How do our proposed models contribute to stories' interest levels?
5.1 Textual Quality
We measure the textual quality of stories using a wide range of automatic evaluation metrics.
Perplexity. For ROCStories, all three model variants can improve Ref. PPL against VANILLA-GEN and TEMPORAL-BART while maintaining good Gen. PPL. The weak Gen. PPL of MEGATRON may be attributed to its sentence-by-sentence generation pipeline, whereas our models generate an entire story in an integrated step. For Writing-Prompts, both model variants improve Gen. PPL over VANILLA-GEN and CONTENTPLANNING while maintaining good Ref. PPL.
Token Diversity. For ROCStories, RL-based model improves the VANILLA-GEN by 0.18 per
Models	Ref. PPL (↓)	Gen. PPL (↓)	Distinct Ratio (↑)	BLEU (↑)	ROUGE L (↑)	Tok. Len. (↑)	Pearson Corr. (↑)	Temporal Coherence (↑)	Interest Level (↑)
CONTENTPLANNING	-	25.52	1.80	3.46	14.40	252.3	0.04	0.57	2.20
VANILLA-GEN	31.04	11.17	3.50	0.67	9.43	160.2	0.09	0.52	2.49
+ STRUCTURED-PROMPT	30.77	9.30	2.86	1.44	10.95	208.6	0.56	0.49	2.49
+ RL	30.98	9.50	2.83	1.39	10.78	203.8	0.57	0.55	2.62
Table 3: Evaluation results for WritingPrompts. Pearson correlation approximates the effectiveness of prompts.
Distinct Ratio. MEGATRON achieves the highest token diversity as it incorporates external knowledge-bases that make the generated stories contain novel tokens. For WritingPrompts, we observe longer stories are associated with poorer scores. However, the large increases in Distinct Ratio suggest that the token usages in our proposed models are diverse.
BLEU and ROUGE $_L$ . For ROCStories, the proposed models perform on-par with VANILLAGEN and TEMPORAL-BART while outperforming MEGATRON, which generates the shortest stories among all compared models. For WritingPrompts, CONTENTPLANNING performs the best partially due to its usage of BART-large models.
The overall performances across these three types of automatic metrics suggest that using temporal prompts in the Plan-and-Write framework can produce stories with high textual quality.
5.2 Effectiveness on Flashback Generation
The second research question probes the effectiveness of using temporal prompts on generating flashbacks. For ROCStories, all models can generate stories with the same number of events/sentences as the gold stories. This allows annotators to judge pairwise event relations in the generated stories and help us check whether the generated events have relations truthfully reflecting the temporal prompts used. Accuracy is the perfect metric for this. As Table 1 shows, the final RL-based model achieves the highest score, which indicates the strongest effectiveness of generating flashbacks.
However, temporal prompts are not used in the baselines and VANILLA-GEN. So we compute an approximate measures of effectiveness, temporal diversity, which indicates how many non-(before) relations (after) prompt can induce. Table 1 shows that STRUCTURED-PROMPT, PRETRAINED and RL-based models can help improve VANILLA-GEN with more than  $80\%$  generated (before) relations. MEGATRON achieves the highest score due to the largest amount  $(29\%)$  of (vague) relations (complex or undetermined) annotated by MTurkers shown in Figure 3, which is associated with its lowest temporal coherence score.
Figure 3: Relation annotation distribution by MTurkers for compared models trained on ROCStories.
For WritingPrompts, stories are long and can contain dialogues or short phrases without events at all. These make the sentence or event alignments between the gold and generated stories worse than ROCStories, i.e.  $e_{i,k}, r_{i,k}$  may not correspond to the k-th sentence in  $\mathcal{V}_i$ . Therefore, accuracy cannot be computed. To obtain an approximate metric, we use the tool described in Sec. 3.3 to annotate neighboring event temporal relations in the generated test stories for all the compared models. Slightly different from temporal diversity, we calculate the total number of machine annotated  $\langle after\rangle$  relations, denoted as  $\hat{N}_{i,A}$  in each  $\hat{S}_i'$ . Let  $\{N_{i,A}\}$  denote the number of  $\langle after\rangle$  temporal prompts extracted in gold stories. We compute the Pearson correlation (Benesty et al., 2009) between the sets  $\{\hat{N}_{i,A}\}$  and  $\{\hat{N}_{i,A}\}$  as the measure.
As Table 3 shows, for CONTENTPLANNING and VANILLA-GEN without temporal prompts, the correlations are weak; whereas when temporal prompts are used in both STRUCTURED-PROMPT and RL-based models, the correlations are strong. Although using models' temporal annotations for the generated stories is not as precise as human annotations, the large differences in correlation provide another piece of evidence that our proposed methods are effective at generating flashbacks.
5.3 Temporal Coherence
Generating flashbacks requires a system to disrupt the monotonic  $\langle before\rangle$  sequence, which is the dominant temporal pattern generated by VANILLAGEN (see Figure 3). In other words, flashbacks with at least one  $\langle after\rangle$  are minor patterns that can be
hard to learn in our data, which may result in event sequences violating our temporal commonsense. Thus, we need to check that stories generated with flashbacks maintain good temporal coherence. As shown in Table 1 and 3, our proposed models with temporal prompts can achieve on-par or slightly lower scores, suggesting little trade-off of temporal coherence in generating flashbacks. We will discuss this more in the error analysis (Sec. 6).
5.4 Contributions to the Interest Level
As we can observe in Table 1, the impact of temporal diversity and coherence on the interest level appears to be complex. To better understand the dynamics among these metrics, we run ordinary least square regressions (OLS) (Kenney and Keeping, 1965) by setting interest level as the target variable and temporal diversity, coherence and the number of  $\langle after\rangle$ s as predictors. Since all of these metrics apply to each of the compared stories, the total instances are 530 and 308 for ROCStories and WritingPrompts, respectively.
As Table 4 shows, for ROCStories, holding other metrics constant, adding 1 unit to temporal coherence and diversity leads to a 0.609 increase and 0.532 decrease of the interest level. The former result implies that a story lacking event temporal coherence tends to be less interesting. The latter result suggests that increasing temporal diversity may lead to less interesting stories, which we hypothesize could be attributed to two factors: 1)  $\langle before\rangle$  is dominant in ROCStories, and by using  $\langle after\rangle$  as prompt, we force models to generate relations less seen in data. 2) Figure 3 shows temporal diversity can increase with more  $\langle vague\rangle$  relations. Since  $\langle vague\rangle$  is an undetermined temporal relation even for our annotators, it could make the storyline confusing and thus lead to less interesting stories. The coefficient for the number of  $\langle after\rangle$  indicators is positive with strong statistical significance. It suggests that holding the other two metrics constant, adding the number of  $\langle after\rangle$  indicators by 1 contributes to 0.387 increases of the interest level.
For WritingPrompts, although we are not able to conclude that the estimates are statistically significant, the coefficients have the same signs as ROCStories. Also, we observe that the p-value of the number of  $\langle after\rangle$  indicators is much lower than the other two variables, which implies a relatively stronger (positive) impact.
Since temporal prompts in human evaluations all contain at least one  $\langle \text{after} \rangle$ , these results show
ROCStories	WritingPrompts
Coef.	p-value	Coef.	p-value
Temporal Coherence	0.609	0.000*	0.006	0.963
Temporal Diversity	-0.532	0.004*	-0.279	0.410
# ⟨after⟩ prompt	0.387	0.000*	0.034	0.238
Table 4: OLS regression results on temporal coherence, diversity and number of  $\langle \text{after} \rangle$  indicators. The coefficients for the intercept are omitted. * means statistically significant with 99% confidence.
that when  $\langle after\rangle$  prompt successfully produces event pairs with  $\langle after\rangle$  relation in the final stories, it makes stories more interesting. Now, we can answer the final research question: improving temporal diversity can help interest level when  $\langle after\rangle$  prompts are effective at generating  $\langle after\rangle$  relations in stories; that is, when flashbacks truly work, stories become more interesting.
In Table 1, we observe that our final models cannot outperform VANILLA-GEN for Temporal Coherence and fall behind MEGATRON for Temporal Diversity. We show examples below to provide feasible explanations.
Temporal Coherence. Table 5 shows three comparisons between our final RL-based model and the VANILLA-GEN baseline. In all three pairs, human evaluators mark VANILLA-GEN's generated stories as temporally coherent but not for RL's stories. As we can see in these examples, the red highlighted events temporally contradict the preceding context. In Input 1, the context shows "I have listened to the new album" so "can't wait to listen to the new album" shouldn't follow the former event. For Input 2, if "Anna had gotten her hair permed," it does make sense that the event "went to apply it (the perm)" follows. In Input 3, if the officer "gave [a] ticket" already, it does quite make sense to give "a warning" afterward as it is a weaker penalty than a "ticket." However, we observe that in all three cases,  $\langle after\rangle$  relations are successfully reflected (flashback). Since stories with  $\langle after\rangle$  relations are minority cases in the data, we hypothesize that they have not been perfectly learned by our proposed models. We leave more rigorous investigation for future research efforts.
Temporal Diversity. Table 10 in the appendix shows three comparisons between our final RL-based model and the MEGATRON baseline. In all three pairs, MEGATRON's generated stories are more temporally diverse based on the predicted relations provided by human evaluators. However, MEGATRON's stories are either contradict-
Input 1
prompt	[s1] the new album from my favorite pop star came out today.
<after> <before> <before>
VANILLA	[s2] i was so excited to listen to it.
[s3] i listened to it all day long.
[s4] i couldn't believe how good it was.
[s5] red i can't wait to listen again.
RL	[s2] i had never heard the album before.
[s3] i went to the store and bought the album.
[s4] when i got home, i listened to the whole thing.
[s5] i can't wait to listen to the new album.
Input 2
prompt	[s1] anna had just gotten her hair permed.
<before> <before> <after> <before>
VANILLA	[s2] she was excited to see how it looked.
[s3] but then she saw that her hair had grown out.
[s4] anna was horrified!
[s5] she had to go to the salon to get it fixed.
RL	[s2] she was excited to see how it looked.
[s3] but when she went to apply it, it was terrible.
[s4] her hair had grown way too long!
[s5] anna had to go to the salon to get her hair fixed.
Input 3
prompt	[s1] last week we saw a police car in front of a parked car.
<after> <before> <before> <before>
VANILLA	[s2] the police officer pulled over the car.
[s3] he asked the driver if he had insurance.
[s4] the driver said he did not.
[s5] the officer gave him a ticket.
RL	[s2] the car was parked in the parking lot.
[s3] the police officer pulled over the car.
[s4] he gave the driver a ticket.
[s5] the officer gave him a warning.
Table 5: Examples from our RL-based model that are not as temporally coherent as examples generated by the VANILLA-GEN model. Highlighted sentences contain an event that started earlier than the preceding event. Red text indicates temporal incoherence.
tory (Input 1), incoherent (Input 2) or repetitive (Input 3), resulting in higher ambiguous event relations, i.e.  $\langle \text{vague} \rangle$  annotations (consistent with Figure 3). Therefore, despite lower temporal diversity, our proposed models can still produce stories with higher quality, which is demonstrated via other metrics such as Temporal Coherence and Interest Level.
Generating flashbacks has been studied in a few prior works. Bae and Young (2008) is one of the early efforts proposing a planning-based approach to generate flashbacks to evoke surprise in the readers. Follow-up works proposed a cognitive-based model that finds the best location in the original stories to insert a past event (Wu et al., 2016). Our work differs from this line of research by using temporal prompts with pretrained language models to generate integrated flashback in stories. Hoek et al. (2014) studies flashback in game narrative generation, which is remotely related to our work.
Plan-and-Write framework has been shown to be an effective method to enhance the explainability and controllability of story generation. Yao et al. (2019) enables machines to produce a sequence of
keywords prior to generating stories. Follow-up works leverage commonsense or external knowledge to enhance the quality of stories (Guan et al., 2020; Xu et al., 2020; Tan et al., 2021). Goldfarb-Tarrant et al. (2020) is one of our compared works that incorporates SRL extracted event representations in storylines and train models with several event-related decoding objectives. Our work differs from it by explicitly encoding temporal prompts in event plots that facilitates flashback.
Structured representation such as discourse structure (Guan et al., 2021), story keywords (Peng et al., 2018; Goldfarb-Tarrant et al., 2019) and event/plot graph (Ammanabrolu et al., 2019, 2021) have been widely used in story generation to enable models to output diverse stories, but they are remotely related to our flashback generation task.
Reinforcement learning has also been explored in two-stage story generation such as Xu et al. (2018) and Tambwekar et al. (2019). Our motivation of using RL-based generation is to enhance the effectiveness of temporal prompts.
Event temporal reasoning helps the construction of the temporal prompts. It has been studied in story understanding (Han et al., 2019b), information extraction (Ning et al., 2017; Han et al., 2019c,a, 2020; Ma et al., 2021), QA (Ning et al., 2020; Zhou et al., 2019), and event generation (Lin et al., 2021; Li et al., 2021). Our proposed method is the pioneer work to introduce event temporal prompts in flashback generation.
We propose to generate flashbacks in stories by encoding temporal prompts in structured storylines. Experimental and evaluation results show our proposed systems can produce fluent, temporally coherent, and more interesting stories. Future work can focus on improving temporal prompts so that they handle minority cases (i.e. prompts with one or more  $\langle$  after  $\rangle$  indicators) more effectively. Designing more powerful rewards in reinforcement learning is another promising research direction.
We thank the PlusLab members and the anonymous reviewers for their constructive feedback. This work is supported in part by IARPA, via Contract No. 2019-19051600007, the DARPA Machine Common Sense (MCS) program under Cooperative Agreement N66001-19-2-4032, and a CISCO research award.
ACE. 2005. The ace 2005 (ace 05) evaluation plan evaluation of the detection and recognition of ace entities, values, temporal expressions, relations, and events 1.
L. Alwitt. 2002. Maintaining attention to a narrative event. Shohov, S.P. (eds.) Advances in Psychology Research, 18:99-114.
Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, and Mark O Riedl. 2021. Automated storytelling via causal, commonsense plot ordering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 5859-5867.
Prithviraj Ammanabrolu, Ethan Tien, Wesley Cheung, Zhaochen Luo, William Ma, Lara J. Martin, and Mark O. Riedl. 2019. Story realization: Expanding plot events into sentences. In Proceedings of AAAI 2020.
Byung-Chull Bae and R. Michael Young. 2008. A use of flashback and foreshadowing for surprise arousal in narrative using a plan-based approach. In *Interactive Storytelling*, pages 156–167, Berlin, Heidelberg. Springer Berlin Heidelberg.
Morteza Behrooz. 2019. *Curating Interest in Open Story Generation*. Ph.D. thesis. Copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; Last updated - 2021-05-25.
Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009. Pearson correlation coefficient. In Noise reduction in speech processing, pages 37-40. Springer.
W.F. Brewer and E. H. Lichtenstein. 1981. Event schemas, story schemas, and story grammars. Long, J. and Baddeley, A. (eds.) Attention and Performance, 9:263-279.
W.F. Brewer and E. H. Lichtenstein. 1982. Stories are to entertain: A structural-affect theory of stories. Journal of Pragmatics, 6:473-483.
Taylor Cassidy, Bill McDowell, Nathanael Chambers, and Steven Bethard. 2014. An annotation framework for dense event ordering. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 501-506, Baltimore, Maryland. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898, Melbourne, Australia. Association for Computational Linguistics.
Angela Fan, Mike Lewis, and Yann Dauphin. 2019. Strategies for structuring story generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2650-2660, Florence, Italy. Association for Computational Linguistics.
Siddhant Garg, Thuy Vu, and Alessandro Moschitti. 2020. Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7780-7788.
Tsegaye Abie Gebeyehu. 2019. Analysis of literary techniques employed in the novel the revelation: Flashback in focus. Journal of Literature, Languages and Linguistics, 63.
Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. 2020. Content planning for neural story generation with aristotelian rescoring. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4319-4338, Online. Association for Computational Linguistics.
Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng. 2019. Plan, write, and revise: an interactive system for open-domain story generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 89-97, Minneapolis, Minnesota. Association for Computational Linguistics.
Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, and Minlie Huang. 2020. A knowledge-enhanced pretraining model for commonsense story generation. Transactions of the Association for Computational Linguistics, 8:93-108.
Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wenbiao Ding, and Minlie Huang. 2021. Long text generation by modeling sentence-level and discourse-level coherence. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6379-6393, Online. Association for Computational Linguistics.
Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, and Nanyun Peng. 2021a. ESTER: A machine reading comprehension dataset for reasoning about event semantic relations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7543-7559, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.