Narrative Order Aware Story Generation via Bidirectional Pretraining Model with Optimal Transport Reward
Authors: Zhicong Lu
Li Jin
Guangluan Xu
Linmei Hu
Nayu Liu
Xiaoyu Li
Xian Sun
Zequn Zhang
kaiwen wei
Venue: EMNLP 2023 Findings (2023)
Keywords: story generation
narrative order
optimal transport

Abstract: To create a captivating story, a writer often plans a sequence of logically coherent events and ingeniously manipulates the narrative order to generate flashback in place. However, existing storytelling systems suffer from both insufficient understanding of event correlations and inadequate awareness of event temporal order (e.g., go to hospital <after> get ill), making it challenging to generate high-quality events that balance the logic and narrative order of story. In this paper, we propose a narrative order aware framework BPOT (Bidirectional Pretraining Model with Optimal Transport Reward) for story generation, which presents a bidirectional pretrained model to encode event correlations and pairwise event order. We also design a reinforcement learning algorithm with novel optimal transport reward to further improve the quality of generated events in the fine-tuning stage. Specifically, a narrative order aware event sequence model is pretrained with the joint learning objectives of event blank infilling and pairwise order prediction. Then, reinforcement learning with novel optimal transport reward is designed to further improve the generated event quality in the fine-tuning stage. The novel optimal transport reward captures the mappings between the generated events and the sentences in the story, effectively measuring the quality of generated events. Both automatic and manual evaluation results demonstrate the superiority of our framework in generating logically coherent stories with flashbacks.

Narrative Order Aware Story Generation via Bidirectional Pretraining Model with Optimal Transport Reward
Zhicong Lu , Li Jin', Guangluan Xu', Linmei Hu', Nayu Liu
Xiaoyu Li', Xian Sun', Zequn Zhang', Kaiwen Wei!.2
Key Laboratory of Network Information System Technology (NIST), Acrospace Information Research Institute, Chinese Academy of Sciences
School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences Beijing Institute of Technology School of Computer Science and Technology, Tiangong University (nazaritelzc. jinlimalls gmails.com
Abstract
Input
to eatch neighborhood thief. the police weaning mask. the mask
tuned Bart (Lowis et al., 2020)
the polies cought the thief. he was arrested.
To create a captivating story, writer often plans a sequence of logically coherent events and ingeniously manipulates the narrative or- der to generate flashback in place. However, existing storytelling systems suffer from both insufficient understanding of event correlations and inadequate awareness of event temporal or- der (e.g.. go to hospital <after> get ill), making it challenging generate high-quality events that balance the logic and narrative order of story. this paper, we propose narrative order aware framework BPOT (Bidirectional Pretraining Model with Optimal Transport Re- ward) for story generation, which presents bidirectional pretrained model to encode event correlations and pairwise event order. We also design reinforcement learning algorithm with novel optimal transport reward to further im- prove the quality of generated events in the fine-tuning stage. Specifically, narrative order aware event sequence model is pretrained with the joint learning objectives event blank in- filling and pairwise order prediction. Then, re- inforcement learning with novel optimal trans- port reward is designed to further improve the generated event quality in the fine-tume fine-tuning stage. The novel optimal transport reward captures the mappings between the generated events and the sentences in the story, effectively measuring the quality of generated events. Both automatic and manual evaluation results demonstrate the superiority of our framework in generating log- ically coherent stories with flashbacks.
Megatron- 122M 124M (Xu et al., 2020)
cell month. juil
had arvested him they him jail
full people.
they brought him to juil cell.
the thae hud stolen waller neighbor's sentenced to community service. now, the poliic are trying to appreticnd the thief!.
Flashback
the police arrested
(Han et al., 2022)
Table 1: Stories generated by existing storytelling sys- tems. The red texts and brown texts indicate logical incoherence and flashback respectively.
Fhabbuck Generatios
ofter
ofter
34] tee (keg svien walle srighter can.
Aypour
the polloc amount ttier.
Cnndrctiioal Renosing
Ukdiroctinaal Hcmontig
An example of flashback generation. Tem- poral prompts (<hefore>, <afier>) hint at the pairwise event order, thereby manipulating the narrative order to generate flashbacks (the brown texts) in place.
Figure 1:
can generate stories like human writer is meaning- ful. Story generation aims to generate a series of events and organize them into a reasonable story given limited information (Huang et al., 2021).
Recently, a significant process has been made in story generation. Similar to human writers, Martin et al. (2018); Yao et al. (2019); Xu et al. (2018) introduced a planning- ning sketch prior to writing story. To improve the controllability of story generation (Kong et al., 2021; Xie al., 2022), different attributes (e.g.. style, psychology) were added to sketch. Besides, Guan et al. (2020); Xu et al. (2020) leveraged ex- ternal knowledge bases to generate commonsense stories. Although existing methods can generate fluent stories related to the desired attribute, they pay little attention to event correlations. resulting in insufficient understanding of event correlations. This makes it difficult maintain logical coherence as a whole story (Zhou et al., 2022c). Moreover, they rarely consider the crucial role narrative order in story generation. For example in Table I,
method, namely plan-
1 Introduction
As the fundamental element of numerous literary genres, stories have played a significant role in var- ious social domains, such as literature, education, and entertainment. However, manually creating high-quality stories is often time-consuming and laborious. Therefore, developing an Al system that
*Corresponding author.

---

all storytelling systems exhibit logical incoherence and only the last system reverses narrative order in place to generate flashback.
based on the mappings captured by OT.
3) Extensive evaluation results demonstrate the superiority of our framework in generating logi- cally coherent stories with flashbacks.
Flashback is a common writing technique that describes the past events to provide context for the current narrative. It arouses reader's interest and deepens their understanding of the narratives. To generate flashback in story, Han et al. (2022) proposed temporal prompt. As shown in Figure 1, <afler> hints at the pairwise event order, which trig- gers model reverse the narrative order describe the past events for flashback generation ("catch a neighborhood thief" occurs later than "had stolen a wallet"). However, they face the problems of inefficient prompt and deteriorating logic in some cases (shown in section 4.5). It is probably because model always unidirectionally reasons event from order, lacking the process of inversely inferring or- der from pairwise events, resulting in inadequate awareness of event temporal order. Additionally, when generating high-quality events in story, is intrinsically more challenging to balance the logic and narrative order than merely focusing on logic.
2 Preliminaries
This section provides description of the sketch used in this work, followed by an introduction the basic concept of OT.
2.1 Sketch Details
We follow the line of the planning-based method (Martin et al., 2018; Yao et al., 2019) for story gen- eration. Our sketch S is composed of alternating splicing of event and temporal prompt t, where I (<hefore>, <after>, <vague>). It hints at the pairwise event order, thereby manipulating the narrative order. Note that <vague> represents arbi- trary event order, Formally, an event e a structure (a", v, a²), which extracted from the correspond- ing sentence in story Y by semantic role labeling (SRL) tools (Gardner et al., 2018). v is the trigger (Wei et al., 2021) describing the event, a. and a' are its relevant arguments (Wei et al., 2022). We only consider one event in each sentence for sim- plicity. Let Ct,x (a., Di,n, a!,x) denotes the n-th event in i-th sketch Sj, li, denotes the event order between the n-th event and the (n+ 1)-th event in S,. Then a sketch with m events can be represented as S; = { ti,1,,,,,,,,...... ti.,,...., Ci,ms <eoe>}; J, where <eoe> refers to event ending. A sketch with five events is shown at the bottom of Figure 3.
In this paper, we propose narrative order aware framework BPOT for story generation, which presents bidirectional pretrained model to encode event correlations and pairwise event order. We also design reinforcement learning (RL.) algo- rithm with novel optimal transport (OT) reward to further improve the quality of generated events in the fine-tune tuninnnnng stage. Specifically, a narrative order aware event sequence model is pretrained with the joint learning objectives of event blank infilling and pairwise order prediction. Then, we employ OT to capture the mappings between the generated events and the sentences in the story. Based the map- pings, we construct novel reward to effectively measure the quality of generated events, further improving the generated event quality under the optimization of the RL algorithm. In summary, our main contributions include:
2.2 Optimal Transport
OT has recently been introduced to numerous tasks in NLP (Wei et al., 2023). Its original objective is find the OT maps between two data distri- butions with minimum cost (Kantorovich. 2006). Formally, given two complete metric spaces X and Y, let u(x) and v(y) denote two discrete probabil- ity distributions on X and Y respectively, where E u(z;) = Σm v(y)) = 1. Then the OT maps T between u(z) and v(y) is obtained by solving the optimization problem (1):
1) We propose a narrative order aware frame- work BPOT for story generation, which pretrains an event sequence model with the joint learning objectives of event blank infilling and pairwise or- der prediction encode the event correlations and pairwise event order.
T arg min
Ls c(x,,
8.f. Σ fu = u(x.)
Ei (1 I....nn
2) We design an RL algorithm with novel OT reward to further improve the quality of generated event in the fine-tuning stage. The novel OT reward effectively measures the generated event quality
())
Σ by v(y)) Ej E m1.
E R m

---

where e is the cost function. To effectively solve (1), researchers have proposed some approximation algorithms, such as Sinkhorn (Cuturi, 2013), IPOT (Xie et al., 2020). In this work, we adopt IPOT.
Ereat Inftllinn
Order Prodiction
Mask
Bocovers
Sequence
Rg
3 Methodology
i-ds kanperal pruzpe
raa
In this section, we detail our proposed narrative order aware framework BPOT for story generation, which presents bidirectional pretrained model to encode event correlations and pairwise event or- der. As shown in the upper Figure 3, an RL algorithm with novel OT reward is designed to further improve the generated event quality. Par- ticularly, a narrative order aware event sequence model is pretrained with the joint learning objec- tives of event blank infilling and pairwise order prediction shown in Figure 2. In the fine-tuning stage. we employ OT to capture the mappings be- tween the generated events and the sentences the story. Based on the mappings, novel reward constructed to effectively measure the quality of generated events, further improving the generated event quality under the RL optimization.
Figure 2: An illustration of bidirectional pretraining of event blank infilling and pairwise order prediction
3.2 Bidirectional Pretraining
Intuitively, the quality of sketch greatly influences the generated story. However, due to suffering from both insufficient understanding of event corre- lations and inadequate awareness of event temporal order, it is challenging for sketch model to generate high-quality events that balance the logic and nar- rative order of story. Therefore, present joint pretraining of event blank infilling and pairwise order prediction encode event correlations and pairwise event order as shown in Figure 2.
Event Blank Infilling demands model to reason blank event from the given pairwise event order. requires the generated event to not only be rea- sonable in event correlations but also conform to a specific event order. Concretely, for each event è in Sg, we mask with a probability of 0.25 to obtain Sy and keep all temporal prompts visible. If no events are selected for masking, we randomly choose
3.1 Vanilla Pipeline Generation
The vanilla pipeline generation of our framework is shown at the bottom of Figure 3. Following the line of the planning-ban based method, we plan sketch prior to writing story. Firstly, golden sketch S; is extracted from golden story Y; mentioned in section 2.1. Temporal prompts are obtained by identifying the pairwise event order between sen- tences by ECONET (Han et al., 2020) in advance, Then all events are masked in S, except for the in- put event to obtain input xf. Last, we require sketch model to generate sketch S; given on of and story model to unfold S; into story Y;- The pretrained BART-base (Lewis et al., 2020) is served as both sketch model and story model. Formally, Let a and 0 denote the parameters of sketch model and story model respectively, per sample loss for two models during training can be expressed as (2), (3):
mask. By doing so, the model is able pay more attention understanding event correlations conditioned on specific order.
Pairwise Order Prediction requires model to rea- order from the given pairwise events within the generated paradigm. Concretely, we mask all tem- poral prompts i in S,, where <eoe> is also masked to facilitate model to understand the story ending. To prevent excessive deviation from the sketch gen- eration, we additionally mask an event to obtain St. If an event is chosen to mask, the corresponding temporal prompt or <eoe> will be restored. Since only one event is masked, model reasons pairwise event order in most cases, which facilitates model to have better awareness of event temporal order. To jointly encode event correlations and event (3) temporal order, we first obtain S and St from Š, for each sample. Then, we execute different learning objectives in parallel and combine their loss in a varying ratio to jointly guide optimization. Per sample loss Ly can be represented as (4):
18a1
L. = -logp(S:]a)) =
(S l Sc.ck)
(2)
Lo = -logp({((Ss) = (Y My. P2.e)
We use St rather than S to reduce the discrepancy between training and inference.

---

Reward (
Transport Matrix
Optimal Transport Reward Construction
To
Tu
Ey
Syc
TER
IPOT
CE
Ta
Tx
Transport Matrix
Similarity Metric
To
To
To
Co
R R
RBF Keme!
Expansion
Reward Computing
Cost Matrix
Final R
normalize
Hg. Hg
Event Semantic Extraction( e)
Sentence Semantic Extraction(H.))
Sentence Loss Computing(
he
ba
huu
hes
h
h
h
hg
L
Lg
Vanilla Pipeline Generation
BPretraied Sketch Model
Story Model
Generate Sketch
Input Sketch
Output
Jkity oponed her eyporraf
{ Jeljkity opened her eyes<hyfhre:>
Jainy opened her eyes and found that sho was hospital.
[e2] mmllk-oroff
[62]she felt confused and in painsigfler>
[s2 juhe confused and pain her body.
|e3jche accident-ägtore
[s3] recalled
accident night.
(fed) doggeres
[o4]tee doctor cameefeffrre?
[s4]the
examined her condition.
[ed) [ef)emask--aaee 2o0e
[e5]she had mild concusion--ooe>
[s5] sold her she bad mild concussion and noeded rost
Input
Sketch
Story
Figure 3: An illustration of our framework for story generation. We take example of five-sentence story. The bottom shows the vanilla pipeline generation. The upper shows the specific details of constructing the novel OT reward. For illustration purposes, we add tag ([ea), [s.]) for each event sentence (i.e., not required reality).
Therefore, the core idea is to design an effective reward R; that guides sketch model to generate het- ter sketch. A naive approach is regarding negative sentence loss as the corresponding event reward. Because low-quality event in sketch will make it harder for story model to reconstruct the original sentence story, leading to higher sentence loss and smaller event reward. However, the mappings between each event e; in sketch and the sentence sj in story modeled by the naive approach are one- to-one as the reward for e; is only determined by 8;. But they should be one-to-many contribute multiple sentences. To overcome this barrier, we design a novel OT reward.
18:1 Lp = (nn onloopl Sic
(4)
Ylooop (S,
where and are weight factors, varying with the pretraining process.
3.3 Optimal Transport Reward
An apparent problem with the planning-base method is sketch model can't adjust with story model the sketch generation is differentiable. This leads sketch model not know- ing how the generated sketch affects the final story. To overcome this barrier, we design an RL algo- rithm with novel OT reward, further improving the generated event quality.
event may
As shown in the upper of Figure 3, extract the event semantics HI hhei and sentence semantics H, {h.g) _ by averaging the repre- sentations of the included tokens from the last hid- den state of sketch model and story model. Here, le and la represent the number of events in sketch and sentences in story respectively. The sentence loss Lg [T.. ) is also computed by adding up the loss of the included tokens. Later, view the pro- of unfolding sketch into story moving the event semantic distribution sentence semantic distribution, and employ OT to capture the map- pings between them. Concretely, the cost matrix C E Rlexts is first calculated based on the simi-
We briefly introduce how to utilize RL to opti- mize sketch model. Particularly, reward ft is con- structed based on the feedback from story model. Later, the policy gradient (Sutton et al., 1999) is adopted to optimize sketch model through maxi- mizing the expected reward E. [ft.) in (5). The gradient of sketch model can be represented as (6) according to the policy gradient theorem, which can be approximated with the sampling techniques.
E. [B:] Elî.. (a)), [((a) E. V.ogp(8, (.l,,
(6)

---

Algorithm 1: Training And Optimization
4.1 Datasets
Input: Dirwwin {x., Y. 1-1---1 weight
To verify the effectiveness of our framework in generating both short and long stories, we choose ROCStories (ROC) (Mostafazadeh al., 2016) and WritingPrompts (WP) (Fan et al., 2018) as bench- mark datasets. For pretraining data, we adopt the event sequences provided by Lin et al. (2020) and further process it as follows. For each pairwise events, the event triggers and contexts are fed into ECONET (Han et al., 2020) to obtain their pairwise event order. Last, we repeat the operations in sec- tion 3.1 and obtain a total of 100k event sequences for pretraining. More details about the datasets and pretraining data are shown in appendix A.I.
parameters of sketch model and story model.
pretrain sketch model
sumple J [xx
e Dtrasn
Compute loss Le,, Lg by (2), (5).
Extratt H., H., Ly from the output
Compute cost matria by (7)
Compute maneport matrix 7 by IPOT
Compute compact reward Rt OT Loss Lae by (9)
Expand RG get final reward RG
(((a)= = RR- (S.
10
Vo (J(a) L.
Lot optimize the entire model based on Vo, V8
V0
12
end
larity between events and sentences measured by RBF kernel as (7).
Gij
cxp(
2,52
(7)
4.2 Compared Models
It means the closer the semantics of hei and haj. the smaller the transportation cost between them. Then, IPOT adopted to compute the transport matrix (maps) E It based on C, He and H. Tij can be understood as the semantic transportation or contribution from e; to sj. After obtaining the mappings T, we construct the compact reward R (ri)le and compute the OT loss Lot as below:
Baselines. (1) Bart (Lewis et al., 2020) typi- cal seq2seq model for natural language generation. We directly fine-tuned Bart-base on both datasets. (2) Bart (planning) adopted the planning- -baed method based on (1). (3) TemporalBart (Lin et al., 2020) is pretrained with temporal event ordering and event infilling tasks. We utilize its pre-trained weights to initialize
sketch model. (4) Mega- tron (Xu et al., 2020) fine-tuned GPT2 (Radford et al., 2019) and leveraged external knowledge bases to enrich the generated stories on ROCSto- ries. (5) ContentPlanning (Goldfarb-Tarranl et al., 2020) adopted the planning- based Bart-large model on WritingPrompts. (6) Flashback (Han et al., 2022) utilized temporal prompt to generate flash- back in place, which is the strongest baseline for our framework. We also compare with its three variants for fair, which are only RL, only Pre- trained and the integration of RL and Pretrained (PR), respectively. For RL, it regarded the negative story loss as the final reward R. For Pretrained, it adopted an autoregressive manner at pretrain- ing stage. Note that Flashback and its variants all served event arguments mask unit. (7) Chat- GPT is selected as its powerful ability in natural language generation. More details about the base- lines are in Appendix A.3.
R. -TL. Lot = tracee("
(8)
(9)
where r; indicates the reward for ca. Last, we pand each r; make its dimension consistent with the number of tokens contained in the correspond- ing event to get the final reward Rt. In this way, r; is not only determined by the sentence loss 8;, but also by other sentence loss. The reward intensity of 8j to 6; depends on the semantic contribution (Tiz) from e to sj captured by OT. Consequently, each event reward comprehensively considers the feed- back of all sentences, thus effectively measuring the quality of events in sketch. Furthermore, sketch model can perceive the generated event quality and understand event correlations through comparing the reward of different events, thereby adjusting itself accordingly. It facilitates sketch model to improve the quality of generated events. The entire flow is shown in Algorithm 1.
Ablation Variants. In addition to the baselines,
we present the ablation variants of our framework. (1) Vanilla performs vanilla pipeline generation shown in the bottom of Figure (excluding bidi- rectional pretraining)
4 Experiments
In this section, we first introduce the datasets, the compared models, and the evaluation measures in the experiments. Then, we show the experimental results and provide a detailed analysis.
(2) BP adopts bidirectional pretraining of event blank infilling and event tem-
poral order based on (1). (3) OTRL attaches the RL algorithm with OT reward based on (1).

---

ROCStories
WritingPrompit
Models
PPL.
B-3t
RLt
R-24
D-4t
PPL4
B-3t
Ru1
R-34
D-4t
Tkst
Bart
20.24
4.98
19.11
47.78
62.44
31.15
0.57
9.28
20.92
59.37
148.6
Bart (planning)
27.30
5.13
19.29
49.77
61.80
31.04
0.67
9.43
23.40
60.27
160.2
TemporalBart
24.65
5.01
19.12
50.62
62.13
Flashhack
22.85
19.39
46.11
63.42
30.77
1.44
10.95
23.70
59.83
208.6
Pretrained
21.16
5,06
19.43
49.16
62.79
30.70
1.68
11.13
25.90
58.90
226.7
+ RL
15.67
5.12
19.41
49.43
64,10
30.98
1.39
10.78
23.60
59.12
203.8
PR
15.45
5,20
19.49
50.05
64,76
30.73
1.64
11.03
24.20
58.86
222.4
Vanilla
25.51
19.40
1,97
25.20
$7.89
BP
25.35
5.06
19.43
46.02
64.21
30.62
2,10
11.51
24.80
59.99
255.7
OTRL
14.73
5.27
19.61
49.01
65.09
30.41
2.09
11.51
26.10
60.15
256.9
ВPOT*
14.85
19.64
48.17
65.51
30.54
2.04
11.43
24.60
60.39
253.9
Table 2:
Automatic evaluation results. represents that it includes pretraining stage. The bottom shows the results our framework and its ablation variants. Best in bold, the runner-up with an underline.
4.3 Evaluation Measures
stories based on the overall quality of the story and their preferences. Also, they line up stories together if they are fairly similar in quality. For each of stories, we have 5 workers annotate it. Appendix A.4 shows more details on automatic and manual evaluation.
Automatic Evaluation We use the following auto- matic metrics evaluate stories. PPL represents the model's perplexity of the stories. Repeat-n (R-n) (Shao et al., 2019) reflects the redundancy of the stories by computing the ratio of the stories that repeat at least one n-gram. Distinct-n (D-n) measures the diversity of the stories by computing the ratio of n-gram types to all generated grams. Tks is the average length of the stories. We also re- port standard BLEU-3 (B-3) (Papineni al., 2002) and ROUGE, (R2) (Lin, 2004).
4.4 Experimental Results
Automatic Evaluation Results are reported in Ta- ble BPOT surpasses other baselines all metries except for repetition, which indicates the generated stories are more fluent and diverse, overlapping more with the reference stories. For repetition, our model outperforms other baselines under sim- ilar configurations (e.g., RL OTRL) ROC- Stories, while showing slightly worse results WritingPromptss, probably because of the much longer generated stories. Concretely, when com- pared to the strongest baselines (Flashback and its variants), Vanilla performs worse than Flashback on ROCStroies. It could be that Flashback serves event arguments mask unit, which helps learn the dependencies between event arguments. Instead, Vanilla is better than Flashback on WritingPompts, especially in Tks. It may be because the stories in WritingPrompts are much longer. Thus, the de- pendencies between event arguments are complex
Manual Evaluation We conduct manual eval- uation ROCStories to verify whether the sto- rytelling system can generate the high-quality events that balance the logic and narrative or- der in generating stories with flashbacks. Specif- ically, we randomly choose 100 stories that have <after> in test sets. For each story, we obtain six versions which are generated by the ablation vari- ants of our framework and three strong baselines. Then, the annotators required evaluate stories on three aspects: Narrative order, Coherence and Overall. For Narrative order, we ask them to label all pairwise event orders in each story. Then the ratio each narrative order calculated, and we further compute the entropy to represent Narrative Order Diversity (NOD). Meanwhile, Narrative Order Accuracy (NOA) is measured by calculat- ing the ratio of the annotated results that are consis- tent with the given temporal prompts. For Coher- ence, we have the annotators rate each story (1-5) according the inter-sentence logic and whether the generated story deviates from the given input. For Overall, the annotators are required to rank
Models
B-3f
R,."
R-2,
R-3)
12-2f
Megatron (ROC)
19.29
60.75
85.42
ContenpPlaiing (WP)
14
95.60
78.16
CTRL (ROL)
5.27
19.61
49,01
65,09
OTRL (Bart large-
3.26
13.21
30,90
61.29
323.4
Table 3: Comparison results two datasets. PPL is missing as it not reported in original paper.

---

Models
N
NOA 1
Coherence
Ovemil
Astogras Exent Tetrectional hatraney
Temporallart
0,913
0.903
2.543
3,398
PR
L037
0.913
2,923
2,975
Vanilla
0,954
0.900
2,400
3,555
OTRL
0,985
0.915
3.425
2,590
Pupplcky (P9
CharGPT
0:474
0.745
4,403
1,530
BPOT
600
Table Manual evaluation results on ROCStories. Best in bold, the runner-up with underline.
accuraty before accurRey
50.05%
Figure 5: The Perplexity (PPL) of different strategies in pretraining stage.
0200
16.27%
55.45%
12.755
Models
PPL.
B:31
R,
R-2|
D-41
0T9L
Autoregressive
15.35
5.22
19:53
49.06
64.97
9.25%
Event Blank Infilling
15.34
5.23
19:55
49.94
65.11
Pairwise Order Prediction
15.04
5.26
19:48
48.73
64.46
BPOT"
14.85
5.31
19.64
48.17
65.51
15.87
5.16
19:40
50.31
62.76
60
6%6
Naive-LL
15.37
5.12
19:48
46.74
63.57
OTRL
14.73
5.27
19.61
49.01
65.09
Figure 4: Detailed statistics on the annotated results of narrative order. For each set, the first line shows the ratio of different narrative orders, the second and third line shows the NOA on <after> and <hefore> respectively.
Table 5: The results of comparative experiments. The upper represents different pretraining strategies com- bined with OTRL. The bottom represents different re- wards, the specific difference is shown in Appendix 8.
and hard to learn, while serving event as mask unit makes it easier learn event correlations, thereby generating longer stories. When attach- ing OT reward, OTRL outperforms all variants of Flashback. Besides, although our pretraining data is about 0.1 of Pretrained (I million), BP performs better, especially in diversity and repetition. BP achieves a growth of 1.66 and 2.1 in Distinction and a decline of 1.66 and 0.4 in Repetition on two datasets, while Pretrained are -0.63, -0.93 and 3.05, 2.2 respectively. The superior perfor- mance on two datasets demonstrates the effective- ness of bidirectional pretraining and OT reward.
narrative order are shown in Figure 4. We can find that the dominant narrative order of the generated story is straightforward (<before>), which is con- sistent with human writing habits and the findings in Ning al. (2018). BPOT performs best in NOD (1.115) and NOA (0.958). proves that inversely inferring pairwise order through bidircctional pre- training facilitates model to be more better aware of <after>, thus reversing narrative order to gener- ate flashback (highest after ratio 24% and highest after accuracy 88%1.) Moreover, OTRL greatly boosts the coherence and overall quality. We be- lieve that the reason that OT reward effectively measures the quality of the generated events. There- fore, sketch model can adjust with reward to gener- ate higher- events. Besides, the bidirectional pretraining also improves the coherence and overall quality. We believe that it is because event correla- tions encoded by bidirectional pretraining facilitate sketch model to plan reasonable event logic, thus alleviating the logical incoherence. Particularly, al- though ChatGPT performs better in coherence and overall due to its huge number of parameters and rich training corpus, it is difficult for ChatGPT to break the dominant narrative order <hefore> and
Moreover, since the experimental settings of Megatron (Xu et al., 2020) and ContentPlanning (Goldfarb- Coorrfatb-Tarrant et al., 2020) are different from other baselines, we conducted separate experiments to compare with them. The results are shown in Ta- ble 3. Megatron achieves better results in diversity as it served GPT2 as the backbone and leveraged external knowledge bases to insert novel words into stories. Content Planning outperforms ours in some metrics because it used more training data and designed a series of classifiers to refine sketch. Manual Evaluation Results are shown in Table 4. More detailed statistics on the annotated results of

---

Input prompt
[s]] serry was bed sleeping.
PR sometimes generates correct flashback. But it faced with logical incoherence. Seeing the upper example, "terry rushed to the hospital" follows "it was a car accident". But the accident is not related to terry so that there is no need for him to rush to the hospital. Instead, BPOT generates the correct flash- back ("had attacked terry's house." occurs in the past and explains why "hearing a loud bang") and maintains the reasonable event logic. Besides, for PR faces
satdenly,
lond crast.
PR
[:]] he jumpeed out hadl vae what wase
[:4] (f was our accident.
[6]] terry rushed the hospina.
[3]] suddenly. the alarm clock blaned.
ChaGPT
up.
realtied be wad
important meeting
[63] framicallyy he rushed get dressed and left the house.
[6]] soaddenly he heard lood bang.
BPOT
bar looked out the wink and sone bear.
bear had Red
[55] Jenry outside called poliic
Input pronatt
friend
up.
[2] ple had a haggtood whoo was cleaatin on has,
complex situations with multiple <after>, PR the problems of inefficient prompt and deteriorat- ing logic. As shown in the bottom example, fails to generate flashback with second <after> and the conflicting relationship with that man has shifted from my friend to me. In contrast, BPOT generates two correct flashbacks ("had a boyfriend who was cheating on her" occurs in the past and explains "broke up". "cheated on her" occurs in the past and explains "asked her why"). Both cases demonstrate the effectiveness of our framework in generating logically coherent stories with flashbacks. More cases are shown in Appendix.
[:] thed to talk to him. but he wouldn't 't talk to me.
PR
lasked him why cheatee
friends with kennooled him Ice creams movies
ChociP
[33] later, be admitted was for the best.
[+[] after few montss, he found somcone new
now: he's huppier ihan before.
had huyfrioud
olecating
RPOT*
[s3] siee rold oe that he cheated on her with another wousen.
asked why.
[65] 1 felt haa for her.
Table 6: The generated stories from different models.
The brown and the yellow represent the correct and failed generated flashhacks respectively. The red repre- sents the logical incoherence.
follow <after> to generate flashback. It is reflected in the poor performance of NOD (0.474), lowest after ratio 8.25% and NOA (0.745), lowest after accuracy 16.36%.
5 Related Work
Story Generation was first approached by sym- bolic and logical planning (PÉrez and Sharples, 2001; Martens et al., 2014). Recently, significant progress has been made in applying deep neural networks in story generation. Fan et al. (2018); Mao et al. (2019) reused seq2seq model to trans- late the prompt into story. Xu et al. (2018); Yao et al. (2019); Martin et al. (2018) introduced the planning-based method, namely planning sketch prior to writing story. Then, numerous works were devoted to designing the format of sketch (Fan et al., 2019; Chen et al., 2020) and enriching sketch with external knowledge bases (Tan et al., 2021; Xu et al., 2020). Morcover, researchers explored controllable story generation through fine-grained control of sketch, such as writing style (Kong et al. 2021), protagonist's personality (Zhang al., 2022). However, they faced the problem of logical incoherence and rarely considered the crucial role of narrative order in story generation. Although Han et al. (2022) proposed the temporal prompt to generate flashback. it faced the problems of ineffi- cient prompt and deteriorating logic in some cases. In this work, we focus on generating logically co- herent stories with flashbacks.
Analysis of BP and OTRL To further verify the effecttivnness of BP and OTRL, we conduct compar- ative experiments on ROCStories. As shown Fig- ure 5 and Table 5, we compare various pretraining strategies and rewards under the same experimen- tal settings (serving event as mask unit). We can find that BP achieves the lowest PPL during pre- training stage. After combing with OTRL, BPOT outperforms other control groups in all metrics. It proves that the bidirectional reasoning supe- rior to unidirectional reasoning and the autoregres- sive manner. Besides, when compared to RL and Naive RL, OTRL performs best. It demonstrates the effectiveness of measuring the event quality in fine-grained way and further considering the one- to-many mappings between the generated events and the sentences in the story. Both comparative experiments verify the superiority of bidirectional pretraining and OT reward.
4.5
Case Study
Two cases are shown in Figure 7. We find that Chat- GPT is almost able to generate reasonable event logic. However, fails to follow the <after> and re- verse narrative order to generate flashback in place. Moreover, for simple situations with one <after>,
Event Correlations and Temporal Order have been proven useful in many event-related tasks

---

(Chen et al., 2023; Liu et al., 2022). Recent works paid attention to utilizing pretraining to encode event correlations and event temporal order. Han et al. (2020) learned to identify the temporal re- lationship between events through masking event triggers and temporal indicators. Lin et al. (2020); Zhou et al. (2022a) explored temporal event or- dering and event infilling tasks for mining tem- poral knowledge. Zhou et al. (2022b,e) adopted event-level pretraining with contrastive learning to capture event correlations. However, they rarely in- volve mutual reasoning between pairwise event and event order. In this work, we jointly encode event correlations and pairwise event order through the bidirectional reasoning between event and order.
IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing. 29:585-593.
Xiuying Chen, Mingzhe Li, Shen Gao, Zhangming Chan, Dongyan Zhao, Xin Gao, Xiangliang Zhang, and Rui Yan. 2023. Follow the timeline! generating an abstractive and extractive timeline summary in chronological order. ACM Transactions on Informa- rion Systems, 41(1):1-30
Marco Cuturi. 2013. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neu- ral information processing systems, 26.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), pages 889- 898.
6 Conclusions
Angela Fan, Mike Lewis, and Yann Dauphin. 2019. Strategies for structuring story generation. In Pro- ceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 2650- 2660.
In this paper, we propose a narrative order aware framework BPOT for story generation, which presents bidirectional pretrained model to encode event correlations and pairwise event order. We also design an RL algorithm with novel OT reward further improve the generated event quality in the fine-tuning stage. Both automatic and manual evaluation results demonstrate the superiority of our framework in generating logically coherent sto- ries with flashbacks. In the future, we will explore how to control the narrative order of long texts (paragraphs) or other narrative modalities (video).
Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Pe- ters, Michael Schmitz, and Luke Zettlemoyer. 2018. Allennlp: A deep semantic natural language process- ing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1-6.
Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. 2020. Content plan- ning for neural story generation with aristotelian rescoring. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 4319-4338.
Limitations
Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, and Minlie Huang. 2020. A knowledge-enhanced pre- training model for commonsense story generation. Transactions of the Association for Computational Linguistics, 8:93-108.
The performance of our proposed framework is re- lated to the used pretrained language model (PLM). Applying our proposed framework stronger PLM may lead to further improvements. Besides, the temporal prompts are obtained by ECONET through majority voting with different random seeds. This inevitably introduces some noise in the data, possibly affecting the final performance.
Rujun Han, Hong Chen, Yufei Tian, and Nanyun Peng. 2022. Go back in time: Generating flashbacks in stories with event temporal prompts. In Proceedings of the 2022 Conference of the North American Chap- of the Association for Computational Linguistics: Human Language Technologies, pages 1450-1470.
A Acknowledgements
Rujun Han, Qiang Ning, and Nanyun Peng. 2019. Joint event and temporal relation extraction with shared representations and structured prediction. In Pro- ceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing (EMNLP-IJCNIP),, pages 434 444.
This research was funded by the National Natural Science Foundation of China (62206267). We sin- cerely thank Shiyao Yan. Changyuan Tian, and Wei Jia for their constructive collaboration.
References
Rujun Han, Xiang Ren, and Nanyun Peng. 2020. Econet: effective continual pretraining of language models for event temporal reasoning. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5367-5380.
Gang Chen, Yang Liu, Huanbo Luan, Meng Zhang,
Qun Liu, and Maosong Sun. 2020. Learning to gen- erate explainable plots for neural story generation.

---

Qingbao Huang, Linzhang Mo. Pijian Li, Yi Cai, Qing- guang Liu, Jielong Wei, Qing Li. and Ho-fung Leung. 2021. Story ending generation with multi-level graph convolutional networks over dependency trees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13073-13081.
Nasrin Mostafazadeh. Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational I. Linguistics: Human Language Technologies, pages $39-849.
Leonid Vitalevich Kantorovich. 2006. On a prob- lem of monge. Journal of Mathematical Sciences,
4(133):1383-1333.
Qiang Ning, Hao Wu, Haoruo Peng, and Dan Roth. 2018. Improving temporal relation extraction with globally acquired statistical resource. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guisticx: Human Language Technologies, Volume (Long Papers), pages 841
Xiangzhe Kong, Jialiang Huang, Ziquan Tung, Jian Guan, and Minlie Huang. 2021. Stylized story gen- cration with style-guided planning. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 2430-2436.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2020. Bart: pre-training for nat- ural language generation. translation, and comprehen- sion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311-318.
Denoising
Rafael PÊrez Y Pérez and Mike Sharples. 2001. Mex- ica: A computer model of cognitive account of creative writing. Journal of Experimental Theoret- ical Artificial Intelligence 13(2):119-19,
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei. Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9.
Shih-Ting Lin, Nathanael Chambers, and Greg Durrett. 2020. Conditional generation of caaporally-orderee event sequences. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the I 1ith International Joint Conference on Natural Language Processing, pages 7142-715
Zhihong Shao, Minlie Huang. Jiangtao Wen, Wenfei Xu, and Xiaoyan Zhu. 2019. Long and diverse text gen- eration with baaningg-bssed hierarchical variational model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP), pages 3257-3268.
Nayu Liu, Kaiwen Wei, Xian Sun, Hongfeng Yu, Fang- long Yao, Li Jin, Guo Zhi, and Guangluan Xu. 2022. Assist non-native viewers: Multimodal cross-lingual summarization for how2 videos. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages
Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approxima- tion. Advances in neural information processing systems, 12.
Huanru Henry Mao, Bodhisattwa Prasad Majumder,
Julian McAuley, and Garrison W Cottrell. 2019, Im- proving neural story generation by targeted common sense grounding. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural anguage Processing (EMNLP-IJCNIPP) pages
Bowen Tan, Zichao Yang, Maruan AI-Shedivat, Eric P Xing, and Zhiting Hu. 2021. Progressive generation of long text with pretrained language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 4313-4324.
Chris Martens, Joao F Ferreira, Anne-Gwenn Bosser, and Marc Cavazza. 2014. Generative story worlds as linear logic programs. Intelligent Narrative Tech- nologies, 7:51.
Kaiwen Wei, Xian Sun, Zequn Zhang, Li Jin, Jingyuan Zhang, Jianwei Lv, and Zhi Guo. 2022. Implicit event argument extraction with argument-argummnt relational knowledge. IEEE Transactions on Knowl- edge and Data Engineering.
Lara Martin, Prithviraj Ammanabrolu, Xinyu Wang. William Hancock, Shruti Singh, Brent Harrison, and Mark Riedl. 2018 Event representations for auto- mated story generation with deep neural nets. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Kaiwen Wei, Xian Sun, Zequn Zhang, Jingyuan Zhang, Guo Zhi, and Jin. 2021. Trigger not sufficient: Exploiting frame-aware knowledge for implicit event

---

argument extraction. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th 11th International Joint Confer- ence on Natural Language Processing (Volume I: Long Papers), pages 4672-4682.
model for event correlation reasoning. In Proceed- ings of the ACM Web Conference 2022, pages 850- 859.
Yucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long, and Daxin Jiang. 2022c. Claret: Pre-training a correlation-aware context-to-event transformer for event-centric generation and classification. In Pro- ceedings of the 60th Annual Meeting of the Associa- tion for Computational Linguistics (Volume I: Long Papers), pages 2559-2577
Kaiwen Wei, Yiran Yang. Li Jin, Xian Sun, Zequn Zhang, Jingyuan Zhang, Xiao Li. Linhao Zhang, Jintao Liu, and Guo Zhi. 2023. Guide the many- to-one assignment: Open information extraction via iou-aware optimal transport. In Proceedings of the 61 Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 4971-4984.
A Appendix
A.1 Details on Datasets and Pretraining data
Yujia Xie, Xiangfeng Wang. Ruijia Wang, and Hongyuan Zha. 2020. A fast proximal point method for computing exact wasserstein distance. In Uncer- tainty in artificial intelligence. pages 433-453, Pro- ceedings of Machine Learning Research (PMLR).
ROCStories (Mostafazadeh et al., 2016) It con- tains 98,162 five-sentence short stories. The average length of the story is about 42 words. Following (Xu et al., 2020; Han et al., 2022) we split the data into 88,344/4.90//4.99 for train/validationneesst sets.
Yuqiang Xie, Yue Hu, Yunpeng Li, Guanqun Bi, Luxi Xing. and Wei Peng. 2022. Psychology-guice trollable story generation. In Proceedings of the 29th International Conference on Computational Linguis- tics, pages 1480-6492.
con-
Writing Prompts (Fan et al., 2018) It contains 30,335 pairs of prompts and stories. The average length of the story is over 700 words. Thus, the stories in WritingPrompts those in ROCStories. Following (Han et al., 2022), we select stories with a maximum of 500 words, leading to a total number of 96,488 training and 5,784 validation prompt-story pairs, respectively. For the test set, we utilize the 1000 prompt-story pairs provided by the compared baseline (Goldfarb- Tarrant et al., 2020). The temporal prompts in both datasets are provided by Han et al. (2022).
Jingjing Xu, Xuancheng Ren, Yi Zhang, Qi Zeng. Xi- aoyan Cai, and Xu Sun. 2018. A skeleton-based model for promoting coherence among sentences in narrative story generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4306-4315.
much longer than
Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, and Bryan Catanzaro. 2020. Megatron-entrll Controllable story generation with external knowledge using large-scale language models. In Proceedings of the 2020 Con- ference Empirical Methods in Natural Language Processing (EMNLP)), P),, pages 2831-2845.
Pretraining data For pretraining data, we utilize the event sequences provided by Lin et al. (2020). It is originally used to pretrain with the tasks of temporal event ordering and event infilling. Thus, the event sequences are full of the knowledge of event correlations and event temporal order. We further process it as follows. For each event se- quence, we segment all its pairwise adjacent events. For each pairwise event, we separately restore their original sentences and detect their event triggers by Han et al. (2019). Then we feed their event trig- gers and contexts into three ECONET (Han et al., 2020) with different random seeds to get pairwise event order. We only adopt the result when the event orders judged by the three models are the same. Otherwise, we label its pairwise event order as <vague>, which means arbitrary event order. Note that there may be multiple results of semantic role labeling for a sentence in datasets, we choose the results whose event trigger is the ROOT node in the syntactic analysis as our event. By doing so,
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan- and-write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages
Zhexin Zhang, Jiaxin Wen, Jian Guan, and Minlie Huang. 2022. Persona- Persona-guided planning for control- ling the protagonist's persona in story generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 3346-3361.
Bo Zhou, Yubo Chen, Kang Liu, Jun Zhao, Jiexin Xu, Xiaojian Jiang, and Qiuxia Li. 2022a. Generating temporally-ordert event sequences via event op- timal transport. In Proceedings of the 29th Inter- national Conference on Computational Linguistics, pages 1875-1884
Yucheng Zhou, Xiubo Geng, Tao Shen, Guodong Long, and Daxin Jiang. 2022b. Eventbert: A pre-trained

---

we finally obtain total of 100k event sequences as our golden sketches for pretraining.
model can focus more on pairwise order predic- tion in the early stage of pretraining, then utilizing the awareness of event temporal order to generate better events in event blank infilling.
A.2 Implementation Details
Unless specifically mentioned, use the pre- trained Bart-base" as our sketch model and story model. The initial vocabulary of BART contains 50265 tokens. If the compared model which could be baselines or ablation variants of our frame- work adopts the planning-basee method, we add an event ending (<eoe>) token. If it inserts temporal prompts into sketch, we add three additional narra- tive order tokens (<hefore>, <after>, <vague>). On ROCStories, we serve the event in the first sen- tence of each story as the input event. On Writing- Prompts, we serve the prompt as the input event. For ROCStories, the syper-parameters are learning rate: 5e 5e-6; 5; batch size: 10; 3: 0.7; gradient accu- mulation: 1. To fairly compare with (Han et al., 2022), we use three random seeds (5, 9998, 20016) and report the average performance of all models evaluated on ROCStories. For WritingPrompts, the hyper-parameters are learning rate 25-5; batch size: 64; 3:0.7, gradient accumulation: 8. For both datasets, we fine-tune on a single Tesla V100 GPU with 32G memory. When compared with the baseline ContentPlanning (Goldfarb-Tarm) 2020), we fine-tune Bart-large on a single A 100 GPU with 80G memory. The training time for per epoch ROCStories and WritingPromptt is 2.5- 3.5 hours and 24 hours respectively. We train our framework and its ablation variants for 10 epochs and save the model with the best evaluation perplex- ity. For pretraining, dyper-parameters are learning rate 1e ; batch size: 10; gradient accumulation: 10. We also pretrain 10 epochs and save the model with the best evaluation perplexity. The weight factor m,, 172 during pretraining are computed as follows:
A.3 Details on Baselines
Bart: (Lewis et al., 2020) It is a typical seq2seq model for natural language generation. We fine- tune it on both datasets. We directly serve the first sentence of story on ROCStories and prompt of story on WritingPrompts the input, and the entire story as labels to fine-tune the model.
Bart (planning): It adopts the planning-based method based on Bart. We first use sketch model to generate sketch and then use story model to gen- erate story given the generated sketch. The de- tailed process is consistent with the vanilla pipeline generation at the bottom of Figure except that we replace all temporal prompts with <eoe>.
TemporalBart: (Lin et al., 2020) It is designed with the tasks of temporal event ordering and event infilling. We pick it as it implicitly encodes event correlations and event temporal order. Note that the temporal prompt is not used during pretraining. Concretely, we use its pre-trained weights to initial- ize the sketch model and then adopt the workflow of Flashback (Han et al., 2022) to complete the subsequent fine-tuning.
et al..
Megatron: (Xu et., 2020) It used GPT2(Radford et al., 2019) and leveraged external knowledge bases to generate the commonsense stories. We picked it as it outperforms previous systems (Guan et al., 2020) on ROCStories. Due to its operation of delexicalization that replaces names and enti- ties with [MALEJ, [FEMALE], and [NEUTRAL], when compared with it, we strive to recover its original entities and name.
ContentPlanning: (Goldfarb-Tamm) Tarrant et al., 2020). It adopted the planning based method on Writing- Prompts. As reported in Han et al. (2022), our final training data is about two-thirds of theirs. More- over, they do not adopt end-to-end training as they additionally design a series of classifiers to refine the sketch.
1/1 3- (global step/total step)
(10)
1/2 1 + (global step/totat step)
(11)
where the global step represents the current number of iterations that have been updated and the total step represents the total number of iterations to be updated. It means that the influence of event blank infilling will gradually increase during pretraining, while the influence of pairwise order prediction will gradually decrease. It is because we hope
Flashback: (Han et al., 2022). It first consid- ered the crucial role of narrative order in story generation and proposed temporal prompt to gen- erate flashback. It is the strongest baseline for our work. The detailed process consistent with the bottom of Figure 3 except it served event ar- guments as mask unit. Concretely, each masked

---

event in input is represented as "<mask>> ; <mask> <mask> <Temporal Prompt>" rather than "<mask><Tempora" Prompt>" in our framework. We serve event as mask unit as we deem that it fa- cilitates model to directly learn event correlations, especially in long stories. The results of automatic evaluation on WritingPrompts also demonstrate it. To further verify the effectiveness of our methods, we compare with its three variants. The only RL simply regards the negative story loss as reward R, which means that it returns an identical reward for all events in the generated sketch. The Pretrained used I million event sequences which are ten times for our pretraining data. Moreover, it adopts au- toregerssive manner during pretrianing stage. The autoregressive manner represents that it unidirec- tionally reasons all event arguments from left to right. The PR is the integration of RL and Pre- trained, which means that it uses the pre-trained weights to initialize sketch model and combines with RL in the fine-tuning stage.
ware you drosstalee you story no more then wors, which are logicall y coharant and conter apecific namathvv order.
centricc short
Contect: The nammatte order of story is hirted by serverlloaivionn event ordens. "Te hope twfi over oreer inclades "etofireod" "suttrc" land "eregush 5. Speckooury sarteecalicteerroroeee
rndicasestte moans ownnt
semencel
the
event senteneed: wwtperootttttttteeeeeeeee than
later
Saartenstt saguar
coneenooe2 arbittary essence "f"ttterr" nocriing nummatiee ordur so dlee the past svants for Blattbeck generation laking centences "hhe policee were trying to catch ne gritofooottheeee the thee thd stolen Eat Usay linalll unodted trant and commussetng mogiing namraiie order" "ullurneiuffrrr" la coampple "tyynng to cotrh a neightorhoodttee rt noru bew than "had a cort, "hoo stolees oocura oar lor "arnssted tinn" Tharctane for fiee sentamec stary well provids notion evenn to hirt narenntattt order ohthee Please calanoe log order
event insentencet and
Irpuu sentence "wino nescied tnlood doseet, opes namatiiv <oaforeicction wnteetteeeteteteeeeeee. staet with the seetiennc and gerrrattt fro-saettanco b lf short otory no more than 48 worls, whitch flys the stooe speecllo nan roline order.
story."
Cumput it newdedt nodt teets done, dha hartherr trelingumwnt for wreke the doctor onderedit to determens no cause, whenit was away, ralee washed her the bload results came back pertectiv
mine
speattic
s-bbotoree-sttteee weteeeettttt-tttteeeeee setot-ttoooeei start with serienice penerate five sontience
fo af short altry more then 48 raa which th the above apacc nemmative orderr.
outpost:
mine just up with waw for the best After tew morthe he found comeonn new how, het happierthon ever
Figure 6: The instructions for ChatGPT on ROCStories.
ChatGPT: we pick it because of its powerful abil- ity in NLP. We only compare with it on ROCStories. Because the stories in WritingPrompts are much longer and contain dialogue or short phrases with- out events, which makes it hard for annotators to judge the order of pairwise event. Referring to the tutorial* ?, we design the instruction for ChatGPT as shown in Figure 6. Note that we prompt ChatGPT to generate stories no more than 48 words as the average length of the story on ROCStories is about 42. If the length is not constrained, the stories gen- erated by ChatGPT are much longer than 42 words, which is adverse to fair comparison. Moreover, al- though we hint the story should be five-sentence. ChatGPT sometimes generates more than five sen- tences. In this case, we only choose the first five sentences as the final story.
training. For WritingPrompis, is hard for models to generate such long stories in training sets due to the limitation of the model's capacity. Therefore, the Tks can reflect the performance of the model to some degree.
Manual Evaluation Measures: We randomly sam- ple 100 stories from the test sets of ROCStories. For each story, we get 6 versions which are gen- erated by different storytelling systems. Each set of stories is evaluated by 5 annotators on three as- pects: Narrative Order, Coherence, Overall. It should be noted that all annotators have linguistic backgrounds and the order of storytelling systems in each set is shuffled. The specific instructions are shown in Figure 7 and Figure 9. Specifically, we clearly introduce the definition of the task and the evaluation method for each metric. Moreover, we provide an example with the detailed explanation for the corresponding task. For each set of stories, we require four annotators to rate coherence and rank overall. Then, average their values and obtain the final results reported in the manual eval- uations. We also ask one annotator label the pairwise event order to get the narrative order.
A.4 Details on Evaluation Measures
Automatic Evaluation Measures: We find that our models can achieve nearly 0 in Repeat-3 on ROCStories and in Repeat-4 on WritingPrompts, which is in line with the findings in Han et al. (2022). Therefore, we report Repeat-2 on ROCSto- ries and Repeat-3 on WritingPrompts. For Tks, we only report on WritingPrompis. Because the stories on ROCStories are relatively short so that the gener- ated stories are almost the same length through full
A.5 Comparison of Reward
The comparison of various rewards is shown in Figure 8. The RL in (Han et al., 2022) is con- structed at the sevel, which means that all
design

---

Instruction for [taskl: judge narradive order] this tack yse will
Instruction for [cask2: rate coherence and rank overall)
DI couttis Drevssattaee stoies geseuaee by difireall ducyiellau sysuus daay nila saau aquat dossted as
You aee reguoeed jaäge the uaneiiee odder of ssch atoy se, The temoriie order of the svey refleced
la liis tok, you will be jives ott of doxics Susaiiis vix Grescaltace shoike praccsted by differeat stepteling ay aloen with the sane upsat densted A. K, C, DD F, F. You requie own [1-5 enternnze on de logical corclatie of iaateeattaae and wheles dae jescuate shay deciatts Lieus la given aegat You ski regaieed mak (1-6) the dtries acomniing fhei coweall quility hoteally it cuttieel you malee a guurtiariiv jroggeet on the oveail quality the sweyy We hepe you can mad the qualiy la shey based ou your owa prefermace slier consederisa dioeccy innreet the story Ityon deso turt quality the stodiss is sinulae you smige thon fhe wene reaking, Hare is dksilldd reformoc comppl with explesntion.
Tlarefor, you soquiod to label parrnis cnders each siery. The types svert onder techsde twriie: "wher" "oague" Specifially. evear, ocenrs excler that event), fies the eveat oider berteen thees "betber" event secur lata than cventz, fie
onks
evey
strry
"sffrr you the onter bertowes parnti snout this you bial "rager" Time is detuled cofeennce secople will exp eeplannison. Not fast the sarkare saneancilly" "f, 7P
slent
Retiereced Story.
desssate was knowa to bonile tanker, cen day, chesiadd to look inn twinlinn's is hemme. loske horse
the ehere kay now toke, when be got fane. be roslindd diart tove lis toke. he weat laack home and changed las tire nows douatai dimal have tik
System doniniic was bnown be trrable naber one decided
frings dourzie's is pareats vey rppce. They
soouaded hau fix soarls
Systear dkuaisi
lesa boalle sadess oae sky, dosidtdl intor with
Restrence Assoraton Besals: "betbee", "before", "befare" : "tefare"
gest the pot was irvekia with the potice the peloo amededd decuma nowe, cherezae dowarh boalll suyuooe
[orrience 142) Deplonotien: The event serrenca) in "was bnowe be mobble rambn" the eveat sestaaoo2 "docided break knto neighoo''s 's bous"" can obain time indeowe "one day" that eveest aoemed beter flos evertt.
Systear domaisic
be tookle takes uee day, deessled tank vilke. zade vodeo of hitsell throwing a rock ar neighber's window the crigitor sow the video and cilled die yolize. the yelicr asoste deniisic for anilin the válee. Systess
Ivniiane 28,3 Explastion, lasee", fee cocnta anttoneel is "boke into barad" and "voie fiis thine'' it ckor that "deeeeer to book ieto" stured oo foo torckn yosr" and waite all bac tiveee" Ioaakaace 3&41) Explazaiion The his things The evest in verencel in "were vey speat" th obsices that "wee owyy sppaet by "tudir Lato the boose" and all oclis linge" Tass "tosk iato the bose" stared audar.
varissee "Mosided to lonk seiallon's
ducikted break seiailio''s house he booke into home and stede ell ofhis things domisoe' paccrts wase vayy spect dayy arvended laa for mocth. Syrtein domiic
be busall tanker
vtalaadd "look
liste" "nide
mobile maber one day,
go the police
ataikos the golice ollice fold his that le had wannet for lis smot dosisii wast le jull le cnsionned communttt weare Systen F.
oue day. aiiz intriaad,, le calocked on beasure lasa le Sellowed dac clacs degaing up the badkyaad. fuall, be assomeedd ches Uiedd
now
ki- mep hatffee
[onteecee Eegionntiinnn The ecent oestencel is "wore very spoat! The event oeriencd "rooadddd hin'. & dker that "Mase ugueff stared cadier thas "aronddd hin". Thas onliee.
actiick
Expl explicnttoo (Red aate oxlicti mucebmnennt Percolanccs, ween find that Sodtmm As were (aastkereat wholl remsining system compurabla (Il wik aciiiis wark logic is intereenteess to the givea aqpit) As resut, the nessecre of coleeiane shoild to and Dy (Dl and E and F3-- A.
Figure 7: Instruction for narrative order
and Systea perfonus best. The perfiruaacee
ruloned
difiernnes of eveull qelity quite distifiou basel ceherence's parfermanee Cradd 10 can be forther daringrsshed baed on panveni per perfomanca.
Figure 9: Instruction for coherence and overall
Na
OT-XL
Expat prompt
[=]] my friend jou hes kisnay probleme.
Figure 8: Comparison of different rewards in reinforce-
<hefiee sonffooosafllero-o-wwaa <sogurss
The is lrying ln gei hetter.
ment learning
PR*
[+4] the doctor sold joe he had kidney stones.
k3] be went to the doesce for dieckup
[s5] joe said
problem.
events in the generated sketch receive the identi- cal reward. The reward is regarded as the negative story loss. Therefore, it cannot measure the quality of the generated events because there is no dif- ference between their rewards. The Naive-RL is constructed at the event-level, which means that different events in the generated sketch receive the different rewards. The reward is regarded as the negative sentence loss. Therefore, the sketch model can perceive the quality of generated event through comparing the reward of differrnnt events. However, the mappings between the generated events and the sentences in the story are modeled by one-to- one. But they actually should be one-to-many as an event may influence multiple sentences in story. Therefore, can't effectively measure the event quality as the mappings are inaccurate. The OT- RL is constructed at the event-level. It also means that different events in the generated sketch receive the different rewards. The reward is regarded as the weighted sum of sentence loss. The weights are the mappings between the generated events and the sentences in the story, which are captured by OT.
[-]] unllrggees smasesfal trampphent
ClaGPT
[-3| feels like new perions
[s4] be starts volunteering at the hospital.
[s5] be insprts ethers to give thegift life.
[x]] he want to the hospitil in his kilney chackeal ond
BPOT
doctor sold joe had kidney stones.
[s4] joe was la
of pain.
[-5| be sargery to remove the stones.
Ixpai prompt
[+]] kim hasd il humnng sansation her bladder
caefore--after>>s cafters---s-eeeeeeeee
<before>
[62] sibe west to the bothmom to find out what was.
[s]] she realized she had flushed the tollet,
PR*
[54] who cailked the plunher.
[-5 he came and fised the toiles
[62] she rushed to the restroom but it was occupied.
CharGiT
[s3] the discomfert gew inbearabla)
[-]] finllly shu found relief and sighed nolief. take [=2] sie went to the bathromm see what was wrong.
her health.
[s3] the realized that she had a clogged bladder.
BPOT
[-4] kim hadl to gu la the emergenc room.
[-5] the doctor gave her some medication to help her.
Inpet
|st bom was depressed.
prompt
sefter>-hhfoee--ssa <booore
[2] he didn hove such money.
PR*
k]] he decided to buy a lottryy ticket.
[s4] be spent a lot of money.
[s5] tom won the jackpot.
[s]] her dial'' ' ferl like doing anything.
ChaiGPT
lay hed all day, senolling through social media.
[s4] the day went on, he wasse.
[s5] be decided to call a friend for help.
[<]] lor diuto 's have anysking to cheer hiim up.
BPOT
[<]] he decisled to the doctor.
[s4] the doctor gave prescription antidepressants.
[s5] tom anchh better.
Table 7: More case studies on ROCStories. The brown and the yellow represent correct and failed flashbacks. The red represents the logical incoherence.


Reviews:
[{"review_id": "FXc9cRc4NL", "reviewer": "Reviewer_qVqz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your positive feedback and constructive comments.  \n\n(1) Regarding the relation between our proposed method and building the flashback stories.  \n    ​\t​    ​\tRes: Thank you for your detailed comments. In this paper, we adopt the temporal prompt (*<after>*) to reverse the narrative order to describe the past event for flashback generation (e.g., given \"have a rest *<after>* <Mask>\", the \"<Mask>\" might be infilled with flashback \"feel tired\" as \"feel tired\" occurs earlier than \"have a rest\"). However, building flashback stories requires generating high-quality events that balance the logic and the narrative order rather than simply adopting temporal prompts in the specified position. To better balance the logic and the narrative order, we propose a bidirectional pretraining model. It pretrains event blank infilling (order2event direction: aims to reason event from pairwise event order) and pairwise order prediction (event2order direction: aims to reason order from pairwise events) to facilitate the model to more sufficiently understand the event correlations and event temporal order. To generate higher-quality events, we propose a reinforcement learning algorithm where a novel optimal transport reward is designed to effectively measure the quality of generated events. It aims to guide the model to improve the quality of events by punishing low-quality events that receive low rewards. Through extensive experiments, it can be observed that our method can improve the flashback ability and outperform other methods (88.18% vs. 72.73% flashback accuracy in Figure 4 and 4.113 vs. 2.923 coherence score in Table 4). We believe that our method is highly correlated with generating flashback stories.\n\n(2) Regarding proving the improvement of flashback ability through experiment.  \n    ​\t​    ​\tRes: Thank you for your detailed comments. In section 4.3 of our paper, we conduct a manual evaluation and specially design two metrics: Narrative Order Diversity (NOD, the higher the NOD, the more flashback stories are generated) and Narrative Order Accuracy (NOA, the higher the NOA, the higher the efficiency of generating flashback stories) to evaluate the flashback ability of different methods. The results in the first two columns of Table 4 demonstrate that our method consistently outperforms other baselines and ablation variants. Specifically, our method achieves the highest score on NOD (1.115) and NOA (0.958). To further evaluate the flashback ability of different methods, the statistical results of various narrative orders (\"after\" represents flashback) are presented in Figure 4. In the last bar of Figure 4, it can be observed that our method achieves the highest after ratio (24%) and highest after accuracy (88.18%), which confirms the strong flashback ability of our method. Additionally, a series of case studies are provided in Table 6 and Table 7, showcasing the generated flashback stories by our method. All of the above proves that our method can improve the flashback ability and outperform other methods.\n\n(3) Regarding building the narrative order and the overemphasis on the concept of \"narrative\".  \n    ​\t​    ​\tRes: Thank you for your constructive suggestions. To the best of our knowledge, building the narrative order is a relatively new issue that has hardly been covered in previous research. It is also indeed related to many aspects (e.g., sentiment, role). In this paper, we follow the pioneering work [1], which reduces building the narrative order to modeling the pairwise event order. Besides, we also consider event correlations to help build the narrative order. The case studies in Table 6 and Table 7 indicate that the pairwise event order can effectively influence the narrative order, which proves the rationality of this simplified modeling approach. Additionally, in the final version, we will pay more attention to distinguishing the differences between narrative order and pairwise event order, avoiding overemphasizing \"narrative\". A discussion of this simplified modeling approach will also be added in the \"Limitation\" section. In the future, we will consider more aspects to comprehensively build the narrative order.   \n[1] Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts.  2022 NAACL\n\n(4) Regarding the reproducibility and typo error.  \n    ​\t​    ​\tRes: Thank you for your detailed comments. Due to the double-blind policy, we can't directly post our code in the article, but we have submitted our code as an attachment to the review system. Additionally, we promise to make the code publicly available and describe more details for reproduction. We will also thoroughly proofread our paper and correct typos or grammatical errors in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The overall work is solid and the experiments are sufficient.\n\n2. The problem is interesting and so is the solution, especially the event masking tasks.\n\n3. The paper writing is qualified to clearly introduce the work.", "reasons_to_reject": "1. The authors claim that the proposed model can handle the narrative order and build the flashback stories. However, in my opinion the keys of the proposed method, BPOT, lie on the event masking based prediction tasks and the RL based fine-tuning, which both do not embody the motivation for building the flashback stories. Also the experiments cannot prove the flashback ability, instead of just showing the improvement of general story generation quality.\n\n2. In terms of building the narrative order ability, I think it does not just mean the aspect of event. Other aspects include sentiment, role, storyline and so on. So the authors may over-claim the concept of ‘narrative’. I think this paper should be focused only within the scope of ‘event aware story generation’. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper investigates the claimed problem of narrative order aware story generation in a two-stage paradigm, i.e., pre-training the model with novel event-level mask pridiction tasks and fine-tuning the model with a OT reward based RL algorithm.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Some typos or grammatical errors in Line 50 52 142 198…"}, "extra_text": {}}, {"review_id": "WcpVZNuJry", "reviewer": "Reviewer_bja9", "summary": "", "questions": "The overall experimental setup of the paper is relatively weak, with few baseline models. It is hoped that more comparative experiments, such as [1], can be added. \n\n[1] Follow the Timeline! Generating an Abstractive and Extractive Timeline Summary in Chronological Order", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your positive feedback.\n\n(1) Regarding the addition of a baseline.  \n    ​\t​    ​\tRes: Thank you for your constructive suggestions. we carefully study this meaningful work [1], which proposed a unified abstractive and extractive timeline summarization framework. It first leverages a graph-based encoder to model the mutual influence among events. Then, it innovatively designs the time-event memory and evolutionary attention to learn temporal information and guide the generation. With that in mind, we integrate their core innovations into the \"vanilla\" version of our model to construct a corresponding baseline. The specific details are as follows:  \n* First, we utilize a pretrained sketch model to generate sketches with temporal prompts. Later, we input the generated sketches into the BART encoder to extract word representation and event representation from the last hidden state of the encoder.  \n* Next, following [1], we adopt a graph-based encoder to encode the global event representation.  \n* Then, following [1], we denote the representation of temporal prompts as the key of the time-event memory as it explicitly expresses time series information. We also serve the local event representation and global event representation as the value of the time-event memory.  \n* Last, following [1], we replace bart decoder with RNN and realize the evolutionary attention in the process of decoding. \n \nDue to time constraints, we have not yet completed the experiments. It is because RNN can't perform parallel operations like Transformer, resulting in lower efficiency. Actually, when serving Transformer as the decoder, it takes about 30 hours to complete the experiment on ROCStories as reported in our paper. However, it is estimated to take about 3.5 days when serving RNN as the decoder.  Furthermore, there are some essential differences between our work and [1]. Specifically, their temporal modeling is monotonic (always \"from the past to the future\"), while our temporal modeling is diverse (flashback includes \"go back to the past\"). Therefore, our temporal modeling is inherently more complex and challenging as it models more diverse temporal relations. Besides, we also consider event correlations to help temporal modeling. In the final version, we will complete the experiment and cite this meaningful work [1] as our additional baseline model.  \n[1] Follow the Timeline! Generating an Abstractive and Extractive Timeline Summary in Chronological Order 2022 TOIS", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The motivation and experiment setting is reasonable. ", "reasons_to_reject": "The overall experimental setup of the paper is relatively weak, with few baseline models. It is hoped that more comparative experiments, such as [1], can be added. \n\n[1] Follow the Timeline! Generating an Abstractive and Extractive Timeline Summary in Chronological Order", "questions_to_authors": "The overall experimental setup of the paper is relatively weak, with few baseline models. It is hoped that more comparative experiments, such as [1], can be added. \n\n[1] Follow the Timeline! Generating an Abstractive and Extractive Timeline Summary in Chronological Order", "paper_topic_and_main_contributions": "The paper introduces the BPOT (Bidirectional Pretraining Model with Optimal Transport Reward) framework for story generation, aiming to tackle the challenges related to event correlations and narrative order in existing storytelling systems. To address these challenges, the framework utilizes a bidirectional pretrained model, which captures event correlations and pairwise event order. Additionally, a reinforcement learning algorithm with a novel optimal transport reward is employed to enhance the quality of the generated events. The results showcase the superior performance of the framework in generating logically coherent stories with flashback. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "t1MXPm9ngq", "reviewer": "Reviewer_tdcS", "summary": "", "questions": "1. How to compute PPL for planning-based models? And how to compare PPL of different models with different formats (e.g., with or without temporal prompts)\n2. Can you clarify the intuition and motivation of the OT reward? I am not familiar with IPOT. It is better to let readers know how to compute T.\n3. When the sketch is model-generated, how do you compute the story loss?\n4. What is the relation between Eq.8 and Eq.9? How are they used, and what are they used for?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your positive feedback and constructive comments.\n  \n(1) Regarding the difficulty of following the method and reproducibility.  \n​\t​\tRes: Thank you for your detailed comments. Actually, the obscurity of the concept of \"narrative order\" itself makes it more difficult to understand the method. With that in mind, we have made great efforts to clearly express our approach. For example, the section \"Preliminary\" is set up to help understand the basis of our method. Each core design of our method is equipped with a figure to explain (Figure 2 for bidirectional pretraining, Figure 3 and Figure 9 for optimal transport reward). We also provide pseudo code to illustrate the entire algorithm. In the camera-ready version, we will further polish our paper to make it easier to understand. We will also make the code publicly available for comprehension and reproduction.\n\n(2) Regarding the research significance of story generation.  \n​\t​\tRes: Thank you for your detailed comments. It is well known that ChatGPT is a very powerful natural language generation tool. However, recent studies have shown that current ChatGPT performed poorly in event temporal reasoning [1] [2]. And its poor performance in narrative order story generation reported in our paper (0.474 in Narrative Order Diversity, 0.745 in Narrative Order Accuracy) also supports this view. This is because narrative order story generation requires the model to have temporal reasoning capabilities to understand the pairwise event order. Therefore, current ChatGPT might perform well in general story generation. However, it still has a gap in the performance of fine-grained controllable story generation that combines narrative order, causality, sentiment, etc. Thus, we believe that story generation is still worth researching. To a certain extent, it can also reveal the shortcomings of LLM and make improvements.  \n[1] Chatgpt evaluation on sentence level relations: A focus on temporal, causal, and discourse relations.  2023 arXiv  \n[2] Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models.  2023 ACL \n\n(3) Regarding the computation and comparison of PPL.  \n​\t​\tRes: Thank you for your detailed comments. PPL in this paper represents the model's perplexity of the golden stories. Formally, let $S_i$ , $Y_i$ denote the $i$-th generated sketch and $i$-th golden story in planning-based model, respectively. Then, the perplexity of $i$-th golden story $PPL_i$ can be computed as below:  $$PPL_i = exp(-\\sum_{k=1}^{|\\hat{Y_i}|}logp(\\hat{Y_{i,k}}|S_i,\\hat{Y_{i,<k}}))$$\n$p(\\hat{Y_{i,k}}|S_i,\\hat{Y_{i,<k}})$ is the probability of the golden token $\\hat{Y_{i,k}}$, which is computed by the last hidden state of  bart decoder. The overall PPL of the golden stories in testset is the average of all $PPL_i$, which can be represented as: $PPL = \\frac{1}{N}\\sum_{i=1}^{N}PPL_i$, where N is the number of the golden stories in testset. Note that for different models, their formats of $S_i$ may be different (with or without temporal prompts), but their target output remains consistent (pure golden story $Y_i$ without temporal prompts). As a result, we can use the same method to calculate PPL for different models and directly compare them.  \n\n(4) Regarding the intuition and motivation of OT reward.  \n​\t​\tRes: Thank you for your detailed comments. The intuition and motivation of OT reward is to model the one-to-many mappings between the generated events and the sentences in story so as to effectively measure the quality of the generated events. Specifically, the model is optimized with the reward of RL in fine-tuning stage. And we hope that lower event quality leads to lower reward. Then the reward can guide model to generate higher-quality event by punishing low-quality event that represents low reward.  Thus, effectively measuring the event quality is critical as the optimization can be misleading if the reward is not accurate. However, existing rewards can't effectively measure the generated event quality. Briefly, they either fail to differentiate the quality of generated events (return identical reward for all generated events), or mistakenly model the mappings between the generated events and the sentences in story as one-to-one. The specific discussions have been written in the line259~272 and Appendix A.5 of our paper. As a result, we propose a novel OT reward to effectively measure the quality of generated events. Also in the camera-ready version, we will supplement the details of IPOT and explain how to utilize IPOT to compute the optimal transport matrix (T).  \n\n(5) Regarding the computation of story loss when the sketch is model-generated.  \n​\t​\tRes: Thank you for your detailed comments. The story loss is actually $L_{\\theta}$ in the equation 3 of our paper. Specifically, let $S_i$ , $Y_i$ denote the $i$-th generated sketch and $i$-th golden story, respectively. The story loss is computed by summing up the negative log-likelihood loss of all included tokens as below:  $$L_{\\theta} = -log(\\hat{Y_i}|S_i)=-\\sum_{k=1}^{|\\hat{Y_i}|}logp(\\hat{Y_{i,k}}|S_i,\\hat{Y_{i,<k}})$$ We will provide a detailed description of the process of computing the story loss in the camera-ready version.  \n\n(6) Regarding the equation 8 and equation 9 of our paper.   \n​\t​\tRes: Thank you for your detailed comments. Actually, there is no apparent connection between equation 8 and equation 9. Formally, they utilize the optimal transport matrix ($T$) to compute the reward in reinforcement learning and the optimal transport loss, respectively. The purpose of equation8 is to make reward consider the one-to-many mappings between the generated events and sentences in story. The specific details are illustrated in the upper left corner of Figure 3 and Appendix A.5. After executing equation 8, the reward for $i$-th event is not only determined by the $i$-th sentence but also by other sentences. This is because $r_i$ is the weighted sum of $L_s=\\\\{L_{s1}, L_{s2}, ..., L_{sk}\\\\}$ rather than only $-L_{si}$. As a result, one event will receive different rewards from different sentences in story so that the mapping is one-to-many (one event to many sentences). The purpose of equation9 is to measure the cost of moving the event semantic distribution in the sketch to sentence semantic distribution in the story. It is subsequently used to optimize the whole network, pushing these two distributions closer in semantics [1].   \n[1] Generative Modeling with Optimal Transport Maps.  2022 ICLR", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The reward design is novel\n2. The experiment is sufficient and convincing.", "reasons_to_reject": "1. The method part is difficult to follow, including the wording, structure, and graphs.\n2. Zero-shot ChatGPT is still very strong. So I am suspectible of the research significance of story generation.", "questions_to_authors": "1. How to compute PPL for planning-based models? And how to compare PPL of different models with different formats (e.g., with or without temporal prompts)\n2. Can you clarify the intuition and motivation of the OT reward? I am not familiar with IPOT. It is better to let readers know how to compute T.\n3. When the sketch is model-generated, how do you compute the story loss?\n4. What is the relation between Eq.8 and Eq.9? How are they used, and what are they used for?", "paper_topic_and_main_contributions": "This paper proposes a narrative order aware framework BPOT for story generation, which presents a bidirectional pretrained model to encode event correlations and pairwise event order. They also designed a RL algorithm with novel OT reward to further improve the generated event quality in fine-tuning stage. Both automatic and manual evaluation results demonstrate the superiority of BPOT in generating logically coherent story with flashback.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 142, zh"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work presents a novel method of keeping generating stories while keeping narrative order in mind through a two step pretraining and finetuning with RL+optimal transport rewards. All the reviewers agree that the method is sound and though the experimental setup is rather weak and can be improved. I would encourage the authors to update the paper with such results as they have stated in their rebuttal to reviewer bja9.

(Also a minor suggestion to edit the title Narrative Order Aware Story Generation via a Bidirectional Pretrained Model and Optimal Transport Reward)
Decision: Accept-Findings