SlowGuess/NAACL_2022_OCR huggingface dataset:
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier, and Timothy P. Lillicrap. 2020. Compressive transformers for long-range sequence modelling. In International Conference on Learning Representations.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1-67.
Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao. 2020. Plotmachines: Outline-conditioned generation with dynamic plot state tracking. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4274-4295.
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902-4912, Online. Association for Computational Linguistics.
Stuart Rose, Dave Engel, Nick Cramer, and Wendy Cowley. 2010. Automatic keyword extraction from individual documents. Text mining: applications and theory, 1:1-20.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490, Online. Association for Computational Linguistics.
Zhihong Shao, Minlie Huang, Jiangtao Wen, Wenfei Xu, and Xiaoyan Zhu. 2019. Long and diverse text generation with planning-based hierarchical variational model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3257-3268, Hong Kong, China. Association for Computational Linguistics.
Christian Stab and Iryna Gurevych. 2017. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619-659.
Mark Tappan and Lyn Mikel Brown. 1989. Stories told and lessons learned: Toward a narrative approach to moral development and moral education. Harvard Educational Review, 59(2):182-206.
Paul C Vitz. 1990. The use of stories in moral development: New psychological reasons for an old education method. American psychologist, 45(6):709.
Svitlana Volkova, Kyle Shaffer, Jin Yea Jang, and Nathan Hodas. 2017. Separating facts from fiction: Linguistic models to classify suspicious and trusted news posts on twitter. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 647-653.
Tianming Wang and Xiaojun Wan. 2019. T-CVAE: transformer-based conditioned variational autoencoder for story completion. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 5233-5239. ijcai.org.
Thomas Wolf, Julien Chaumont, Lysandre Debut, Victor Sanh, Clement Delangue, Anthony Moi, Pierric Cistac, Morgan Funtowicz, Joe Davison, Sam Shleifer, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45.
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7378-7385.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 654-664.
Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, and Xiaoyong Du. 2019. Uer: An open-source toolkit for pretraining models. EMNLP-IJCNLP 2019, page 241.
A STORAL Construction
A.1 Data Source
We show the full list of web pages used for constructing STORAL in Table 11. We initially collect 52,017 Chinese and 2,630 English raw texts from the web pages. Then we de-duplicate the texts by removing those texts which overlap with others more than twenty words. After de-duplication, we finally collected 19,197 Chinese and 2,598 English texts. And we construct STORAL based on these texts.
A.2 Data Annotation
Table 9/10 shows the examples given to the annotators to inform them of the requirements for stories/morals, respectively. If the constraints are not met, we ask annotators to refine the story and moral. All workers were paid more than $7 per hour on average.
Example 1: Come on Bear! What a beautiful day! Go for a walk with your father! Take a deep breath and smell the flowers. But don't pick the flowers. Listen to the birds sing. But don't scare them. How beautiful the world is. Isn't it, dear Bear?
Example 2: When I was a child, I heard a story that felt very regrettable. I felt sorry for the protagonist of the story. Long ago, there lived ... Such trees are now found all over Uganda.
Example 3: I have a well-off friend. When she first entered college, she had many good wishes and thought she could achieve her goals. . . . Now she felt very painful under the strong mental pressure. I can understand her feelings. . . . If magnifying your own pain, you will get trapped in the mire of your pain, and even feel that life is too unfair to you.
Example 4: Raul sat at his door, frowning. ... His father told Raul a true story: A wild wolf escaped into a cave after being wounded by a hunter's arrow. ... After hearing the story, Raul cheered up immediately. ...
Table 9: Examples of stories provided for the annotators. Each example does not meet one of the following requirements in order: (1) having a clear beginning and ending; (2) not stating anything irrelevant to the main plot; (3) not stating any explicit arguments for the moral; and (4) not telling the story in a nested form. The sentences causing the above issues are in italic.
Example 1: If you saw a thief in a crowded bus, would you bravely stop him? Please reflect on yourself instead of just complaining that our world is becoming worse. Without the foothold for dirt, the flower of civilization is bound to be fragrant.
Example 2: The story tells us: we should remember that we should become a polite person and communicate with others carefully.
Example 3: As long as you keep your sanity and make right judgments, all the barriers will not become an obstacle, just like the beautiful girl in the story.
Table 10: Examples of morals provided for the annotators. Each of the examples does not satisfy one of the following constraints in order: (1) describing only the main standpoint and not stating any sub-arguments or proofs, and (2) not stating anything irrelevant to the moral, and (3) not involving any specific characters in the story. We highlight the sentences leading to the above issues in italic.
Links	Number
http://www.qbaobei.com/jiaoyu/yegs/yygs/	14,674
https://www.517gj.com/yuyangushi/	14,474
https://www.etgushi.com/zgyy/	6,691
https://www.chazidian.com/gushi_1/	3,457
http://wwwfeel-bar.com	3,329
http://www.xiaole8.com/renshengzheli/	2,509
http://www.zuowen.com/sucai/zheli/	2,421
http://www.rensheng5.com	2,092
https://www.yuyangushi.com/lz/xgsdl	1,886
http://www.gushi88.cn/ErTong/ZhongGuoYuYan_1	484
Grand Total	52,017
Links	Number
https://moralstories26.com	799
https://english.7139.com/2539/	552
https://kidsfables.com	193
http://read.gov/aesop	145
http://www.taleswithmorals.com	108
https://www.studentuk.com/category/fable	101
http://www.english-for-students.com/Moral-Stories.html	97
https://www.advance-africa.com/English-Moral-Stories.html	65
https://www.gutenberg.org/files/25512/25512-h/25512-h.htm	52
Others	518
Grand Total	2,630
Table 11: List of source web pages used for constructing STORAL-ZH (Top) and STORAL-EN (Bottom). Numbers in the right column means the number of raw texts initially collected from the corresponding web page.
(a) Stories in STORAL-ZH
(b) Morals in STORAL-ZH
(c) Stories in STORAL-EN
(d) Morals in STORAL-EN
Figure 3: Top 50 most frequent nouns for stories and morals in STORAL-ZH and STORAL-EN. The numbers in the legend show the percentages of the total frequency of the nouns of the same type among the 50 nouns.
A.3 Analysis of High-Frequency Words
To investigate the topic features of STORAL, we count the top 50 most frequent nouns in STORAL (excluding stop words) as shown in Figure 3. We roughly categorize these words into four types: (1) Animals: animals are popular as protagonists in moral stories since they usually have various but clear characteristics (e.g., "sly foxes"), which embody rich commonsense knowledge; (2) Relationships: such nouns are used to describe the inter-character relationships in a story (e.g., "friend"), which are useful for modeling characters' motivation and behavior; (3) Concrete nouns: they refer to physical entities that can be observed, such as "water"; and (4) Abstract nouns: they re
fer to abstract concepts, such as "difficulty". We manually check the proportional distribution of the four types for stories and morals, respectively. The results in Figure 3 demonstrate that morals contain significantly less concrete nouns and more abstract nouns than stories. And morals contain little animal words but almost as many relationship words as stories, indicating that morals may be independent of specific characters but relate to general interpersonal relations. The result shows that morals are more abstract than stories.
Furthermore, Table 12 shows the most frequent 4-grams in STORAL, further indicating that morals are more abstract than stories. Each of the 4-grams in Table 12 comprises less than  $0.01\%$  of all 4-
Stories	Morals
Dataset: STORAL-ZH
as one is walking	we should be a
say to him that you	everyone has
say after thinking	everyone has his own
the most in the world	has own
all the animals	each of us
all the persons	we should know to
a place far away	for anything, we should
the dad of the pink pig	be one who knows to
this is my	for anything, be
in the forest there lived a	is a true
Dataset: STORAL-EN
once upon a time	we should try to
upon a time there	the best way to
a time there was	it is better to
time there was a	it is easy to
there was once a	we should learn to
once there was a	those who help themselves
was not able to	with what we have
as soon as he	be happy with what
and asked him to	we should not judge
did n’t want to	look before you leap
Table 12: Top 10 most frequent 4-grams in STORAL-ZH and STORAL-EN respectively. The Chinese 4-grams in STROAL-ZH are translated into English.
grams in the corresponding dataset, showing the diversity of STORAL.
A.4 Discussion about STORAL
The high-quality examples in STORAL are full of commonsense and discourse relations. As exemplified in Table 1 in the main paper, the common sense is mainly regarding the characters' reaction and intention (e.g., "the cows dispersed" and then the "tiger" and "lion" intend to kill them), as well as the nature of physical objects and abstract concepts (e.g. "cows" may be the food of "lions" and "tigers", and "unity" refers to "keeping together for a common goal"). Additionally, the stories usually have a specific discourse structure, i.e., the premise to introduce the story settings (e.g., the characters "four cows" and the location "a meadow"), the right or wrong behavior ("stay together or not") and the endings ("living well or being killed"). We believe it is an essential topic of future work to develop a better approach to model such commonsense and discourse relations.
B.1 Implementation
We implement the pretrained models used in our experiment mainly based on the register models of
HuggingFace (Wolf et al., 2020). Table 13 shows the names of the used register models. Note that we use LongLMbase (Guan et al., 2022) as the T5 model for experiments on STORAL-ZH, which has not been registered on HuggingFace.
All results in the main paper and the appendix are based on one NVIDIA Tesla v100 (16G memory). All reported results are based on one single running. The CPU is Intel Xeon Gold 5218. It cost less than 5 hours for fine-tuning each model on STORAL. We set the hyper-parameters following the default parameters of HuggingFace.
B.2 Automatic Evaluation for Moral Faithfulness
We follow Guan and Huang (2020) to train a learnable metric to evaluate moral faithfulness. Specifically, we fine-tune  $\mathrm{RoBERTa}_{\mathrm{BASE}}$  as a classifier to distinguish whether a story matches a moral. We regard ground-truth examples as positive where the story and moral are matched, and construct negative examples by replacing the story or moral with a randomly sampled one. Finally, the classifier achieves an accuracy of  $77.32 / 79.21\%$  on the data constructed based on the test set of STORAL-ZH/STORAL-EN respectively. Then we calculate the faithfulness score as the average classifier score of all generated texts for the inputs.
Table 14 presents the evaluation results. We can see that pretrained models achieve better faithfulness than the non-pretrained models as shown by the much higher faithfulness scores. However, we also observe that the faithfulness score of the ground-truth texts is lower than some models (e.g., T5) when generating morals. Therefore, it is still necessary to manually evaluate faithfulness.
Results on Validation Sets We show the performance of several baselines and RA-T5 on the validation sets of the understanding tasks and the generation tasks in Table 15 and Table 16, respectively.
B.3 Manual Evaluation Instruction
We show the manual annotation interface in Figure 4. To ensure that the annotators guarantee a consistent standard in the annotation process, we asked annotators to rate four examples with the same input at the same HIT (human intelligence task). In these four examples, one is written by humans and three are generated by models (i.e., Fusion, T5 and RA-T5). We paid each annotator $0.2 on average for annotating each example.
Datasets	STORAL-ZH	STORAL-EN
BERT	bert-base-chinese (Devlin et al., 2019)	bert-base-uncased (Devlin et al., 2019)
RoBERTa	hfl/chinese-roberta-wwm-ext (Cui et al., 2020)	roberta-base (Liu et al., 2019)
GPT2	uer/gpt2-chinese-cluecorpussmall (Zhao et al., 2019)	gpt2 (Radford et al., 2019)
T5	LongLM (Guan et al., 2022)	t5-base (Raffel et al., 2020)
Table 13: Names of register models used in our experiment.
Models	ST2MO	MO2ST
ZH	EN	ZH	EN
ConvS2S	31.92	28.68	33.44	35.59
Fusion	33.85	25.23	38.81	35.16
GPT2	68.52	73.73	50.49	64.90
PM	N/A	N/A	52.41	62.53
T5	89.20	90.57	56.11	63.45
T5-Post	90.98	91.87	58.58	75.67
RA-T5	86.50	88.69	59.50	74.92
Truth	77.49	80.03	77.49	80.03
Table 14: Automatic moral faithfulness scores. The score of PM for the ST2MO task is N/A since we do not experiment with PM for this task.
Models	#P	MOCPT	MOPREF
ZH	EN	ZH	EN
BERT w/o Story	110M	20.71	21.69	72.64	77.62
BERT	110M	65.24	54.08	85.37	79.36
RoBERTa	110M	66.90	61.69	90.49	80.52
RoBERTa-Post	110M	67.14	55.77	89.27	84.01
T5	220M	74.52	62.25	78.05	77.91
T5-Post	220M	74.05	67.61	81.22	81.10
RA-RoBERTa	110M	66.43	63.94	88.54	86.63
RA-T5	220M	74.05	67.61	80.73	80.23
Table 15: Accuracy (%) for MOCPT and MOPREF on the validation set.
Models	ST2MO	MO2ST
ZH	EN	ZH	EN
Fusion	14.44/1.80	10.78/0.92	16.06/6.44	16.74/6.72
T5	18.54/4.08	13.17/2.05	18.98/10.17	28.87/15.48
RA-T5	18.68/3.64	14.49/4.47	23.98/13.17	31.72/17.97
B.4 Significance of Manual Evaluation Results
Table 17 shows the  $p$ -values (sign test) when comparing the manual evaluation results (Table 8 in the main paper) between each pair of the ground truth, Fusion, T5 and RA-T5.
B.5 Evaluating Value Preference Alignment
Although we have used MOPREF to evaluate whether machines can capture the value preference
1. Read the moral, first sentence and four stories.
2. Comparing the stories with one another in terms of fluency, coherence and faithfulness to the input.
3. Answer the question for each story. Please choose the reasons if your answer is "no" when evaluating coherence and faithfulness.
Table 16: BLEU-1/BLEU-2 for ST2MO and MO2ST on the validation set.
Moral: Nothing can be gained without effort.
First Sentence: There was a farmer who had three sons.
story 1: All of his sons were very lazy ...
story 2: The farmer loved his sons very much ...
...
story 5: The farmer said: "It's a good job is that ..."
Evaluating Story 1
Q1: Is the Story Fluent? Yes No
Q2: Is the Story Coherent? Yes No
□ Repetition Unrelatedness
□ Conflicting logic Chaotic Scenes
□ Others
Q3: Is the Story faithful to Moral? Yes No
□ Not a moral story Unrelated concepts
□ Conflicting value preference
□ Others
Evaluating Story 5
Figure 4: A simplified version of the manual annotation interface for MO2ST. The interface for ST2MO is similar.
of a story, the automatically constructed dataset may bias machines to focus on distinguishing general standards of good behaviour without considering story plots. Therefore, in this section, we construct examples manually to test this ability beyond the token level. Specifically, we randomly sampled 50 examples from the test sets of STORAL-ZH and STORAL-EN respectively. For each example, we manually rewrote the moral to convey a synonymous or anonymous value preference. For example, a synonymous moral with "unity is strength" in Table 1 can be "we are powerful as long as we unite with each other" and an anonymous one can be "everyone can also be powerful enough." Then we expect a model to be able to accept the synonymous moral but reject the anonymous one. We use three typical models, including BERT w/o Story, RA-RoBERTa and RA-T5, to compute the winning
Tasks	Models	Flu	Cohc	Faith
Dataset: STORAL-ZH
ST2MO	T5 vs. Fusion	8.55e-14	3.82e-11	1.55e-6
RA-T5 vs. T5	0.03	0.75	0.23
Truth vs. RA-T5	6.10e-5	2.91e-11	2.35e-14
M02ST	T5 vs. Fusion	2.35e-3	2.44e-4	0.38
RA-T5 vs. T5	0.14	0.04	0.11
Truth vs. RA-T5	2.27e-12	1.08e-19	1.1e-24
Dataset: STORAL-EN
ST2MO	T5 vs. Fusion	3.71e-11	2.92e-12	7.92e-9
RA-T5 vs. T5	0.27	0.06	4.18e-3
Truth vs. RA-T5	7.20e-3	4.08e-3	4.92e-5
M02ST	T5 vs. Fusion	0.04	0.09	0.04
RA-T5 vs. T5	0.5	0.25	7.81e-3
Truth vs. RA-T5	1.46e-11	1.42e-14	1.71e-8
rate of pair-wise comparisons between any two of ground-truth, synonymous and anonymous morals. These models are trained on the training set of the MOPREF task.
Table 17:  $p$ -values (sign test) when comparing each pair of the ground truth and three models for the manual evaluation results. We highlight the  $p$ -values larger than 0.1 in **bold**, which indicates A has an insignificant superiority w.r.t. B for "A vs. B".
Models	True vs. Syn	True vs. Ant	Syn vs. Ant
Dataset: STORAL-ZH
BERT w/o Story	52% (0.89)	46% (0.67)	58% (0.32)
RA-RoBERTa	40% (0.21)	36% (0.06)	48% (0.89)
Dataset: STORAL-EN
BERT w/o Story	54% (0.67)	54% (0.67)	48% (0.89)
RA-RoBERTa	64% (0.06)	34% (0.03)	40% (0.20)
Table 18 shows the evaluation results. We observe that BERT can not distinguish different types of morals without input stories. RA-RoBERTa fails to accept the synonymous morals on STORAL-EN (winning rate of only  $36\%$  w.r.t the ground truth,  $p < 0.1$ ), and can not distinguish synonymous and antonymous morals on both STORAL-ZH and STORAL-EN (winning rate near  $50\%$  with  $p > 0.1$ ). Additionally, it prefers antonymous morals to the ground truth significantly on both datasets (winning rate less than  $50\%$  and  $p < 0.1$ ). The results indicate that existing models still struggle to capture the value preference of moral stories.
Table 18: Winning rates of pair-wise comparisons which require selecting a correct moral from two candidates. Each candidate is a ground-truth (True), synonymous (Syn), or anonymous (Ant) moral. The number in the parenthesis is the corresponding  $p$ -value (sign test).
Models	NAM	UNREL	CONF	Others
Task: ST2MO
Fusion	27%	23%	7%	2%
T5	19%	9%	12%	0%
RA-T5	15%	7%	6%	0%
Truth	3%	4%	2%	0%
Task: MO2ST
Fusion	25%	13%	6%	1%
T5	19%	9%	10%	1%
RA-T5	16%	10%	10%	0%
Truth	2%	1%	1%	0%
Table 19: Percentage of the texts annotated with a certain error in all annotated 100 texts in terms of moral faithfulness.
C Error Analysis and Case Study
In this section, we conducted a case study and investigated the errors of existing models on the proposed tasks to provide insight into future work. We show several typical error cases in Table 20.
C.1 Understanding Tasks
The example in Table 20 for MOCPT shows that the model may not grasp abstract concepts such as "good will" and "good acts" and align them to the story plots. It makes predictions possibly based on only token-level features such as relations between "ask after" and "attention". On the other hand, the example for MOPREF indicates that the model can not capture the value preference of the story in terms of "whether it is intelligent to regard others are illiterate". The results demonstrate the necessity of introducing concept knowledge and modeling high-level semantic information.
C.2 Generation Tasks
Table 21 shows cases generated by several baselines and our model for the generation tasks. We can see that retrieval can provide effective guidance for both moral and story generation. Baseline models including GPT2 and T5 tend to generate unrelated concepts or non-moral texts.
However, as shown by the manual evaluation results, there is still a big gap between RA-T5 and humans. To provide quantitative error analysis, in the process of manual evaluation on STORAL-EN, we required annotators to annotate the error type of a text when it exhibit an unfaithful moral. We summarize three main error types as follows: (1) Not a moral text (NAM): not stating or imply
Table 20: Typical error cases predicted by RA-T5 (for the understanding tasks) or sampled from RA-T5 (for the generation tasks). For the generation tasks, the error types in terms of moral faithfulness include "not a moral text" (NAM), "unrelated concepts" (UNREL) and "conflicting value preference" (CONF). The underlined words are improper concepts/events which leads to corresponding errors. Bold words for MO2ST are the given first sentence and the outline of multiple phrases.
ing what is right or what is wrong; (2) Unrelated concepts (UNREL): containing unrelated concepts with the input; and (3) Conflicting value preference (CONF): conveying a value preference conflicting with the input despite related concepts. In addition, we also provide annotators with another option Others. The annotators are allowed to annotate a text with multiple errors. When at least
two of three annotators annotate the text with some error, we decide it has the error. We show the distribution of the error types in Table 19, suggesting that existing models still struggle to generate meaningful morals and stories, and align the concepts and value preferences between them.
Furthermore, as exemplified in Table 20, when generating morals, we can see from Case 1 that the
Generation Task: ST2MO