A Corpus for Understanding and Generating Moral Stories (2022)

Abstract: Teaching morals is one of the most important purposes of storytelling. An essential ability for understanding and writing moral stories is bridging story plots and implied morals. Its challenges mainly lie in: (1) grasping knowledge about abstract concepts in morals, (2) capturing inter-event discourse relations in stories, and (3) aligning value preferences of stories and morals concerning good or bad behavior. In this paper, we propose two understanding tasks and two generation tasks to assess these abilities of machines. We present STORAL, a new dataset of Chinese and English human-written moral stories. We show the difficulty of the proposed tasks by testing various models with automatic and manual evaluation on STORAL. Furthermore, we present a retrieval-augmented algorithm that effectively exploits related concepts or events in training sets as additional guidance to improve performance on these tasks.

## introduction stories play an essential role in one's moral development @xcite . for example, individuals usually learn morals from life experiences or literature such as fables and tell their morals by representing their lived experience in a narrative form @xcite . accordingly, it is a crucial ability for humans to bridge abstract morals and concrete events in stories. however, this ability has not yet been investigated for machines. there have been many tasks proposed for evaluating story understanding and generation, including story ending selection @xcite and story generation from short prompts @xcite . unlike these tasks, which focus on reasoning plots from context, we emphasize the ability to associate plots with implied morals. as exemplified in furthermore, we collected a new dataset named storal composed of 4k chinese and 2k english human-written stories paired with morals through human annotation to address the above challenges. we call the chinese dataset storal-zh and the english dataset storal-en, respectively. and we construct datasets for the proposed tasks based on storal. our focus of morals is on the social set of standards for good or bad behavior and character, or the quality of being right, honest or accept-able @xcite . we conduct extensive experiments on the proposed tasks. furthermore, we present a retrieval-augmented algorithm to improve model performance by retrieving related concepts or events from training sets as additional guidance. however, the experiment results demonstrate that existing models still fall short of understanding and generating moral stories, which requires a better modeling of discourse and commonsense relations among concrete events and abstract concepts 1 . ## storal dataset we collected storal from multiple web pages of moral stories. all stories are allowed to use and redistribute for research and have been reviewed by the website editors as stated on the pages. we show the full list of links to these pages in section a.1. after de-duplication, we collected 19,197 chinese and 2,598 english raw texts. then we adopted human annotation for decoupling the story and moral ## retrieval augmentation a critical challenge for tackling the proposed tasks is the sparsity of morals and events makes it difficult to learn relations between them. prior studies have shown that retrieval improves performance towards infrequent data points across various tasks such as open-domain question answering @xcite ) and text classification @xcite . we present a retrieval-augmented algorithm that exploits the moral-event relations in training sets. we illustrate our model for the mopref task in fig- input encoder [c] be . ... ... [s] [c] be . one . ... candidate 1 candidate 2 story query encoder ... any idea that ... self help is ... before doing ... in life, many ... ... help fail life training sets ... 𝐡 !" 𝐡 !# 𝐡 $ hidden states input tokens retrieved tokens dot product story index morals ure 2. our models for other tasks are similar. for both mocpt and mopref, we encode the story and candidates using an input encoder, and then predict a probability distribution over the candidates by normalizing the dot-product scores between the representations of the story and each candidate. we optimize the model by minimizing the cross-entropy loss. we insert special tokens [s] and [c] before the story and each candidate, respectively, and take the corresponding hidden states as their representations. furthermore, we propose to retrieve related concepts from the training set using the input story. we encode the story using a query encoder, then take the output as the query to retrieve m most related stories based on a story index, i.e., a set of dense vectors as the representations of stories in the training set. we adopt bert @xcite followed by a mean-pooling layer to build the query encoder and story index, which are frozen in the training stage. finally, we extract the nouns, verbs, adjectives and adverbs from the morals of the top-m stories and lemmatize them as the retrieved concepts. we feed the concepts together with the original input to the input encoder. for example, the retrieved concepts for the story in the retrieval-augmented algorithm can easily adapt to the generation tasks. for st2mo, we take the input story paired with the retrieved concepts into the encoder and then generate the output using the decoder. and for mo2st, we use the input moral as the query to retrieve top-m stories, and regard their outlines as the retrieved additional information to guide the subsequent story generation. ## conclusion we present storal, a collection of chinese and english moral stories. to test the ability to bridge concrete events and abstract morals, we propose new understanding and generation tasks based on storal, including selecting the correct moral from several candidates with different topics or opposite value preferences, concluding the moral of a story and generating a story to convey a moral. extensive experiments prove these tasks still to be challenging for existing models. we propose a retrieval-augmented algorithm to improve performance by retrieving related concepts or events from training sets. although it is possible to further increase the dataset size, we expect to make meaningful progress by developing better representations of commonsense and discourse relations among events and abstract concepts in future work.