Improving C hinese Story Generation via Awareness of Syntactic Dependencies and Semantics (2022)

Abstract: Story generation aims to generate a long narrative conditioned on a given input. In spite of the success of prior works with the application of pre-trained models, current neural models for Chinese stories still struggle to generate high-quality long text narratives. We hypothesise that this stems from ambiguity in syntactically parsing the Chinese language, which does not have explicit delimiters for word segmentation. Consequently, neural models suffer from the inefficient capturing of features in Chinese narratives. In this paper, we present a new generation framework that enhances the feature capturing mechanism by informing the generation model of dependencies between words and additionally augmenting the semantic representation learning through synonym denoising training. We conduct a range of experiments, and the results demonstrate that our framework outperforms the state-of-the-art Chinese generation models on all evaluation metrics, demonstrating the benefits of enhanced dependency and semantic representation learning.

## introduction story generation presents a challenging task, primarily due to the difficulty that end-to-end neural models experience in maintaining logical coherence during long text generation @xcite . these challenges are even more prominent for the task of story generation in chinese, due to increased levels of ambiguity stemming from the absence of explicit delimiters for character separation @xcite . recent works, on the other hand, have aimed to improve long text generation through the proposal of more efficient frameworks @xcite @xcite , or through augmenting existing frameworks with pre-training and the in-* equal contribution. † corresponding author. jection of extra knowledge @xcite @xcite . encoder decoder outlines stories dataset dependency tagger semantic denoising hanlp simbert longlm 他们 游历 了 所有 的 国家 他们游历了所有的国家 they visited all the countries nsubj root dobj they visit countries 他们游览过所有的国家 they visited all the countries figure 1 : the overview of our framework. for the stories, the words in blue denote the semantic roles in a sentence (e.g., the subject (nsubj)), whilst the words in green denote the expressions that are replaced with synonyms. however, we observe that current generation models still struggle to generate fluent and coherent chinese stories, which may be the result of the inefficient capturing of features in written chinese. for example, chinese characters have a range of morphological parsing strategies, e.g., " 小心地 滑" can be understood as "小心 地滑" (caution wet floor) or "小心地 滑" (carefully slide), whose meaning is highly dependent on context @xcite . this may cause important sentential roles such as subjects, predicates, and objects to be difficult to identify and process by neural models. additionally, when neural networks learn the semantics of an utterance, synonymous expressions may lead to confusion, damaging the robustness of the generation model, e.g., "游历", "周游", and "游览" are different chinese words but all express "travelling" in chinese. we therefore propose to train neural networks to learn the semantic-level features contained in context, rather than the low-level features of characters. to this end, we propose a novel data augmented story generation framework illustrated in figure 1 , including a longlm @xcite based con-ditional generator, a dependency tagger, and a semantic denoising module. the generator, longlm @xcite , is a sota pre-trained model that has been demonstrated to be effective at multiple chinese nlg tasks. the dependency tagger, powered by hanlp foot_0 @xcite , recognises the root of a sentence, usually the verb, as well as related subjects and objects via dependency parsing, all of which are essential in expressing the event represented within a sentence. the semantic denoising module, based on simbert foot_1 @xcite , generates a range of different, yet essentially synonymous sentences, to force the neural network learn the semantic representations of key entities and different expressions. overall, our proposed framework enhances the ability for language understanding in written chinese via training to capture the dependencies and semantics contained within sentences, in order to then generate stories. we conduct a range of experiments on the latest public benchmark for chinese story generation @xcite , and the results indicate that the model trained with our framework substantially outperforms the state-of-the-art (sota) baselines on all metrics. 3 this indicates that our framework improves the generated stories via enhanced capturing of syntactic dependencies and semantic features. ## methodology we formulate our story generation task based on the outgen task from lot @xcite , a chinese story generation benchmark. the definition of the task is: an outline x, which contains an unordered list of an arbitrary number of chinese phrases concerning characters and events, is given as the input. the model is required to generate a coherent story @xmath0, y 2 , ..., y n } where y i denotes the i-th token (chinese character) in the story. ## conclusion we propose a novel story generation framework for chinese, which includes a dependency tagging module, a semantic denoising module, and a neural conditional generator. we aim to improve the generation of chinese through more effectively incorporating the features of dependencies and semantics. the performance improvements shown in our experiments and ablation study demonstrate that these features significantly benefit the task of chinese story generation.