That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory (2022)

Abstract: The evolution of language follows the rule of gradual change. Grammar, vocabulary, and lexical semantic shifts take place over time, resulting in a diachronic linguistic gap. As such, a considerable amount of texts are written in languages of different eras, which creates obstacles for natural language processing tasks, such as word segmentation and machine translation. Although the Chinese language has a long history, previous Chinese natural language processing research has primarily focused on tasks within a specific era. Therefore, we propose a cross-era learning framework for Chinese word segmentation (CWS), CROSSWISE, which uses the Switch-memory (SM) module to incorporate era-specific linguistic knowledge. Experiments on four corpora from different eras show that the performance of each corpus significantly improves. Further analyses also demonstrate that the SM can effectively integrate the knowledge of the eras into the neural network.

## introduction as a human-learnable communication system, language does not remain static but instead evolves over time. the rate of change between different aspects of language, such as grammar, vocabulary, and word meaning, vary due to language contact and many other factors, which has led to the diachronic linguistic gap. an example of this can be seen in, "that slepen al the nyght with open ye (that sleep all the night with open eye)," which is a sentence from the canterbury tales, written in middle english by geoffrey chaucer at the end of the 14th century. it is difficult for people without an understanding of middle english to make sense of this sentence. furthermore, some discourses the chinese language has the honor of being listed as one of the world's oldest languages and, as such, has seen several changes over its long history. it has undergone various incarnations, which are recognized as archaic (ancient) chinese, middle ancient chinese, near ancient chinese, and modern chinese. notably, most chinese nlp tasks skew towards modern chinese. previous research has primarily focused on addressing the cws problem in modern chinese and has achieved promising results, such as chinese word segmentation (cws) @xcite @xcite @xcite @xcite . although cws for ancient chinese has been recognized in recent years, the processing of languagehybrid texts is still an open question. as shown in to address this need, we propose crosswise (cross-ear segmentation with switch-memory), which is a learning framework that deals with crossera chinese word segmentation (cecws) tasks. the framework integrates era-specific knowledge with the switch-memory mechanism to improve cws for era-hybrid texts. more specifically, we utilized the abilities of both cws and sentence classification tasks to predict segmentation results and era labels. we also incorporated the switchmemory module to include knowledge of different eras, which consists of key-value memory networks @xcite and a switcher. in order to store era-specific knowledge by several memory cells, key-value memory networks are used. the sentence discriminator is considered to be a switcher that governs the quantity of information in each memory cell that is integrated into the model. for each memory cell, we map candidate words from the dictionary and word boundary information to keys and values.. the main contributions of this paper are summarized as follows: ## related work chinese word segmentation is generally considered to be a sequence labeling task, namely, to assign a label to each character in a given sentence. in recent years, many deep learning methods have been successfully applied to cws @xcite @xcite @xcite @xcite @xcite @xcite @xcite . among these studies, some indicate that context features and external knowledge can improve cws accuracy @xcite @xcite @xcite . studies from @xcite and @xcite leveraged the dictionary to improve the task; n-gram is also an effective context feature for cws @xcite @xcite . the use of syntactic knowledge generated by existing nlp toolkits to improve cws and part-of-speech (pos) has been established by @xcite . furthermore, @xcite incorporated wordwood information for neural segmenters and achieved a stateof-the-art performance at that time. it is common practice to jointly train cws and other related tasks based on a multi-task framework. @xcite took each segmentation criterion as a single task and proposed an adversarial multitask learning framework for multi-criteria cws by extracting shared knowledge from multiple segmentation datasets. @xcite investigated the effectiveness of several external sources for cws by a globally optimized beam-search model. they considered each type of external resource to be an auxiliary classification, and then leveraged multi-task learning to pre-train the shared parameters used for the context modeling of chinese characters. @xcite jointly trained the cws and word classification task by a unified framework model. inspired by these successful studies, this study also incorporated ideas from the multi-task framework, and jointly trained the cws task and the sentence classification task to enhance the performance of cross-era cws. recently, some studies have noticed the linguistic gap due to the differences between eras. @xcite proposed a time-aware recontextualization approach to bridge the temporal context gap. @xcite reframed the translation of ancient chinese texts as a multi-label prediction task, then predicted both translation and its particular era by dividing ancient chinese into three periods. the use of key-value memory networks were introduced to the task of directly reading documents and answering questions by @xcite , which helped to bridge the gap between direct methods and the use of human-annotated or automatically constructed knowledge bases. @xcite applied this mechanism to incorporate n-grams into the neural model for cws. encouraged by the above works, this study designed a multi-task model for cross-era cws by jointly training the sentence classification task and cws through the use of a unified framework model. key-value memory networks are used to integrate era-specific knowledge into the neural network, as was done in research by @xcite . ## conclusion in this study, a flexible model, called crosswise, for cross-era chinese word segmentation is proposed. this model is capable of improving the performance of each dataset by fully integrating era-specific knowledge. experiments on four corpora show the effectiveness of this model. in the future, the incorporation of other labeling tasks into crosswise, such as pos tagging and named entity recognition, may prove to be insightful.