Incorporating Inner-word and Out-word Features for M ongolian Morphological Segmentation (2020)

Abstract: Mongolian morphological segmentation is regarded as a crucial preprocessing step in many Mongolian related NLP applications and has received extensive attention. Recently, end-to-end segmentation approaches with long short-term memory networks (LSTM) have achieved excellent results. However, the inner-word features among characters in the word and the out-word features from context are not well utilized in the segmentation process. In this paper, we propose a neural network incorporating inner-word and out-word features for Mongolian morphological segmentation. The network consists of two encoders and one decoder. The inner-word encoder uses the self-attention mechanisms to capture the inner-word features of the target word. The out-word encoder employs a two layers BiLSTM network to extract out-word features in the sentence. Then, the decoder adopts a multi-head double attention layer to fuse the inner-word features and out-word features and produces the segmentation result. The evaluation experiment compares the proposed network with the baselines and explores the effectiveness of the sub-modules.

## introduction mongolian is a morphologically rich language and its words are formed by attaching suffixes to roots @xcite . each word has a root and zero or more suffixes, which are called mongolian morphemes. the morphemes in a word indicate the basic word features and provide grammatical and semantic relations among words in the sentence. mongolian morphological segmentation aims to split mongolian words into their morphemes, which facilitates the mongolian nlp tasks, such as name entity recognition @xcite , information retrieval @xcite , machine translation @xcite , and speech synthesis @xcite . there are about 60 thousand of morphemes in mongolian, and the number of their formed words is more than 7 million. it becomes a tendency to process mongolian text on morphemes rather than on words to make full use of the morpheme information in mongolian nlp tasks. besides, segmenting mongolian words into morpheme can alleviate the data-sparse problem and out-of-vocabulary (oov) problem. therefore, mongolian morphological segmentation is an essential preprocessing step and effects the downstream mongolian nlp tasks. mongolian morphological segmentation is closely related to the words themselves and their context. this study proposes a novel approach to mongolian morphological segmentation, which addresses challenges by incorporating inner-word and out-word features. the proposed network consists of two encoders and one decoder. first, a standard self-attention network is utilized as an inner-word encoder, which conducts connections between two arbitrary characters in a word and draws the inner-word features directly. meanwhile, a bidirectional lstm (bilstm) network is used as the out-word encoder to extract the out-word features of the word in the sentence. finally, a doubly attentive decoder is employed to fuse the inner-word features and out-word features and produce the segmentation result. the evaluation experiment compares the proposed network with the baselines and explores the effectiveness of the sub-modules. the contribution of this paper is as follows. this paper proposed a network for mongolian morphological segmentation according to the mongolian characteristics. two well-designed encoders were introduced in the network to extract the inner-word-level feature information and the out-word-level information between the target word and other words in the sentence. a doubly attentive decoder distinguishes and balances the inner-word and out-word information utilization in the decode stage. the experiment demonstrates that our approach achieves the sota performance. ## approach this paper proposes a neural network incorporating the inner-word and the out-word features for mongolian morphological segmentation, as shown in figure 1 . the network consists of two encoders and one decoder. the inner-word encoder uses the self-attention mechanisms to capture the inner-word features of the target word. the out-word encoder employs a two layers bilstm network to extract out-word features in the sentence. the decoder adopts a multi-head double attention layer to fuse the inner-word features and out-word features and produces the segmentation result. the following sections describe our approach in detail. ## conclusion this paper proposed a neural network incorporating inner-word and out-word features for mongolian morphological segmentation. we employ a standard san to encode the inner-word features between the characters in the word and a bilstm to encode the implicit out-word features between the target word and other words in the whole sentence. to distinguish and balance utilizing the inner-word and out-word features, we apply a doubly attentive decoder to decode two-level features jointly. the experimental results show that (1) the self-attention mechanism introduced to capture the inner-word information is shown to be more flexible in representing and focusing on the crucial information;