Language-Independent Approach for Morphological Disambiguation (2022)

Abstract: This paper presents a language-independent approach for morphological disambiguation which has been regarded as an extension of POS tagging, jointly predicting complex morphological tags. In the proposed approach, all words, roots, POS and morpheme tags are embedded into vectors, and contexts representations from surface word and morphological contexts are calculated. Then the inner products between analyses and the context’s representations are computed to perform the disambiguation. The underlying hypothesis is that the correct morphological analysis should be closer to the context in a vector space. Experimental results show that the proposed approach outperforms the existing models on seven different language datasets. Concretely, compared with the baselines of MarMot and a sophisticated neural model (Seq2Seq), the proposed approach achieves around 6% improvement in average accuracy for all languages while running about 6 and 33 times faster than MarMot and Seq2Seq, respectively.

## introduction morphological disambiguation (md) is the task of jointly predicting lemma/root, part of speech (pos) @xcite , and morpheme tags. for a turkish word "yeni" (new), it can be analyzed as: 1)yen+noun+[a3sg, pnon, acc]; 2)yen+noun+ @xcite @xcite . if one counts analyses as tags, md can be cast as a tagging problem with an extremely large tagset. this fact discourages direct application of the state of the art approaches designed for small fixed tagsets. for instance, many approaches treat each analysis as a tag, and apply sequence labeling models to perform tagging @xcite @xcite . treating each analysis as a tag leads to an oversized tagset and corresponding data-sparsity issues, which can be a concern for morphologically complex languages such as turkish and kazakh, where the number of morphological analyses is theoretically unlimited @xcite . to address this problem, a sequence to sequence (seq2seq) @xcite based approach was proposed, which treated each morphological analysis as a sequence of a composite tags and explicitly modeled their internal structure. this approach was inspired by the neural sequenceto-sequence models for machine translation @xcite . the lstm networks were applied to model morphological analyses and context as a pair of sequences, which involves more sophisticated architectures, namely using double layers of bilstm, one for characters and another for words. this approach was almost challenging to simplify the architecture if one wanted to keep the performance as the original seq2seq has. because all types of architecture of recurrent neural networks are more fit to the nature of sequence to sequence problems @xcite . this paper presents a language-independent md approach that applies an uncomplicated neural architecture and obtains comparable results in accuracy and speed with the current best. a sequence of morphological analysis is an expansion sequence of its surface words with morphological information. the idea of the approach is to measure the distance between analyses and surface word context by embedding each morphological analysis and the surface word context into a single vector space. the underlying hypothesis is that the vector representation of the correct analysis should be closer to the context vector. two types of contextual embedding for words are presented: i) surface word context; ii) morphological context, which improves the model's performance significantly. in the following, the proposed approach is referred to as language-independent morphological disambiguation (limd). our contribution amounts to the following: i) a general language-independent approach for md, its neural architecture is simple, decoding is fast and can be implemented easily in practice. it achieves comparable results with the current best. ii) two types of context representation are explored: word and morphological context representations. ## related work morphological disambiguation/tagging has been studied extensively for decades, and here we review the work most relevant to this paper. we categorize the common approaches into three groups: i) modeling the structure of complex morphological labels with structured prediction models @xcite @xcite . the work @xcite presented a pruned crf (pcrf) for tagging and proposed to use coarse-to-fine decoding and early updating to train the higher-order crf. experiments on six languages show that the pcrf gives significant improvements in accuracy. we evaluate this model on our data-sets as one of the baselines. @xcite compared the performance of the most important representations that can be used for across-domain mt. one of their findings is that the representations similar to brown clusters perform best for pos tagging and that word representations based on linguistic morphological analyzers perform best for tagging. the study @xcite combines neural networks and graphical models presented a framework for cross-lingual tagging. instead of predicting full tag sets, the model predicts single tags separately and modeling the dependencies between tags over time steps. the model is able to generate tag sets unseen in training data, and share information between similar tag sets. this model is about crosslingual tagging and we do not make comparisons with monolingual tagging models. ## approach this section describes the proposed md approach, which embeds a context and its morphological analysis into a vector space, then calculates similarity scores to rank them for performing disambiguation. ## results for morpheme, limd gives comparable accuracy with marmot and seq for the most of the languages. again, it shows promising results for the smallest (kazakh, the improvement is ≈ 25%) and the second smallest (turkish, the improv. is ≈ 7.5%) datasets. for german morpheme prediction, seq2seq (88.44%) gives 1.88% improvement over marmot (86.56%), and limd yields 92.23% accuracy in this case. compared to other languages, limd achieves a larger improvement over the baselines on the german data that is highest ambiguous among all datasets. for pos+morpheme joint prediction, limd performs much higher than marmot and seq2seq for the german, turkish, kazakh data, and for other languages, they give very competitive accuracies. cross-task comparisons (morpheme vs. pos+morpheme and pos vs. pos+morpheme) reveal that the morpheme tagging is the most challenging part for all models, as it can be observed that morpheme's accuracies are much lower than pos one. it worth noting that seq2seq applies double-layer of bilstm network as encoder to model the character and word embeddings for context. this architecture has been applied recently to context representation learning for md and achieved the notable results @xcite @xcite . ## analysis and discussion analysis of surface word context. to explore the influence of different window-sized surface contexts, we fixed the morphological context with the leftmost and rightmost ones and tuned the window size only for the surface context. we choose the german dataset for the exploration because it is the most complex data in its ambiguous analysis in this work. ## conclusion this paper presents a language-independent morphological disambiguation approach, limd. it embeds surface word and morphological context into vector representations, then calculates cosine similarly scores of two to perform disambiguation. experimental evaluations show that limd outperforms other sophisticated models in both accuracy and speed. results indicate that limd works well on morphologically complex languages with many analyses per token and the approach suffers less from the issue of lack of data. possible future work in this direction is to apply different methods to the model's output instead of a computing dot product for disambiguation. also, there is still room for the improvement in the model's architecture, such as better capturing surface word context or modeling morphological analyses with more advanced architectures.