Morpheme boundary Detection & Grammatical feature Prediction for G ujarati : Dataset & Model (2021)

Abstract: Developing Natural Language Processing resources for a low resource language is a challenging but essential task. In this paper, we present a Morphological Analyzer for Gujarati. We have used a Bi-Directional LSTM based approach to perform morpheme boundary detection and grammatical feature tagging. We have created a data set of Gujarati words with lemma and grammatical features. The Bi-LSTM based model of Morph Analyzer discussed in the paper handles the language morphology effectively without the knowledge of any hand-crafted suffix rules. To the best of our knowledge, this is the first dataset and morph analyzer model for the Gujarati language which performs both grammatical feature tagging and morpheme boundary detection tasks.

## introduction as natural language processing is increasingly becoming an active area of research with many important applications, most of the research is fo cused only on a few languages. developing nlp tools for under resource languages is an essential task, as it not only opens considerable economic perspectives but also prevents its extinction and foster its expansion @xcite in this paper we present a morph analyzer for gujarati. gujarati is an indoaryan language, spo ken mainly in the gujarat state of india. it is the 26th most widely spoken language with ap proximately 55 million speakers across the world. we identify various grammatical features of gu jarati morphology and prepare a gold data set of 16527 unique words. we have developed bi lstm based morph analyzer inspired from @xcite and @xcite . for the training and evaluation of the morph anal ysis task, we have built the dataset for gujarati language in the standard unimorph format @xcite . the neural architecture proposed in this paper does not require any language specific rules and captures linguistic characteristics of the language effectively. the remaining of the paper is organized as fol lows : section 2 describes related work. section 3 describes the dataset details. section 4 describes proposed approach. section 5 describes experi ments and observations. section 6 describes result analysis from the linguistic perspective. in section 7, we discuss conclusion and future research direc tion. ## related work morphological analysis is the task of analyzing the structure of the morphemes in a word and is gener ally a prelude to further complex nlp tasks such as parsing, machine translation, semantic analysis. existing approaches to build a morphologi cal analyzer can be broadly classified as rule based approaches and machine learning based ap proaches. recent breakthroughs in the field of deep learning has motivated researchers to apply the neural models to the morphological analyzer problem. twolevel morphology @xcite ) was the first practical general model in the history of computational linguistics for the analysis of morphologically complex languages. the current c version twolevel compiler, called twolc, was created at parc @xcite . (kenneth r. @xcite introduced xfst, a finite state morphology tool. finite state transducer based approach have been used to develop morph analyzer for many languages @xcite @xcite . @xcite developed a morphological analyzer for hindi using this approach. following hindi, mor phological generators were developed for other indian languages e.g. kannada (melinamath and mallikarjunmath, 2011), oriya @xcite , etc. @xcite described another popular approach named paradigm based approach for morphological analysis. a paradigm defines all the word forms that can be generated from given root along with grammatical feature set. apart from the rule based techniques, there have been some efforts to develop machine learning based methods to develop morph analyzer for in dian languages. @xcite have defined the morphological analysis problem as classification problem and experimented with var ious kernel methods to capture non linear relation ships of the morphological features using svm tool. @xcite developed statistical morphological analyzer for the indian languages using linguistic features. @xcite proposed a neural lemmatizer for the bengali language. the pro posed lemmatizer makes use of contextual infor mation of the surface word to be lemmatized. @xcite investigated character based neural morphological tagger for morphologically rich languages having large tag set. they pre sented various neural architectures. the work is extended in @xcite by experi menting on 14 different languages. @xcite introduced deep neural network based method for lemmatization. this method works by identifying a correct edit tree for the word lemma transformation. @xcite de veloped a deep learning approach for detecting morpheme boundaries. they studied agglutinative nature and sandhi splitting process of malayalam language. @xcite explore 3 different neural architectures named simple multi class multilabel model, hierarchical model and se quence model for morphological tagging. the en coder used for this work is same as the one used in @xcite . @xcite ) evalu ated various neural morpholgical taggers for san skrit with the focus on the fact that good result for morphological tagging should be achieved without an extensive linguistic knowledge. there have also been effort to develop unsu pervised approaches for morphological tagging task. (j., 2005) described an automatic morpho logical analyser which can be adopted for dif ferent languages. morphessor @xcite is another widely used tool which performs unsu pervised morphological segmentation. @xcite and @xcite have also proposed unsupervised morphological ana lyzer using trie based approach and log linear methods. to the best of our knowledge, very less work is reported in the area of developing a morph ana lyzer for the gujarati language. @xcite built a stemmer using handcrafted suffix list along with unsupervised learning. @xcite devel oped rule based lemmatizer for gujarati by hand crafting of suffix rules. the language independant models such as morfessor can not be used to de velop full fledge morph analyzer as they only give morphological segmentation and do not perform morphological feature tagging. the model sug gested by @xcite can not be directly used as the language specific training data for gu jarati language is not available in penn treebank dataset. ## gujarati morphology and data set generation gujarati is a verbfinal language and has a rela tively free word order, it is an inflectional lan guage foot_0 . words are formed by successfully adding suffixes to the root word in series. when suffixes are attached to the root, several morphophonemic changes take place. in this section, we describe the format and the details about the data set cre ation for the training and evaluation of morpho logical analyzer. for the creation of the dataset, we did a survey of available corpus for the gu jarati language. for the morphological analysis, it is preferable to have a pos tagged data, hence we have selected gujarati monolingual text corpus ilciii corpus for the creation of the dataset. the dataset is obtained from tdil. foot_1 . we now discuss gujarati morphology and details of dataset created for noun, verb and adjective pos categories. ## proposed approach we propose a morphological analyzer for gu jarati which performs morpheme boundary detec tion and grammatical feature tagging of a given in flected word. gujarati is a morphologically rich language and manual hand crafting of rules is cum bersome process, hence we propose a deep learn ing based approach for this problem so that the features of an inflected word can be learned with out supplying hand crafted rules. the morpheme boundary segmentation model inspired by (pre mjith et al., 2018) and feature tagging module is based on the work reported in @xcite . figure 1 shows the overall architecture of the proposed system. we first prepare the training data and perform the model training for both tasks. the input word is passed to both models to get the cor responding morpheme segmentation and linguis tic feature tagging outputs. for the preparation of training data, we represent inflected word as binary string and mark "1" in the position of the split char acter, rest all characters are marked as "0". fig ure 2 shows morpheme splitting example. due to the rich morphological nature of gujarati lan guage, different word forms can be constructed by attaching various suffixes to the root word. in the linguistic feature tagging module, we predict morphological features associated with an inflected word. ## experiments and results we use keras python library for the implementa tion of the system. our model is sequential model with the first layer as embedding layer followed by a bidirectional lstm layer followed by a dense layer for output prediction. we use adam optimizer and binary cross entropy and categorical cross entropy loss for morpheme segmentation and feature tagging task respectively. we keep 80:20 ratio for training and testing of the model. to study the effect of pos category on the re sults, we repeat the experiments individually for each pos category. we also compare the results of the neural mor phological analyzer with an existing unsupervised morph analyzer morphessor. morphessor @xcite is a family of methods for unsupervised morphological segmentation. the first version of morfessor, called morfessor baseline, was devel oped by @xcite and its soft ware implementation, morfessor 1.0 was released by @xcite . we have tested our dataset on morfessor implementation and com pared the results of the neural model and the unsupervised model. for the gujarati language, morphessor implementation is available in indic nlp @xcite library which is very popu lar nlp library for indian languages. however, the limitation of morphessor is that it only per forms morpheme segmentation task, whereas our proposed neural morphological analyzer performs both morpheme segmentation and morph feature tagging tasks. due to this limitation, we are able to compare results for neural and unsupervised morph analyzer for only the morpheme segmenta tion task. we observe that the neural morphologi cal analyzer outperforms unsupervised model by a large margin. 6 result analysis 6.1 morpheme boundary detection using the lstm based morpheme segmentation module, the system predicts a correct segmentation point for 3613 words out of 4058 total words in the test data set. even though the morphemes splitting in all above cases are correct, it is observed that the first portion of the split may not be the valid root word every time. we make an observation that the rules to form a valid root word are different for each word. these rules depend on pos category of the word and other grammatical features. we supply pos category of the word as an input to the system and obtain the grammatical features using morph tagging module. using this informa tion, accuracy of the morpheme boundary detec tion task can be further enhanced. it is also observed that due to ambiguities in the word formation rules, in some cases, the system is not able to identify correct segmen tation. for example, words િવદે શો (vidēśō) is spitted correctly as િવદે શ +◌ો (vidēśa + ō) and word જબરો (jabarō) is spitted correctly as જબર +◌ો(jabara + ō) . system tries to split the words ખુ લાસો(khulāsō) and િકનારો(kinārō) us ing similar method leading to incorrect outputs ખુ લાસ(khulāsa) and િકનાર(kināra). the issue here is that system considers ◌ો (ō) as the suf fix but in some words◌ો(ō) is part as the root word not as a suffix. the similar issue is observed in many other inflected words ending with suffix ◌ી(ī) as highlighted in the ## conclusion and future scope in this paper we have proposed a bilstm based morphological analyzer for the gujarati language. we have prepared the dataset and evaluated the proposed system. the system effectively performs morpheme boundary detection and morphological feature tagging tasks. with the proposed system, morphological analysis of unknown inflected word can be performed without the knowledge of lin guistic rules. we have done result analysis from the linguistic perspective. we also conclude that the proposed model performs better than the exist ing unsupervised model. in future, we aim to expand the dataset, and im plement other neural architectures such as seq2seq model. we also aim to study sentence level depen dency for morphological analysis.