Evaluating Neural Morphological Taggers for S anskrit (2020)

Abstract: Neural sequence labelling approaches have achieved state of the art results in morphological tagging. We evaluate the efficacy of four standard sequence labelling models on Sanskrit, a morphologically rich, fusional Indian language. As its label space can theoretically contain more than 40,000 labels, systems that explicitly model the internal structure of a label are more suited for the task, because of their ability to generalise to labels not seen during training. We find that although some neural models perform better than others, one of the common causes for error for all of these models is mispredictions due to syncretism.

## introduction sanskrit is a fusional indo-european language with rich morphology, both at the inflectional and derivational level. the language relies heavily on morphological markers to determine the syntactic, and to some extent the semantic roles, of words in a sentence. there exist limited and partly incompatible solutions @xcite @xcite for morphological tagging of sanskrit that heavily rely on lexicon driven shallow parsers and other linguistic knowledge. however recently, neural sequential labelling models have achieved competitive results in morphological tagging for multiple languages @xcite @xcite . we therefore explore the efficacy of such models in performing morphological tagging for sanskrit without access to extensive linguistic information. most recent research treats morphological tagging as a structured prediction problem where the morphological class of a word is either treated as 1 code and data available at https://github.com/ ashim95/sanskrit-morphological-taggers a monolithic label or as a composite label with multiple features @xcite . @xcite ; @xcite model the morphological tags as a sequence of individual morphological features. recently, @xcite proposed to generate this sequence of morphological features using a neural encoder-decoder architecture. @xcite shows a significant improvement in performance for morphological tagging in sanskrit by using a monolithic tagset with recurrent neural network based tagging model. in systems using monolithic labels, multiple feature values pertaining to a word are combined to form a single label @xcite , which leads to data sparsity for morphologically rich languages such as czech, turkish and sanskrit. the sparsity issue can be addressed by using composite labels which model the internal structure of a class as a set of individual features @xcite . @xcite use a neural factorial crf to model inter-dependence between individual categories of the composite morphological label. however, as the decision for monolithic vs. composite labels is one of the central design choices when tagging morphologically rich languages, we use sanskrit as a test case for a systematic evaluation for this choice. for this evaluation, we consider several neural architectures with different modelling principles. for the monolithic tag model, the neural architecture is based on a bidirectional lstm with a linear crf layer stacked on top of it @xcite . for composite labels, we explored a neural generation model that generates a sequence of morphological features for each word in the input sequence. in order to explicitly capture the interdependencies between the morphological features, we use a model based on a factorial conditional random field (crf) @xcite . additionally, independent classifiers trained under a multi-task setting with sharing of parameters are also explored @xcite . our experiments specifically focus on the following problems and questions: this raises an important point: models that perform marginally better in terms of evaluation metrics are supposedly not superior, since we see similar performances on special test sets targeting particular statistical phenomenon (unseen tags) and linguistic phenomenon (syncretism). ## problem formulation and models tagset: sanskrit, similar to hungarian @xcite and turkish (c ¸etinoglu and kuhn, 2013), relies on suffixes for marking inflectional information. as sanskrit has a rich inflectional system, the size of the tagset plays a relevant role. @xcite uses a tagset with 86 possible labels that merges some grammatical features based on linguistic considerations. @xcite use an extended tagset of 270 features, by adding the feature tense, but only to finite verbs. as the systems tested in this paper do not use external linguistic information that could restrict the range of applicable features, we choose a tagset consisting of the features shown in notation: given a sequence of tokens @xmath0, x 2 ..., x t , we aim at predicting a sequence of labels @xmath1, y 2 , ....y t , one for each token. each label y i is a composite label y @xmath2, y i1 , ...y it } and consists of a collection of grammatical features for x i . encoder: all neural sequence labelling models tested in this paper (see below) use an encoder that generates the input representations of words as follows. given a sequence of tokens as input, for each token x @xmath3, its vector representation is obtained by concatenating its word embedding with a sub-word character embedding obtained from a bidirectional lstm similar to @xcite . these word representations are passed through a word level bi-lstm to obtain hidden state h i for each token in the sequence. ## conclusion and future work in this work, we evaluated various neural models for morphological tagging of sanskrit, concentrating on models that are capable of using composite labels. we find that all the composite label models outperform monseq by significant margins. these models, with an exception to mtl-shared, achieve overall competitive results when enough training data is available. a major problem for all the sequence labelling models studied in this paper is syncretism of morphological categories, which should constitute the main focus of future research.