Tackling the Low-resource Challenge for Canonical Segmentation (2020)

Abstract: Canonical morphological segmentation consists of dividing words into their standardized morphemes. Here, we are interested in approaches for the task when training data is limited. We compare model performance in a simulated low-resource setting for the high-resource languages German, English, and Indonesian to experiments on new datasets for the truly low-resource languages Popoluca and Tepehua. We explore two new models for the task, borrowing from the closely related area of morphological generation: an LSTM pointer-generator and a sequence-to-sequence model with hard monotonic attention trained with imitation learning. We find that, in the low-resource setting, the novel approaches out-perform existing ones on all languages by up to 11.4% accuracy. However, while accuracy in emulated low-resource scenarios is over 50% for all languages, for the truly low-resource languages Popoluca and Tepehua, our best model only obtains 37.4% and 28.4% accuracy, respectively. Thus, we conclude that canonical segmentation is still a challenging task for low-resource languages.

## introduction morphological segmentation denotes the task of dividing words into their constituting morphemes, i.e., their smallest meaning-bearing units, and has been studied extensively in natural language processing (nlp) @xcite . the most common form of segmentation consists of separating morphemes at the surface level. however, this is not always well suited: in fusional languages, morphemes are merged during word formation and, thereby, change their surface forms. thus, in this paper, we tackle the task of canonical segmentation @xcite , which consists of segmenting a word while restoring the original forms of its morphemes. considering, e.g., the english word collision, its surface segmentation is collis+ion, while its canonical segmentation is collide+ion. figure 1 provides examples for all five languages we experiment on. neural models have shown to perform well on this task when large amounts of training data are available @xcite . nevertheless, datasets with morphological annotations are difficult to obtain, since they require expert annotators. furthermore, many languages with complex morphology are spoken by a limited number of people or are listed as endangered languages @xcite , which reduces the possible annotator pool even more. however, morphological segmentation is important for downstream tasks like machine translation @xcite , dependency parsing @xcite , or semantic role labeling @xcite . moreover, high performance on these tasks can yield more language independent nlp models @xcite . here, we focus on low-resource canonical segmentation. we propose two new models for the task, which have recently been successfully applied to a related morphological generation task called morphological inflection. the approaches we in-vestigate are (i) an lstm pointer-generator model @xcite , and (ii) a neural transducer trained with imitation learning (il; @xcite . since both canonical segmentation and morphological inflection are characterlevel string transduction tasks, we hypothesize that models which can learn one from limited data, will also be able to do so for the other. we experiment on three benchmark datasets in german, english, and indonesian, but simulate a low-resource scenario by reducing the number of training examples. we further evaluate our models on datasets for two truly low-resource languages: popoluca and tepehua. we find that our new models indeed outperform previous approaches on all languages. for additional insight, we also evaluate the performance of all models for varying amounts of training data from the high-resource languages and find that the neural-transducer with imitation learning outperforms all other models in all but one setting with up to 600 training examples. using the entire training set for english, german, and indonesian, the state-of-the-art lstm sequenceto-sequence model performs best. however, the difference to our proposed models is below 3.3% accuracy for all languages and models. contributions. (i) inspired by recent advances in the area of morphological generation, we propose two new models for the task of low-resource canonical segmentation, which outperform all baselines. (ii) we introduce two canonical segmentation datasets for the truly low-resource languages popoluca and tepehua. (iii) we compare all models under multiple different conditions, highlighting their strengths and shortcomings, and conduct an analysis of the errors made by all neural models. ## related work the task of morphological segmentation was introduced by @xcite . most work has considered the surface segmentation task, for which unsupervised methods like linguistica @xcite and morfessor @xcite @xcite played an important role. the latter was further extended to a semisupervised version @xcite . over the last years, supervised methods have attracted more attention: @xcite cast the task as a sequence labeling problem using conditional random fields (crfs; @xcite . a similar approach was suggested by @xcite , who employed a long short-term memory network (lstm; @xcite for tagging. semi-markov crfs were also proposed @xcite . @xcite modeled the task as a sequence-to-sequence problem. supervised methods for surface segmentation were shown to perform acceptably even in the low-resource setting @xcite . recent work also included context to improve morphological disambiguation @xcite . @xcite proposed a pointer network to find surface segmentation boundaries. for fusional languages, surface segmentation is not very effective. therefore, restoring morphemes to their canonical form was previously discussed in linguistics @xcite as well as in the nlp literature. previous approaches include unsupervised @xcite , as well as joint models for segmentation and transduction @xcite and neural encoder-decoder models @xcite . however, up to now, supervised models have only been explored in the high-resource setting. we aim at closing this gap. for low-resource morphological segmentation, rule-based approaches have been used frequently, since they do not need large amounts of data. they have been developed, e.g., with finite state transducer (fst) tools like foma @xcite or @xcite . however, this kind of system requires both time and linguistic knowledge. our aim is to explore data-driven approaches for the low-resource setting in order to overcome this limitation. ## datasets for popoluca and tepehua we release two new datasets for low-resource canonical segmentation in popoluca and tepehua foot_0 . in this section, we briefly introduce the languages, before describing our datasets. we use these two languages to shed light on polysynthetic languages that also exhibit fusional phenomena. the highresource datasets introduced by @xcite cover fusional (german), analytic (english), and agglutinative (indonesian) languages. ## models inspired by recent successes of two models for lowresource morphological inflection, we propose to apply these architectures to canonical segmentation with limited training data. in this section, we introduce the models. ## experiments we now describe the experiments we conduct to explore the performance of our models both in the high-resource and in the low-resource setting. ## error analysis to get a better understanding of the results obtained with our neural models, we perform an error analysis on the output for the development sets of all folds. by manual inspection, we identify five not mutually exclusive types of errors: oversegmentation (overseg.) arises when the number of morpheme boundaries in the prediction is higher than in the gold standard annotation. undersegmentation (underseg.) occurs when the number of morpheme boundaries is lower than in the gold in the low-resources experiments, il excels over all other models for oversegmentation and overrestoration, and for all languages with except to indonesian for wrong segmentation errors. this low error rate explains the important gains that this model shows for low-resource languages. pgnet shows, however, better performance avoiding undersegmentation errors in all languages. it also performs better for popoluca and tepehua for restoration errors, while s2s has the lowest restoration errors for english, german, and indonesian. finally, we also perform an error analysis of joint (cf. supplementary material). in our lowresource simulation experiments, we notice a surprisingly good performance of joint for german. the data for this language is special since all words contained in the set are segmentable. we find that joint has no undersegmentation errors at all. also, it makes very few copy errors (9.5%, compared to 21.7% of pgnet). for our new datasets, this model obtains a high rate of wrong segmentation (88.87% for popoluca and 91.57% for tepehua). it further seems to not easily be able to decide which words should or should not be segmented. this is shown by the high undersegmentation rate (50.14% for popoluca and 62.43% for tepehua). thus, the low performance of joint on those languages can be explained by this error type and the high morphemes-per-word rate of those languages as shown in ## conclusion we proposed two new models for the task of canonical segmentation in the low-resource setting: an lstm pointer-generator model and a neural transducer trained with imitation learning. we evaluated the performance of both models against multiple state-of-the-art baselines on five languages of different morphological typology: english, german, indonesian, tepehua, and popoluca. in emulated low-resource settings with up to 600 training examples, our best proposed model outperformed all baselines in all but one setting. we obtained a similar picture for experiments on the truly lowresource languages popoluca and tepehua: our best approach outperformed the best baseline by 11.4% and 6.5% accuracy. for large training sets, our systems performed close to the state of the art. however, we find a large gap between the emulated and the real low-resource scenarios: while accuracy is above 50% for all high-resource languages even with reduced amounts of training data, for popoluca and tepehua, our best model only obtains 37.4% and 28.4% accuracy, respectively.