SIGMORPHON 2022 Shared Task on Morpheme Segmentation Submission Description: Sequence Labelling for Word-Level Morpheme Segmentation (2022)

Abstract: We propose a sequence labelling approach to word-level morpheme segmentation. Segmentation labels are edit operations derived from a modified minimum edit distance alignment. We show that sequence labelling performs well for “shallow segmentation” and “canonical segmentation”, achieving 96.06 f1 score (macroaveraged over all languages in the shared task) and ranking 3rd among all participating teams. Therefore, we conclude that sequence labelling is a promising approach to morpheme segmentation.

## introduction this paper describes our participation in the sig-morphon 2022 shared task on morpheme segmentation @xcite . building on previous work on word segmentation and transliteration by @xcite , we propose a sequence labelling approach to morpheme segmentation. the shared task consists of 2 tracks: word-level morpheme segmentation and sentence-level morpheme segmentation. data for this shared task was taken from @xcite and @xcite . although our approach is applicable to both word-level and sentence-level morpheme segmentation, we focus on word-level segmentation. we only evaluate the zero-shot performance of our word-only segmentation models on sentencelevel morpheme segmentation. sequence labelling approaches can claim several advantages over the main alternative, namely (neural) encoder-decoder approaches: sequence labelling does not require beam search for inference, may allow for smaller models, and defines a direct alignment between the input and predictions. the latter property may make models more interpretable and help with error analysis. however, sequence labelling is less flexible than encoderdecoder approaches and requires special handling of cases where the input and target sequences are of different length. however, due to the local structure of morphology, sequence labelling may be sufficient to model morpheme segmentation despite being less expressive than encoder-decoder approaches. ## related work morpheme segmentation is a well-established task in computational linguistics (cf. @xcite ). recently, two definitions of morpheme segmentations have emerged: "shallow segmentation" and "canonical segmentation" @xcite . "shallow" segmentation means segmenting the input word surface string into morphemic substrings. this kind of segmentation is called "shallow", because no orthographic restoration of morphemes to their "canonical" form is performed @xcite . "canonical segmentation", instead, attempts to restore a standardised form of morphemes. as noted by @xcite , this is necessary for synthetic languages where multiple morphemes may be merged. another source of morpheme merging may arise from phonological or orthographic constraints of the language. the present shared task features both shallow segmentation data (e.g. czech, latin), and canonical segmentation (e.g. italian, english). since canonical segmentation is a strict generalisation of shallow segmentation, methods that work for all languages in this shared task have to be able to perform canonical segmentation. however, shallow segmentation allows for a conceptually easier approach, namely sequence labelling @xcite . canonical segmentation has hitherto been defined as a sequence-to-sequence task @xcite . of course, various improvements for the sequence-to-sequence setup have been proposed, for example reranking of output hypotheses @xcite , multi task learn-ing @xcite , pointer-generator networks @xcite , and imitation learning @xcite . in fact, @xcite explicitly doubts that canonical segmentation can be approached as a sequence labelling task. however, other approaches have already worked towards approaching canonical segmentation as a sequence labelling task: cotterell et al. (2016) take a middle ground by allowing only for a maximum number of insertions. @xcite train a model to first predict insertion positions in the input sequence. then, they use a sequence labelling model on the augmented input string to predict the labels. while similar to our approach, we augment the labels instead of the input string. therefore, our method remains endto-end trainable. finally, @xcite propose a sequence labelling approach to sanskrit word segmentation, which includes restoring original forms that have been merged due to a phonological process called sandhi. therefore, our work extends the method proposed by hellwig and nehrdich and thereby shows that canonical morpheme segmentation can be approached effectively as a sequence labelling task. ## results word-level segmentation official test set results foot_0 for word-level segmentation are in in our results, we do not see any trends regarding a relationship between number of generated labels and performance. the language with weakest performance, english, has the 2nd highest number of generated labels, but the language with highest number of generated labels, russian, is the language with second best performance. czech, the number with the lowest number of generated labels, is the language with 3rd worst performance, but latin, the language with 2nd lowest number of generated labels, is the language with best performance. this suggests that our data preprocessing method does not obscure the segmentation difficulty inherent in a language. remember that differences in the amounts of labels is due to different annotation approaches in the data: for czech and latin, only "shallow" morpheme boundaries are annotated, i.e. where morpheme boundaries are in the input string. for other languages, restored morphemes are annotated that are contracted when forming the word. for example, the english word "entabulates" is segmented as "en @@table @@ate @@s" where "u" is inserted to form the word, but the "e" in "table" is deleted. sentence-level segmentation official test set results for sentence-level segmentation are in ## error analysis frequent errors as claimed in section 1, our proposed sequence labelling method allows for direct comparison of the predicted labels to labels created by our preprocessing. here, we provide a short analysis of the most frequent errors made by our english word segmentation model. to this end, we apply the preprocessing method described in section 3.1 to the test set released by the shared e n t a b u l a t e s c c @@ + c c c d c + e @@ + c c c @@ + c the most common errors are predicting morpheme boundaries where actually no morpheme boundaries are, i.e. predicting " @@ + c" instead of "c", which happens 3820 times, and missing to predict morpheme boundaries, i.e. predicting "c" instead of " @@ + c", which happens 3786 times. an example is "lemming": our models predicts "lem @@ing" instead of "lemming". a morpheme boundary was overlooked in "sanity": our model predicts "sanity" instead of "sane @@ity". the next most frequent errors are missing to insert an "e", i.e. predicting "c" instead of "c+e", which happens 499 times, and inserting a superfluous "e", i.e. predicting "c + e" instead of "c", which happens 414 times. for example, our model predicts "wok @@ism" instead of "woke @@ism" for "wokism" and "ominouse @@ity" instead of "ominous @@ity" for "ominosity". the last error type in the top 5 most frequent errors is not deleting an input character, i.e. predicting "c" instead of "d", which happens 448 times. for example, our model predicts "charr @@y" instead of "char @@y" for "charry". please note that there can be multiple errors in the predictions for a single input word. however, in most cases (5873), there is only one incorrectly predicted label. in 2008 cases, there are 2 incorrectly predicted labels. the extreme case is "alsakharovite", for which 9 of the predicted labels are incorrect: our model simply predicts to copy each character, but the ground truth is given as "aleksey @@ite". this shows that our model struggles with proper names, which is not surprising. in conclusion, this analysis shows that the largest gains in performance can be expected from improving the shallow part of segmentation, while there are fewer individual morpheme reconstruction errors. therefore, a possible future extension of our proposed model is switching to a multi task setting, where one task is to predict morpheme boundaries and the other task is to reconstruct partial or missing morphemes. in the setting evaluated here, these tasks are approached jointly. label embeddings additionally, we inspect the learned (english) label embeddings and try to see whether any patterns emerge. to this end, we retrieve the 50 most frequent labels (accounting for 98% of all labels in the dataset excluding the simple copy label) and their label embeddings (columns in the final linear prediction layer). we cluster the label embeddings by affinity propagation. the advantage of affinity propagation is that we do not have to specify the number of clusters. we use the scikit-learn implementation of affinity propagation @xcite with default parameters. the discovered clusters are in in total, the clustering generates 7 clusters, of which 4 clusters contain multiple labels and 3 clusters contain only 1 label. no cluster is completely pure, but we can observe the following trends: cluster 0 mostly represents morpheme boundary labels followed by a copy operation, i.e. "insert morpheme boundary before this character". cluster 2 mostly represents substitutions and clusters 1 and 3 mostly represent insertions. we cannot say anything definite about the 1 element clusters. from these observations we conclude that the model learns to distinguish different edit operations (insertion, substitution) and also learns to distin-guish inserting morpheme boundaries from other edit operations. this provides further evidence that changing our approach to a multi task setting may be worth exploring. generalisation to unseen substitutions finally, we want to address the problem that the finite number of labels generated by our data preprocessing method (see section 3.1) may not allow the model to generalise to substitutions not seen in the training data. 2 to collect evidence concerning this problem, again for the english test set, we find all words that cannot be generated by our model because generating them would require labels that were not generated from the training data. in total, we find 35 such words (of 57755 test words in total). furthermore, upon manual inspection, we find many of these cases either caused by proper names with irregular or non-english segmentation, for example "staffie" is segmented as "stafford @@shire @@ie" or "lebos" is segmented as "lebanon @@ese @@o @@s", or annotation errors, for example "unlid" is segmented (in the gold data) as "un#etymology_2 @@lid" or "perfosfamide" is segmented as "hydroperoxy @@fosfamide". however, we also discover a genuine problem of our model, namely that it does not have any labels to generate hyphens ("-"). this proves that the problem can be substantial, if there had been more hyphenated words in the test data. on the other hand, hyphens do not appear as character in any train set segmentation, so it is generally hard to anticipate peculiarities of the test set. the case of proper names could perhaps be handled by external resources, but this does not scale well. summing up, we acknowledge that this is an issue not solved entirely by our approach. however, it does not cause many problems for this shared task. this being said, this shared task provides a lot of data for the featured languages, so the missing label problem may become more serious for low resource languages or settings. one step towards approaching this issue could be not only generating labels from one alignment of the word to its morpheme segmentation string (see section 3.1), but from multiple alignments. this could potentially also regularise the model or allow for different training strategies than the standard supervised training. another possibility could be to equip the model with the ability to generate new cluster id labels 0 "c", " @@ + c", " @@e + c", " @@a + c", " @@i + c", " @@o + c", "c + t", " @@l + c", "d + n", " @@ + c + e", "c + @@s", "c + te", "c + ic", " @@is + c", " @@i + c + e", "c + m", " @@en + c", " @@a + c + e", "c + l", " @@t + c" 1 "d + y", "d + @@y", "c + um", "d + e", "c + s", "d + e @@y", "d + o", "d + a", "d + d" 2 "c + e", "c + y", "c + a", "c + o", "c + us", "c + on", "d + sis", "c + ion", "c + ous", "d + p" 3 "d", "d + i", "c + @@y", "d + is", "d + x", "c + n", "d + s" 4 "c + ne" 5 "d + ce" 6 "d + de" ## conclusion we presented a sequence labelling approach for word-level morpheme segmentation. models trained with this approach yield strong performance on all languages (word-level) despite of not using ensembling and using a simple bilstm encoder. error analysis for english reveals that the model is often only wrong in 1 single place, struggles with proper names, and most frequently errors are caused by incorrect prediction of morpheme boundaries.