Sharing Data by Language Family: Data Augmentation for R omance Language Morpheme Segmentation (2022)

Abstract: This paper presents a basic character level sequence-to-sequence approach to morpheme segmentation for the following Romance languages: French, Italian, and Spanish. We experiment with adding a small set of additional linguistic features, as well as with sharing training data between sister languages for morphological categories with low performance in single language base models. We find that while the additional linguistic features were generally not helpful in this instance, data augmentation between sister languages did help to raise the scores of some individual morphological categories, but did not consistently result in an overall improvement when considering the aggregate of the categories.

## introduction morpheme segmentation is a task in which individual words are divided into meaningful sub-units called morphemes. it is a difficult task, particularly in synthetic languages which have more complex morphological systems, but morphological analysis is an important sub-component of various downstream nlp related tasks, such as lexicography, terminology management, and semantic parsing. previous approaches to morpheme segmentation include unsupervised methods @xcite , and more recently there have been neural approaches @xcite . this paper is a submission to the sigmor-phon 2022 shared task on morpheme segmentation, which aims to benefit the nlp community with improvements for subword-based tokenization through morpheme segmentation @xcite . the shared task includes word-level and sentence-level morpheme segmentation subtasks for various development languages. we focus on the subtask for word-level morpheme segmentation, specifically for the three romance languages among the development languages: french, italian, and spanish. in this paper, we experiment with adding character based features to a sequence to sequence neural model, and we also experiment with sharing training data between sister languages. the structure of the of the paper is as follows: in section 2 we give an overview of the base system architecture of our approach. section 3 describes the character based features we experimented with during development, and section 4 describes our methods for data sharing between sister languages. section 5 presents the results from our various models, and section 6 provides the accompanying discussion. finally, section 7 offers a brief conclusion. ## additional features while sequence-to-sequence neural models have a tremendous ability to learn patterns that are la-tent in the raw text data on which the models are trained, there is still value in leveraging additional knowledge sources to provide features that may be linguistically important to morpheme segmentation that cannot be gleaned from the raw text of the training data alone. this is particularly true for languages where training data is limited and for morphological categories that are represented with low frequency in the data. in order to train extra features in sequence to sequence modeling, we can combine our features into a single input vector with the individual input character representations @xcite . we do this by concatenating vectorized character input with a vectorized representation of our character based features. for simplicity's sake, we experimented with a series presence/absence features, which could be represented with a binary 1 or 0 encoding and easily concatenated to the one-hot representation of the text of the input character. we experimented with adding a series of binary features to indicate whether the substrings that would be created by making a morpheme boundary at a given character would contain a known prefix or suffix. we created character based features for the following rules (yes-1, no-0): if the given character were the start of a new morpheme: for instance, the word enthrallments would have the feature vector 000001 for the character m, as visualized in figure 1 . this is because ment is a known suffix that starts a character m where we are imagining a morpheme boundary to be, which means that "yes" is the answer for question six. the answer for the rest of the questions is "no", so the rest of the digits in the vector are 0. such short feature vectors were generated for every character in every word of the provided data sets for our focus languages by referencing against previously complied language specific prefix foot_2 and suffix foot_3 lists compiled from wiktionary. we created various models with subsets of the training data and tested on subsets the development data for validation in order to gauge the merit of these features. in this instance, the inclusion of various combinations of the above features frequently led to degradation in performance compared to our base models when evaluated on the development data. as such, the features described above were not included in our final models trained on the full data set for most of our focus languages. for the sake of comparison, in section 5.3 we include the results of a model trained on the full french training data which also incorporates a subset of the features outlined above. this model shows marginal improvement over the base french model on the test data. ## sister language data sharing data augmentation for low-resource languages has been well researched area for various nlp tasks, such as machine translation @xcite and speech recognition @xcite . while data is provided by the shared task for all of the development languages, the number of training instances varies considerably, both in total amount and in the proportion of different morphological categories attested. sharing data between languages is one means of evening out the representation of these underrepresented morphological categories. this type of data sharing is an instance of transductive transfer learning, where the domains are initially distinct (different languages), but the task in question remains the same (morpheme segmentation), and the knowledge in one domain is used to increase the task performance in the other domain @xcite . sister languages descend from a common ancestral language and are as such part of the same language family. languages from the same language family are more likely to bear a strong resemblance to one another with regard to various linguistic aspects, including morphological structure, than sets of unrelated languages. our focus languages in this paper (french, italian, and spanish) are all a part of the romance language family, and as such, we may posit that they share enough similarity in their morphological structure for there to be some benefit in sharing data between the languages during training. in order to test this conjecture, we make a comparison between base models for each of our focus languages, which only contain training data from one language, and augmented models, which are trained on the full training data for one language and supplemented with training data from the other two romance languages for select morphological categories. for several of the development languages, including all three romance languages, training and development data for the word-level subtask included additional annotation which indicated the morphological category of the word, and the evaluation scripts provided by that shared task also offered a breakdown by morphological category. the morphological categories provided in the shared task data are shown in in order to decide which morphological cate-gories should be augmented with data from sister languages for each of the romance languages, we evaluate our base models, which were each only trained with data from one language. for each language, we examine the base model's performance on the development data for the task, and we identify the four morphological categories with the lowest performance. for these categories, we add supplemental data from the other two romance languages to train our augmented models. the identification of these categories for each of our augmented models and the results of their performance is detailed in section 5.4. ## results the shared task for word-level morpheme segmentation uses precision, recall, and f-measure as evaluation metrics for correctly predicted morphemes, as well as the average levenshtein edit distance between the predicted instance and the reference instance. overall scores are reported, as well as scores for individual morphological categories. the following subsections go through the baseline results provided by the shared task for our focus languages, as well as the results for our models. all scores are on the test data sets for individual languages. overall, we find that all of our models make a significant improvement over the baseline. ## discussion while all of base models made significant improvements from the baseline scores provided for the word-level subtask, we note that our additional experimentation resulted in only modest improvements. we also note that our experimenting with additional features frequently led to score degradation on the development data. we did not expect to see the general degradation in our scores with the inclusion of the known affix presence/absence based features that we saw in our experiments predicting on the development data. however, we did see the marginal improvement we expected in the results of the fully trained french model predicting on the test data, as described in section 5.3. on possible explanation for these inconsistent results is that the inclusion of single character or two character affixes created feature vectors with too many false positives to be of use in the model's learning for our small scales experiments predicting on the development data. further error analysis is needed to conclude the reason for such inconsistency. the fact that the improvements seen in fully trained french model were marginal suggest that the base architecture of our models may be independently capable of learning information encoded in our linguistic features. the sharing of language data between sister languages gave modest gains in our experiments, indicating that there is some potential to leverage available data from morphologically similar languages for morpheme segmentation. in future experiments we want to experiment with different methods of deciding what/how much data should be shared in order to maximize this potential. additionally, rather than just assuming that being in the same language family indicates enough morphological similarity between languages for data sharing to be of use, we believe that is would be beneficial to make a closer study of the morphological similarities and differences between sets of languages that will be used for data sharing. ## conclusion in this paper we presented a basic approach to morpheme segmentation at the word-level for the sig-morphon 2022 shared task for french, italian, and spanish. all of our presented models considerably improved upon the baselines for the shared task. while the extra character based features we experimented with generally did not prove useful in this instance, we did find some evidence that sharing data between morphologically similar languages could result in minor improvements in the segmentation of words in morphological categories which were augmented with additional data.