Modelling the Reduplicating L ushootseed Morphology with an FST and LSTM (2023)

Abstract: In this paper, we present an FST based approach for conducting morphological analysis, lemmatization and generation of Lushootseed words. Furthermore, we use the FST to generate training data for an LSTM based neural model and train this model to do morphological analysis. The neural model reaches a 71.9% accuracy on the test data. Furthermore, we discuss reduplication types in the Lushootseed language forms. The approach involves the use of both attested instances of reduplication and bare stems for applying a variety of reduplications to, as it is unclear just how much variation can be attributed to the individual speakers and authors of the source materials. That is, there may be areal factors that can be aligned with certain types of reduplication and their frequencies.

## introduction a significant proportion of the world's languages face the threat of endangerment to varying degrees. this endangered status poses certain constraints on the extent to which modern nlp research can be conducted with such languages. this is due to the fact that many endangered languages lack extensive textual resources that are readily accessible online. furthermore, even with available resources, there is concern about the quality of the data, as it may be influenced by various factors such as the author's level of fluency, accuracy of spelling, and inconsistencies in character encoding at the most basic level (see @xcite . reduplication appears in many languages of the world @xcite . while full reduplication is observed as a repeated word form, partial reduplication is associated with extensive variety both regular and irregular. this paper focuses on a finitestate description of the partial reduplication patterns found in the lushootseed language forms (lut) and (slh). the most predominant forms of reduplication in lushootseed are distributive (distr) and diminutive (dim), which can, in fact, appear in tandem, but there are restrictions delimiting their use (see @xcite @xcite . in addition to distr and dim, however, we also find a third and slightly less frequent random or out of control distributive (oc) (see @xcite . the base of these three types of reduplication can be found in the initial two to three phonemes of the word root most often referred to with the notation c 1 vc 2 , but the authors of this paper will surround the vowel with parentheses to indicate the possibility of its absence: c 1 (v)c 2 and thus accommodate the radical cc mentioned in @xcite . the radical consist of simple and compound letters alike, e.g., qw , g w , λ', all of which add to the issues of facilitating the extensive variation in lushootseed reduplication. first, the concept of compound letters involved in regular reduplication segments is a very import part of finitestate description for lushootseed. although the 46 phonemes canonize the extensive alphabet, they create their own demands on the description. our facilitation of lushootseed reduplication with a finite-state machine foot_0 is based on the use of a five-place holder segement concatenated directly before the radical. we number these right-toleft away from the radical {p5}{p4}{p3}{p2}{p1} where the odd-numbered place holders represent consonants, and the even-numbered ones vowels. the system is set up so that the place holders {p3}{p2}{p1} are used with distr, dim and oc reduplication, whereas the more remote place holders {p5}{p4} are used to deal with distr + dim combinations. albeit, theory sees the distributive losing the third phoneme due to a principle of antigemination (see @xcite referencing also (hess 1967: 7) and @xcite . we have assumed the absence of geminates and therefore have left them out of the equation. perhaps, further studies will require their addition to our finite-state description of reduplication in permeating the lushootseed vocabulary. ## related work several different methods are currently in use to model morphology of endangered languages computationally. in this section, we will covers some of the existing rule-based, statistical and neural approaches. our method embraces the rule-based tradition because machine-learning based methods rely on a lot of annoated data we currently do not have for lushootseed. in the rule-based research, morphology has mainly been modelled using a finite-state transducer (fst) using one of several technologies such as hfst @xcite , openfst @xcite or foma @xcite . such an approach has been successful in describing languages of a variety of different morphological groups such as polysynthetic languages (e.g. plains cree @xcite , east cree @xcite and odawa @xcite ), agglutinative languages (e.g. komi-zyrian @xcite , san mateo huave (tyers and castro, 2023), skolt sami @xcite , sakha @xcite and erzya @xcite ) and fusional languages (e.g. akkadian @xcite and arabic @xcite ). for statistical approaches, @xcite has done research on english morphology by an approach that comprises two interrelated components, which are morphological rule learning and morphological analysis. the morphological rules are acquired by means of statistical learning from a list of words. on another line of work, @xcite has developed a machine learning technique that utilizes sequence labeling and kernel methods for training, which enables the model to effectively capture the non-linear associations between various aspects of the morphological features found in tamil. with the emergence of unimorph @xcite , which continues to include only partial morphological descriptions of each language, a great deal of neural based research has emerged to conduct morphological analysis. the typical models that are used are lstm @xcite and transformer (see @xcite ) based models. ## materials and methods the materials used for this paper come from the lushootseed dictionary of @xcite and language learning binders by zalmai zahir and peggy k w ipalq ahvakana (book 1 d z ix w first, book 2 d@g w i you, book 3 s.p@ì@d food, book 4 palpal house) as well as a binder of transcriptions to recordings from the university of washington archives received in 2003 on the muckleshoot reservation. the method involves a mnemonic descriptive approach, implemented for a decidely deterministic machine and human-friendly solution -if there is such a thing. to this end, we adhere to a threephoneme segment approach to lushootseed description and simply start with the labeling 123. here ‹1› indicates the first consonant of the radical (root), ‹2› the vowel (which seems to be absent/latent in at least a few roots), and ‹3› the second consonant. we then introduce a series of five ordered place holders to precede the root. the insertion of place holders is convenient in this finite-state description if they come before the root. although there are numerous segments of regular morphology, inserting a series of five place holders immediately before the root can be seen as just another step in regular concatenation. here it might be mentioned that theoretic distinctions between inflection and clitics do not come before consideration for orthographic practices (cf. @xcite . the five place holder, numbering away from the first three letters of the root is set so the odd numbers correlate with the consonants and the even numbers with the vowels. thus, {p3} correlates with k w , {p2} with a, and {p1} with t {p5} {p4} {p3} {p2} {p1} k w a t k w atač: k w atač 'climb' s‹k w atač: sk w atač 'mountain' s ‹ {p5}:0 {p4}:0 {p3}:k w {p2}:a {p1}:0 k w a t a č : sk w ak w @tač 'mountains' s ‹ {p5}:0 {p4}:0 {p3}:k w {p2}:a {p1}:0 k w a:0 t a č : sk w ak w tač 'hill' with this as a point of departure, we can then enumerate four predominant tendencies, one -total reduplication, one -partial to the left, two partial to the right. first, total reduplication is 123123, which is extremely regular and typically distributive in meaning. second, comes the diminutive with extensive variation: 1213, 12123, 1i13, 1i123, 1iq13. third, and less frequent in the materials are 123 ## fst models the finite-state description of lushootseed involves several layers of experience. it addresses issues involving orthography, morphophonology, concatenation and symmetric tagging for subsequent machine readability. the orthography, which is canonized by the language's reduplication patterns, uses lower-case letters with multiple diacritics, as no precomposed letters are available for nearly half of the alphabet. the concatenative morphology, which with the exception of the possessive person marking strategy, is symmetric but involves abbreviated or short-hand forms for some consecutive morphemes. the variation in multiple reduplication patterns appears to be partially monolectic or geographic in nature, but there is definitely also breathing room for variation in where individual derivations are used. in general, both preposed and postposed affixing is present, and, in particular, there is asymmetry in the possessive person marking strategy. for language-independent comparison, we use flag diacritics in our models, which allows us supersegmental concatenation and facilitates regular tagging practices for use in downstream language technology, even work with python libraries. ## current state presently the lexicon is extremely small. it contains 110 verbs and 283 nouns, which might explain the low coverage rate of 70%, i.e., 1822 unrecognized tokens out of a total of 6186 tokens in the test corpus. the two-level model has 31 rules governing reduplication copying patterns in the place holders and vowel loss or permutation in the root. the vowel system has be complemented by vowels with acute and grave accents, which might be useful in pedagogical use of the language model, and in work with language variation across the continuum of the language community. source the lexc continuation lexica number at 135. these continuation lexica provide coverage for regular nominal and verbal inflection, which utilizes a mutual set of morphology controlled partially with flag diacritics. ## neural extension no matter how extensive an fst transducer is, it still cannot cover the entire lexicon of a language. for this reason, we also experiment with training neural models to do morphological analysis based on the fst transducer described in this paper. the goal is not to replace the fst we have described in this paper, but to develop a neural "fallback" model that can be used when a word is not covered by the fst. we follow the approach suggested by hämäläinen et al. (2021), we use the code that has been made available in uralicnlp @xcite . this approach consists of querying the fst transducer for all the possible morphological forms for a given lemma. for a given input, the fst will thus produce all possible inflections and their morphological readings. we limit our data to nouns only, and we use a list of 214 lushootseed nouns to generate all the possible morphological forms for. this way, we produce a dataset consisting of around 756,000 inflectional form-morphological reading tuplets. this means that we have an average of 3536 inflectional forms for each lemma. we split this data into 70% training, 15% validation and 15% testing. the test data has words that are completely unseen to the model in the training data. this means that in the testing, the model needs to analyze based on lemmas and word forms it has not seen before even in a partial paradigm. for the model itself, we use a python library called opennmt @xcite and use it to train an lstm based recurrent neural network architecture with the default settings of the library. the task is defined as a character-level neural machine translation problem where each word form are split into characters separated by a white-space in the target side and the morphological readings produced by the fst are split into separate morphological tokens. examples of the training data can be seen in the overall accuracy of the model is 71.9%. this is measured by counting how many full morphological readings the model predicted correctly for each word form in the test corpus. the results per morphological tag can be seen in ## discussion and conclusions in order to further test the accuracy of our lushootseed description, more test data and descriptions of regular inflection will be needed. the challenge is to continue with the outline given for an inflectional complex (see @xcite and define what can actually be described as regular. more time will be required to model more recent reanalyses of the morphological complexes. this means we may need to establish whether a sixplaceholder segment is required to aptly describe lushootseed reduplication and put our description in line with a hypothesis of antigemination. the idea of describing morphological complexes as series of aligned clitics is very interesting (see @xcite . this will actually provide fuel for future work with syntax, since most of the semantic information is already present in the word roots where the clitics conglomerate.