SIGMORPHON 2020 Task 0 System Description: ETH Z ürich Team (2020)

Abstract: This paper presents our system for the SIGMORPHON 2020 Shared Task. We build off of the baseline systems, performing exact inference on models trained on language family data. Our systems return the globally best solution under these models. Our two systems achieve 80.9% and 75.6% accuracy on the test set. We ultimately find that, in this setting, exact inference does not seem to help or hinder the performance of morphological inflection generators, which stands in contrast to its affect on Neural Machine Translation (NMT) models.

## introduction morphological inflection generation is the task of generating a specific word form given a lemma and a set of morphological tags. it has a wide range of applications-in particular, it can be useful for morphologically rich, but low-resource languages. if a language has complex morphology, but only scarce data are available, vocabulary coverage is often poor. in such cases, morphological inflection can be used to generate additional word forms for training data. typologically diverse morphological inflection is the focus of task 0 of the sigmorphon shared tasks @xcite , to which we submit this system. specifically, the task requires the aforementioned transformation from lemma and morphological tags to inflected form. a main challenge of the task is that it covers a typologically diverse set of languages, i.e. languages have a wide range of structural patterns and features. additionally, for a portion of these languages, only scant resources are available. our approach is to train models on language families rather than solely on individual languages. this strategy should help us overcome the problems frequently encountered for low-resource tasks, e.g., overfitting, by increasing the amount of training data used for each model. the strategy is viable due to the typological similarities between languages within the same family. we combine two of the neural baseline architectures provided by the task organizers, a multilingual transformer @xcite and a (neuralized) hidden markov model with hard monotonic attention @xcite , albeit with a different decoding strategy: we perform exact inference, returning the globally optimal solution under the model. ## background neural character-to-character transducers @xcite define a probability distribution p θ (y | x), where θ is a set of weights learned by a neural network and x and y are inputs and (possible) outputs, respectively. in the case of morphological inflection, x represents the lemma we are trying to inflect and the morphosyntactic description (msds) indicating the inflection we desire; y is then a candidate inflected form of the lemma from the set of all valid character sequences y. note that valid character sequences are padded with distinguished tokens, bos and eos, indicating the beginning and end of the sequence. the neural character-to-character transducers we consider in this work are locally normalized. specifically, the model p θ is a probability distribution over the set of possible characters which models p θ (• | x, @xmath0. by the chain rule of probability, p θ (y | x) decomposes as the decoding objective then aims to find the most probable sequence among all valid sequences: this is known as maximum a posteriori (map) decoding. while the above optimization problem implies that we find the global optimum y , we often only perform a heuristic search, e.g., beam search, since performing exact search can be quite computationally expensive due to the size of y and the dependency of p θ (• | x, @xmath1. for neural machine translation (nmt) specifically, while beam search often yields better results than greedy search, translation quality almost always decreases for beam sizes larger than 5. we refer the interested reader to the large number of works that have studied this phenomenon in detail @xcite @xcite . exact decoding effectively stretches the beam size to infinity (i.e. does not limit it), finding the globally best solution. while the effects of exact decoding have been explored for neural machine translation @xcite , to the best of our knowledge, they have not yet been explored for morphological inflection generation. this is a natural research question as the architectures of morphological inflection generation systems are often based off of those for nmt. ## data we use the data provided by the sigmorphon 2020 shared task, which features lemmas, inflections, and corresponding msds (following unimorph schema @xcite ) for 90 languages in total. data was released in two phases; the first phase included languages from five families: austronesian, niger-congo, uralic, oto-manguean, and indo-european. data from the second phase included languages belonging to afro-asiatic, algic, australian, dravidian, germanic, indo-aryan, iranian, niger-congo, nilo-sahan, romance, sino-tibetan, siouan, tungusic, turkic, uralic, and uto-aztecan families. the full list of languages can be found on the task website: https://sigmorphon.github.io/ sharedtasks/2020/task0/ . due to scarcity of resources available to the task organizers, many of the languages had only a few morphological forms annotated. for example, zarma, a songhay language, had only 56 available inflections in the training set and 9 in the development set. ## system description our systems are built using two model architectures provided as baselines by the task organizers: a multilingual transformer @xcite and a (neuralized) hidden markov model (hmm) with hard monotonic attention @xcite . we then perform exact inference on the models. the following subsections explain the two components separately. ## results on the shared task test data results on the test data from sigmorphon 2020 task 0 can be found in ## conclusion we perform exact inference on two baseline neural architectures for morphological inflection, a transformer and a (neuralized) hidden markov model with hard monotonic attention, to find the inflections with the globally best score under the model. on test data, the hidden markov model showed better results: on average, it achieved 80.9% accuracy and a levenshtein distance of 0.5, while the transformer performed worse with 75.6% and 0.62 respectively. overall, exact decoding of morphological inflection generators does not appear to significantly affect model performance compared with greedy search. this is notable when compared with nmt systems, for which exact search often leads to performance degradation.