The U ni M elb Submission to the SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection (2020)

Abstract: The paper describes the University of Melbourne’s submission to the SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection. Our team submitted three systems in total, two neural and one non-neural. Our analysis of systems’ performance shows positive effects of newly introduced data hallucination technique that we employed in one of neural systems, especially in low-resource scenarios. A non-neural system based on observed inflection patterns shows optimistic results even in its simple implementation (>75% accuracy for 50% of languages). With possible improvement within the same modeling principle, accuracy might grow to values above 90%.

## introduction according to wals database 80% of the world's languages morphologically mark verb tense and 65% mark grammatical case @xcite . still, until recently most research in natural language processing was focused on a few welldocumented languages with modest amount of morphological marking. a great variety of typologically diverse low-resource languages were left outside of nlp investigation and modeling. at the same time, neural systems outperformed nonneural ones on many benchmarks(cite) while being evaluated on a limited (and often not typologically representative) sample of languages. nevertheless, some of such systems or architectures were stated as "universal". but are they universal? how well models trained a certain sample of language families can generalize outside of it? for instance, a model trained on indo-european languages might be biased towards suffixing and will be working less well on languages that use infixing or prefixing. the sigmorphon 2020 shared task 0 ("typologically diverse morphological inflection"; vy-lomova et al. ( 2020 )) aims at evaluation of the generalization ability of models. it continues recent trend of increasing linguistic diversity: starting with 10 well-documented languages in @xcite up to 103 in @xcite . these shared tasks demonstrated that neural models outperform non-neural ones but generally struggle in low-resource settings. therefore, the 2019 shared task focused on cross-lingual transfer @xcite and explored transfer of morphological information from a high-resource to a low-resource language. in this paper, we describe three models submitted to the shared task 0. we investigate both generalization ability of models and their performance in low-resource languages. we propose a variation of data hallucination technique that significantly improves the results of neural models in low-resource settings. ## task description the task was organized in three stages: development, generalization and evaluation. in the development stage participants were provided with initial set of 45 development languages that were used to develop their systems. in the next stage, generalization, an extra and more diverse set of 45 languages was released, and participants were asked to fine-tune and optimize their systems on these languages. in both stages, only training and development datasets were released. test splits for both development and generalization languages were provided in the final, evaluation, stage. systems were then evaluated and ranked based on the test set predictions. ## baseline systems two types of baseline systems were provided: neural and non-neural. the non-neural baseline was essentially the same as in previous years' tasks @xcite . more specifically, it first extracts possible lemma-form alignments and associates them with corresponding target tags, then majority classifier chooses the most frequent transformation and applies it to a given lemma. the neural baselines include a hard monotonic attention model @xcite and a character level transformer @xcite . both were trained in monolingual and multilingual modes. organizers also provide a variation of the model that uses data hallucination technique from @xcite to improve performance in low-resource languages. ## evaluation the systems were evaluated in terms of test accuracy and levenstein distance between predicted and gold forms. unlike in earlier shared tasks where systems were ranked based on macroaveraging, here systems were ranked based on statistical significance of differences in their performance. ## system description in terms of the shared task, we experimented with three systems, two neural and one non-neural. sub-1 tungusic and turkic sections below provide a short description of each. ## conclusion we proposed and tested (1) multilingual training, and (2) pattern-based hallucinated inflections as possible enhancements of sequence-to-sequence morphology modeling for diverse low-resource languages. we also developed a simple non-neural approach based on multi-variant search of common inflection patterns. we explored its suitability for different language families and proposed further improvement options.