The CMU - LTI submission to the SIGMORPHON 2020 Shared Task 0: Language-Specific Cross-Lingual Transfer (2020)

Abstract: This paper describes the CMU-LTI submission to the SIGMORPHON 2020 Shared Task 0 on typologically diverse morphological inflection. The (unrestricted) submission uses the cross-lingual approach of our last year’s winning submission (Anastasopoulos and Neubig, 2019), but adapted to use specific transfer languages for each test language. Our system, with fixed non-tuned hyperparameters, achieved a macro-averaged accuracy of 80.65 ranking 20th among 31 systems, but it was still tied for best system in 25 of the 90 total languages.

## introduction morphological inflection is the process that creates grammatical forms (typically guided by sentence structure) of a lexeme/lemma. as a computational task it is framed as mapping from the lemma and a set of morphological tags to the desired form, which simplifies the task by removing the necessity to infer the form from context. for an example from asturian, given the lemma aguar and tags v;prs;2;pl;ind, the task is to create the indicative voice, present tense, 2 nd person plural form aguà. let @xmath0. . . x n be a character sequence of the lemma, @xmath1. . . t m a set of morphological tags, and @xmath2. . . y k be an inflection target character sequence. the goal is to model p (y | x, t). the problem has been studied in various settings through the sigmorphon shared tasks @xcite @xcite , with the 2019 edition focusing in particularly challenging low-resource scenarios. the 2020 edition @xcite focused on generalization of systems across typologically diverse languages, regardless of data size. in our submission we built upon our previous work @xcite , utilizing cross-lingual transfer from related languages, data hallucination, and a series of training techniques and regularizers. the defining change was that we attempted to create language-specific regimes for each test language, depending on the particular characteristics of the language, on the data availability for the particular test language and the availability of other related language data. as a result, for some high-resource languages we submitted systems without cross-lingual transfer, for some we used a single related high resource language, and for some we used multiple related languages. last, for a few test languages we augmented our datasets with romanized versions of the training data, an approach that has shown promising results in concurrent work @xcite . our submissions are very competitive in 25 of the 90 test languages, with performance statistically significant similar to the best performing system, but fall behind in many other languages. we suspect that this is due to our not tuning of the system's hyperparameters towards higher-resource settings. ## system description our system is the same as the one of anastasopoulos and neubig (2019): a neural multi-source encoder-decoder (which reads in the lemma and the tag sequences in a disentangled manner using two separate encoders) with a task-specific attention mechanism. we skip providing further redundant information and we direct the interested reader to @xcite for all details. it is important to note, however, that we did not tune any model hyperparameters for our submissions (which we suspect contributed to the poor performance of our system in some languages); we used the default parameters from the system's distribution foot_0 which are tuned towards extremely low-resource settings. here, we provide an exhaustive list of modifications to the general pipeline that we devised for specific languages and language families. ## results the two languages where our system performs the worst are algic (cree) and tungusic (evenki). we suspect this is due to the fact that the data hallucination technique, which is crucial for such low resource settings, is not appropriate for capturing the vowel harmony of evenki along with its agglutinating morphological patterns -the hallucinated data do not follow these patterns and hence do not guide the model towards learning them. as for cree, we suspect that the problem lies again in the data hallucination process: the polysynthetic and fusional nature of cree verb inflected forms is too complicated to be modeled by the simple characterlevel alignment model which is the first step for hallucination. ## conclusion and future work the performance of our system in the 2020 sig-morphon shared task leaves many questions unanswered and several avenues to explore in future work. regarding the choice of languages to use for cross-lingual transfer, we will further in-vestigate the use of automatic suggestion systems such as the one of @xcite . with regards to modeling, we will update our model to use sparsemax @xcite , which can facilitate exact search and hopefully lead to better results @xcite . as we anticipate and hope the shared task and the whole community will become more multilingual in the future, in the future we will employ the language/task selection method of @xcite , which will allow us to tune the systems in a small subset of languages that will generalize well in all others. similarly, we will employ more sophisticated techniques for learning in multilingual settings, such as differential data selection @xcite which will allow us to optimize a single model to multiple model objectives (namely, each target language).