Character Alignment in Morphologically Complex Translation Sets for Related Languages (2020)

Abstract: For languages with complex morphology, word-to-word translation is a task with various potential applications, for example, in information retrieval, language instruction, and dictionary creation, as well as in machine translation. In this paper, we confine ourselves to the subtask of character alignment for the particular case of families of related languages with very few resources for most or all members. There are many such families; we focus on the subgroup of Semitic languages spoken in Ethiopia and Eritrea. We begin with an adaptation of the familiar alignment algorithms behind statistical machine translation, modifying them as appropriate for our task. We show how character alignment can reveal morphological, phonological, and orthographic correspondences among related languages.

## background languages such as turkish, arabic, and swahili are characterized by a wide variety of productive ways in which morphemes are combined to form words. in these morphologically complex languages (mcls) -and there are thousands of them-many more specific word forms are possible than in languages such as english and chinese. this very high type-to-token ratio results in a severe data sparsity problem, leading the field of machine translation (mt) to begin to take morphology seriously once such languages were considered @xcite . as far as we know, however, no one has investigated the translation of individual words between mcls. why would such a task be worth pursuing? first, in mcls, words may correspond to entire phrases in languages such as english or chinese. thus when translating from, say, arabic to turkish, the capacity to translate individual nouns or verbs, segmented into their constituent morphemes, could play a significant role. we suspect this will be especially true for translation between closely related mcls. in such cases, syntactic differences may be limited, with word order and long-distance dependencies largely preserved, while the within-word differences may remain challenging. second, focusing on word translation pairs may reveal other potentially useful information, especially for related languages, such as language distance or correspondences between characters or phonemes. finally, a system that translates complex words can support applications such as crosslinguistic information retrieval, dictionary creation, and language instruction. in this paper, we consider the more general problem of translation among a set of words in related mcls. as an illustration, see the words in column (b) of figure1, each a translation of 'i was listening' in one of four closely related romance languages of the iberian peninsula. by examining multiple related languages, we eventually hope to be able to capitalize on inter-relationships among the languages. we are specifically interested in the word translation task for languages with few computational resources. our task is related to other work in mt. we could, for example, approach the translation of pairs of words within a set using one of the state-of-the-art neural sequence-to-sequence methods, which have also been shown to be applicable to other morphological tasks @xcite . however, we may be just as interested in the details of what is learned -where the boundaries between morphemes are, what morphophonological rules apply -as we are in the performance itself, and it may be difficult or impossible to extract this knowledge from a neural system. for now, we will look for inspiration in earlier mt work, specifically variants of statistical mt (smt) and leave comparison with neural approaches to future work. a further related area of work, which actually resembles our task more closely, concerns morphological paradigms @xcite @xcite . a partial example of a spanish paradigm appears in column (a) of figure 1 . of particular interest to us is the "paradigm cell filling" problem, that of learning the inflections that comprise a paradigm given all forms for a set of lemmas @xcite . if we compare word translations in a set of related languages with cells in a morphological paradigm within one language, the relationship between our task and the paradigm cell filling problem becomes clear. however, this is only the case for the translation of words with similar roots. figure 1 illustrates this point with an example paradigm from spanish and two example translation sets from four romance languages. 1 when the roots are related, as in the translation set in column (b), the set resembles a paradigm; when they aren't (or aren't obviously), as in the translation set in column (c), only the affixes resemble one another. we will be focusing on the case where roots are similar; thus we will need to address the question of how to distinguish such sets from those like the one in column (c). our longer-term goal is translation: given a word in one language, generate a plausible word in one or more of the other related training languages. in this paper, however, we focus on the subtask that durrett and denero (2013) begin with in their paradigm cell filling research, that of aligning the characters of the words in a translation set. the rest of the paper is organized as follows. in section 2, after an introduction to the ethio-eritrean semitic (ees) languages, we define our task in more detail, looking specifically at possible types of morphological relationships between related languages, with focus on the ees languages. next, in section 3, we discuss how some cross-linguistic morphological relationships can be captured in terms of character alignment and show how character alignment within word translation pairs in closely related languages can be be constrained. next, in section 4, we outline our approach to character alignment within word translation sets, in particular, how we adapt smt word alignment methods to handle character-level alignment. then, in section 5, we describe our experiments applying the method to data from the semitic languages. in section 5.4, we discuss results, both in terms of alignment performance and in terms of useful information that the alignments provide. finally, in section 6, we conclude, looking forward to the next steps in the long-term project of the translation of morphologically complex words. ## semitic languages of ethiopia and eritrea the languages of the world differ greatly with respect to how much morphological elaboration they make available to speakers @xcite . what is interesting for our purposes (and those of this workshop) is the apparent tendency for languages within a closely related family to agree in morphological complexity, though not necessarily in the morphological details. in this paper we focus on one of these families, the ethio-eritrean semitic (ees) languages, consisting of the subgroup of semitic languages spoken in ethiopia and eritrea @xcite . there is no general agreement on the number of ees languages, but, judged by mutual intelligibility, there are between 10 and 15. of these, two, tigrinya (hereafter, ti) and tigre (hereafter, te), are spoken in eritrea, by roughly 85% of the population, and the remainder, in addition to ti, are spoken in ethiopia, as native languages of roughly 38% of the population. these include amharic (hereafter am), the official language of the federal government of ethiopia. the ees languages are divided into northern and southern subgroups, with ge'ez, ti, and te in the northern subgroup, am and the remaining languages in the southern subgroup. ## morphological alignment and character alignment it is beyond the scope of this paper to discuss morphological complexity @xcite . we base our consideration of morphology in the remainder of the paper on the structure of verbs in ees languages and on general notions familiar from basic morphological research. we assume that for each word in a mcl, there is a stem, based on a root; zero or more affixes; and possibly instances of reduplication. for each of these elements in a word, there may be a correspondence within the word's translation in another language. there may be similarities in the order of the morphemes, their forms, and the morphophonological processes that take place at their boundaries. some of these correspondences are illustrated in figure 3a for an am-ti-te translation set. the arrows indicate corresponding morphemes, with the phonetic relationships relatively clear, except in the case of the first prefix. within the stem, the root consonants appear in boldface. though this may not be apparent, the am and ti roots in this case are systematically related: the ti consonant /è/ corresponds regularly to zero in am, and the ti consonant /s'/ corresponds often (though not always) to /t'/. clearly translation between words such as those in figure 3a can benefit from recognizing the relationships indicated by the arrows in the figure. alignment is the process of determining such relationships. alignment-identifying corresponding elements in source and target-is fundamental to the training of mt systems; the progress in neural mt resulting from the inclusion of attention mechanisms is just the most recent example @xcite . within smt, alignment is a separate step, and because everything else depends on the quality of the alignment, a great deal of emphasis is given to alignment mechanisms @xcite . how is character alignment constrained for word translation pairs in closely related languages? with rare exceptions (see below), we can expect it to be monotonic. that is, for word translation pair s and t, if the character in position i in s is associated with the character in position j in t, then the character in position i + 1 in s is either (1) not associated with any character in t or (2) associated with a character in position k in t such that @xmath0. this is illustrated for the am-ti pair in figure 3b . informally, monotonicity means that none of the association arrows (as in the upper diagram in figure 3b ) cross one another. given this constraint, we can represent character alignments in the simpler format shown in the lower diagram in the figure, with aligned characters above one another. what would it mean for the monotonicity constraint to be violated for word pairs in related languages? as far as we can tell, the only situation where this could occur is one in which one of the languages has undergone a diachronic or synchronic metathesis process, by which a pair of characters has swapped their positions. for example, imagine a word gabla in language 1, which translates as the word galba in language 2. while we are not aware of any such cases, they are certainly possible, but in this paper we restrict ourselves to the computationally simpler, and obviously much more common, monotonic cases. morphemes can of course consist of multiple characters, and a morpheme in one language can correspond to a morpheme of a different length in another; there are several examples in figure 3a . however, as in @xcite approach to paradigm cell filling, we begin with the alignment of single characters, constraining the alignment to be one-to-one in both directions. as can be seen from the alignment shown in figure 3b , these one-to-one character alignments usually do not represent morpheme correspondences, but they do represent a step towards the identification of corresponding morphemes. following character alignment, durrett and denero merge neighboring aligned characters into candidate morphemes. this step will be the topic of the next phase of the project and a future paper. ## experiments the goal of the experiments reported on below is to investigate whether our em alignment algorithm (1) results in character alignments that reflect the correspondences between affixes and root consonants ## conclusions the main contributions of this paper are (1) the introduction of a new task, that of translation of sets of words in morphologically complex, closely related languages and (2) the demonstration that a simple alignment algorithm, based on work within smt, is a promising first step in approaching this task. in particular, we have shown that the parameters that are learned during em training support alignments that reveal (1) correspondences between affixes and root consonants in pairs of ees languages and (2) relationships between orthographic or phonetic segments that we know to be valid. of course we have not yet shown to what extent alignment actually supports translation of morphologically complex words. to that end, our next step will be to proceed roughly as in the work of @xcite , extracting a set of transformation rules based on the alignments and then using a discriminative sequence model to learn the contexts for the rules.