Morphology Without Borders: Clause-Level Morphology (2022)

Abstract: Morphological tasks use large multi-lingual datasets that organize words into inflection tables, which then serve as training and evaluation data for various tasks. However, a closer inspection of these data reveals profound cross-linguistic inconsistencies, which arise from the lack of a clear linguistic and operational definition of what is a word, and which severely impair the universality of the derived tasks. To overcome this deficiency, we propose to view morphology as a clause-level phenomenon, rather than word-level. It is anchored in a fixed yet inclusive set of features, that encapsulates all functions realized in a saturated clause. We deliver MightyMorph, a novel dataset for clause-level morphology covering 4 typologically different languages: English, German, Turkish, and Hebrew. We use this dataset to derive 3 clause-level morphological tasks: inflection, reinflection and analysis. Our experiments show that the clause-level tasks are substantially harder than the respective word-level tasks, while having comparable complexity across languages. Furthermore, redefining morphology to the clause-level provides a neat interface with contextualized language models (LMs) and allows assessing the morphological knowledge encoded in these models and their usability for morphological tasks. Taken together, this work opens up new horizons in the study of computational morphology, leaving ample space for studying neural morphology cross-linguistically.

## introduction morphology has long been viewed as a fundamental part of nlp, especially in cross-lingual settings-from translation @xcite to sentiment analysis @xcite -as languages vary wildly in the extent to which they use morphological marking as a means to realize meanings. recent years have seen a tremendous development in the data available for supervised morphological tasks, mostly via unimorph @xcite , a large multi-lingual dataset that provides morphological analyses of standalone words, organized into inflection tables in over 170 languages. indeed, unimorph was used in all of sigmorphon's shared tasks in the last decade @xcite . such labeled morphological data rely heavily on the notion of a 'word', as words are the elements occupying the cells of the inflection tables, and subsequently words are used as the input or output in the morphological tasks derived from these tables. however, a closer inspection of the data in unimorph reveals that it is inherently inconsistent with respect to how words are defined. for instance, it is inconsistent with regards to the inclusion or exclusion of auxiliary verbs such as ''will'' and ''be'' as part of the inflection tables, and it is inconsistent in the features words inflect for. a superficial attempt to fix this problem leads to the can of worms that is the theoretical linguistic debate regarding the definition of the morpho-syntactic word, where it seems that a coherent cross-lingual definition of words is nowhere to be found @xcite . relying on a cross-linguistically ill-defined concept in nlp is not unheard of, but it does have its price here: it undermines the perceived universality of the morphological tasks, and skews annotation efforts as well as models' accuracy in favor of those privileged languages in which morphology is not complex. to wit, even though english and turkish exhibit comparably complex systems of tense and aspect marking, pronounced using linearly ordered morphemes, english is said to have a tiny verbal paradigm of 5 forms in unimorph while turkish has several hundred forms per verb. moreover, although inflection tables have a superficially similar structure across languages, figure 1 : in word-level morphology (top), inflection scope is defined by 'wordhood', and lexemes are inflected to different sets of features in the bundle depending on language-specific word definitions. in our proposed clause-level morphology (bottom) inflection scope is fixed to the same feature bundle in all languages, regardless of white-spaces. concretely, in this work we propose to reintroduce universality into morphological tasks by side-stepping the issue of what is a word and giving up on any attempt to determine consistent word boundaries across languages. instead, we anchor morphological tasks in a cross-linguistically consistent set of inflectional features, which is equivalent to a fully saturated clause. then, the lexemes in all languages are inflected to all legal feature combinations of this set, regardless of the number of 'words' or 'white spaces' needed to realize its meaning. under this revised definition, the inclusion of the swahili form 'siwapendi' for the lexeme penda inflected to the following features: prs;neg;nom(1,sg);acc(3,pl), entails the inclusion of the english form 'i don't love them', bearing the exact same lexeme and features (see figure 1 ). we thus present mightymorph, a novel dataset for clause-level inflectional morphology, covering 4 typologically different languages: english, german, turkish, and hebrew. we sample data from mightymorph for 3 clause-level morphological tasks: inflection, reinflection, and analysis. we experiment with standard and state-of-theart models for word-level morphological tasks @xcite @xcite and show that clause-level tasks are substantially harder compared to their word-level counterparts, while exhibiting comparable cross-linguistic complexity. operating on the clause level also neatly interfaces morphology with general-purpose pretrained language models, such as t5 @xcite and bart @xcite , to harness them for morphological tasks that were so far considered non-contextualized. using the multilingual pre-trained model mt5 @xcite on our data shows that complex morphology is still genuinely challenging for such lms. we conclude that our redefinition of morphological tasks is more theoretically sound, crosslingually more consistent, and lends itself to more sophisticated modeling, leaving ample space to test the ability of lms to encode complex morphological phenomena. the contributions of this paper are manifold. first, we uncover a major inconsistency in the current setting of supervised morphological tasks in nlp ( §2). second, we redefine morphological inflection to the clause level ( §3) and deliver mightymorph, a novel clause-level morphological dataset reflecting the revised definition ( §4). we then present data for 3 clause-level morphological tasks with strong baseline results for all languages, that demonstrate the profound challenge posed by our new approach to contemporary models ( §5). in practice, we did not inflect english clauses for number in 2nd person, so we did not use the y'all pronoun and it is given here for the illustration. the morphological inflection task is in fact a specific version of a more general task, which is called morphological reinflection. in the general case, the source of inflection can be any form rather than only the lemma. specifically, a source word-form w l i b j from some lexeme l i is given as input accompanied by its own feature bundle b j , and the model reinflects it to a different feature bundle b k , resulting in the word w l i b k (2a). in (2b) we illustrate for the same french lemma finir, a reinflection from the indicative present tense with a first person singular subject 'finis' to the subjunctive past and second person singular 'finisses'. ## the proposal: word-free morphology in this work we extend inflectional morphology, data, and tasks, to the clause level. we define an inclusive cross-lingual set of inflectional features {b j } and inflect lemmas in all languages to the same set, no matter how many white-spaces have to be used in the realized form. by doing so, we reintroduce universality into morphology, equating the treatment of languages in which clauses are frequently expressed with a single word with those that use several of them. figure 1 exemplifies how this approach induces universal treatment for typologically different languages, as lexemes are inflected to the same feature bundles in all of them. the inflectional features our guiding principle in defining an inclusive set of features is the inclusion of all feature types expressed at word level in some language. this set essentially defines a saturated clause. concretely, our universal feature set contains the obvious tense, aspect, and mood (tam) features, as well as negation, interrogativity, and all argument-marking features such as: person, number, gender, case, formality, and reflexivity. tam features are obviously included as the hallmark of almost any inflectional system, particularly in most european languages, negation is expressed at the word level in many bantu languages @xcite , and interrogativity-in, for example, inuit @xcite and to a lesser degree in turkish. perhaps more important (and less familiar) is the fact that in many languages multiple arguments can be marked on a single verb. for example, agglutinating languages like georgian and basque show poly-personal agreement, where the verb morphologically indicates features of multiple arguments, above and beyond the subject. for example: trans: ''we will let you go'' ind;fut;nom(1,pl);acc(2,sg) b. spanish: dímelo trans: ''tell it to me'' imp;nom(2,sg);acc(3,sg,neut);dat(1,sg) c. basque: dakarkiogu trans: ''we bring it to him/her'' ind;prs;erg(1,pl);abs(3,sg);dat(3,sg) following @xcite feature layering approach, we propose the annotation of arguments to be done as complex features, that is, features that allow a feature set as their value. 5 so, the spanish verb form dímelo (translated: 'tell it to me'), for example, will be tagged as imp;nom(2,sg);acc(3,sg,neut);dat(1,sg). 5 this is reminiscent of feature structures in unification grammars @xcite such as gpsg, hpsg, and lfg @xcite @xcite . for languages that do not mark the verb's arguments by morphemes, we use personal pronouns to realize the relevant feature-bundles, for example, the bold elements in the english translations in (4). treating pronouns as feature realizations keeps the clauses single-lexemed for all languages, whether argument incorporating or not. to keep the inflected clauses single-lexemed in this work, we also limit the forms to main clauses, avoiding subordination. although we collected the inflectional features empirically and bottom-up, the list we ended up with corresponds to @xcite suggestion for clausal inflections: ''[for vp:] auxiliaries, tense markers, and pronominal elements representing the arguments of the clause; and determiners and possessive markers in np''. thus, our suggested feature set is not only diverse and inclusive in practice, it is also theoretically sound. 6 to illustrate, the tasks to formally complement our proposal, we amend the task definitions in section 2 to refer to forms in general f l i b j rather than words w l i b j : see ## the mightymorph benchmark we present mightymorph, the first multilingual clause-level morphological dataset. like uni-morph, mightymorph contains inflection tables with entries of the form of lemma, features, form. the data can be used to elicit training sets for any clause-level morphological task. the data covers four languages from three language families: english, german, turkish, and hebrew. 7 our selection covers languages classified as isolating, agglutinative, and fusional. the languages vary also in the extent they utilize morpho-syntactic processes: from the ablaut extensive hebrew to no ablauts in turkish; from fixed word order in turkish to the meaningconveying word-order in german. our data for each language contains at least 500 inflection tables. our data is currently limited to clauses constructed from verbal lemmas, as these are typical clause heads. reserved for future work is the expansion of the process described below to nominal and adjectival clauses. ## limitations and extensions of clause-level morphology our revised definition of morphology to disregard word boundaries does not (and is not intended to) solve all existing problems with morphological annotations in nlp of course. here we discuss some of the limitations and opportunities of this work for the future of morpho(syntactic) models in nlp. the derivation-inflection divide. our definition or clause-level morphology does not solve the long-debated demarcation of boundary between inflectional and derivational morphology (e.g., @xcite . specifically, we only refered here to inflectional features, and, like unimorph, did not provide a clear definition of what counts as inflectional vs. derivational. however, we suggest here that the lack of a clear boundary between inflectional and derivational morphology is highly similar to the lack of definition for words that operate as the boundary between morphology and syntax. indeed, in the theoretical linguistics literature, some advocate a view that posits no boundary between inflectional and derivational morphology @xcite . although this question is out of scope for this work, we conjecture that this similar problem may require a similar solution to ours, that will define a single framework for the entire inflectional-derivational morphology continuum without positing a boundary between them. overabundance. our shift to clause-level morphology does not solve the problem of overabundance, where several forms are occupying the same cell in the paradigm (for example, non-mandatory pro-drop in hebrew). as the problem exists also in word-level morphology, we followed the same approach and constructed only one canonical form for each cell. however, for a greater empirical reach of our proposal, a further extension of the inflection table is conceivable, to accommodate sets of forms in every cell, rather than a single one. implications to syntax. our solution for annotating morphology at the clause level blurs the boundary between morphology and syntax as it is often presupposed in nlp, and thus has implications also for syntactic tasks. some previous studies indeed emphasized the cross-lingual inconsistency in word definition from the syntactic perspective @xcite . our work points to a holistic approach for morpho-syntactic annotation in which clauses are consistently tagged in a morphology-style annotation, leaving syntax for inter-clausal operations. thus, we suggest that an extension of the approach taken here is desired in order to realize a single morpho-syntactic framework. specifically, our approach should be extended to include: morphological annotation for clauses with multiple lexemes; realization of morphological features of more clause-level characteristics (e.g., types of subordination and conjunction); and annotation of clauses in recursive structures. these are all fascinating research directions that extend the present contribution, and we reserve them for future work. polysynthetic languages. as a final note, we wish to make the observation that a unified morpho-syntactic system, whose desiderata are laid out in the previous paragraph, is essential for providing a straightforward treatment of some highly polysynthetic languages, specifically those that employ noun incorporation to regularly express some multi-lexemed clauses as a single word. for example, consider the yupik clause mangteghangllaghyugtukut translated we want to make a house 14 containing 3 lexemes. its treatment with the current syntactic tools is either non-helpful, as syntax only characterizes interword relations, or requires ad hoc morpheme segmentation not used in other types of languages. conversely, resorting to morphological tools will also provide no solution, due to the lexemeinflection table paradigm that assumes singlelexemed words. with a single morpho-syntactic framework, we could annotate the example above by incorporating the lemmas into their respective positions on the nested feature structure we used in this work, ending up with something similar to yug;ind;erg(1;pl);comp(ngllagh;abs(-mangtegha; indef)). thus, an annotation of this kind can expose the predicate-argument structure of the sentence while also being naturally applicable to other languages. equipped with these extensions, our approach could elegantly deal with polysynthetic languages 14 example adopted from yupik ud @xcite . and unlock a morpho-syntactic modeling ability that is most needed for low-resourced languages. ## conclusions in this work we expose the fundamental inconsistencies in contemporary computational morphology, namely, the inconsistency of wordhood across languages. to remedy this, we deliver mightymorph, the first labeled dataset for clause-level morphology. we derive training and evaluation data for the clause-level inflection, reinflection and analysis tasks. our data analysis shows that the complexity of these tasks is more comparable across languages than their word-level counterparts. this reinforces our assumption that redefinition of morphology to the clause-level reintroduces universality into computational morphology. moreover, we showed that standard (re)inflection models struggle on the clause-level compared to their performance on word-level tasks, and that the challenge is not trivially solved, even by contextualized pretrained lms such as mt5. in the future we intend to further expand our framework for more languages, and to explore more sophisticated models that take advantage of the hierarchical structure or better utilize pretrained lms. moreover, future work is planned to expand the proposal and benchmark to the inclusion of derivational morphology, and to a unified morpho-syntactic framework.