Interpretability for Morphological Inflection: from Character-level Predictions to Subword-level Rules (2021)

Abstract: Neural models for morphological inflection have recently attained very high results. However, their interpretation remains challenging. Towards this goal, we propose a simple linguistically-motivated variant to the encoder-decoder model with attention. In our model, character-level cross-attention mechanism is complemented with a self-attention module over substrings of the input. We design a novel approach for pattern extraction from attention weights to interpret what the model learn. We apply our methodology to analyze the model’s decisions on three typologically-different languages and find that a) our pattern extraction method applied to cross-attention weights uncovers variation in form of inflection morphemes, b) pattern extraction from self-attention shows triggers for such variation, c) both types of patterns are closely aligned with grammar inflection classes and class assignment criteria, for all three languages. Additionally, we find that the proposed encoder attention component leads to consistent performance improvements over a strong baseline.

## introduction with the rise of deep learning, neural networks have been nowadays used in the process of decision making in various domains, as different as trading, medicine and government. ethical considerations of such decisions have led to an increasing need for interpreting neural models which is a vivid research topic in machine learning community @xcite . although interpretability research in nlp is partly driven by ethics (jacovi and goldberg, 2020), there is a growing body of work exploring what linguistic properties emerge in neural models @xcite . the latter line of work aims to aid and scale up linguistic research, which is also the topic of this paper. lin-guistics research focuses on uncovering patterns and regularities in language. retrieving and analyzing structures of languages learned by neural agents can systematize our knowledge and, ideally, help us to come up with new regularities. recent advances @xcite in testing hypotheses about human language processing using the growing suite of modern interpretable nlp are, indeed, inspiring, but still relatively limited to few languages. to scale up linguistic research, we require truly language-independent models developed for languages other than english @xcite . in return, understanding the model's decisions can lead to new ideas, how to improve the performance of the model on hard cases, i.e. on a particular linguistic phenomenon or language. in our work, we concentrate on interpretability methods suitable for examining what knowledge of inflection morphology is captured by neural networks. specifically, we consider a neural model that learns a mapping from a lemma and an abstract morpho-syntactic definition (msd) to its inflected form. msd comprises a part-of-speech (pos) tag as well as language-specific inflection tags. for example, given the italian lemma scolorire "discolor" and msd v;ind;prs;3;pl (verb, 3rd person plural present indefinite form), the output is the word form scoloriscono. datasets for this task of morphological reinflection are available for many languages and provide an opportunity to study a broad set of inflection phenomena. character-level encoder-decoder neural models with attention achieved very high performance on this task across many languages (see @xcite @xcite for the results of the recent shared tasks). nevertheless, this class of models is typically not interpreted, and if it is so, the interpretation is limited to visualizing attention heatmaps on selected examples (see e.g. @xcite . figure 1 : example of a heatmap visualizing attention weights learned by the model of @xcite . the inflected form is generated from top to bottom. we argue that per-example heatmaps provide very limited insight into what neural agent learns about a specific inflection phenomenon and how neural learning process can be related, in a systematic way, to the linguistic theory. indeed, consider the previous italian example: how would humans reason to convert the lemma to its inflected form? foot_0 in this work, we assume that humans apply the rules of grammar (implicit for native speakers, and explicit for language learners) when they perform this task. specifically to our example, human reasoning could look as follows: the verb scolorire ends with the suffix -ire which determines its inflection class foot_1 , according to which we construct the inflected form by copying the stem scolorand adding an inflection suffix -iscono. what does a typical character-level model of an encoderdecoder class do? we visualize learned attention weights for an example of such a model in fig. 1 . by analyzing the most prominent alignments, we conclude that the model's character-level decisions can be combined into a) copying a substring of characters corresponding to the stem and b) generating characters for a substring corresponding to the inflection suffix. however, the decision why the model chose a particular inflection class is not visible. to reach interpretability of models for inflection, we require methods that satisfy three conditions. first, the inflection model's decisions have to be aligned more closely to human reasoning by separating two kinds of operations: determining inflection class versus generating a string (given the assignment to a class). second, a systematic analysis of model's decisions requires the extraction of inflection rules that are interpretable to humans. finally, both of these factors require working with subword units rather than individual characters, the latter being the prevailing practice for inflection models. in this paper, we propose a methodology -subword modification for a typical inflection model and interpretation method -which meet all three requirements. our experiments on three typologically-different inflection phenomenon show that the linguistic rules elicited with our framework are highly consistent with linguistic knowledge (approximated by grammars). we evaluate the effectiveness of the proposed subword modification and find that, apart from its direct impact on interpretability, it leads to consistent performance improvements. to facilitate the use of our methods for linguistic research, we share our code 3 . ## methodology: interpretability for inflection how to make a character-level neural model for inflection more interpretable? we take the stance that, to make current models more interpretable, we should analyze their decisions in terms of subwords, i.e. clusters of characters rather than individual characters. interpreting character decisions is outside of human intuition because of the double articulation principle @xcite which postulates that single phones are uninterpretable to humans, whereas the clusters of them form a mental linguistic representation of meaning in the speaker's mind. in writing, this distinction maps to the one between single characters and morphemes. are ere/ire ire 1sg o o isco 2sg i i isci 3sg a e isce 1pl iamo iamo iamo 2pl ate ete ite 3pl ano ono iscono we propose to extract human-interpretable rules from an encoder-decoder model with attention. specifically, to make it more interpretable, we modify such model by complementing the crossattention mechanism with a novel component ( §3) for self-attention over the subwords of the lemma. the task of this component is to help identify the morphological class. to extract the rules, we design a pattern extraction method ( §4) that aggregates learned attention weights a) over a span of characters in a word and b) over a range of words in the same inflection category. pattern extraction applied to character-level cross-attention weights retrieves linguistic rules satisfying requirements p1 and p2, whereas pattern extraction applied to subword-level self-attention weights, targets the requirements p3 and p4. ## neural model cross-att ch self-att sub in this section, we introduce our novel component for self-attention over subwords of the lemma (self-att sub ). this module can be integrated into any variant of encoder-decoder system for inflection with character-level cross-attention (cross-att ch ). in this work, we show such integration to a sparse two-headed model of @xcite . 5 to explain the two-headed attention mechanism of the baseline model as well as our novel attention component, we first introduce the terminology of an abstract attention head module. ## pattern extraction to extract linguistic rules from the trained model cross-att ch self-att sub , we represent its knowledge of inflection as a database. to populate it, we analyze predictions of the model on a dataset. the latter can be the original task data or a dataset collected to study a specific inflection phenomenon. then, for each example in the dataset, we populate the knowledge database with the example itself and two patterns, which are extracted from the learned attention weights. the first one is a transformation pattern ( §4.1) obtained by applying our pattern extraction method to learned cross-attention weights (cross-att ch component). this method can be applied to any inflection model embedded into encoder-decoder paradigm with attention. the second pattern is over the lemma subwords ( §4.2) which is obtained from attention weights of the novel self-att sub component. finally, we explain how populated in this way knowledge database can be queried to study inflection phenomena ( §4.3). ## experiments and results we perform three case studies, introduced in §2.1, to demonstrate how our framework allows querying patterns learned by an inflection neural model. the goal of our experiments is to assess how well the extracted patterns correspond to known inflection rules. to see whether our modifications to the inflection model affect its performance, we check the inflection accuracy on the analyzed languages and compare it with the original character-level model. we use data from sigmorphon shared task: 2018 edition for italian and finnish (10k/1k/1k examples in train/development/test data), and 2020 edition for tagalog (1,870/236/478). for each language, we train cross-att ch self-att sub model with batch size 4, beam size 1 and other hyperparameters as reported in @xcite . to produce segmented lemma input, we use the bpe method @xcite with 1k merges on a token list (100k examples) extracted from wikipediadumps articles. 10 . using model's predictions on the concatenation of train, development and test set, we query cross-att ch and self-att sub patterns as a filtering function, we keep only nonzero weights for self-att sub patterns, whereas we choose max-pooling for cross-att ch ones, as on average sparse activations assign nonzero weight to one input feature. to systematically examine whether the classes of patterns extracted are correct and cover the data adequately, we report two metrics, namely a) number of examples selected with a query and how many of them are grouped by each pattern, and b) model accuracy (correct predictions) with respect to the number of examples per selection with a query and per group pattern. cross-att ch : transformation patterns for each language, we define specific queries: 3rd person plural present tense for italian (@xmath0;ind;prs;3;pl), 3rd person plural present positive imperative for finnish (@xmath1;act;prs;pos;imp;3;pl) and imperfective aspect with agent semantic role for tagalog (@xmath2;ipfv;agfoc). the choice of msds is rather arbitrary: for illustrative purposes we select grammatical categories that contain enough examples to represent form variation of the corresponding morpheme. we observe that in all three cases, the patterns recover inflection morphemes listed in grammars for studied grammatical categories as well as their form variation. for finnish, the model correctly identifies morpheme -koot as well as its variantkööt because of vowel harmony. additionally, the patterns (3) and (4) display morphophonological processes on morphemes boundaries: if a stem ends with -d or -l, this ending is removed from the final form. in italian, the morphemes for all three inflection classes are present in the patterns: -ano (-are class) -ono (-ere class) and -scono (-ire class). besides, the separation of reflexive ending si is visible (pattern (2)) for the -are class. tagalog patterns de- ## discussion and future work in the following, we discuss our proposed methodology in terms of two aspects, namely, interpretability for inflection (in terms of typological parameters) and ideas for performance improvement. ## conclusion we propose a novel approach for interpreting neural inflection models by extracting patterns from attention weights. to enhance the interpretability of this class of models, we design a linguistically motivated attention component over subwords that leads to a systematic performance improvement. our experiments with linguistic rules induction illustrate the great potential of our methodology for linguistic research scaled to diverse typology.