Picking Up Where the Linguist Left Off: Mapping Morphology to Phonology through Learning the Residuals (2024)

Abstract: Learning morphophonological mappings between the spoken form of a language and its underlying morphological structures is crucial for enriching resources for morphologically rich languages like Arabic. In this work, we focus on Egyptian Arabic as our case study and explore the integration of linguistic knowledge with a neural transformer model. Our approach involves learning to correct the residual errors from hand-crafted rules to predict the spoken form from a given underlying morphological representation. We demonstrate that using a minimal set of rules, we can effectively recover errors even in very low-resource settings.

## introduction modern standard arabic (msa) and classical arabic are heavily studied and privileged with many resources. arabic dialects, on the other hand, are considered to be low-resource. some dialects, such as egyptian and gulf arabic, have dedicated resources for morphological analysis but lack corresponding phonological representations. a paired representation of morphological analysis with full phonological representation is important for downstream tasks like text-to-speech (tts) and dialect identification. moreover, given a morphological representation in msa as an output from a morphological analyzer, generating the phonological representation is straightforward using a handful of rules due to the relative transparency of the standard orthography in msa. this is, however, challenging for dialects, which generally lack conventionalized consistent orthography, and in which the mapping from underlying to phonological forms may be less transparent. while grapheme-to-phoneme models seem suitable for this task, they require large amounts of data to train, which is not readily available for most arabic dialects. in arabic, a large number of phonological processes are sensitive to morphological structure. it is, therefore, reasonable to map morphophonological properties and utilize them in generating phonological representations. theoretical linguistics offers a rich literature discussing various frameworks to describe these processes efficiently. however, these frameworks are traditionally based on what is assumed to be a representative subset of the language data and are not tested against large corpora. consequently, different analyses may diverge both in terms of the posited underlying representations and the rules or constraints that relate these representations to the spoken forms. in this work, we explore the task of generating spoken forms of words from their underlying morphological representations for dialectal arabic, specifically the cairene variety of egyptian arabic. our approach utilizes the expertise of a linguist to provide morphophonological rules that map a word's morphological representation to its spoken form. we then train a character-level neural transformer model to learn the residual errors, i.e., those errors that remain after applying the linguist's rules. thus, by comparing the linguist's rule-based predictions and the actual spoken forms, this model serves as an additional correction step. ## related work modern morphological analyzers for arabic @xcite @xcite @xcite provide many features beyond morphosyntactic ones such as morphological tokenization, diacritization, and root and pattern information, but phonological representation is usually absent. although there is some degree of phonotactic modeling, it is typically constrained by orthography. when fully diacritized, the standard orthography of msa is phonologically transparent to a large degree, making phonological transcription of inflected words relatively simple, as demonstrated by @xcite and @xcite . however, dialectal arabic lacks conventionalized orthography and its phonological forms may be quite distant from the standard orthography. @xcite modeled phonotactics for dialectal arabic for the dialect identification task; however, it was based on speech signals directly. @xcite provides phonological transcription for some core lexicon entries, although those were elicited from native speakers and are not modeled to generate phonological representations for new forms productively. in our own recent work, we generate phonological forms by learning morphophonological mappings automatically @xcite ; however, while explainable, our approach produces a large number of rules and so far has modest performance as an end-to-end system. to the best of our knowledge, there is no effort focusing primarily on producing phonological representations from morphological ones for dialectal arabic. in this work, we take inspiration from zamani et al. (2018): we train a model to correct the predictions of hand-crafted rules. ## linguistic background phonology is one of the most salient differences between msa and dialectal arabic. while most phonotactics for msa can be derived through morphotactics on the level of orthography, the same is not necessarily true for dialects. naturally occurring dialectal text is usually spelled spontaneously, and even in conventional spelling, it is undiacritized, and thus far less reflective of phonology compared to msa. in this work, we focus on modeling the mapping from morphology to phonology through morphophonological mappings, specifically at the phonemic level rather than the phonetic one. in addition to some morpheme-specific rules, most phonological processes in egyptian arabic are triggered by strict requirements on how segments are organized into syllables. one such requirement is that each syllable must begin with exactly one consonant; when an underlying representation begins with a vowel, that onset consonant is supplied either by resyllabification of the final consonant of a preceding word or by insertion of a glottal stop (hamza). a second requirement is that, except in phrase-final position, syllables may end with no more than one consonant; thus, when con-catenation of morphemes creates a triconsonantal sequence, a vowel is inserted following the second consonant. a third requirement restricts long vowels to either high or low positions in the vocal tract and to stressed syllables; underlyingly long vowels are therefore shortened when unstressed and raised if underlyingly mid. similarly, long vowels are restricted to open syllables except in word-final position. finally, a word-internal short high vowel [i] is deleted when the consonants surrounding the vowel can be reassigned to neighboring syllables. ## data in this work, we chose the egyptian colloquial arabic lexicon (ecal; @xcite to acquire the spoken forms (sf). ecal also provides detailed morphological analysis for its entries but only in the form of a complex part-of-speech (pos) tag and has minimal segmentation. since our goal is to map the morphological representation (mr) of the word, we need to generate a fine-grained tokenized mr. to do so, we use the calima egy morphological analyzer @xcite with the analyzer engine from cameltools @xcite . we utilize the undiacritized orthographic form provided by ecal and run it through calima egy and extract the tokenization from the best matching analysis based on the similarities between the pos tags and lemmas provided by both resources. calima egy uses a conventional orthographic representation for its entries that balances both phonology and etymology @xcite . we map the different morphs in the tokenized form from their orthographic form to their phonemic form. we note that this phonemic form is hypothesized based on certain assumptions about the underlying representation of the different morphs (chomsky and halle, 1968; hyman, 2018). for example, word-final vowels are considered to be underlyingly short @xcite @xcite although others have argued in favor of underlyingly long final vowels in various dialects @xcite @xcite . while we acknowledge the diverse hypotheses concerning underlying representations, our focus here is on comparing the performance of different systems, assuming a specific set of underlying representations. since rules consider context from the underlying representation, adopting a different set of underlying representations would not necessarily affect the final pre-dictions. in addition to these fundamental choices about the underlying representation, we perform other orthographic-specific changes to convert the orthography-oriented representation coming out of calima egy into a phonemic representation: a) long vowels that are represented orthographically with a short vowel and a glide are normalized into a single symbol: 'iy','uw' -> 'i','u' foot_0 , b) all glottal stop shapes are normalized into a bare hamza, c) consonants with unconditional sound change (i.e., /q/, /j/, /d/, and /t/ , , , and ) preserve their etymological spelling as it is not central to morphophonological processes. while converting the single morphs from orthography to phonology is relatively straightforward, generating the full spoken form is not due to the complex interactions as explained in §3. the resulting dataset contains pairs of mr and sf. an example entry for the word /kita:bha/ 'her book' we followed the splits for train/dev/eval that were provided by ecal. ## linguist rules linguistic formalisms, such as phonological rules or constraints, provide an efficient framework for describing generalized linguistic phenomena. trained, well-versed linguists develop formal grammars mapping posited underlying representation to spoken forms based on extensive study of collected data points representing spoken language. in this work, we use a minimal set of morphophonological rewrite rules provided by broselow, a linguist experienced in egyptian arabic phonology. six rules in total were provided based on @xcite , as well as syllabification and stress assignment operations. because rules that concern the organization of segments into syllables may interact, they must be applied in order. the rules we use are as follows (in order): ## approach: learning the residual preliminary analysis of the rule predictions on a sample of data has shown that while the rules cover a wide range of phenomena, they don't predict everything correctly. this is due to several factors: a) a mismatch between the expectation of the form of the mr and the automatically generated mr; b) less frequent phonological processes that were not addressed by the current set of rules; c) some of the mrs which we automatically generated from the ecal corpus have non-trivial errors that require extensive manual adjustments. to correct for these cases, we train a character-level neural transformer to learn the residual between the rule-predicted sf and the gold sf. there are two principal goals to this effort: a) to understand the coverage of the set of rules proposed by the linguist on naturally occurring data. b) to evaluate the efficacy of learning the residuals when presented with a small amount of training data, as most arabic dialects are low-resource. ## evaluation and discussion we evaluate different configurations using oov-dev. for the first set of experiments, we evaluate the overall performance of all systems across the different learning curve training sizes. trnf is trained on mr-sf pairs for each of the splits, while trnf-res is trained on rsf-sf pairs after applying rules at each split. the accuracy across the different sizes is shown in figure 2 . we see that at the lowest setting (100-1,000), the rules alone perform best. however, within the same setup, trnf-res performs notably better than trnf, which indicates the value of learning the residuals. one could argue that learning the residual may seem easier since the pairs are more similar. however, the significantly lower performance of trnf compared to donothing suggests that this family of models is not well equipped to learn to copy. as the data size increases, both neural models quickly catch up and the difference becomes negligible. the second set of experiments targets understanding the effects of using a limited number of rules. this setup is expected when we have a true low-resource scenario with a very limited number of examples and linguistic knowledge: does devising a small number of hand-crafted rules make a difference? we train trnf-res on the residuals of applying the rules incrementally, starting with the first rule and up to all six rules, following the order provided by the linguist. we focused on the lower training sizes as shown in first, we observe that across the board, trnf is consistently behind all trnf-res models, which suggests that in low-resource settings, any number of rules improves performance over a transformer alone. second, we see that in the lowest half of the learning curve, rules alone outperform their trnf-res counterparts. however, starting at 600 examples, trnf-res transformers catch up to or outperform the rules which they have learned to correct. ## conclusion we study the problem of generating the spoken form of words from their underlying morphological representation for egyptian arabic. our approach uses state-of-the-art character-level neural transformer models to train a "residual" model which mitigates the shortcomings of using rules based on linguistic knowledge. 3 we find that a comprehensive set of hand-crafted linguistic rules is more accurate than any neural system (full or residual) in very low-resource settings (< 1,000 words of training data). however, when we do not have a full rule set, but we do have around 600 to 1,000 training examples, the residual model outperforms both the rules it has been trained to improve on, and a neural model trained without rules. we aim to explore this effort in the context of child language acquisition, where children efficiently hypothesize and generalize rules from limited input. furthermore, we will examine scenarios with limited data from one dialect and linguistic knowledge from a closely related dialect and analyze how learning the residuals can facilitate knowledge transfer.