A Cautious Generalization Goes a Long Way: Learning Morphophonological Rules (2023)

Abstract: Explicit linguistic knowledge, encoded by resources such as rule-based morphological analyzers, continues to prove useful in downstream NLP tasks, especially for low-resource languages and dialects. Rules are an important asset in descriptive linguistic grammars. However, creating such resources is usually expensive and non-trivial, especially for spoken varieties with no written standard. In this work, we present a novel approach for automatically learning morphophonological rules of Arabic from a corpus. Motivated by classic cognitive models for rule learning, rules are generalized cautiously. Rules that are memorized for individual items are only allowed to generalize to unseen forms if they are sufficiently reliable in the training data. The learned rules are further examined to ensure that they capture true linguistic phenomena described by domain experts. We also investigate the learnability of rules in low-resource settings across different experimental setups and dialects.

## introduction discovering patterns and generalizing them is the core concept behind learning in the vast majority of nlp models throughout time regardless of how they are learned or represented. tasks such as morphological (re)inflection and grapheme-tophoneme conversion have direct parallels with language learning in humans, and there is often a desire to compare the performance of modern systems (especially deep neural networks) to that in humans due to the relatively salient patterns in the transformations that the learners (machine or human) learn. representing such transformations with explicit rules would further enhance the efforts on language acquisition modeling and reduce the gap between nlp and domain experts such as linguists and cognitive scientists. additionally, in low-resource settings in nlp, rule-based resources continue to withstand the test of time when it comes to downstream tasks; however, creating such resources is a tedious task and often labor-intensive. moreover, neural networks are opaque and require additional efforts to extract human-interpretable patterns from them. therefore, there is a crucial need for rule-learning systems that produce well-generalizable rules and are able to learn rules from a small amount of data. in this paper, we present a theory-backed rulelearning approach that produces a set of generalizable rules given a dataset. we use arabic morphophonology as our case study for rule learning because it is a morphologically rich language. additionally, arabic is a continuum of related but clearly morphologically distinct dialects, most of which are very low-resourced. our primary goal of this study is not to achieve the best results on a specific nlp task per se, but rather to derive an optimal set of rules from data automatically. since we are studying morphophonology, we explicitly concentrate on transcribed speech, using the egyptian dialect of arabic as our prime example. transcribed speech itself is data that is costly to obtain so the low-resource setting is extreme: we are not in a situation where we have lots of unannotated data but little annotated data; instead we have little data altogether. therefore this is an ideal setup for this study. in a previous publication @xcite , we introduced the problem, the dataset, and an initial system which in this paper we call simple. this paper's main contributions are as follows: ## pruned abundance rule learning algorithm (parla) in this section, we introduce parla, an algorithm that produces generalizable rules from a dataset of input and output pairs. we show how we use it for egyptian arabic morphophonology in section 4. parla approaches rule learning as a spacepruning problem. we assume the starting point to be an abundant number of rules that are generated from every data point found in the data with the goal being to select the most productive rule with respect to the data. the core mechanism in determining the productivity of a rule is an evaluation metric that examines the scope of the rule. the result will be a set of rules and exceptions that represent the linguistic phenomena found in the data. parla has two independent components; the first generates all possible hypothesized rules according to certain configurations, and the second evaluates those rule hypotheses to determine their generalizability. this section provides an abstract view of parla. ## parla for egyptian arabic morphophonology in this section, we describe parla configuration details for the task of deriving the surface form, i.e., transcribed utterance, from an underlying representation. ## conclusion and future work we presented parla, an effective cognitivelymotivated rule-learning algorithm. parla is a rules+exceptions model that produces the most productive rules from a given input-output style dataset according to a productivity criterion. we used egyptian arabic morphophonology as a case study for parla. our two configurations use the tolerance principle productivity criterion (parla-tp) and accuracy at a fixed threshold (parla-acc). we conducted experiments to evaluate the overall performance, the performance at low-resource settings, and the transferability of the acquired knowledge from one dialect to another. parla-@xmath0.4 was the best performer overall. when compared to a state-of-the-art neural transformer designed for such tasks, both configurations outperformed the transformer in extremely low-resource settings. egyptian-trained parla was also effective when tested on sudanese arabic, even in extremely lowresource settings. we also show that the rules produced by parla capture the same linguistic phenomena described by an experienced linguist. in future work, we plan on further developing the rule generation component by adding more ways to configure it, including a finer-grained generalization mechanism based on phonological features, different context window sizes, and using a decomposed representation of the rules rather than complex ones. we will extend the number of arabic dialects, and languages, we test parla on, and use the produced rules to create multi-dialectal morphophonological lexicons and analyzers. we also plan to specifically examine parla-tp's performance and errorful predictions and compare it to the performance and errors of children acquiring their native languages. furthermore, we plan to study state-of-the-art neural morphological (re)inflection models and extract rule-like representations from them and evaluate them in a similar fashion to this study. additionally, for the task of learning morphophonology rules, we plan to experiment with automatically transcribed data and ways to automatically produce underlying representations since data for many dialects only exists in that form.