When is Wall a Pared and when a Muro?: Extracting Rules Governing Lexical Selection (2021)

Abstract: Learning fine-grained distinctions between vocabulary items is a key challenge in learning a new language. For example, the noun “wall” has different lexical manifestations in Spanish – “pared” refers to an indoor wall while “muro” refers to an outside wall. However, this variety of lexical distinction may not be obvious to non-native learners unless the distinction is explained in such a way. In this work, we present a method for automatically identifying fine-grained lexical distinctions, and extracting rules explaining these distinctions in a human- and machine-readable format. We confirm the quality of these extracted rules in a language learning setup for two languages, Spanish and Greek, where we use the rules to teach non-native speakers when to translate a given ambiguous word into its different possible translations.

## introduction with increasing globalization there is a widespread prevalence and need for good materials and tools to help people learn languages. curating such content manually requires a large time and cost investment which poses a challenge particularly for languages where protection and revival efforts are ongoing @xcite . one of the most important and challenging processes in learning a new language (l2) is vocabulary acquisition @xcite , which is generally made easy by associating l2 words with words from the first language (l1) @xcite . in many cases, l1 words or word senses can be unambiguously associated with l2 words. for example "linguistics" and "lingüística" essentially form a one-to-one mapping between english and spanish. however, different languages carve up the semantic space of the world in different ways leading 1 https://github.com/aditi138/ lexselection es: muro en: wall es: pared figure 1 : semantic subdivision for the concept 'wall' results in different lexical manifestations in spanish: 'muro' for outside wall and 'pared' for inside wall whereas in english both are referred as 'wall'. to semantic subdivisions, distinctions made in one language not made in another. for example, "wall" in english is manifested differently in spanish as "pared" or "muro", as shown in figure 1 , and for an l1 english speaker it may not be immediately obvious when one should be used over the other. a skilled teacher or comprehensive language learning resource may be able to provide explanations that resolve this ambiguity. for example, robertson (2020) present word definitions in-context for finnish learners, while cavoca @xcite takes a learner through various stages of the word acquisition process including word usage, syntax. in this work, we propose a method to automatically discover rules regarding fine-grained lexical distinctions and present l2 learners with concise descriptions derived from them in an interactive framework. research in l2 vocabulary acquisition @xcite has shown that it is effective to combine strategies using explicit definitions and examples in context. however, our work contrasts to most prior work in this field such as cavoca @xcite or duolingo, 2 which use learning content manually created by subject matter experts. this necessity for curation makes it difficult to comprehensively scale this approach to many languages. specifically, our framework consists of two steps: i) use a parallel corpus to identify words in l1 which have different lexical manifestations owing to a semantic subdivision in l2, and ii) create human-and machine-readable concise descriptions that allow for easier interpretation of each lexical distinction. first, we extract source (l1) and target (l2) parallel sentences for each shortlisted l1 word. we then extract lexical and semantic features, as well as a label encoding the lexical choice in the target language from these parallel sentences for each l1 word. finally, we train a prediction model that distinguishes between the lexical choices, and extract human-understandable descriptions from this model. these descriptions could either be used as-is, or could be used as a starting point for further curation by educators. to confirm the quality of the extracted descriptions, we conduct a study where we use them to teach english native speakers lexical distinctions arising from semantic subdivisions in spanish and greek. we make our study interactive by presenting the learning content in the form of cloze tests @xcite where the english word to be taught is presented to the learner in context along with extracted concise description. the learner is then required to select the most appropriate lexical choice from the given set. the main methodological contributions therefore are automated methods to: ## problem formulation for the purpose of this paper, we define the task of lexical selection as choosing contextually correct translations from a set of target translations for an ambiguous word in the source language @xcite . we first define some variables: ## identifying semantic subdivisions in this section, we describe in detail the procedure for identifying l1 words that have different lexical manifestations in l2 owing to semantic subdivisions. for the purpose of this work, we refer to these different lexical manifestations in l2 as lexical choices and the corresponding l1 words as focus words. our work is "loosely inspired" by contrawsd @xcite and semeval-2013 @xcite which construct a dataset for cross-lingual word sense disambiguation, using a semi-automatic approach combining frequency-based heuristics with human supervision. these datasets are restricted to a subset of manually selected nouns (20 for semeval-2013 and 70-80 for contrawsd). in contrast, our approach is fully automated going beyond using just frequency-based filters. furthermore, we do not restrict to any one word class leading to words being identified across different word classes (nouns, verbs, adjectives, adverbs) for both spanish and greek. 4 we start with a parallel corpus @xmath0, y 1 ), • • • , (x |d| , y |d| )} where (x m , y m ) denote the source and target sentence pair. next, we extract word alignments automatically using a word aligner that finds sets of pairs of source and target words a @xmath1, y j : x @xmath2, y @xmath3, where for each word pair x i , y j , x i and y j are semantically similar to each other within this context. to focus on translations of the underlying content, as opposed to morphological variations, we then lemmatize all words in both the source and target sentence pairs. thus, v x and v y refer to the lemmatized vocabulary of the source and target language. going forward, all words refer to their respective lemmatized forms. we perform automatic part-of-speech (pos) tagging, dependency parsing and word sense disambiguation (wsd) on the source side data, resulting in a pos tag and word sense associated with each source word, tag(x i ) ∈ t x and sense(x i ) ∈ s x where t x is the set of pos tags and s x is the word sense vocabulary in the source language. in order to identify the focus words, we extract a list of lemmatized l1 word types v x filtered by their part-of-speech (pos) tags t x giving us tuples of the form v x , t x . this ensures that we don't conflate meanings across pos tags, because in many languages the semantics of a word can vary widely across its different pos tags. 5 we refer to the extracted tuples v x , t x as focus words for simplicity. we then extract the focus words with their respective lexical choices as follows: finally, retain the source tuples whose target translations all have the same sense, giving us l lexical choices trans(v x , t x ) = {v y 0 , . . . , v y |l| } for a source tuple (v x , t x ). ## lexical selection model after identifying a set of focus words in the source language, we train a lexical selection model parameterized by θ vx,tx for each focus word v x , t x . we extract the parallel sentences from d that include the focus word and its corresponding lexical choices, denoting them with d vx,tx . the model takes as input the source sentences x vx,@xmath4,tx and predicts the contextually correct target translation v y from a set of possible translations trans(v since we aim to induce concise, humanunderstandable explanations of semantic distinctions that can be presented to learners to help them better understand the lexical selection process, we train a prediction model which allows us to easily extract such descriptions for each lexical choice v @xmath5, t x ). in this paper, we use humanreadable descriptions of the features learned by a linear model, where these features are defined over a set of lexical and semantic features extracted from the source sentences in d vx,tx . for designing features, we take inspiration from prior work which uses extracted contextual information to improve cross-lingual sense disambiguation in machine translation systems @xcite . ## automated validation since our main research goal is to aid human learners in their learning, we focus on two approaches of evaluation: (a) automated validation, a preliminary evaluation where we validate to what extent our interpretable model can perform cross-lingual lexical selection, and (b) human evaluation ( §6) which answers our main question of whether it can teach human learners the usage of l2 words. for the automated evaluation in particular, we verify several things. first, we check whether our interpretable lexical selection model is able to learn cross-lingual lexical selection at all by measuring its performance compared to selecting the most frequently occurring translation in the corpus for a given focus word ("frequency"). we also compare with another alternative interpretable model, decision trees (dtree) trained using the same features as linearsvm, to validate the choice of svms as an interpretable model over other alternatives. further, we check how our interpretable linear svm model compares with a "performance skyline"; a less interpretable bert-based neural model @xcite that extracts representations of the source sentence from bert and trains a classifier to predict the correct lexical choice. ## evaluation with human learners we move to our main evaluation where we examine how effective our extracted rules are in aiding human learners in understanding the distinctions in l2 words. ## related work computer-assisted language learning call systems have been increasingly using nlp for creating learning content. both smille @xcite and werti @xcite aim to help the text understanding process by highlighting linguistic structures using hand-written rules and automatically acquired syntactic analysis. apertium @xcite , a rule-based mt system, while not aimed at language learning, does use human-and machine-readable rules, whose formalism can account for only fixed-length ordered contexts restricting their application. further, these rules use a combination of only lemma and pos tags while our framework uses more features. cross-lingual word sense disambiguation cl-wsd disambiguates a word in-context by providing appropriate translation across languages. @xcite construct a dataset (25 ambiguous english nouns across six languages) semi-automatically from parallel corpora which are then verified by expert translators. such lexical choice tasks have been created also for evaluating mt systems @xcite . however, these methods cover a limited set of words (mostly nouns) and require some manual intervention during the data creation process. to the best of our knowledge, our proposed pipeline is the only fully automated one that extracts several ambiguous words across multiple pos tags. ## future work while we have demonstrated the efficacy of our extracted rules in teaching new words for two languages, we plan to apply our framework on much less-resourced languages which have fewer available learning resources where learners would benefit more from an automated system. we also plan to use automated methods such as selection using model confidence to select 'representative' examples for the learning setup instead of using the native speakers. furthermore, multimodal features have proven their utility in automatic methods for lexical acquisition @xcite , and we plan to examine their effectiveness for l2 learning.