Employing distributional semantics to organize task-focused vocabulary learning (2021)

Abstract: How can a learner systematically prepare for reading a book they are interested in? In this paper, we explore how computational linguistic methods such as distributional semantics, morphological clustering, and exercise generation can be combined with graph-based learner models to answer this question both conceptually and in practice. Based on highly structured learner models and concepts from network analysis, the learner is guided to efficiently explore the targeted lexical space. They practice using multi-gap learning activities generated from the book. In sum, the approach combines computational linguistic methods with concepts from network analysis and tutoring systems to support learners in pursuing their individual reading task goals.

## introduction learning vocabulary is a major component of foreign language learning. in the school context, initially, vocabulary learning is typically organized around the words introduced by the textbook. in addition to the incrementally growing vocabulary lists, some textbooks also provide thematically organized word banks. when other texts are read, the publisher or the teacher often provides annotations for new vocabulary items that appear in the text. a range of tools has been developed to support vocabulary learning, from digital versions of file cards to digital text editions offering annotations. while such applications serve the needs of the formal learning setting in the initial foreign language learning phase, where the texts that are read are primarily chosen to systematically introduce the language, later the selection of texts to be read can in principle follow the individual interests of the student or adult, which boosts the motivation to engage with the book. linking language learning to a functional goal that someone actually wants to achieve using language is in line with the idea of task-based language teaching (tblt), a prominent strand in language teaching @xcite . naturally, not all authentic texts are accessible to every learner, but linguistically-aware search engines, such as flair @xcite , make it possible to identify authentic texts that are at the right reading level and are rich in the language constructions next on the curriculum. where the unknown vocabulary that the reader encounters in such a setting goes beyond around 2% of unknown words in a text that can be present without substantial loss of comprehension @xcite , many digital reading environments provide the option to look up a word in a dictionary. yet, frequently looking up words in such a context is cumbersome and distracts the reader from the world of the book they are trying to engage with. relatedly, one of the key criteria of tblt is that learners should rely on their own resources to complete a task @xcite . but this naturally can require pre-task activities preparing the learner to be able to successfully tackle the task @xcite . but how can a learner systematically prepare for reading a text or book they are interested in reading? in this paper, we explore how computational linguistic methods such as distributional semantics, morphological clustering, and exercise generation can be combined with graph-based learner models to answer this question both conceptually and in practice. on the practical side, we developed an application that supports vocabulary learning as a pre-task activity for reading a self-selected book. the conceptual goal is to automatically organize the lexical-semantic space of any given english book in the form of a graph that makes it possible to sequence the vocabulary learning in a way efficiently exploring the space and to visualize this graph for the users as an open learner model @xcite showing their growing mastery of the book's lexical space. lexical learning is fostered and monitored through automatically generated multi-gap activities @xcite that support learning and revision of words in the contexts in which they occur in the book. in section 2 we discuss how a book or other text chosen by the learner is turned in to a graph encoding the lexical space that the learner needs to engage with to read the book, and how words that are morphologically related as word families @xcite are automatically identified and compactly represented in the graph (2.1.1). in section 3 we then turn to the use of the graph representation of the lexical semantic space of the book to determine the reader's learning path and represent their growing lexical knowledge as spreading activation in the graph. in section 4, the conceptual ideas are realized in an application. we discuss how the new learner cold-start problem is avoided using a very quick word recognition task we implemented, before discussing the content selection and activity generation for practice and testing activities. section 6 then provides a conceptual evaluation of the approach and compares it with related work, before concluding in section 7. ## constructing a structured domain model for the lexical space of a book going beyond the benefits of interactivity and adaptivity of individualized digital learning tools, supporting learner autonomy is known to be important for boosting motivation and self-regulation skills @xcite . this includes the choice of reading material a learner wants to engage with, where the texts prepared by a teacher or publisher cannot reflect the interests of individual students, the topics and genres they want to explore in the foreign language. the freedom of choosing a text that the learner wants to engage with also identifying a clear functional goal for learning vocabulary -learning new words to enable us to read a text of interest, so that the interest in the content coincides with the interest in further developing the language skills. in that sense learning vocabulary becomes a pre-task activity in the spirit of task-based language learning. organizing vocabulary learning in this way also helps turn the otherwise open-ended challenge of learning the lexical space of a new language to the clearly delineated task of mastering a sub-space. this functionally guided approach contrasts with the approach of other vocabulary learning tools selecting random infrequent lexical items from the language to be learned, which given their rare and often highly specialized nature are likely to only be useful for impressing friends when playing foreign language scrabble. to make text-driven vocabulary learning work, we need to map the text selected by the learner into a structured domain to support systematic and efficient learning of the lexical space as used in the book. we distinguish the process of structuring the vocabulary used in the book, independent of the learner's background, from the representation of the individual learner's knowledge. the former is tackled in this section and can be regarded as our domain model, while the latter is a learner model that essentially is an overlay over the domain model, and will be discussed in section 3. since vocabulary learning is about establishing form-meaning connections, in principle the basic unit best suited for this would be word senses. at the same time, full automatic word sense disambiguation is complex, error prone, and often domain-specific -and in the context of a given book, a given word will often occur with the same meaning. we, therefore, limit ourselves to only disambiguating homographs in terms of their partof-speech (pos), following @xcite . throughout our approach, we therefore @xmath0, @xmath1. to pos annotate the book selected by the user, we use the spacy nlp tools ( http://spacy.io ). given our focus on learning the characteristic vocabulary of the book, we eliminate stop words as well as word-pos pairs appearing less than five times in the given book. ## representing the lexical knowledge of a learner: an open learner model with a structured domain model established for the vocabulary space to be explored by the user, we want to make use of it to efficiently guide the learner to cover the space and track learning in a learner model. the learner model is an overlay on the domain model that helps us track the learner's vocabulary knowledge in terms of a mastery score associated with each word family. on the basis of the learner model, we then can propose the next set of words to be practiced in a way that reduces the number of interactions required to cover the vocabulary space. it also serves as an open learner model @xcite ) by allowing the user to view and explore the lexical space of the book as a graph, with each node being colored according to the current mastery score. in this section, we discuss how this is achieved. ## evaluation to empirically evaluate the approach, we first want to establish that it works as described. there are a number of components and parameters involved in generating the graph, selecting the nodes to be practiced, and updating the learner model. so we ran experiments with simulated users who performed the activities at different levels of accuracy. in addition to a a cold-start setting, where the system initially knows nothing about the learner, we also performed warm-start simulations for users at different proficiency levels. as a second step, we envisage conducting studies with language learners in authentic learning contexts. testing educational tools in real-life contexts is crucial for establishing that an approach is effective in the complex authentic education contexts with the rich set of cognitive, motivational, and social variables at stake there. while in @xcite we illustrate the feasibility of conducting such randomized controlled field studies, this clearly is an endeavor of its own, beyond the scope of this paper. in the first set of experiments, we cold start the system with the learner model set to the default .5 chance level for every word family, and we simulated learners with performance levels of 60%, 70%, 80%, 90%, and 100%. as baseline approach for comparison, we include a traditional flashcard setup tackling words in a linear fashion, one by one, where each word is independent of the other words. throughout, we assume that mastery is achieved when a word has reached .8 or more. in the second set of experiments, we assume an accuracy of 90% and instead consider the effect of proficiency differences as indicated by the learner's cefr level. instead of simulating the web-based book-specific vocabulary test we implemented as discussed in section 4.1, we base are simulation experiments on meara and milton's estimation of the knowledge of the most frequent 5000 lemmatized english words for learners at different cefr levels as reported in @xcite . simplifying their estimates for the number of known words distributed over the frequency bands to the upper bound given for the number of words learned in the first four proficiency levels (a1: 1500, a2: 2500, b1: 3250, b2: 3750), we started the simulation by setting the mastery score of those nodes to 0.75 for which the head word of the family occurs frequently enough (as determined by reference to subtlex-us; @xcite to be included in the set for the given proficiency level. the learner model thus encodes that the learner is likely to know the word, but the positive bias alone is not sufficient to cross the 0.8 level indicating mastery so that additional evidence is required to mark them as known. for example, an a2 learner only needs 47 sessions to master the learning targets for book b) assuming an accuracy of 90% in completing the exercises, whereas in the cold start condition we saw in ## related work while the experimental evaluation provides some insights into the practical viability of the approach, given the conceptual nature of our proposal, we here also contextualize and compare the approach with related work to discuss where it conceptually advances the state of the art. our approach can be characterized by the following aspects: first, the user can select what they want to learn the vocabulary for; they pick the text of the book they want to be able to read, i.e., the functional task goal. second, the system automatically creates a domain model graph representing the lexical semantic space to be learned. third, a learner model is created as an overlay of the domain model graph and records the mastery of the concepts by the learner, with updates to the learner model spreading activation through the graph to indirectly activate related concepts as a way to avoid explicit interaction for every word. fourth, it determines in which order the words can be learned in such a way that the lexical space is efficiently explored, prioritizing the words that are central nodes. fifth, the system compactly represents word families to allow the visualization and open learner model to be concise and usable with minimal number of interactions. sixth, the system supports learning of the words using multi-gap activities using sentences drawn from the actual book to be read. putting this approach into context of the related work on vocabulary learning, there is a large number of applications designed to support vocabulary learning -though, as we will see, the above characteristics clearly seem to set our approach apart from what is offered in this domain. foreign language textbooks systematically provide a list of vocabulary items per chapter and there are many specialized or general file card applications for memorizing these sentences including phase-6.de, quizlet.com, or ankiweb.net. other tools offer more language-related functionality. lextutor ( https://lextutor.ca ) is a website offering a collection of tools to learn vocabulary using lexical resources such as frequency-based vocabulary lists and corpus data. list learn supports learners in choosing words from frequency-based word lists and working with corpus concordances. grouplex lets the learner select from a 2k crowd-sourced word list and practice them in fill-in-the-blank activities, with hints based on dictionary definition and pos tags. flash employs cards showing words on one side and lexical support on the other. apart from word meaning and usage, morpholex supports learning regular inflectional and derivational affixes based on the word family levels of bauer and nation @xcite . other lextutor tools target reading texts with support from concordances and dictionaries. resource assisted reading lets the user choose a pre-processed book, but hyper text allows the learner to upload their text. while lextutor offers a variety of tools and corpus resources, none of them offer personalized learning, performance tracking, or structured vocabulary spaces. memrise.com is a commercial flashcard-based vocabulary learning application focused on beginners, with learning units grouped by theme with little freedom for the learner to choose contents of interest. duolingo.com is a strictly guided application supporting the users to learn a foreign language using various learning activities offering some gamification elements but no personalized vocabulary learning for texts or domains of personal interest. vocabulary.com is a gamified free vocabulary list learning application that lets learners choose from collections and the literature to practice the words in multiple-choice questions activities to choose the correct meaning phrase for the given word usage. the literature only is a source of vocabulary though, it is not used as testing context or learning goal, and the vocabulary domain is not semantically structured or to construct a structured learner model. cabuu.app supports learning of vocabulary lists scanned from books by associating each item with gestures. overall, while there is a rich landscape of applications supporting vocabulary learning, the six characteristics of the method presented in this paper set our approach apart -especially the use of distributional semantic methods to create a graph representation for any book or text the user wants to read, to efficiently organize and individually support and track the learning in this lexical space. ## conclusion in this paper, we discussed the methodological basis and realization of a tool allowing the learner to systematically learn the lexical material needed to be able to read a book they are interested in. automatically structuring the lexical space and sequencing the learning is achieved through distributional semantic methods, the automatic identification of word families, and concepts from network analysis. the graph-based domain model that is automatically derived from the given book serves as the foundation of a learner model supporting the selection of an efficient learning path through the lexical space to be acquired. multi-gap activities are automatically generated from the targeted book and used for practice and testing activities. in addition to self-guided learning for people interested in reading specific books, which may be particularly useful in the context of so-called intensive reading programs, the approach is particularly well-suited for the english for specific purposes context, where both the language and the particular content domain are of direct importance. given this kind of integration of language and content learning, a similar affinity exists to so-called content and language integrated learning @xcite .