Multitasking Framework for Unsupervised Simple Definition Generation (2022)

Abstract: The definition generation task can help language learners by providing explanations for unfamiliar words. This task has attracted much attention in recent years. We propose a novel task of Simple Definition Generation (SDG) to help language learners and low literacy readers. A significant challenge of this task is the lack of learner’s dictionaries in many languages, and therefore the lack of data for supervised training. We explore this task and propose a multitasking framework SimpDefiner that only requires a standard dictionary with complex definitions and a corpus containing arbitrary simple texts. We disentangle the complexity factors from the text by carefully designing a parameter sharing scheme between two decoders. By jointly training these components, the framework can generate both complex and simple definitions simultaneously. We demonstrate that the framework can generate relevant, simple definitions for the target words through automatic and manual evaluations on English and Chinese datasets. Our method outperforms the baseline model by a 1.77 SARI score on the English dataset, and raises the proportion of the low level (HSK level 1-3) words in Chinese definitions by 3.87%.

## introduction helping language learners understand words in doubt is an important topic in the field of intelligent computer-assisted language learning (icall) @xcite @xcite . in recent years, researchers attempted to automatically generate definitions for words rather than formulating predefined worddefinition inventories @xcite @xcite . there are two reasons for this. firstly, it can be difficult for users to distinguish which sense is appropriate in the a notice, picture or film telling people about a product, job or service. simple definition in oald a notice or announcement in a public medium promoting a product, service, or event or publicizing a job vacancy. current context because of the cognitively inaccurate nature of discrete sense boundaries @xcite @xcite . secondly, the predefined inventories need to be updated manually by lexicographers, which is time-consuming and causes dictionaries to lag behind the ever-changing language usage. different from previous work @xcite @xcite ) that focused only on how to generate definitions, we further propose a novel task of simple definition generation (sdg). making the definitions easier to read and understand could benefit the language learners, low literacy readers, as well as helping people with aphasia or dyslexia. for example, compared with the oxford dictionary (od), the oxford advanced learner's dictionary (oald) has simpler definitions, which are specifically designed for language learners. as shown in figure 1 , the definition of the word advertisement in oald does not contain difficult words or phrases such as announcement and public medium. the goal of sdg task is to generate simple definitions for languages that lack learner's dictionary. for example, chinese as second language (csl) learners do not have suitable dictionaries. as @xcite pointed out, since the difficulty of definitions is not considered, the existing dictionary cannot meet csl learner's needs. the sdg task is challenging because it requires a model to learn from a standard dictionary containing complex definitions and then generate simple ones, and hence fully unsupervised. a seemingly feasible solution is to generate definitions first and then simplify them, i.e., the generationsimplification pipeline. however, the simplification task requires dataset with complex-simple sentence pairs, and such data is also difficult to find in languages other than english @xcite . besides, the pipeline methods do not perform well due to accumulated errors (section 6.1). to solve this dilemma and bridge the gap between practical needs for simple definitions and current trivial definition generation systems, we present a novel method for the sdg task. as illustrated in figure 2 , our method leverages a multitasking framework simpdefiner to generate simple definitions by performing three sub-tasks at the same time, which are definition generation, text reconstruction, and language modeling tasks. the framework consists of a fully shared encoder and two partially shared decoders. we disentangle the complexity factors from the text by designing a parameter sharing scheme. particularly, we share parameters in complexity-dependent layer normalization and complexity-dependent query projection of the transformer architecture @xcite to control the complexity (section 3.3). through joint learning and sharing parameters between the decoders, the simpdefiner is able to generate complex and simple definitions simultaneously. main contributions of our paper are listed below: ## method we integrate three sub-tasks of definition generation, text reconstruction, and language modeling into the simpdefiner. this section first gives a formal definition of the sdg task, then introduces each sub-task, and finally the parameter sharing scheme. ## datasets we evaluate the proposed multitasking framework on both english and chinese datasets. each language has a definition generation dataset and a simple text corpus. ## experiments this section presents the experimental settings and evaluation methods. ## conclusion in this work, we propose the sdg task, a novel task of generating simplified definitions in a zero-shot manner. to this end, we leverage a multitasking framework simpdefiner to tackle this task. we introduce a text reconstruction task to the framework to control the text complexity, and a language modeling task to enhance the decoder. for evaluation, we construct a novel test set in english by manually aligning the two dictionaries of od and oald. the automatic and manual evaluations indicate that the our proposed framework can generate more accurate and more straightforward definitions than other models and the generation-simplification pipelines. in the future, we will try to combine the current method with prompt learning methods, aiming to let users condition the complexity of generated definitions.