Understanding Jargon: Combining Extraction and Generation for Definition Modeling (2022)

Abstract: Can machines know what twin prime is? From the composition of this phrase, machines may guess twin prime is a certain kind of prime, but it is still difficult to deduce exactly what twin stands for without additional knowledge. Here, twin prime is a jargon - a specialized term used by experts in a particular field. Explaining jargon is challenging since it usually requires domain knowledge to understand. Recently, there is an increasing interest in extracting and generating definitions of words automatically. However, existing approaches, either extraction or generation, perform poorly on jargon. In this paper, we propose to combine extraction and generation for jargon definition modeling: first extract self- and correlative definitional information of target jargon from the Web and then generate the final definitions by incorporating the extracted definitional information. Our framework is remarkably simple but effective: experiments demonstrate our method can generate high-quality definitions for jargon and outperform state-of-the-art models significantly, e.g., BLEU score from 8.76 to 22.66 and human-annotated score from 2.34 to 4.04.

## introduction jargons are specialized terms associated with a particular discipline or field. to understand jargons, a straightforward approach is to read their definitions, which are highly summarized sentences that capture the main characteristics of them. for instance, given jargon twin prime, people can know its meaning by reading its definition: "a twin prime is a prime number that is either 2 less or 2 more than another prime number." recently, acquiring definitions of words/phrases automatically has aroused increasing interest. there are two main approaches: extractive, corresponding to definition extraction, where definitions are extracted from existing corpora automatically @xcite @xcite ; and abstractive, corresponding to definition generation, where definitions are generated conditioned with the target words/phrases and the contexts in which they are used @xcite @xcite @xcite . in this paper, we study jargon definition modeling, which aims to acquire definitions for jargon automatically. jargon definition modeling is important since definitions of jargon are less likely to be organized in an existing dictionary/encyclopedia and such terms are difficult for non-experts to understand without explanations @xcite . this is particularly true for new jargon from fast-advancing fields. for instance, neither oxford dictionary @xcite nor wikipedia foot_1 includes few-shot learning-an important setup in machine learning. however, to acquire definitions for jargon, both extractive and abstractive approaches may fail. extracting high-quality definitions would be difficult due to the incompleteness and low quality of data sources (this issue is more serious for jargon since jargon is usually less frequently used than general words/phrases). e.g., a good definition may not be available in the corpus; even if it existed, it might be difficult to select from a large set of candidate sentences @xcite . generating definitions for jargon would be challenging since jargons are usually technical terms that need domain knowledge to understand, while the contexts in which they are used cannot provide sufficient knowledge. for instance, it is almost impossible for a model to generate the definition for twin prime only with context "proof of this conjecture would also imply the existence of an infinite number of twin primes" since the context does not explain twin prime, and the specific meaning is difficult jargon: few-shot learning ## related work definition extraction. definition extraction, which aims to extract definitions from corpus automatically, has been studied for a long period. existing works for definition extraction can be roughly divided into three categories: 1) rule-based, which extracts definitions with defined linguistic rules and templates @xcite @xcite ; 2) machine learning-based, which extracts definitions by statistical machine learning with carefully designed features @xcite ; 3) deep learning-based, the state-of-the-art approach for definition extraction, which is based on deep learning models such as cnn, lstm, and bert (anke and schockaert, 2018; @xcite @xcite . definition generation. definition generation, or definition modeling, has aroused increasing interest in recent years. the first study on definition generation was presented in @xcite , which aims to generate definitions of words with word embeddings. later works on definition generation put more emphasis on generating definitions of words/phrases with given contexts @xcite @xcite @xcite @xcite . for example, @xcite apply pre-trained bart @xcite for definition generation with a simple context encoding scheme. @xcite employ three t5 models @xcite for definition generation with a re-ranking mechanism to model specificity of definitions. @xcite study the graphaware definition modeling problem by incorporating biomedical ontology. @xcite study the problem of generating definitions of scientific and medical terms with varying complexity. @xcite propose to generate definitional-like sentences to describe relations between entities. there are also recent works on definition modeling for other languages, e.g., chinese, by incorporating the special properties of the specific language @xcite . however, although definition extraction and definition generation are quite relevant tasks, surprisingly, existing works do not connect and compare them. in this work, we report the first attempt to combine them. ## methodology our framework for jargon definition modeling consists of two processes: extraction, which extracts self-and correlative definitional information of the target jargon from the web; and generation, which generates the final definition by incorporating the extracted definitional information. the overview of the framework is shown in figure 1 . ## discussion in this work, we focus on jargon definition modeling. the proposed framework can be further extended to general words/phrases in a context-aware setting @xcite . for instance, to retrieve the definitional information, we can incorporate the context the target word/phrase used in. e.g., the bert extractor can be trained with a modified encoding scheme: "[cls] word/phrase [sep] context [def] sentence". similarly, the generator can produce the final definition conditioned on the context. e.g., the input of the generator can be encoded as "word/phrase [sep] context [def] sent 1 [sep] sent 2 ... [sep] sent k [def] sent ′ 1 [sep] sent ′ 2 ... [sep] sent ′ k ′ " ## conclusion we present the first combination of definition extraction and definition generation. we show that, by incorporating extracted self-and correlative definitional information, the generator can produce high-quality definitions for jargon. experimental results demonstrate the effectiveness of our framework, where the proposed method outperforms recent baselines by a large margin. we also publish several datasets for jargon definition modeling. in future work, we plan to improve our framework as discussed in section 5 and apply our methods to construct several online domain dictionaries.