Fine-grained Contrastive Learning for Definition Generation (2022)

Abstract: Recently, pre-trained transformer-based models have achieved great success in the task of definition generation (DG). However, previous encoder-decoder models lack effective representation learning to contain full semantic components of the given word, which leads to generating under-specific definitions. To address this problem, we propose a novel contrastive learning method, encouraging the model to capture more detailed semantic representations from the definition sequence encoding. According to both automatic and manual evaluation, the experimental results on three mainstream benchmarks demonstrate that the proposed method could generate more specific and high-quality definitions compared with several state-of-the-art models.

## introduction when readers find some expressions unfamiliar during reading a text, machines can help. the task of definition generation (dg) aims to generate a textual definition for a given word or phrase (the target), according to a surrounding context (the local context) @xcite . in addition to assisting readers in comprehending expressions, the task of dg is also useful for generating definition when building dictionaries. recently, pre-trained encoder-decoder models have achieved great successes on this task @xcite . despite their successes, the definitions produced by these pretrained models often contain several types of errors @xcite . according to ## method in this section, we present our method of using contrastive learning to enhance target words' representation for definition generation. specifically, we first formulate the definition generation task and introduce the denotations (section 3.1). then we provide a preliminary description of the definition generation processing based on t5 (section 3.2). finally, we explain how to apply the contrastive loss in the training process to solve the under-specific problem and improve the generation quality (section 3.3). figure 1 depicts the overview pipeline of our method. ## experiments in this section, we compare our method with several state-of-the-art methods and conduct a series of experiments to verify the effectiveness of our method in addressing the under-specific problem in definition generation.