Glyph Enhanced C hinese Character Pre-Training for Lexical Sememe Prediction (2021)

Abstract: Sememes are defined as the atomic units to describe the semantic meaning of concepts. Due to the difficulty of manually annotating sememes and the inconsistency of annotations between experts, the lexical sememe prediction task has been proposed. However, previous methods heavily rely on word or character embeddings, and ignore the fine-grained information. In this paper, we propose a novel pre-training method which is designed to better incorporate the internal information of Chinese character. The Glyph enhanced Chinese Character representation (GCC) is used to assist sememe prediction. We experiment and evaluate our model on HowNet, which is a famous sememe knowledge base. The experimental results show that our method outperforms existing non-external information models.

## introduction in linguistics, sememes are defined as the minimum semantic units for human language @xcite , which describe the semantic meaning of concepts. hownet @xcite is one of the most well-known sememe knowledge bases (kb), which has been widely used in many nlp tasks @xcite , such as semantic similarity computation @xcite , sentiment analysis @xcite , language modeling @xcite , word representation learning @xcite and short text matching @xcite . in order to free human experts from the laborious sememe annotating job, @xcite propose the task of sememe prediction, which intends to automatically select related sememes from a closed sememe set for each word. they propose two frameworks based on word embedding and matrix factorization. but these methods usually fail to * the corresponding authors are lu chen and kai yu. deal with the prediction problem of low-frequency words. motivated by this, @xcite present character-enhanced sememe prediction (csp), taking advantage of both internal character information and external context information of words. however, csp is an ensemble model which still relies on word and character representation, and ignores the fine-grained information. for internal structural information of words, many researchers believe that only using characters is not sufficient for capturing the semantic information @xcite @xcite . for instance, the words "森林(forest)" and "木头(wood)" are semantically related. but these two words share no information since they consist of different characters. to address this problem, we split each chinese character into several components, and regard component as the minimum unit to express the meaning of the character. we believe that fine-grained units can share more information between semantically related words, which helps model prediction. take figure 1 for example, the characters of word "濒海(near the sea)" have components "步(step)" and "氵(water)", which are related to its sememes, namely "靠近(benear)" and "水域(waters)", respectively. in order to better incorporate the internal information of chinese character, we pre-train a glyph ## methodology in this section, we first introduce the architecture of pre-training model. then, we describe how to incorporate pre-trained representation into sememe prediction task. ## conclusion in this work, we pre-train a glyph enhanced chinese character embedding (gcc) for sememe prediction. the model is evaluated on hownet sememe kb and outperforms existing non-external information models. our experiments show that glyph information can enhance the semantic expression of words, and has a better performance on low-frequency words.