End to End C hinese Lexical Fusion Recognition with Sememe Knowledge (2020)

Abstract: In this paper, we present Chinese lexical fusion recognition, a new task which could be regarded as one kind of coreference recognition. First, we introduce the task in detail, showing the relationship with coreference recognition and differences from the existing tasks. Second, we propose an end-to-end model for the task, handling mentions as well as coreference relationship jointly. The model exploits the state-of-the-art contextualized BERT representations as an encoder, and is further enhanced with the sememe knowledge from HowNet by graph attention networks. We manually annotate a benchmark dataset for the task and then conduct experiments on it. Results demonstrate that our final model is effective and competitive for the task. Detailed analysis is offered for comprehensively understanding the new task and our proposed model.

## introduction coreference is one important topic in linguistics @xcite , and coreference recognition has been extensively researched in the natural language processing (nlp) community @xcite @xcite . there are different kinds of coreferences, such as pronoun anaphora and abbreviation @xcite @xcite . here we examine the phenomenon of chinese lexical fusion, also called separable coreference @xcite , where two closely related words in a paragraph are united by a fusion form of the same meaning, and the fusion word can be seen as the coreference of the two separation words. since the fusion words are always out-of-vocabulary (oov) words in downstream paragraph-level tasks such as reading comprehension, summarization, machine translation and etc., which can hinder the overall understanding and lead to degraded performance, the new task can offer informative knowledge to these tasks. ## chinese lexical fusion lexical fusion refers to the phenomenon that a composition word semantically corresponds with related words nearby in the contexts. the composition word can be seen as a compact form of the separation words in the paragraph, where the composition word is referred to as the fusion form of the separation words. the fusion form is often an oov word, which brings difficulties in understanding of a given paragraph. we can use a tuple to define the phenomenon in a paragraph: w f , w a1 , w a2 , • • • , w an (@xmath0, where w f denotes the fusion word, and w a1 , w a2 , • • • , w an are the separation words. we regard the w f as a coreference of w a1 , w a2 , • • • , w an . more detailedly, the number n equals to the length of word w f , and the ith character of word w f corresponds to the separation word w ai as one fine-grained coreference. the task is highly semantic-oriented, as all the fine-grained coreferences are semantically related. notice that chinese lexical fusion has the following attributes, making it different from others: by the above characters, we can easily differentiate the chinese lexical fusion with several other closelyrelated linguistic phenomena, as an example, the abbreviation, the illustrated negative examples in item (1) and ( 3 ) are both abbreviations. for simplicity, we limit our focus on @xmath1, because this situation occupies over 99% of all the lexical fusion cases according to our preliminary statistics. thus we can call tuples as triples all though this paper. ## method we built an end-to-end model for the lexical fusion recognition task. the recognition of lexical fusion consists of two steps. first mention detection is performed to collect all mention words, including the fusion words and their separations. by applying the bio tagging scheme associating with mention types, this subtask can be converted into a typical sequence labeling task @xcite . second, we conduct pair-wise character-word clustering, obtaining one-one fine-grained coreferences and reformatting them to triples of w f , w a1 , w a2 . the end-to-end model alleviates the problem of error propagation in pipeline methods through a joint way. the model consists of an encoder, a crf decoder for mention recognition and a biaffine decoder for coreference recognition. further, we enhance the encoder with sememe knowledge from hownet. ## related work coreference recognition has been investigated extensively in nlp for decades @xcite @xcite . lexical fusion can be regarded as one kind of coreference, however, it has received little attention in the nlp community. our proposed models are inspired by the work of neural coreference resolution @xcite @xcite @xcite . we adapt these models by considering task-specific features of chinese lexical fusion, for example, enhancing the encoder with a gat module for structural sememe information. another most closely-related topic is abbreviation @xcite . there have been several studies on abbreviation prediction, recovery and dictionary construction @xcite @xcite . lexical fusion is different from abbreviation in many points. for example, abbreviation always refers to one inseparable mention, which is not necessary for lexical fusion. besides, lexical fusion should abide by certain word construction rules, while abbreviation is for free. bert and its variations have achieved the leading performances for glue benchmark datasets @xcite @xcite . for the close tasks such as coreference resolution and relation extraction, bert representations have also shown competitive results @xcite , which inspires our work by using it as basic inputs aiming for competitive performance. the sense and sememe information has been demonstrated effective for several semantic-oriented nlp tasks @xcite @xcite @xcite . hownet offers a large knowledge base of sememe-based @xcite , which has been adopted for sememe extraction. we encode the sememes by the form of a graph naturally to enhance our task encoder. ## conclusion in this work, we introduced the task of lexical fusion recognition in chinese and then presented an end-to-end model for the new task. bert representation was exploited as the basic input for the models, and the model is further enhanced with the sememe knowledge from hownet by graph attention networks. we manually annotated a benchmark dataset for the task, which was used to evaluate the models. experimental results on the annotated dataset indicate the competitive performance of our final model, and the effectiveness of the joint modeling with the sememe-enhanced encoder. the analysis is further offered to understand the new task and the proposed model in-depth.