Augmenting C hinese W ord N et semantic relations with contextualized embeddings (2019)

Abstract: Constructing semantic relations in WordNet has been a labour-intensive task, especially in a dynamic and fast-changing language environment. Combined with recent advancements of contextualized embeddings, this paper proposes the concept of morphology-guided sense vectors, which can be used to semi-automatically augment semantic relations in Chinese Wordnet (CWN). This paper (1) built sense vectors with pre-trained contextualized embedding models; (2) demonstrated the sense vectors computed were consistent with the sense distinctions made in CWN; and (3) predicted the potential semantically-related sense pairs with high accuracy by sense vectors model.

## introduction chinese wordnet(cwn) @xcite has been one of the most important lexical resources in chinese. through years of rigorous works from linguists and lexicographers, cwn covers large amount of chinese words, senses distinctions, and various lexical semantic relations. however, the linguistic knowledge cwn tries to incorporate is far more than a static snapshot of the language usage from a given time. as a lexical resource which aims to facilitate better nlp applications, the current version of cwn has intended to incorporate the complicated and dynamic relations that language implicitly encodes. this is a challenging task for resource maintainer, for they have to manually edit the database, in order to keep up the the neologisms and ever-changing novel word usage. recent algorithmic advancements shed lights on how we can augment lexical resources, at least semi-automatically. thanks to the bloom of internet and social media, voluminous textual data are easily available, where emergent concepts and their relations could be discovered from the real-world and most updated data. this process is further facilitated by recent development of deep learning and machine learning models, such as pretrained language model @xcite , word embeddings @xcite , or contextualized embeddings. these computation resources allows us to leverage the ample data, without going through considerable efforts to actually collect, and store the vast amount of data, and setup a model training infrastructure. in this paper, we took advantage the recent development on contextualized embeddings. specifically, we used a pre-trained bidirectional encoder representations from transformer (bert) @xcite , basing on which we semi-automatically predicted new related senses in cwn. the predictions were only possible with the constraints encoded in chinese morphology, where the semantic relationship between the whole word and its composing sub-word were suggested @xcite . we introduced how we applied bert to construct sense vectors from existing example sentences in each cwn senses, and how to use sense vectors and heuristics rule s regarding chinese word morphology to semi-automatically generate new relationships (hyponymy/troponymy pairs) among cwn senses. we evaluated these sense vectors with a simulation study and conducted an experiment on the model-predicted sense relation pairs. the procedures described in this paper was shown in figure 1 . ## building sense vectors word sense is closely related to the context the word resides in, and the contextualize embeddings is meant to encode the context. if we can characterized the context through contextualized embeddings, the context vector was then a formal representation of a word sense. we therefore computed sense vectors from the contextualized embeddings of the target word located in an disambiguated context. in this section, we first identified the lemmas (and their senses) to be included in current analysis and the experiment in following section. secondly, we built sense vectors from example sentences of each sense. thirdly, in order to explore the nature of the sense vectors, we conducted a simulation study over the computed sense vectors. ## experiment with sense vectors as a computable representation of word senses, we aimed to semiautomatically discover potential hypernymyhyponymy sense relations in cwn, guided by chinese morphology. previous study argued that chinese two-character words with inner structure of two nouns and two verbs, were likely a hyponymy of the second character (when used as a one-character word). that is, at lemma level, we could discover semantic relations leveraging chinese word morphology. however, semantic relations in wordnet are relationship among word senses. given there are multiple word senses in each lemma, manually found them would be a daunting task. with help of sense vectors, we could try to find senses among which relations existed. ## conclusion this paper combines recent advancements of contextualized embeddings and existing cwn resources to build sense vectors. we have demonstrated these sense vectors followed the sense distinctions made in cwn, and showed sense vectors, guided by chinese morphol-ogy, provided valuable suggestion discovering hypernymy/troponymy. the semi-automatic procedures greatly facilitate the on-going development of cwn in the fast-paced language environment.