Distant Supervision for Relation Extraction with Multi-sense Word Embedding (2018)

Abstract: Distant supervision can automatically generate labeled data between a large-scale corpus and a knowledge base without utilizing human efforts. Therefore, many studies have used the distant supervision approach in relation extraction tasks. However, existing studies have a disadvantage in that they do not reflect the homograph in the word embedding used as an input of the relation extraction model. Thus, it can be seen that the relation extraction model learns without grasping the meaning of the word accurately. In this paper, we propose a relation extraction model with multi-sense word embedding. We learn multi-sense word embedding using a word sense disambiguation module. In addition, we use convolutional neural network and piecewise max pooling convolutional neural network relation extraction models that efficiently grasp key features in sentences. To evaluate the performance of the proposed model, two additional methods of word embedding were learned and compared. Accordingly, our method showed the highest performance among them.

## introduction relation extraction refers to the task of extracting the relation between two entities in a sentence. for example, a relation extraction system extracts 'founder(facebook, mark zuckerberg)' from the sentence "mark zuckerberg is the founder of facebook". in recent years, the importance of knowledge bases has emerged, and studies for constructing large-scale knowledge bases such as dbpedia, yago, and wikidata are actively underway. furthermore, the research on extracting knowledge from web-scale corpus is also underway. however, since many studies use machine learning to design a relation extraction system, there is a high-cost problem in generating a large amount of supervised training data. to solve this problem, the distant supervision assumption is introduced in this paper @xcite . the dis-tant supervision assumption means, "if two entities are linked with a certain relation in the knowledge base and there is a collected sentence that contains both entities from the corpus, then the collected sentences may describe the certain relation between the two entities." figure 1 is an example of automatically collected labeled data using the distant supervision assumption. the distant supervision method is relatively efficient in that it automatically generates training/labeled data between a large corpus and a large knowledge base, but the veracity of the labeled data is sometimes ambiguous. as shown in figure 1 , among the collected sentences that contain both 'facebook' and 'mark zuckerberg', the first sentence means that mark zuckerberg is a founder of facebook, but the second sentence does not. various studies @xcite @xcite have been introduced to solve this problem. however, they use traditional natural language processing (nlp) features such as part of speech (pos) tagging and dependency tree, so the errors occurring in nlp tools propagate to the relation extraction system. therefore, these papers @xcite proposed a relation extraction system that used word embedding and deep neural network (dnn) approaches without the above nlp features, and showed improved performance than previous studies. especially, the piecewise max pooling convolution neural network (pcnn) model introduced in @xcite transforms the convolution neural network (cnn) model into a form more suitable for relation extraction task. however, these studies have a disadvantage in not reflecting the sense of words in word embedding. for example, the word 'bow' could be divided into various meanings such as 'baugreeting' and 'bou -archer's weapon'. therefore, if a relation extraction model is learned with lexical ambiguity, it may result in not properly reflecting the characteristics of the homograph. thus, it is necessary to apply multi-sense word embedding to the relation extraction model. however, to the best of our knowledge, there are no studies applying multi-sense word embedding to relation extraction models. in this paper, we introduce a distant supervision relation extraction model with multi-sense word embedding. we use two relation extraction models, cnn proposed in @xcite and pcnn proposed in @xcite . to learn the multisense word embedding, we use the results of the word sense disambiguation (wsd) module and skip-gram algorithm. to demonstrate the superiority of our method, we compared the relation extraction performances of two other word embedding models. the first is the most common wordtoken-based word embedding, and the second is the morpheme-based word embedding. in chapter 4, we present the experimental results of learning and evaluation of these models based on korean wikipedia and k-box, which extended knowledge base on korean dbpedia. ## methodology in this paper, we propose a relation extraction model using multi-sense word embedding. we use cnn and pcnn for the relation extraction model, and generate multi-sense word embedding using the wsd module. the structure of our relation extraction system is as shown in figure 4 , and it consists largely of the word embedding and distant supervision relation extraction model. first, we take the corpus as input and perform wsd module. next, entitypadding tokenization is performed as described in section 3.1. next, the multi-sense word embedding is learned by the skip-gram algorithm, so that the tokens with the sense number have their own embedding vectors. in this way, the same form of lexical token has different embedding vectors based on the sense number. distant supervision is performed between knowledge base and corpus, and the collected labeled data are word sense disambiguated and tokenized in the same manner. then this data is divided into two groups-one for learning and the other for evaluation.