Unsupervised Knowledge Graph Generation Using Semantic Similarity Matching (2022)

Abstract: Knowledge Graphs (KGs) are directed labeled graphs representing entities and the relationships between them. Most prior work focuses on supervised or semi-supervised approaches which require large amounts of annotated data. While unsupervised approaches do not need labeled training data, most existing methods either generate too many redundant relations or require manual mapping of the extracted relations to a known schema. To address these limitations, we propose an unsupervised method for KG generation that requires neither labeled data nor manual mapping to the predefined relation schema. Instead, our method leverages sentence-level semantic similarity for automatically generating relations between pairs of entities. Our proposed method outperforms two baseline systems when evaluated over four datasets.

## introduction a knowledge graph (kg) is a directed labeled graph in which nodes represent entities and edges are labeled by well-defined relationships between entities. formally, given a set e of entities and a set r of relations, a knowledge graph is a set t of triples, where t ⊆ e × r × e. a triple @xmath0, r, e t ), where e @xmath1, @xmath2, e @xmath3, and e h and e t are referred to as the head entity and the tail entity, respectively. as a structured representation of world knowledge, knowledge graphs have been used in a number of applications such as web search @xcite , question answering @xcite and recommender systems @xcite . knowledge graphs can be constructed automatically from text. most of the automatic kg generation methods are supervised or semi-supervised, where a large set of labeled data is required to train a kg generation model (e.g., pcnn @xcite , ollie @xcite , reverb and our method (right) for the input sentence "barack obama was born in hawaii". @xcite ). however, creating labeled data is labor-intensive and the generated graph is limited to the specific domain of the training corpus. in addition, supervised methods can only extract a predefined set of relations occurring in the training data and the model needs to be re-trained to work with other new relation schemas. unsupervised kg models (e.g., stanford ope-nie @xcite ), on the other hand, do not need labeled training corpus. they often use syntactic parsing and a set of rules to extract relationships between two entities in a sentence. although not normally confined to a predefined set of relations, too many unuseful or inaccurate relations can be generated. in figure 1 , the left graph presents an example kg using triples generated with stanford openie @xcite , while the right graph presents the kg generated using our proposed method, both using the same single input sentence. in addition, in case only relations in a predefined set need to be generated, the unsupervised methods do not normally provide a mechanism to map the extracted relation to a known one in the set of relations in a project to build knowledge graphs from news articles where no labeled data are given, we propose an unsupervised knowledge graph generation method using semantic similarity (kgss) that does not need a labeled set of training data nor a complicated set of syntactic rules for kg generation. the method can work with any set of relations that a user prefers, and uses semantic similarity matching to automatically identify the relation between two entities. a salient feature of our method is the use of a pretrained language model @xcite to compute and measure the similarity between the sentence embedding and the embedding of candidate triples formed by the two entities and a candidate relation. the best matching candidate relation is identified as the relation between the two entities. since most supervised models underperform in low-resource settings where no or very limited labeled data are provided, our proposed unsupervised approach can extract useful relations from unlabeled data and can also be used to create a labeled data set for distant supervised learning, which can potentially lead to better results. in this paper, we focus on describing and evaluating the unsupervised method. the contributions of this paper are as follows: ## related work research on kg construction falls under supervised, semi-supervised, or unsupervised categories. for the supervised methods, we name two of them. @xcite propose the recon model to extract relations from a sentence and align them to the kg, using a graph neural network for obtaining the sentence representations. then a neural classifier is adopted to predict the relation of each entity pair in the sentence. another supervised learning method for kg construction is spert (eberts and ulges, 2020), which is a span-based deep learning model with the attention mechanism, targeting to extract entities and relations jointly. semi-supervised approaches such as reverb @xcite , ollie @xcite , and stanford openie @xcite , to name a few, leverage linguistic features (e.g., dependency trees and pos tags) with many human-defined patterns and existing knowledge bases (e.g., wikidata @xcite , dbpedia @xcite ) to extract triples. these systems have a supervision component. for example, stanford openie uses distant supervision to create a noisy corpus of sentences annotated with relation mentions and train a logistic regression classifier to decide which action to perform on an edge on the parse tree when extracting relations. however, these systems miss many potential triples in a sentence since they use verbs as a signal to identify triples, whereas many relational triples may not be connected with a verb. they also tend to generate redundant triples and require manual mapping of the extracted relations to a fixed relation schema. the earliest unsupervised approaches (i.e., heuristics approaches) @xcite @xcite were applied to wikipedia data, building the pioneering knowledge graphs (e.g., yago, dbpedia, freebase). however, these approaches leverage additional (3) users can select the type of entities to be extracted; if nothing is selected, both named entity and noun will be extracted. (4) a submit button. (5) an interactive kg will be generated and visualized where the users can drag the nodes around to modify the presentation of the graph as desired. ## proposed model: kgss given a document, our system generates a knowledge graph from the document. figure 2 illustrates an overview of our system, kgss, which consists of four modules: entity extraction, entity tuple formation and filtering, relation extraction, and kg storage and visualization, and figure 3 illustrates the user interface of our system and visualizes a kg generated given an input paragraph based on a relation schema in tacred* with 6 additional relations: loc:province_of, loc:country_of, loc:city_of, org:is_part_of, per:position_held and per:friend. since our proposed system is unsupervised, it can flexibly work with any user-specified relation schema. ## evaluation datasets we evaluate our kg system by comparing the generated triples to manually annotated triples from three benchmark information extraction datasets and a new dataset on the news domain, all for english language. ## conclusions we presented a novel unsupervised method for knowledge graph generation without the need for labeled data or manual mapping of extracted relations to a predefined relation schema (as in two previous unsupervised methods). a salient feature of the method is that it uses semantic similarity matching to find relations between entities. in addition, our system can work with any set of relations that the user prefers, flexibility that other methods, especially the supervised ones, do not have. we also created a new data set from news articles that will be shared with the community. our evaluation results demonstrate the effectiveness of our system which significantly outperforms two state-of-the-art unsupervised models over four different datasets. we also develop an open source interactive kg generation and visualization tool. as future work, we will evaluate effectiveness of using our method for bootstrapping knowledge graph generation with distant supervision.