Knowledge Extraction From Texts Based on W ikidata (2022)

Abstract: This paper presents an effort within our company of developing knowledge extraction pipeline for English, which can be further used for constructing an entreprise-specific knowledge base. We present a system consisting of entity detection and linking, coreference resolution, and relation extraction based on the Wikidata schema. We highlight existing challenges of knowledge extraction by evaluating the deployed pipeline on real-world data. We also make available a database, which can serve as a new resource for sentential relation extraction, and we underline the importance of having balanced data for training classification models.

## introduction knowledge extraction aims at discovering semantic information from texts using a knowledge representation schema. this discovered information is used to build a knowledge base (kb), which is a useful resource for structured information. kbs can play an important role in many tasks and systems: domain question-answering systems, recommender systems, natural language generation systems, search result enhancement, and many others. entreprise knowledge bases have recently gained a lot of attention @xcite @xcite . they allow to transform heterogeneous data, both public and private, into knowledge representations, which are effectively used for specific applications. in this paper, we report on a preliminary step for building a company-specific kb, namely how to extract knowledge from texts in the form of (subject, relation, object) triples . we develop a system consisting of several components: entity detection and linking, coreference resolution, and relation extraction (re). for the first two components we use off-the-shelf tools, whereas for re we develop our own module. our re module is based on wikidata @xcite , the existing kb, which contains many pre-defined relations. with the goal to cover as many relations from wikidata as possible, we create a database, which merges several datasets for re and distributes them in a standardised format. we make use of this database to create different training scenarios for the re task and show how balancing existing re datasets impacts the task performance. we finally evaluate our knowledge extraction system on our company's internal data by human evaluation. our system is deployed and is intended to be used on real-world data within the company. since we apply state-of-the-art nlp techniques, it is equally interesting to see the limitations of current approaches witnessed by our evaluation. to summarise, the main contributions of this paper are the following: the literature on information/knowledge extraction is incredibly vast @xcite . there exist many approaches for information extraction from raw texts. here we describe text the abbreviation gdpr stands for "general data protection regulation". the gdpr governs the processing of personal data within the territory of the european union. ... triples (gdpr q1172506 , instance of p 31 , abbreviation q102786 ) (general data protection regulation q1172506 , short name p 1813 , gdpr q1172506 ) (gdpr q1172506 , main subject p 921 , personal data q3702971 ) (gdpr q1172506 , applies to jurisdiction p 1001 , european union q458 ) another important difference in re approaches is the use of an open or closed relation set. a closed, pre-defined set of relations is targeted in relation classification systems, where either a custom pool of relations is used @xcite or the set is defined by an underlying kb, such as wikidata, freebase, dbpedia, etc. the paradigm opposite to closed re is open information extraction @xcite . openie aims at extracting domain-independent relations from large corpora without using a predefined schema. openie systems may extract redundant information due to lexical variations in texts, so while using this paradigm for knowledge extraction, a process called canonicalisation is used to reconciliate their output with a given kb @xcite . we adopt the knowledge extraction approach with binary re on a closed set of relations from wikidata. we hope that this choice will allow us to facilitate entreprise-specific kb construction in the future. ## data our company has an internal wiki in english where different terms are explained. those terms can belong to some general knowledge (e.g., climate change, agile software development) or can be specific to the company. the wiki terms span over several areas: from human resources, marketing, legal affairs to computer science and information technology. those wiki documents, while having valuable information for the company, represent unstructured text with no linguistic annotation; they sometimes exhibit some information overlap or they can have related term descriptions span over several pages not linked between each other. an example of the beginning of such document describing a general term is displayed in our motivation to explore internal documents is as follows: we would like to represent the information in a structured way, that would allow reasoning and better understanding of the company knowledge. moreover, a potential kb may serve in different downstream applications developed in the company: question answering, task-oriented dialogue, knowledge management. we chose wikidata as our initial kb schema because of its steady growth within last years and increased community participation. the wikidata schema may be eventually refined to better suit our needs in the future. for instance, we might add new relations, not yet defined in wikidata. ## approach overview we aim to extract rdf (resource description framework) triples in the form (subject, relation, object) from text. we develop a classical pipeline for triple extraction: sentence splitting, entity detection and linking, coreference resolution, and relation classification. while for the steps 1-3, we used off-the-shelf libraries, for relation extraction we developed an in-house solution, which is described in section 5. ## relation extraction re is a notoriously difficult task because relations, as compared to entities, are not often expressed explicitly, i.e. it is hard to find a precise verbal expression. moreover, relations can be expressed in many different ways in a text. re usually works well when it covers a limited set of well-defined relations. we, on the other hand, have no explicit relations, defined on our data. our goal is to explore texts to possibly find some relations coming from the external closed set (wikidata). relations in wikidata are numerous. there are around 9,500 relations as of january 2022 foot_2 . however, most of them are not exploitable in ordinary texts, since a lot of them are about some id numbers in different catalogues and libraries, e.g. imdb id (p345), swiss parliament id (p1307), etc. we estimate that about 1,500-2,000 relations can be usable in everyday texts. to explore relations in wikidata, we use available datasets for re that are based on wikidata relations. it means that we could not use other popular datasets for re such as nyt @xcite or tacred @xcite , since they use other knowledge bases. ## conclusion in this paper we presented the pipeline for knowledge extraction from text based on wikidata. we showed its utility on real-world data coming from our company's internal wiki. while entity detection and linking tools perform well on unseen data, re still presents significant challenges. we developed the database for sentence-based re with a large coverage of wikidata relations, which we hope will be useful for the community. we also showed that balancing training data is crucial for good performance in re. in future work, we plan to improve the current pipeline, especially the re component. we envisage several possible ways that can be explored: integrating syntactic parsing for better entity detection, using insights from semantic parsing to better represent sentence structure, and annotating some part of our data and using it for fine-tuning.