Relation extraction from a corpus using an information retrieval based procedure

An input entity pair including a first entity and a second entity is received. The first entity and the second entity are associated by a particular relation. A first set of statements containing the first entity and a second set of statements containing the second entity is are received from a corpus. A first set of discriminative words in the first set of statements and a second set of discriminative words in the second set of statements are identified. Perceptions in the first set of discriminative words and the second set of discriminative words are identified. A predetermined number of ranked statements is retrieved from the corpus using the identified perceptions as a query. Candidate entity pairs are extracted from the predetermined number statements. The candidate entity pairs have a relation therebetween analogous to the relation between the first entity and the second entity.

TECHNICAL FIELD

The present invention relates generally to a method, system, and computer program product for relation extraction from a corpus. More particularly, the present invention relates to a method, system, and computer program product for information retrieval based relation extraction from a corpus.

BACKGROUND

Information retrieval in the context of retrieving information from a corpus of text documents is the process of searching the content of text documents to obtain information contained within or conveyed by the text documents. A corpus often includes a large and unstructured collections or sets of text documents such as web pages, news documents, broadcast transcripts, electronic books and other sources of textual information that are stored within one or more document repositories. Entity relation detection is a form of information extraction in which semantic relations between entities are determined from the text of the corpus often using machine learning techniques such as natural language processing (NLP). Examples of entities that may be contained within the corpus include, for example, persons, organizations, companies, locations, objects, and countries. Examples of relations that may exist within entities includes, for example, a person-affiliation and organization-location.

A number of techniques exist to perform relation extraction from a corpus of text documents including supervised relation extraction, open information extraction, universal schema and distant supervision. Supervised relation extraction often requires manual human labeling of entity relationships within existing training data. A significant disadvantage of supervised relation extraction is that it requires a large amount of labelled relations within the training data which is expensive to obtain and often does not generalize for obtaining different relations than those contained within the training data. Open information extraction identifies sequences of words in sentences that denote relations between two entities. However, open information extraction is computationally intensive and does not scale well to larger document sets.

Universal schema relation extraction combines information from an existing knowledge base and open information extraction techniques to perform relation extraction upon a collection of documents using matrix factorization methodologies. Distant supervision for relation extraction uses an existing semantic knowledge base consisting of entities and relations between them to find sentences containing those entities in a large unlabeled corpus and extract linguistic and syntactic features to train a classifier.

The illustrative embodiments recognize that existing procedures for extracting relations from a corpus require a large amount of training data and also include deep learning based implementations that are computationally intensive during the training phase.

SUMMARY

The illustrative embodiments provide a method, system, and computer program product. An embodiment of a method for relation extraction from a corpus includes receiving an input entity pair including a first entity and a second entity, the first entity and the second entity being associated by a particular relation. The embodiment further includes retrieving a first set of statements containing the first entity from a corpus, and retrieving a second set of statements containing the second entity from the corpus. The embodiment further includes identifying, by a processor and a memory, a first set of discriminative words in the first set of statements, and identifying, by the processor and the memory, a second set of discriminative words in the second set of statements. The embodiment further includes identifying perceptions in the first set of discriminative words and the second set of discriminative words, and retrieving a predetermined number of ranked statements from the corpus using the identified perceptions as a query. The embodiment further includes extracting candidate entity pairs from the predetermined number statements. The candidate entity pairs have a relation therebetween analogous to the relation between the first entity and the second entity. Thus, the embodiment provides a solution to existing problems in relation extraction by providing for relation extraction from a corpus using an information retrieval based procedure in which a minimal number of examples of entity pairs are required to be provided to obtain acceptable relation extraction results.

Another embodiment further includes ranking the extracted candidate entity pairs based on a frequency of the entity pairs in the ranked statements to form ranked output entity pairs. Another embodiment further includes outputting a list of the ranked output entity pairs. An embodiment further includes outputting supporting evidence for at least one of ranked output entity pairs, the supporting evidence including excerpts from the corpus indicative of the relation between the entities of the output entity pair.

In another embodiment, the input entity pair is received from a user. In another embodiment, the corpus includes a set of textual documents. In another embodiment, identifying the perceptions in the first set of discriminative words and the second set of discriminative words further includes ranking the perceptions, and selecting a predetermined number of the ranked perceptions.

In another embodiment, identifying the first set of discriminative words and identifying the second set of discriminative words is based upon a term frequency-inverse document frequency (TF-IDF) procedure. In another embodiment, the perceptions include common words in the first set of discriminative words and the second set of discriminative words.

DETAILED DESCRIPTION

The illustrative embodiments recognize that the presently available tools or solutions do not address these needs or provide adequate solutions for these needs. The illustrative embodiments used to describe the invention generally address and solve the above-described problems and other problems related to relation extraction from a corpus using an information retrieval based procedure.

In an embodiment, a pair of words/entities is provide to a relation extraction application and the relation extraction application identifies analogous entity pairs within a corpus of text documents having the same or a similar relation to one another as the input pair entities have to one another. In the embodiment, the relation extraction application extracts the analogous pairs from the corpus using an information retrieval based algorithm as further described herein. In one or more embodiments, the information retrieval based algorithm uses statistics from the corpus to perform relation extraction from the corpus. In an embodiment, the relation extraction application ranks the extracted entity pairs based upon a frequency of occurrence of the entity pairs and outputs a ranked list of the output pairs from the corpus. In a particular embodiment, the relation extraction application further outputs supporting evidence for each extracted output pair including excerpts from the corpus indicative of the relation between the entities of the output pair.

In an algorithm according to an embodiment, the relation extraction application receives an input pair (a, b) from a user in which a first entity (a) and a second entity (b) are associated by a particular relation. In a particular example, entity (a) is a “Person A” and entity (b) is an “Organization B” is which a relation between entity (a) and entity (b) is that “Person A” is the chief executive officer (CEO) of “Organization B”. In the embodiment, the relation extraction application retrieves sentences (A) from the corpus (or a subset of the corpus) containing entity (a) and sentences (B) from the corpus (or a subset of the corpus) containing entity (b). In the embodiment, the relation extraction application identifies discriminative words in sentences (A) and sentences (B) in which a discriminative word is relevant to determining a relation between entities within the corpus.

In a particular embodiment, the relation extraction application uses a term frequency-inverse document frequency (TF-IDF) procedure for determining discriminative words within a corpus. TF-IDF is a numerical statistic that is intended to reflect how important a word is to a document in a corpus. The TF-IDF value increases proportionally to the number of times a word appears in a document of the corpus offset by the frequency of the word in the corpus to compensate for words that appear more frequently in general in the corpus.

In the embodiment, the relation extraction application identifies common words, referred to herein as perceptions (P), in the two sets of discriminative words. In a particular embodiment, relation extraction application identifies perceptions (P) using an argument vector space model (VSM). A VSM is an algebraic model for representing text documents as vectors including a number of terms in which each term represents a single word or phrase from the corpus.

In the embodiment, the relation extraction application retrieves a predetermined number k of top ranked sentences from the corpus using perceptions (P) as a query (D). In the embodiment, the relation extraction application extracts the entity pairs in D and ranks the extracted entity pairs based on the frequency of the entity pairs in the ranked sentences. In an embodiment, the relation extraction application further extracts supporting evidence supporting evidence for each extracted output pair including excerpts from the corpus indicative of the relation between the entities of the output pair.

In one or more embodiments, the extracted output pair are used for slot filling in a fast domain adaptation (FDA) engine for a cognitive application utilizing machine learning residing on a server. Slot filling is a procedure used to complete all known information about a queried entity. Fast domain adaptation is used to train a model for extracting of entity relation information from a corpus.

Various embodiments described herein may provide one or more advantages over existing procedure for relation extraction. One advantage that may be provided by one or embodiments includes that the relation extraction procedure described herein does not require offline training at the time of ingestion. Another advantage that may be provided by one or more embodiments includes that the relation extraction procedure described herein can be implemented on top of existing information retrieval technology to provide for scalability and robustness. Another advantage that may be provided by one or more embodiments includes that the relation extraction procedure described herein can be integrated into a search engine. Still another advantage that may be provided by one or more embodiments includes that a minimal number of examples of entity pairs are required to be provided to obtain acceptable relation extraction results.

The illustrative embodiments are described with respect to certain types of data retrieval and relation extraction processes, network devices, transmissions, validations, responses, devices, data processing systems, environments, components, and applications only as examples. Any specific manifestations of these and other similar artifacts are not intended to be limiting to the invention. Any suitable manifestation of these and other similar artifacts can be selected within the scope of the illustrative embodiments.

Any advantages listed herein are only examples and are not intended to be limiting to the illustrative embodiments. Additional or different advantages may be realized by specific illustrative embodiments.

Furthermore, a particular illustrative embodiment may have some, all, or none of the advantages listed above.

Relation extraction application105of server104implements an embodiment of an algorithm for relation extraction from a corpus as described herein. Cognitive application107implements one or more cognitive learning processes on textual data such as natural language processing (NLP) as described herein with respect to various embodiments.

Data processing system200is also representative of a data processing system or a configuration therein in which computer usable program code or instructions implementing the processes of the illustrative embodiments may be located. Data processing system200is described as a computer only as an example, without being limited thereto. Implementations in the form of other devices may modify data processing system200, such as by adding a touch interface, and even eliminate certain depicted components from data processing system200without departing from the general description of the operations and functions of data processing system200described herein.

With respect toFIG. 3, this figure depicts a block diagram300of slot filling in a fast domain adaptation (FDA) engine for a cognitive application in accordance with an embodiment. In the illustrated embodiment, a user selects two input entities in data as an entity pair that share a relationship for which the user wants to train a cognitive application such as cognitive application107. In a particular example, each of the selected entities includes a person, place, object or company. A fast domain adaptation engine302receives the selection of entities and outputs a list of ranked analogous pairs of entities from a corpus304that have an analogous relationship to one another to that of the input entity pair. In the particular example ofFIG. 3, the user desires to explore a “competitor of” relationship between entities such as a “Playstation 4” being a competitor of an “Xbox One S”. In the example, a “500 GB hard drive” “competitor of” relation to a “WiFi antenna” is not relevant to exploring a “competitor of” relation between the “Playstation 4” and the “Xbox One S”, and a “Playstation Vita” “competitor of” relation to a “Nintendo 3DS” may not be relevant to exploring a “competitor of” relation between the “Playstation 4” and the “Xbox One S”. In the example, FDA engine302outputs ranked pairs from corpus304indicating that the “Playstation 4” is a competitor of “Xbox One”, “Nintendo Wii”, “Nintendo Wii U”, an “Xbox”, an “Xbox 360”, a “Nintendo Switch”, and a “Nyko Playpad”.

With respect toFIG. 4, this figures depicts a block diagram of an example architecture400for relation extraction from a corpus using an information retrieval based procedure in accordance with an illustrative embodiment. The architecture400includes a set of documents402from a corpus that are ingested into an input component404of a cognitive application, such as cognitive application107. In the embodiment, input component404performs processing on textual data within documents402to perform functions such as splitting sentences, identifying entities, extracting features, and building models. In the embodiment, the processed textual data is further processed and stored as one or more of an inverted index406A, a pairs vector space model406B, a type vector space model406C, relational embeddings406D, and a word embeddings406E.

In the embodiment, a user408inputs one or more seed entity pairs to a candidate generator component410, and candidate generator component410extracts candidate entity pairs based upon the seed entity pair and data from inverted index406A and pairs vector space model406B. In a particular embodiment, candidate generator component410utilizes an information retrieval-based procedure as described herein to generate the candidate pairs. Candidate generator component410provides the candidate pairs to a scorer component412. In the embodiment, scorer component412utilizes type similarity, word embedding similarity, and/or relation embedding similarity procedure upon data retrieved from type vector space model406C, relational embeddings406D, and word embeddings406E to determine pair features from the candidate pairs. In the embodiment, scorer component412provides the pair features to a ranker component414. Ranker component414ranks the pair feature to generate a list of ranked pairs. In a particular embodiment, ranker component414ranks the pair features using one or more of unsupervised ranking and learning to rank procedure to rank the pair features to generate the ranked pairs. Ranker component414outputs the ranked pairs as result pairs with evidence416.

With reference toFIG. 5, this figure depicts a flowchart of an example process500for relation extraction from a corpus using an information retrieval based procedure in accordance with an illustrative embodiment. In one or more embodiments, process500can be implemented in relation extraction application105.

In block502, relation extraction application105receives an input entity pair from a user including a first entity (a) and a second entity (b). In a particular embodiment, the first entity (a) and second entity (b) are associated by a particular relation. In a particular example, entity (a) is a “Person A” and entity (b) is an “Organization B” is which a relation between entity (a) and entity (b) is that “Person A” is the CEO of “Organization B”. In block504, relation extraction application105retrieves a first set of sentences or statements (A) from corpus109containing the first entity (a). In a particular embodiment, corpus109includes a set of textual documents such as web pages, electronic books, or articles. In block506, relation extraction application105retrieves a second set of sentences or statements (B) from corpus109containing the second entity (b).

In block508, relation extraction application105identifies discriminative words in the first set of sentences (A) to form a first set of discriminative words and the second set of sentences (B) to form a second set of discriminative words. In particular embodiments, a discriminative word is relevant to determining a relation between entities within the corpus. In a particular embodiment, relation extraction application105uses a term frequency-inverse document frequency (TF-IDF) procedure for determining discriminative words within the first set of sentences and the second set of sentences.

In block510, relation extraction application105identifies perceptions (P) in the first set of discriminative words and the second set of discriminative words, ranks the perceptions, and selects a predetermined number of the ranked perceptions. In a particular embodiments, the perceptions include common words in the first set of discriminative words and the second set of discriminative words. In a particular embodiment, relation extraction application identifies perceptions (P) using an argument vector space model (VSM).

In block512, relation extraction application105retrieves a predetermined number of top ranked sentences (D) from the corpus using perceptions (P) as a query. In block514, relation extraction application105extracts candidate entity pairs in the ranked sentences (D) and ranks the extracted candidate entity pairs based on a frequency of the entity pairs in the ranked sentences (D).

In block518, relation extraction application105outputs a list of the ranked output entity pairs. In block520, relation extraction application105outputs supporting evidence supporting evidence for each extracted output entity pair including excerpts from the corpus indicative of the relation between the entities of the output pair. Process500is ended thereafter.