Extracting Relations Among Entities Using NLP | by Arun Jagota | Towards Data Science

In this post, we introduce the problem of extracting relations among named entities using NLP. We illustrate this problem with examples of progressively increasing sophistication, and muse, along the way, on ideas towards solving them.
Let’s get started.
Consider these sentences.
John Doe works at Google.Apple is located in Cupertino.
We’d like to detect the various entities in these sentences. Them being
Person name = John DoeCompany = GoogleCompany = AppleCity = Cupertino
Fortunately, the NLP of this task (named entity recognition) has matured enough so that this can be done with high quality. See [1].
A logical next step is to be able to deduce the relationships among these entities using NLP. In our case, we hope to get
works_at(person:john doe,company:google)located_in(company:apple, city:cupertino)
Clearly automating the process of detecting not only the entities but the relations among them has considerable value. Using this approach we can extract structured data from unstructured documents (of which there are plenty). Powerful queries can then be performed on the structured data. For example to power question-answering systems.
There are numerous particular applications of this meta use-case. Such as mining documents in specific verticals to uncover entities and relationships among them in that vertical. Similarly, mining documents in a company to uncover what's happening under the hood in its engineering or business processes.
First Observations
Let’s look closely at our sentences. The relations of interest are embedded within works at and is located in. Seems like we should be able to detect these using NLP somehow. Let’s give this problem a name: relational phrase extraction.
We did a part-of-speech analysis of this sentence, at https://parts-of-speech.info/
As we see, the POS tags carry strong signals towards identifying the relational phrases. Also keep in mind that named entity recognition would have already tagged the named entities in these sentences which would additionally strengthen our confidence that, in these examples, the relational phrases correspond to the sequence of POS-tags <verb> <preposition>.
To bring this out more vividly, let’s depict the tokenized sentences along with their tagged sequences as below.
Tokens:   John   Doe     works at          GooglePOS tags: noun   noun    verb  preposition nounNER tags: person person  —     —           companyTokens:   Apple  is    located in          CupertinoPOS tags: noun   verb  verb    preposition nounNER tags: company —    —       —           city
Clearly, in these examples, combining the information in the POS tags with those in the NER tags works well to identify the relational phrases.
A more elaborate example
Next, consider this sentence, taken from [2].
Read The Adventures of Sherlock Holmes by Arthur Conan Doyle online or in your email
The named entities are The Adventures of Sherlock Holmes, a book title, and Arthur Conan Doyle, a person’s name. The former is harder to recognize than the latter. This is because book titles are longer and have a more diverse universe. Interestingly, this book title has a person’s name in it.
For now, let’s bury the complexity of recognizing book titles under the hood. If you are curious, read up on some of the more elaborate approaches such as HMMs in the context of NER in [1].
Assuming the two entities are recognized correctly, we notice that the relational phrase still sits between them. That is, this sentence has the structure
<words> book-title by person-name <words>
Below is the POS analysis of the relevant portion of this sentence.
That the word by sandwiched between the entities is a preposition is assuring. It supports forming the relational phrase from this by.
Let’s use this opportunity to ‘dig’ into by a bit more. It's easy to imagine that in a huge corpus we may come across numerous instances of
book-title by person-name
From these instances, we will be able to also infer that the book title always precedes the by and that the person's name always follows it. Thus in principle, the word by can be used as an additional predictor of the entities themselves, in our case the book’s title and the person’s name. Consider replacing by with written by. The predictive strength of the latter towards the book title and person name entities is now higher.
Next, let’s look at
Wimbledon is a tennis tournament held in the UK in the first two weeks of July every year.
This one could be ‘parsed’ via entity recognition (somewhat broadly interpreted) as
Sports event name is a sport tournament held in the location in the time.
The first three entities are reasonably crisp and can be detected via NER approaches suitably trained. The fourth one, time, is less crisp. In our example, its value is first two weeks of July every year. It might be difficult to train a NER for this less crisp entity. Instead, we might consider recognizing more granular time-based entities. Such as month. Armed with such a detector we have a partial inference of in the time as in month: July.
Let’s now look at location. It makes sense to treat it in a hierarchical fashion. Specifically, to break location down into finer entities such as country, state, city, ... We can then recognize each of these entities and use a taxonomy to relate them. Such as country is-a location, state is-a location, state is-in country, etc.(Note that a particular state may be in multiple is-in associations with countries.)
Okay, now that we have discussed the entities, what relationships can we infer among them? Definitely the is-a relationship below.
Sports event name is-a sport tournament
The other ones need more thought. First, for convenience, let’s abbreviate the sentence with entity variables in it as below.
SEN is a ST held in the L in the T.
In our example, we see that ST held-in L would be a wrong inference. It is SEN=Wimbledon that is held in L=UK, not ST=tennis tournament?
An idea worth exploring is to start by forming three hypotheses, expressed in the form of association rules [3]
ST=tennis tournament, held-in ⇒ L=UKSEN=Wimbledon, held-in ⇒ L=UKSEN=Wimbledon, ST=tennis tournament, held-in ⇒ L=UK
In the above, think of X ⇒ Y as
IF X THEN Y
Given a huge corpus, such as Wikipedia, broken down into individual sentences, we can then compute the so-called supports and confidences of each of these association rules. The support of a rule X ⇒ Y is the number of sentences in which X holds. The confidence of this rule is P(Y|X).
From such an analysis we can reasonably expect to infer that ST=tennis tournament does not imply that it is held in the UK whereas SEN=Wimbledon always implies that it is held in the UK.
Next, consider the in relation involving T (time). We can treat it similarly. Our set of candidate association rules is a bit larger. In X ⇒ Y, Y is always T = first two weeks of July every year. X is any nonempty subset of {SEN=Wimbledon, ST=tennis tournament, L=UK}
Framing this problem as one involving mining for high-confidence association rules lets us use highly scalable algorithms that have been developed for this problem. Such as FP growth or the A’priori algorithm. See [3]. We may indeed confront scale if we are going to do this analysis on all sentences in Wikipedia, many having more than two entities in them (as does ours).
Finally, let’s turn our attention to the following sentence.
In 2019, the men’s singles winner was Novak Djokovic who defeated Roger Federer in the longest singles final in Wimbledon history.
This sentence is a good one to illustrate going beyond relating the entities that occur in the sentence from its contents alone. Rather, we will infer (or more accurately, collect evidence for) relationships that may be inferred from combining the information in this sentence with a suitable global knowledge model.
We won’t actually specify the knowledge model’s form. Rather we will muse on what we might be able to infer from the information in this sentence were we to also have access to such a model.
From
Novak Djokovic who defeated Roger Federer
we can extract
defeated(Person Name: Novak Dkokovic, Person Name: Roger Federer)
in the manner discussed earlier.
Now consider the word Wimbledon that appears in the same sentence later on. Assume its been correctly recognized as Sports Event Name. We may now be able to link these person names to this event. If so, and assuming our knowledge graph knows that Wimbledon is a tennis event, we can also infer that Novak Dkokovic and Roger Federer are tennis players.
We are listing the inferences in the last sentence of the previous paragraph mainly to illustrate what sorts of things we can deduce. For these particular assertions — that these people are tennis players — there may be stronger, more direct, evidence in other sentences in Wikipedia.
Finally, let’s note that we can even infer the genders of these persons from their first names alone. The mention of men’s earlier in the sentence may then be seen as corroborating evidence.