DEIM Forum 2016 C3-1

Evaluating Semantic Relatedness through Categorical and Contextual
Information for Entity Disambiguation
Yiming Zhang† and

Mizuho Iwaihara‡

早稲田大学大学院情報生産システム研究科〒808-0135 福岡県北九州市若松区ひびきの２－７
E-mail: †zhang_ym@toki.waseda.jp, ‡iwaihara@waseda.jp
Abstract The number of entities in large-scale knowledge bases has been growing in recent years. The key issue to entity
linking using a knowledge base such as Wikipedia is entity disambiguation. The objective of our proposing system is todisambiguate entities in documents and link entity mentions to their corresponding Wikipedia articles. To this end, our systemranks the set of candidate entities based on relatedness by utilizing semantic features derived from Wikipedia category hierarchies
and articles. In addition, to reflect contextual information of Wikipedia, we utilize word embedding for refining the ranking resultof candidate entities. Our experiment results show that these features has given good correlation with human rankings incandidate relatedness ranking and also high disambiguation accuracy on news articles.
Keyword Entity Disambiguation, Semantic Relatedness, Entity Similarity

University of Washington. So this ambiguity problem will

1. Introduction

hinder the accuracy of entity linking. As another example,
given a sentence: “Texas is a pop rock band from Scotland

1.1. Entity disambiguationand its name was from the 1984 Wim Wenders’ movie Paris,
With the expansion of web, more and more text data can

Texas”. But how can a computer distinguish that Texas is abe approached easily. But looking through enormous text

music band, not a state in the United States and that “Paris,

data might distract people’s attention. It is necessary toTexas” denotes a drama film rather than a city in Northeast

extract useful and refined information from mass text. To

Texas or another band called Paris, Texas?

solve this problem, researchers have been studying related
techniques,

such

as

document

summarization,

text

Knowledge Baseclassification, and text clustering. Meanwhile, entity-

Texas (U.S. state)

centric data has led researchers to a new direction of
constructing

anentitybased

network,andGoogleKnowledge Graph is a well-known example. Changes aretaking place in web text data, from keyword-based text
organization to an entity-centric network of knowledge. An

Texas (band)

Texas are a pop rockband from Scotland andthe name was from the1984 Wim Wenders’movie Paris, Texas.

Scotland (Country)
Wim Wenders
Paris, Texas (film)

eminent feature is to replace the outline of documents

Paris, Texas (city inNortheast Texas)

described by simple keywords with a deeper relationship

Paris, Texas (band)

between entities mentioned in documents.

Figure 1 Illustration of entity disambiguation task

Web pages, twitter, blog postings and news articles
contain mentions of entities, such as people, organization

So the aim of entity disambiguation is to solve thisand geo-political places. To help readers easily understandproblem, establishing mappings between the mentions andthe meaning of main concepts and decrease reading

the actual entities from a knowledge base, like the example

difficulties, those mentions are recommended to link to ain Figure 1, showing the final linking result.

corresponding

descriptive

entitypage.

Sometimes

mentions are ambiguous, a name may refer to different

1.2. Wikipedia and its structure

entities in different contexts. For example, a mention of

Wikipedia is a multilingual free internet encyclopedia,

Washington, can be linked to dozens of possible entities inwhich is collaboratively edited by volunteers worldwide

Wikipedia, such as George Washington, Washington, D.C.,

whose efforts have resulted into over thirty million articles.

There are more than five million articles in English

this matrix is the representation of word w, and eachWikipedia. Because of its abundant coverage and relative

column is the word’s context but suffers from highup-to-date resources, Wikipedia is often used as adimensionality. Recent word embedding has brought new

knowledge base for entity disambiguation. So from thisideas to this area.

sense, one Wikipedia page is regarded as an entity, the page
title is regarded as an entity name, and page content is as
entity description or context. By doing this, we can link amention to a Wikipedia page.

1.4. Our contribution
Our work exploits Wikipedia category hierarchicalfeatures, word embedding similarity and Wikipedia in-link

An especially interesting aspect of Wikipedia structure

relations for measuring semantic relatedness between

is the categorization and linkage within its content [1][8].

words, and these bases for semantic relatedness contribute

Categories in Wikipedia have a hierarchical structure

to achieve the entity disambiguation goal. The rest of the(Figure 2), there is a relative root category called Main

paper is organized as follows: Section 2 gives an overview

topic classification, which can be backtracked by eachof various approaches to entity disambiguation and relatedbottom category. Each category can have an arbitrary

work on word embedding. Section 3 describes our system

number of subcategories, and similarly, one category may

methods and features. Section 4 presents an experiment forhave more than one parent categories, while cycles andevaluating sematic relatedness, comparing the effects of

disconnecteddifferent features, and made an analysis of the results. The

categories

arepossible.

One

page

is

explicitly assigned to one or more categories during users’

paper concludes in Section 5.

editing. Categories act as a semantic tag, and articles thathave similar topics usually belong to the same category.
There also exist numerous links between categories andpages,

from

categories

tosub-categories

and

from

categories to pages.

2. Related work
The usage of Wikipedia has provided tremendous
chances for natural language processing, with various
derived huge knowledge bases, such as DBpeida, Freebase,and BabelNet. Text processing tasks including entity

Main topic
classifications

linking have benefited greatly from these resources. The
problem of named entity disambiguation was addressed in
the literature from different perspectives. Using link based

Culturestructure and using text in articles are two main Wikipediabased approaches.

...

Milne and Witten [2] researched semantic relatedness inApple Inc.
Software

Apple Inc.
hardwarethe early time. They are the first to use hyperlink between
articles for semantic relatedness measurement. They
compared two Wikipedia pages by computing the number

Apple Inc.
operatingsystems

OS X

Apple Inc.
mobilephone

IPadof incoming links the two pages have in common based on
Normalized Google Distance (NGD). But this sort oftransformation of NGD ignores out-links and links thatshould appear but missed or ignored during Wikipedia

Mac OSIOS (Apple)

Iphoneediting. In later work [3], Milne and Witten incorporated
machine learning, applying their relatedness measurement

Figure 2 Wikipedia category organization

to train a supervised classifier in entity disambiguation.
Their method has shown effectiveness to some extent.

1.3. Word embedding
Distributed representations for words were proposed
early, but recently have been successfully used in language
models and some natural language processing tasks,
including word embedding learning [7][12]. Traditional
word representations are based on a co-occurrence matrixof size W×C, where W is the vocabulary size, each row of

However,these

approaches

do

not

considerjoint

dependencies among the possible target entities from input
context.
A distinct direction in entity disambiguation focuses on
the effects of mention’s context. The referent entities of amention is reflected by its context. Context similarity
measures how similar the text around the named entity inthe text data and the text in the Wikipedia article are. TF-

three classes, i.e. person, organization, and location. So

IDF measure and cosine similarity function are often used.

firstly we use NER tagger, to obtain person, location and

TagMe [4] uses a light-weight form of context coherence

organization phrases. Then our system will compare theseby averaging relatedness scores over all candidate entities

phrases with Wikipedia titles based on the following rules:

for a mention that co-occurs with a given mention.

-

For a phrase that is an exact match with Wikipedia

As for word embedding, the main advantage of word

page title: If the Wikipedia page is not a redirect

embedding is that the word representation of two similar

page nor a disambiguation page, then this phrase iswords are very close in vector space and the dimension ofregarded as an unambiguous entity and will bewords can be decreased greatly in comparison with thetraditional bag-of-word model. Word2Vec 1 is an open

linked to this Wikipedia page finally.
-

For a phrase that is contained in or contain asource project released by Google which achieves state of

Wikipedia page title (e.g. noun apple and title Applethe art performances. It takes a large text corpus as inputInc.): Then the candidate set will include all of theseand outputs word vectors for distinct words. These word
vectors can be subsequently used in various natural

possible candidate titles.
-

For a phrase that is redirect or disambiguation pagelanguage processing and machine learning applications. In

title

Mikolov et al.’s Word2Vec paper [7], they carried out two

disambiguation page will be retrieved, and append

neuralall out-link page titles to the candidate set.

network

continuous

modelsforbag-of-words

representation

model(CBOW)

learning:
andthecontinuous skip-gram model. It was shown that word
representations can capture semantic information between
words.

-

ofWikipedia:

The

redirect

page

and

For a phrase that is an abbreviation of Wikipedia
titles, these possible titles will be added into the setof candidate entities.

As this part of research is still in process, for now we

There are other researches on the analysis of Wikipedia’s

can just obtain a rough set of candidate entities. Because

category hierarchies. In [8], authors performed a graph-certain phrases are likely to produce a large candidate set,

theoretic analysis of the Wikipedia Category Graph andso pruning work should be done in future in order to reduce

showed that the Wikipedia category graph is well suited tothe computation cost of following disambiguation part.

estimate semantic relatedness between words.
There are also a number of systems presenting complex

3.2. Entity disambiguation

combinations of several methods for entity disambiguation.

After the first step, we can obtain a set of candidate

Lemahnn et al. [9] present a supervised system combining

entities for each mention in the input text. Then in thefeatures based on hyperlinks, categories, text similarity

second step, for every mention its candidate entities need

and relations from info-boxes.

to be ranked by semantic similarity and other approaches.

Our research is based on former research and propose

The candidate that obtained the highest score will be

new methods to make use of these features.

chosen as the final entity the mention referred to.

3. Proposed method

3.2.1. Wikipedia superordinate category based feature

3.1. Mention recognition
The entity disambiguation process is usually divided
into two steps: recognition and disambiguation. In the firststep, the system recognizes potential mentions from the
input text and links them to a set of candidate entities
which are likely to be referred to by the recognized mention.
In some existing entity linking approaches, researchers
usually use an existing named entity recognition (NER)
tagger to implement the entity recognition. One popular
tool is Stanford NER tagger which used conditional random
fields (CRF) Classifier, and it can recognize particularly1

https://code.google.com/p/word2vec/

Categories in Wikipedia have a hierarchical structure,
where every article is assigned to one or more categories
when editor is editing it. Each category can have an
arbitrary number of subcategories, and may have more than
one parent categories. Pages which have common parent
categories usually have high semantic relatedness. Most of
page-category links provide navigation within Wikipedia
contents, furthermore, they also inferred some semantic
relationships between pages.
Let us see an example in Figure 3, depicting page Apple
Inc. having several categories, e.g. Apple Inc., Computerhardware companies. For category Apple Inc., there arethen this mention is unambiguous, so the system will fetchtens of subcategories (such as category Apple Inc.

thishardware and category Apple Inc. advertising) and pages

getAscendantCategory()) and store them in a common(such as Steve Jobs) under it. For category Computercomparison set. To achieve a faster retrieval and remove

hardware companies, there exist dozens of subcategories

too many overlappings between superordinate categories,(such as Motherboard companies) and pages (such as

the depth range is set between 3 and 6. When we set theMicrosoft) under it. Furthermore, subcategory Apple Inc.

depth of recursion to 3, that is to store all the categories of

hardware also has its subcategories (such as IPad) andthis entity and the parents and grandparents of the

pages

under

it.

In

this

way,

super-categories

entity’sallascendant

categories

(by

and

categories. A higher depth of recursion is time-consuming

subcategories construct the category hierarchy. Fromand will not deliver a prominent improvement. Later theexperience, we know that the semantic relationshipsystem will scan all ambiguous mentions and compute

between IPad and Steve Jobs is stronger than IPad andcategory scores for each candidate. If there are more than

Microsoft, and from this category diagram, we can see that

one candidates in the set of mention’s candidate set, an

Steve Jobs and IPad are under the same parent category.

iterative calculation of fetching ascendant categories(cand.getAscendantCategory()) for every candidate of thisPage:
Apple Inc.

mention will be done. Then the system will compute a
category
Category:
Computer hardware
companies

Category:
Apple Inc.

scoreascendant(categ_score)

categories

using(categ_set)

this

and

candidate’s

thecommoncomparison category set (comparison_set) produced by the
first scan. Here our category score calculation uses the
Jaccard similarity function:

Page:
Steve Jobs

Category:
Apple Inc.
hardware
Page:
Ipad

Category:
Motherboard
companies

Category:
Apple Inc.
advertising

Page:
Microsoft

𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽𝐽(𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐_𝑠𝑠𝑠𝑠𝑠𝑠, candidate_𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐_𝑠𝑠𝑠𝑠𝑠𝑠) =| 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐_𝑠𝑠𝑠𝑠𝑠𝑠 ⋂ candidate_𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐_𝑠𝑠𝑠𝑠𝑠𝑠 || 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐_𝑠𝑠𝑠𝑠𝑠𝑠 ⋃ candidate_𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐_𝑠𝑠𝑠𝑠𝑠𝑠 |Category:
IPad

Figure 3 W ikipedia category example

The higher the category score is, the more similar between
the candidate and those unambiguous entities from input

As described in the above, one of our features exploits

text is. Chances are that this candidate has somecategory hierarchies. The categories we use are extracted

relationship with other entities from the input. So there isfrom Wikipedia dump on Feb. 5t h,2015. We clean the

category hierarchy, by removing Wikipedia management

a high possibility of similar entities co-occurring incontext.

categories, words like Wikipedia, mediaWiki, wikiproject,template are included. Then we construct a category set as
described in our algorithm below.

3.2.2. Embedding word similarity
Word embedding is a dense and low-dimensional word
representation.

for each mention in input:
if candidate_set.size()==1:
comparison_set.add(candidate.getAscendantCategory())
for each mention in input:
if candidate_set.size()>1:
for cand in mention.candidate_set:
cand.categ_set=cand.getAscendantCategory()
cand.categ_score=J(categ_set, comparison_set)

Eachdimensionoftheembeddingrepresents a latent feature of the word, capturing usefulsyntactic and semantic properties. We decide to use
Word2Vec model [7]. The Word2Vec algorithm is used forconstructing a word embedding for each unique word in the
corpus. We adopt the hierarchical softmax skip-gram model
which uses 2N surrounding words to predict the middle
word and builds up a Huffman tree to represent probability
distribution of words in the corpus. The vector for each

The system will then scan every mention that has beentagged in the recognition part. The first scan only scansunambiguous mentions, aiming to construct a common
comparison set (comparison_set) for further use. When
there is only one candidate in the mention’s candidate set,

word is a semantic description of how that word occurs incontext. So if two words are usually used similarly in given
corpus,chances

are

that

they

will

obtainsimilarrepresentations. After mapping words into the vector space,
we can compute cosine similarity to find words that have

similar semantics. The trained word embeddings largely

In order to compare the quality of relatedness scores

depend on textual content, that is the training corpus. In

between entities obtained by our approach, we performed

order to capture more semantic information, we use theexperiment

whole

the

relatedness-entity dataset [10], the authors selected a set ofexperiment. We use Gensim 1 under Python to load prebuilt

21 queries which correspond to 21 entities from knowledge

model

Wikipedia

corpusastraining

corpusin2.

onentityrelatedness.

In

the

KORE-

base YAGO. These entities are selected from four different

The similarity in word2vec is defined by the cosine

domains: IT companies, Hollywood celebrities, video

similarity of two word vectors. So we will also use thisgames, and television series (Table 1). For each of the 21

similarity measurement. For a sentence in the user ’s input,

seed entities, they selected 20 candidates from the set ofits similarity with the candidate is calculated by computing

entities linked to the seed’s Wikipedia article. The authors

the mean word vector similarity between these words andused a crowdsourcing platform to obtain the gold standard

the candidate entity. We compute the following score on

ranking of the 20 candidate entities for each seed entity.

entities.

The KORE dataset is composed of 420 entity pairs in total.
We compare our ranking result with the gold standard ofscore(𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖, 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚_𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐)= sim(∑𝑚𝑚𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑉𝑉𝑚𝑚
|𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖|,

human-ranked results and evaluate our method using∑𝑤𝑤𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚_𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑉𝑉𝑤𝑤

|𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚_𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐|

Spearman’s rank correlation coefficient (ρ). Spearman's)

correlation assesses how well the relationship between two

input is user ’s input query or context of a mention in thevariables can be. Spearman correlation ranges from -1 to 1,

processed article (mention is tagged by recognition step).

a perfect positive correlation is represented by the value 1,

m is every word in input. |input| is the number of words inwhile a value 0 indicates no correlation and -1 indicates a

input. If the article is too long, we set the context window

complete negative correlation. From this sense, theto 100 words around the mention. mention_candidate is thecorrelation ρ of our system’s result and gold-standard ranks

preprocessed Wikipedia article for one candidate from theshould be a positive value, and the larger is the better.

mention’s candidate set. w is every word in this candidate
name. |mention_candidate| is the number of words in thiscandidate name.

Seed entity

Ranked Candidate Entities

Apple Inc.
(IT Companies)

Steve Jobs (1), Steve Wozniak (2)Jonathan Ive (3), Mac Pro (4)…
Greenpeace (17), Ginza (18)
Sears (19), Ford Motor Company (20)
Jon Voight (1), Brad Pitt (2)…
Chip Taylor (8),Academy Awards(9)…
2005 Kashmir earthquake (20)
Niko Bellic (1)Grand Theft Auto (series) (2)…
Brooklyn Bridge(14), Metacritic (15)…
Matthew Weiner(1), Jon Hamm(2)Alan Taylor (director) (3)…
Volkswagen Beetle (19)Sesame Street (20)3.2.3. Wikipedia inlink-based feature
Wikipedia inlinks are links from one Wikipedia page toanother Wikipedia page. Inlinks are added during user ’sAngelina Jolie(Hollywood
Celebrities)

editing. We suppose that if two Wikipedia pages have moreinlinks in common, the more pages are related to bothpages, so we can say they are highly correlated. Our link

Grand Theft AutoIV
(Video Games)

relatedness is defined as the conditional probability ofentity c given entity e [5][12]:InR (c|e) =Mad Men(TV Series)| 𝐼𝐼𝐼𝐼(𝑐𝑐) ⋂ 𝐼𝐼𝐼𝐼(𝑒𝑒) || 𝐼𝐼𝐼𝐼(𝑒𝑒) |Table 1 Example of seed entities and gold-standard ranks

4. Experiment and Evaluationof candidate entities

4.1. Entity relatedness experiment4.1.2. Experiment result and analysis

4.1.1. Dataset

Experimental results for the KORE relatedness entity
ondataset are shown in Table 3. The values in the table are

Wikipedia knowledge base, using the snapshot of English

the Spearman correlation between the gold-standard rankWe

implemented

our

proposed

system

Wikipedia dump exported on February 512

https://radimrehurek.com/gensim/
https://github.com/idio/wiki2vec

based

t h , 2015 3 .

and the ranking result generated by our methods.
3

https://dumps.wikimedia.org/

𝑲𝑲𝑲𝑲𝑲𝑲𝑲𝑲𝑳𝑳𝑳𝑳𝑳𝑳+𝑭𝑭 ( ba s e l i n e )Domain

SuC

WoE

InK

IT companies

0.28150.47220.4478Hollywood Celebrities

0.39820.55460.3702

0.522

Video Games

0.58000.47700.47100.499

Television Series

0.20170.11190.3862

0.426

Chunk Norris

0.21330.37890.4992

0.653

Average(21 entities)

0.3571

0.40270.4226

0.425

0.208

Table 3 Spearman correlation of relatedness measures with human ranking

The SuC method is the method based on superordinate

top have more common categories with the seed entity, incategory feature. The WoE method is the word embedding

contrary, candidates that ranked lower have fewer common

similarity method. The InK method is the method based on

categories with the seed entity. But this is not always the

Wikipedia inlink feature. The KORE method in the last

truth, such as for seed entity Apple Inc. and IBM, there iscolumn is one of the methods proposed by the authors oflittle difference between the common categories among

the KORE dataset, and we regard it as our baseline.

candidates. So the result is not good for these seed entities.

In Table 3, we compared the results of rank correlation.

Lastly, we analyze the performance of the Word

The WoE method achieved the best scores of IT companies

embeddingdomain and Hollywood Celebrities domain. The best scores

Celebrities domain. We take Brad Pitt as an example. Tableof Television Series domain and the separate entity Chunk

4 shows the rank comparison of the word embedding result

Norris which do not belong to any domain are obtained by

and gold-standard, and the word2vec similarity between

the baseline KORE method. Our methods outperform theevery candidate entity and the seed entity Brad Pitt. The

baseline on three domains.

name which is unique and has less duplication is easy toGoldstandard
rank

SuC
rank

Num
of
commoncategories
with entity

Related candidate

Grand Theft Auto III

method.

It

works

best

for

Hollywood

distinguish the person. In this example, Brat Pitt is afamous actor, and he and other famous actors, like Rusty
Ryan, co-starred in some movies. So their names often cooccur in Wikipedia pages which can be movies’ pages,

3

1

2

2

1

3

Niko Bellic526

is his current wife. So their names co-occur a number of

7

4

PlayStation 3

318

times. Word2vec captured such co-occurrence and context

4

5

Rockstar Games316

information, making these candidates in the top rank. As

Grand(series)

Theft

845

Auto

689

…
16

17

11

18

18

19

13

20

Eastern Europe

on. Jennifer Aniston is Brad's ex-wife and Angelina Jolie

for the latter candidates in this table, like CNN, Sudan,

Edinburgh
New York City
FederalBureauInvestigation

directors’ pages, and other co-starred actors’ pages and so

of

171

Pakistan, they seldom co-occur with Brad Pitt. So they146

ranked lower.

115
20

Table 2 SuC Rank comparison for Grand Theft Auto IV

Then we focus on the analysis of the feature based onGoldstandard
rank

WoE
rank

Related candidate

Word2vec
similarity

1

1

Angelina Jolie

0.47154

2

Rusty Ryan

0.4217

Wikipedia superordinate category. It obtained best score2

3

Jennifer Aniston

0.3789for Video Games domain. We take the Grand Theft Auto IV

114David Fincher

0.3703

17

17

Nice

0.0313

19

18

CNN

0.0208

18

19

Sudan

0.018520

20

Pakistan-0.0189…

entity as an example, as shown in Table 2. It gained acorrelation score of 0.712, which is the highest scoreobtained by SuC. When we set ascendant category
recursion depth to 4, we can obtain all the superordinate
categories for Grand Theft Auto IV. The candidates ranked

Table 4 WoE Rank comparison for Brad Pitt entity

Word embedding similarity performed badly in TV

We measured the accuracy as the fraction of mentions

Series domain. We take the worst one as an example. Table

that were correctly linked to Wikipedia pages. Our

5 shows the rank comparison result of entity The Wire. Thisdisambiguation experiment result is shown in Table 6. The

entity’s rank correlation obtained -0.44, which means

SuC method is the method based on superordinate category

negative correlation. We can analyze some reasons. Firstly

feature. The InK method is the method based on Wikipedia

the entity name contains a ‘The’, as we know ‘The’ is too

inlink feature. The WoE method is the word embedding

common in Wikipedia, which impairs the word vector

similarity method.

representation for this entity. Also, word2vec is good at

Overall, the WoE method performed the best, it cancomparing the similarity between separate words, but does

correctly link more than 75% mentions to its true entity in

not work well in comparing phrases on which word orders

Wikipedia page and followed by the SuC method.

need to be considered.
Gold
standard
rank
18
7
19

WoE
rank

Related candidate
1
2
3

17

4

Word2vec
similarity

Chicago Sun-Times
HBO
Soul Food (TV series)
Six Feet Under (TVseries)0.27070.26840.25920.2493

…
13
11
6
10

17
18
19
20

Bob Ehrlich
Kurt Schmoke
Idris ElbaBlake Leyh

0.08990.07790.07710.0720

Table 5 WoE Rank comparison for The Wire entity

Mention

SuC

InK

WoE

Air Afrique

0

0

0

BoeingNigeria's
largestinternational
airport

1

1

0

0

0

0

Lagos
Guardian
newspaper

0

2

1

1

0

1

Cameroon Airlines

0

0

0

Nigeria01

1Table 7 Ranking of true entity in the candidate set using
3 methods for one article

Let us take one article (APW19980625_1136) for an

4.2. Disambiguation experimentexample to analyze the results. There are seven mentionsin this article and two mentions (Air Afrique and Cameroon

4.2.1. Dataset

Airlines) are unambiguous.
Original AQUAINT corpus consists of newswire text
data in English, drawn from three sources: the Xinhua
News Service, the New York Times Service and the

In Table 7, 0 means the true

entity is correctly ranked at the first place, and ranking
results other than 0 mean this method could not choose thecorrect entity for this mention.

Associated Press Worldstream News Service. We used a
subset of AQUAINT corpus, manually removing repeatedarticles and mentions that are common words, and verified
links to Wikipedia titles according to specific version. So
our dataset contains 45 news articles and 370 mentions,among which 266 mentions have at least 2 candidates.
There are about average 220 words and average 8 mentions
in every news article.
For some mentions, there are too many candidates. We
only keep the top 10 candidates ranked by their popularities
in Wikipedia and if the true entity is not in the candidate
set, we add the true entity to its candidate set.

5. Conclusion and future work
This paper proposed an entity disambiguation system
based on the Wikipedia knowledge base. It is modeled to
exploit the Wikipedia category hierarchies and Wikipedia
links structure. Moreover, tomeasure the semantic

relatedness, word embedding is used to refine the result.
Based on semantic relatedness, we can obtain a rank ofcandidate entities for one mention. We also evaluated our
methods on disambiguation experiment.
In future, we plan to add more context information to our
method and learn a combination method based on machinelearning algorithms.

4.2.2. Experiment result and analysis
MethodCorrectly linked mention

Accuracy

SuC

257

69.5%

InK

244

65.9%

WoE

280

75.7%

Table 6 Disambiguation results on dataset

Reference[1]

Bairi, Ramakrishna B., Mark Carman, and Ganesh
Ramakrishnan. "On the Evolution of Wikipedia:

Dynamics[2]ofCategories

andArticles."Ninth[12] Mikolov, T., Yih, W. T., & Zweig, G. (2013, June).

International AAAI Conference on Web and Social

Linguistic Regularities in Continuous Space WordMedia. 2015.

Representations. In HLT-NAACL (pp. 746-751).

Witten, I., & Milne, D. (2008, July). An effective,
low-cost measure of semantic relatedness obtained
from Wikipedia links. In Proceeding of AAAIWorkshop on Wikipedia and Artificial Intelligence:an Evolving Synergy, AAAI Press, Chicago, USA (pp.
25-30).

[3]

Milne, D., & Witten, I. H. (2008, October). Learning
to link with wikipedia. In Proceedings of the 17th
ACM conference on Information and knowledge
management (pp. 509-518). ACM.

[4]Ferragina, P., & Scaiella, U. (2010, October). Tagme:
on-the-fly annotation of short text fragments (bywikipedia entities). In Proceedings of the 19th ACM
international

conferenceon

Information

andknowledge management (pp. 1625-1628). ACM.
[5]Ceccarelli, D., Lucchese, C., Orlando, S., Perego, R.,& Trani, S. (2013, October). Learning relatedness
measures for entity linking. In Proceedings of the
22nd ACM international conference on Conference on
information & knowledge management (pp. 139-148).
ACM.

[6]

Agirre, E., Barrena, A., & Soroa, A. (2015). Studying
the Wikipedia Hyperlink Graph for Relatedness andDisambiguation. arXiv preprint arXiv:1503.01655.

[7]

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013).
Efficient estimation of word representations in vector
space. arXiv preprint arXiv:1301.3781.

[8]

Zesch, T., & Gurevych, I. (2007, April). Analysis of
the Wikipedia category graph for NLP applications.
In

Proceedings

of

the

TextGraphs-2

Workshop(NAACL-HLT 2007) (pp. 1-8).
[9]

Lehmann, J., Monahan, S., Nezda, L., Jung, A., & Shi,
Y. (2010). LCC approaches to knowledge base
population at TAC 2010. In Proc. TAC 2010
Workshop.

[10] Hoffart, J., Seufert, S., Nguyen, D. B., Theobald, M.,& Weikum, G. (2012, October). KORE: keyphrase
overlap relatedness for entity disambiguation. In
Proceedings of the 21st ACM international conferenceon Information and knowledge management (pp. 545554). ACM.
[11] Taieb, M. A. H., Aouicha, M. B., & Hamadou, A. B.
(2013).

Computingsemantic

relatedness

using

Wikipedia features. Knowledge-Based Systems, 50,
260-278.