DEIM Forum 2014 C1-1

Semantic Knowledge Base Construction from Domain-Specified
Metadata
Jiyi LI† , Toshiyuki SHIMIZU† , and Masatoshi YOSHIKAWA†† Department of Social Informatics, Graduate School of Informatics,
Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501 Japan
E-mail: †jyli@db.soc.i.kyoto-u.ac.jp, ††{tshimizu,yoshikawa}@i.kyoto-u.ac.jp
Abstract Existing semantic knowledge bases such as WordNet and Yago contain the information of relations
between entities. Although they are not domain-specific, they contain limited information and have limitations on
the usage scopes and domains. For example, they do not hold the information about the domain-specific commonsense relations between concepts like “horse” and “farm” or “ocean” and “atmosphere” which intuitively have close
relations on semantics in the domains of image description or earth observation data description. Such semantic
knowledge are useful in the corresponding domains for various applications such as annotation, recommendation,
search, suggestion and so on. Because in data collections, metadata which is used to describe data is widespreadin various domains, in this paper we propose an approach to collect this kind of relations and construct knowledge
bases for specific domains by mining knowledge of global structure and internal association in the metadata of datacollections.
Key words Knowledge Base, Metadata, Domain-Specific, Relation Extraction

1. Introduction

the domains of image description. Figure 2 shows more examples of domains and the relative concepts in the domains,

Knowledge base (KB) has been an active research area inincluding concepts “ocean” and “atmosphere” in the domain

the past decades. It harvests and manages facts among en-of earth observation data and concepts “knowledge harvest-

tities as knowledge with representation that is readable bying” and “information extraction” in the domain of scholarlymachines to provide knowledge-centric services like natural

document.

language processing assistant, question answering, and se-

Such kinds of relations between concepts have abstract or

mantic search. There are many existing knowledge bases

vague semantics. They seems to obviously and intuitively

and each of them focuses on diﬀerent types and scopes of in-exist but actually are diﬃcult to be represented exactly information, for example, WordNet [1], [2] is a lexical database;words. For the example of “horse” and “farm”, the seman-

YAGO [3], [4], [5] is a semantic knowledge base derived fromtic relations can be explained like “horse” lives on “farm” orWikipedia（注 1）, WordNet and GeoNames（注 2）; DBpedia [6] ex-“horse” and “farm” usually co-exist in same scenes and im-tracts structured content from Wikipedia; Freebase [7] is anages. The relations are known by human beings but notonline collaborative knowledge base mainly by its commu-

easy to be acquired and understood by machine. It cannity people; KnowItAll [14], [15], [16] project aims to auto-

be regarded as a kind of commonsense knowledge and we

matically extract fact collections from Web resources.

concentrate on it in this paper. Here “commonsense” is a

These existing semantic knowledge bases such as WordNet

domain-specific notion because it may be not commonsense

and YAGO contain the information of relations between en-

for all people and may only be commonsense for the people

tities. Although they are not domain-specific, they containwith knowledge background of a specific domain; the rela-limited information and have limitations on the usage scopes

tions exist and are used in specific domains. We can harvestand domains. For example, they do not hold the informa-these knowledge from the data collections in correspondingtion about the relations between the concepts “horse” anddomains, for example, harvesting the relations like that be-“farm” which intuitively have close relations on semantics intween “horse” and “farm” from image datasets in image description domain.

（注 1）：www.wikipedia.org

Because in these data collections, metadata, which is data

（注 2）：www.geonames.org

about data, is widespread applied in various domains andFigure 1 Example of Domains and Concepts with Relations

contain useful information for describing data content. We

involvement on manually relation labeling for both knowl-

therefore extract the relation information from descriptiveedge base construction and training set construction. The

metadata. The descriptive metadata is diverse in format

problem is that our solution need to make eﬀorts on the pre-and content; some metadata like keywords are only list ofcision and recall of the knowledge bases because of the lackwords without syntactic information; some metadata like so-of usable information.

cial tags are user generated and contain various noises; fur-On the other hand, for the community of knowledge basethermore, the commonsense relations we extract are not easyand semantic web, our work extends the research on the fol-

to be represented with special patterns in natural language

lowing aspects. The extension on various aspects is an im-

sentences. Therefore, we convert the textual description in

portant issue for the area of knowledge base.

metadata into bag of concepts. We propose a solution to•Concepts and Relations:

Most existing Knowledge

construct specific knowledge bases for specific domains bybases focus on harvesting relational facts between entities,

mining the knowledge of global structure and internal asso-

for example, unary relations like “X IsA FootballPlayer” andciation in the metadata of data collections.

binary relations like “X hasWonPrize Y”. Our work focuses

The main contributions of this paper are twofolds. On

on harvesting commonsense relations between concepts in a

one hand, for the communities of specific domains like im-

specific domain, this kind of relations is not covered in ex-

age retrieval, earth observation and academic library, theisting knowledge bases.

commonsense concept relations we collect into the knowledge•

Resources and Domains:

Some existing knowledge

bases can be widely used for data management and informa-bases is constructed manually which their resources are hu-tion retrieval in these domains, such as, keyword annotation,

man knowledge and references. Many recent work propose

keyword recommendation, keyword search, query suggestion

approaches to automatically collect the knowledge from text

and so on.

and web sources.

There is no specific domain declared

We do not focus on entities like that used in YAGO andthough there are limitations on the usage scopes. Our work

relational facts between them. In contrast, we focus on con-harvests the knowledge from the metadata of data collections

cepts that used in WordNet and commonsense relations be-

for a specific domain.

tween them. Note that although the data we focus on is•

Relation Extraction Method:

Existing knowledge

domain-specific, the approach we propose is not domain-bases focus on relational facts and their relation extraction

specific. Without loss of generally, we do not use the in-approaches leverage linguistics syntax in the text by using

formation in the detailed content of data in a domain, such

pattern-based gathering or constraint-based reasoning ap-

as image content, earth observation data, body of paper andproaches. Our relation extraction approach do not refer lin-so on. We only use the information of metadata in the format

guistics syntax information and leverage bag of concepts byof bag of concepts. Actually our relation extraction approachanalyzing data characteristics, use heuristic rules and learn

is more general than existing approaches that leverage lin-classifiers.

guistics syntax information, because it is based on bag of•Applications:

Existing knowledge bases enable

concepts and is independent from the sentence structure andknowledge centric applications like natural language process-language, which also means our solution can be regarded as

ing assistant, question answering, and semantic search fora multilingual solution.

entities and relations, while our knowledge bases enhance

Our approach is supervised and without expensive humandata management and retrieval applications like keyword an-notation, keyword recommendation, keyword search, querysuggestion, and so on in specific domains.
The remainder of this paper is organized as follows. In
Section 2 we review the related work. In Section 3, we propose our solution on semantic knowledge base construction
for special domains with their metadata. In Section 4 weTable 1 Relations in Existing Semantic Knowledge Base
subject, object

YAGO

WordNetRelationhorse, equid

subClassof hypernym

direct hypernym

horse, animal“indirect”“indirect” inherited hypernym

horse, zebra“indirect”“indirect”

horse, black

“no”

“no”

descriptive

horse, farm

“no”

“no”

coexist nouns

sister term

report and discuss the experimental results. We give a conclusion in Section 5.

2. Related Workthe relational facts from web sources by using pattern-based
gathering or constraint-based reasoning. Yago [3], [4], [5] andDBpedia [6] are rule-based and extract information from in-

In this section, we brief review the related work. There aremany existing knowledge bases, Ref. [8], [9] list some of them.
Specially, we review two typical and widely used knowledge
bases, WordNet and Yago, which are closely related to themotivation of our work.
WordNet [1] is an important lexicon in information retrieval and natural language processing. The core concept infobox of wikipedia. Some work [10], [11], [12], [13], [14], [15],[16], [17] leverage the linguistics syntax information in the
text. Our relation extraction approach does not refer linguistics syntax information and leverages bag of concepts,
We analyze data characteristics about global structure andinternal association in the metadata of data collections, use
heuristic rules and learn classifiers.

WordNet is the synset. A synset contains one or more word
senses and each word sense belongs to exactly one synset.
In turn, each word sense has exactly one word that represents it lexically, and one word can be related to one or more
word senses. WordNet follows taxonomy and has tree-likestructure. To adopt WordNet in the semantic web researchcommunity, researchers convert it into RDF or OWL representation, e.g., Ref. [2].
YAGO [3], [4], [5] is a core of semantic knowledge in which
the information is extracted from Wikipedia (e.g., categories,
redirects, infoboxes), WordNet (e.g., synsets, hyponymy)
and GeoNames, using rule-based and heuristic methods. It
contains additional knowledge beyond WordNet on individuals like persons, organizations, products, and so on, with
their relations. It has high coverage and contains more than
1 million entities and 5 millon facts. The empirical evaluation of fact correctness by the author shows that it has ahigh accuracy. It contains both hierarchical relations andnon-taxonomic relations. It is compatible with RDF.
WordNet and YAGO do not explicitly contain the relations
like “inherited hypernym” relation between hhorse, animali
and “sister term” relation hhorse, zebrai. These implicit relations in Yago and Wordnet can be generated by additional
inferences. YAGO contains additional information extracted
from Wikipedia, but still doesn’t contain the information weneed. Some relations like that between hhorse, f armi areneighter explicitly nor implicitly contained in WordNet andYAGO. Table 1 lists some examples that YAGO and WordNet contain or not.
For the approaches of relation extraction and knowledge
base construction, WordNet [1], [2] is created manually. Freebase [7] is constructed by collaboratively by its community
people. Many work make eﬀorts on automatically harvesting

3. Semantic Knowledge Base Construction from Metadata
We propose our approach of semantic knowledge base construction in this section. Instead of relational facts between
entities, we focus on domain-specific commonsense relations
between concepts in this work. We first describe the preprocessing method on the data in the collection and then propose the relation extraction approach. After that, we introduce the post-processing method for arranging and managing the extracted relations and concepts in knowledge bases.
Another issue about enrichment, extension and aggregationof the constructed knowledge bases is not yet covered in thispaper and is one of the future work.
3. 1 Definition and Formulation
Definition: Commonsense Relation Rc between two concepts wi and wj in a specific domain is that these two concepts subjectively has intuitive association for the people inthis domain.
We use Rc instead of Commonsense Relation for short.
This Rc in our work has the following properties.
•

It is a binary relation: wi Rc wj .

•It is an abstract and hypernym relation: Rc = {rc }.

Rc is proposed to handle the abstract or vague semantics
that intuitively exist but may be diﬃcult to be represented
exactly in words. It can be interpreted to various detail relations rc in diﬀerent cases. Table 1 shows some examples
of detailed cases and interpretations like “direct hypernym”,“inherited hypernym”, “sister term”, “descriptive” and “coexist nouns”. Specially, in our work, we focus on the relations
that are not contained in existing knowledge bases such asthat between “horse” and “farm” in Table 1.

Figure 2 Overview of Our Solution•

It is a swappable relation: wi Rc wj ⇒ wj Rc wi . Rc

tract the commonsense relations. The social tags of an image

has a definition like wi “hasCommonsenseRelation” with wj

is a group of words. We can convert it to bag of concepts eas-

which is swappable. Although its sub relations like “inheritedily. For other kinds of textual information in metadata such

hypernym” are not non-swappable, e.g., “equid” is the “di-as title or abstract which are in sentences, we can use nat-rect hypernym” of “horse”, according to the definition of Rc ,ural language processing to split them into concepts. There

this example can be interpreted in the mean of that “horse”

are various natural language toolkits available for this task

has Rc (or “direct hypernym”) relation with “equid”.

in each language. How to convert the plain text in a specific•

It is a non-transitive relation: wi Rc wj ∧ wj Rc wk 6⇒

language to bags of concepts is not covered in this paper.

wi Rc wk . The relation between any two concepts of wi , wj

Although the loss of information on syntax, sequence and so

and wk needs to be evaluated separately. In addition, be-on, bag of words model has been proved eﬀective and widely

cause it is a non-transitive relation, an inference based on

used in information retrieval for representing documents. We

existing relations to generate new relations is not available.

therefore use bag of concepts model in our work.

We formulate the notions in our work as follows. In a spe-

The purpose of data pre-processing is to prepare a list ofcific domain, given a data collection D which is composed

candidate concepts, and then our approach can extract rela-of data records, we denote each record as a document di .

tions among them. After we convert the textual description

The data records can be in various forms, such as an image,in the metadata into bag of concepts, we filter these con-a earth observation dataset, an academic paper, and so on.

cepts with ontology-like vocabularies related to the specific

We denote the metadata of di as mi . We convert the tex-

domain. For example, we use WordNet for image description

tual description in the metadata into bag of concepts. We

domain. The concepts with two low frequency are regarded

denote each concept as wx , the set of concepts in metadata

as noises and have been deleted by the authors of NUS-

di as mi = {wx |wx ∈ mi }, and the set of concepts used

WIDE. The concepts that are not contained in the nounin this data collection as W . Our work is to extract thesynsets in WordNet are not included. The concepts withrelations (predicate) pij between two concepts wi and wj :same homomorphy are merged. WordNet stores concepts liketij = hwx , pxy , wy , cxy i. cxy represents the confidence score“horse”, “farm” and so on, and therefore is proper to provideof the relation, which is also used in YAGO [3]. We denotea concept list for this example. Although the concepts in this

this quadruple txy as a term of “tuple” to distinguish theexample based on WordNet are only some words like “horse”,

term of “fact” used in existing knowledge bases.

the concepts can also be some phrases like “information ex-

3. 2 Data Pre-processing

traction” based on the vocabularies and the domains. For

In this sub-section, we describe the pre-processing method

the domain of earth observation data, this vocabulary canon the metadata of data collection. In this paper, we use so-

be GCMD Science Keywords [18]; for the domain of schol-cial image public dataset NUS-WIDE in the domain of image

arly document, this vocabulary can be constructed from thedescription as the example for our solution. The detailed in-

category tree of Wikipedia.

troduction of this dataset and domain is in the experimental

3. 3 Relation Extraction

section. The data in other domains like earth observation

To reduce or avoid expensive human involvement on man-data and scholarly document are not yet collected and pre-ually relation labeling of knowledge base construction, we

processed. Without loss of generality, we use user generated

propose a solution to automatically extract relations fromsocial tags in English in the metadata of the images to ex-

metadata in data collections. Our fundamental idea is that

1 of Selected Concepts
Table 2 Concept Co-occurrence vxy

2
vxy
=

horse building black farm road animal zebra
horse

2399

building

f (wx , wy ),g(wx , wy )

17

68

100

20

187

17

which is also known as Jaccard similarity coeﬃcient.

6995

189

43

190

30

4

g(wx , wy ) is the number of times that wx or wy is used for a

7407

59

141

285

77

data record. An asymmetric measure is defined as

2236

74

175

6

3004

62

7

9553

133

black
farm
road
animal
zebra

663

3
vxy
=

f (wx , wy ) f (wx , wy ),f (wx )f (wy )

which keeps the feature vector of c3xy and c3yx same. Anotherform of this asymmetric measure can be defined as

2 *1000 of Selected
Table 3 Symmetric Relative Co-occurrence vxy

Concepts
horse building black
1000

horse
building

0

3
vxy
=

•

f (wx , wy )1 f (wx , wy )(+).
2f (wx )f (wy )

farm

road

animal zebra

1.81

6.98

22.05

3.72

15.89

5.58

1000

13.30

4.68

19.37

1.82

0.52

1000

6.16

13.73

17.09

9.63

1000

14.32

15.07

2.07

1000

4.96

1.91

1000

13.19

max{logf (wx ), logf (wy )} − logf (wx , wy ).
logM − min{logf (wx ), logf (wy )}1000

M is the total number of the data records. A Monotonically

black
farm
road
animalzebra

Google Distance: There is a semantic similarity mea-

sure proposed based on the hits number of Google searchengine for a given set of keywords. It can also be adapted toused in our scenario.
4
vxy
=

0

4
4
increasing form of this measure is vxy= exp(−vxy), in which4 ) * 10 of Selected Concepts
Table 4 Google Distance exp(-vxyhorse building black
horse

10

farm

road

animal zebra

the concept pairs with higher similarity have higher value.
•

Associated Word Distribution:

We also propose an

2.795

3.703 5.153 3.460

4.347

4.388

10

3.662 3.456 4.485

2.063

2.886

10

3.648 4.144

3.764

4.676

cepts that co-occur with a given concept pair. This measure

10

4.617

4.340

3.733

is computation consuming (n3 matrix, n is the number of

10

3.262

3.646

uniqure concepts) and therefore is not used currently. It

10

4.909

can be used for classifying sub relations in future work. For

10example, we can use light-computation measures to filterbuilding
black
farm
road
animalzebra

original measure that refers the distribution of the other con-out the candidates with relations and then use this heavywhen two concepts are used in same metadata, it is possible

computation measure to classify them into sub relations.

that they have commonsense relations. However, it always

We make an investigation with some samples to checkdoes not true. Although many concepts are used in the meta-

whether such relations measures is possible to evaluate

data of data records, only some of them have relations. We

the relations. We first check the measure of concept co-need to consider the global structure and internal associa-

occurrence. Table 2 shows the co-occurrence of selected con-tion of a concept pair in the data collection to evaluate thecepts to the images in the NUS-WIDE [21] dataset. We com-possibility of their relation. Therefore we come to a series ofpute the number of times that two concepts are labeled torelation measures that may reflect concept relations.

same images. “horse” and “buildings” with no relation has•

Concept Co-occurrence: We denote the the frequency

low co-occurrence value; “horse” and “farm” with a common-of word wx in the data collections as f (wx ), a word multi-

sense relation has high co-occurrence value. It shows that

ple appears in the metadata of a data record is computed

maybe concept co-occurrence can be used for evaluating theas once. The number of times that two concepts are used

relations. A naive relation extraction method is to set an em-

for same data record is defined as f (wx , wy ). The relation

pirical threshold, e.g., θ1 = 30, and select the concept pairsmeasure iswith concept co-occurrence higher than this threshold. Note1vxy= f (wx , wy ).

•Relative Co-occurrence: Concept co-occurrence is thethat although there are some exsiting text analysis tools canvisualize the co-occurrence between words with graph, word
co-occurrence is not equivalent to commonsense relation.

most intuitive measure and shows the global structure of

However, “horse” and “zebra” which has a commonsense

the concept pairs in the data collection. Some other related

relation has same value on this measure with “horse” andco-occurrence measures that consider internal association of“buildings”.

concept pairs can be used. For example, a symmetric mea-

not enough for evaluating the relations. We then investi-

sure is defined asgate other measures.

It shows that only concept co-occurrence isTable 3 and Table 4 show that inthe measures of symmetric relative co-occurrence and Google

α1 , e.g., α1 = 10. The frequency of concept pairs should be

distance the pair of “horse” and “buildings” and the pair ofhigher than a threshold α2 , e.g., α2 = 5. αk is defined as“horse” and “zebra” are distinguishable on the commonsense

thresholds for generating training set from the resources.

relation. Based on above investigations, we draw an assump-

For the second problem, the concept pairs that do not havetions that these measures can be used for evaluating the rela-relations labeled in WordNet cannot be regarded as negative

ktions. We therefore construct feature vectors vxy = {h(vxy)}

samples. To manually construct negative samples with thefor concept pairs with a selection of these relation measures,same order of magnitude of positive samples is expensive.

and then leverage machine learning technologies to classify

How to automatically select negative samples is a problem.

the concept pairs into a class of commonsense relation or

We address it as future work. Currently our solution is tonot. h is some transformations on the basic measures, e.g.,learn the model without negative samples. Some machine

the monotonically increasing form of Google distance.

learning technologies are proposed for such case. We use one

To reduce or avoid expensive human involvement on man-class SVM [20] for learning the model to extract the relations.

ually relation labeling for training set construction. We pro-

3. 4 Post-processing

pose a solution to automatically generate the training set for

After we extract the rough tuples of relations, we serialize

learning the model. The idea is to collect the learning sam-the concept pair matrix, and store the list of concept pairs

ples from available existing knowledge bases. In our example

with commonsense relation into a file to construct the knowl-

on image description, we use the same ontology-like resourcesedge base. It also can be converted to a graph representation

with data pre-processing, i.e., WordNet. We assume thator stored in RDF format. Because our commonsense relation

various sub-commonsense relations have similar characteris-is non-transitive relation, we do not need to carry out an in-tics on the above mentioned measures, specially the relations

ference based on existing detected tuples to generate newlabeled in WordNet and the relations that are not contained

tuples is not available. In the future work, we will also add

in WordNet have similar distributions on these measures. It

methods to improve the precision in the post-processing.

means that we can use the relations in WordNet that are

The predicate pxy in a tuple of two concept is defined as

manually labeled and have high precision and confidence as“hasCommonsenseRelation”. To assign the confidence scoretraining samples to learning the model and threshold for the

cxy for each tuple of concept pairs. If the machine learningcommonsense relations. We therefore scan all candidate con-method can generate probability value on the positive class

cept pairs in the concept space W . We check whether thesefor each concept pair, we can use it as the confidence score.

concept pairs have a selected list of relation types explicitly

If it cannot return probabilistic value and can only returnor implicitly labeled in WordNet and add the positive ones

class labels, e.g., the implement of one class SVM used this

into the training set.

paper, we use the correctness ratio in the testing set for all

There are two problems in this automatic solution. One isconcept pairs instead.

the noises in the training set; the other is the lack of negative

There are some candidate solutions for improving the pre-samples. For the first problem, although WordNet is manu-

cision and recall of the generated knowledge base, for exam-

ally constructed. It only has high precision on its own rela-ple, we can check and add the tuples manually; we also cantion types. Not all relation types labeled by WordNet can be

add more documents or data collections. In this paper, we

regarded as commonsense relation; for a give relation type la-introduce an approach that how to construct the knowledge

beled by WordNet, not all concept pairs labeled in WordNetbase based on one data collection. This knowledge base gen-have commonsense relations. For example, not all concepts

erated from only one data collection has limited information

pairs with “inherited hypernym” relation in WordNet haveto used for the whole specific domain. We can enrich, extend

commonsense relations; many concepts share same hyper-

this knowledge base with more documents, and we can also

nym and therefore have “sister term” relations in WordNet

integrate multiple knowledge bases into one knowledge base.

do not have commonsense relations. Actually, some conceptThis important issue is not yet covered in this paper and ispairs with relations labeled by WordNet have 0 frequency in

one of the future work.

the data collection and therefore have 0 values on all measures. To solve this problem, we set empirical rules on each

4. Experiment

selected relation type to reduce the noises. In this example,

In this experimental section, we first introduce the candi-

we select the relation types of “direct hypernym” and “sister

date domains we select and the candidate available datasetsterm” in WordNet to generate training samples. For “sisterin these domains. We then describe the details of the ex-term” relation, the maximum depth of the concepts in theperimental settings on the example domain and dataset. We

synset tree of WordNet should be higher than a thresholdreport and discuss the experimental results at last.

4. 1 Dataset Resources

concepts in the concept space W at last.

We consider several candidate domains to collect the theIn the stage of relation extraction, the feature vector of

domain-specified metadata for experiments. They are social

12
3
4
concept pairs wx and wy is vxy = {vxy
, vxy
, vxy
, vxy
}. Forimage, earth observation data and scholarly document.

training set construction, we select the relations of “direct0a ) Social Image

hypernym” and “sister term” in WordNet. Because the exist

Social images are from the huge image collections on so-of noisy samples, for the filtering thresholds of generatingcial networking services websites.The metadata of social im-

training set, we set some strict thresholds to ensure the pre-ages contains exchangeable image file format (Exif) informa-cision of the training set. The maximum depth of concepts

tion, user generated textual information, social information

for “sister term” should be higher than 10 (α1 = 10); thesuch as owner, favorites and groups, and so on. We con-frequency of concept pairs should be higher than 5 (α2 = 5).

centrate on the social tags which are labeled by users for aNote that we do not set too large frequency threshold be-

brief description to the images to collect concept relations.

cause the positive testing concept pairs with low frequency

The constructed knowledge base can be used for image an-

may be rejected, and we don’t need to consider the recall of

notation and image retrieval. Some public datasets in which

the training examples. There are 2181 positive samples in thethe images are gathered from social image hosting websites

training set for the 4031 unified concepts, which means 0.54

like Flickr are available, for example, NUS-WIDE [21] withrelations for each concept in average. To learn the model,

269,648 images and MIR Flickr [22] with 25,000 images.

the implement of one class SVM [20] we use on our automat-b ) Earth Observation Data

ically generated positive samples is LibSVM [19]. The “nu”

The domain of earth observation science gather hugeparameter of one class SVM is set to 0.2 because it reaches

amount of data on earth physics, chemistry and biology.

good results on the cross validation on training set and does

The metadata of earth observation data are to describe thenot overfit the training set.

dataset on the information of content, investigator, publica-

The size of a knowledge base is huge and manually eval-tion, and so on. There are many data centers around theuating the whole knowledge base is impractical. We referworld that publish various data, and the candidate data cen-the empirical evaluation of relation precision which is usedters or services we consider to collect data for our work are

by Ref. [3] for evaluation. We sample a subset of the con-the Data Integration and Analysis System (DIAS) project

cept space W and corresponding concept pairs to construct（注 3）

a testing set for evaluation by human judgment. We use the（注 4）which holds hundreds of datasets and Pangaea

whichprovides rich API interfaces to access their data.

24 concepts from the image annotation ground truth in MIRc ) Scholarly Document

Flickr dataset [22]. Because the target applications of our

Scholarly document collections with millions of publica-

knowledge base are data management and retrieval applica-

tions in various scientific areas have entered big data era.

tions like keyword annotation and recommendation, these 24

The metadata of scholarly document, e.g., academic pa-

concept are regarded typical and common concepts in image

pers, contain authors information, paper content informa-

annotation. In addition, we also use additional 6 concepts

tion, publication information, and so on. For the candidatefrom Table 2. There are 30 concepts in total, and we will

datasets, CiteSeer is a public digital library mainly in theevaluate the relations of 435 concept pairs.

areas of computer science and provide OAI interface （注 5） to

The baseline method we compare with is setting empir-

harvest the records in it. Mendeley is reference manage-ical threshold θ1 to the measure of concept co-occurrence.

ment application and provides the public Mendeley’s Data-

According to Table 2, we set θ1 = 30.

TEL dataset（注 6）

for a challenge.

4. 2 Experimental Settings

4. 3 Experimental Results
We evaluate the precision of the constructed knowledge

As we have introduced in Section 3, we use the social tagbase. We define this metric as Knowledge Base Precision

metadata of the data collection NUS-WIDE in the image de-(kbPre). Generally, for many existing knowledge bases like

scription domain as the example for our work. NUS-WIDEYAGO, this metric of evaluating the precision is enough.

has deleted the concepts with too low frequency. After the

They focus more on precision and less on recall because there

pre-processing stage with homomorphy merge and noun ver-is no boundary on the knowledge, and list the huge number

ification on the candidate concept set, there are 4031 unified

of relation records in the knowledge base. On the balance
between precision and recall, precision is more fundamental（注 3）：DIAS: http://www.editoria.u-tokyo.ac.jp/projects/dias/.
（注 4）：Pangaea: http://www.pangaea.de/.
（注 5）：CiteSeerX Data: http://csxstatic.ist.psu.edu/about/data.
（注 6）：Mendeley’s DataTEL: http://dev.mendeley.com/datachallenge/.

for knowledge bases.
The kbPre of our generated knowledge base is 51/66 ≈
77.3%. It is higher than that of baseline 100/269 ≈ 37.2%.

(a)

(b)

(c)

(d)

Figure 3 Sample Results

Figure 3 shows sample results with a graph representation.
Concepts are nodes, and relations are edges.

Figure 3.ashows the relations in the concepts of the testing set. Figure[8]

3.b, 3.c and 3.d show all extracted relations of three concepts.
It shows that the knowledge base generated by our method
has a good precision. Furthermore, on the whole concept

[9]
[10]

space with 4,031 concepts, our approach extracts 845,560
tuples from all 8,122,465 concept pairs and uses these tuples[11]

to construct the knowledge base.

5. Conclusion[12]

In this paper, we discuss how to construct knowledge basefrom metadata in a specific domain. We focus on collecting[13]

the domain specific commonsense relations. The constructed
knowledge bases can be used for annotation, recommendation, search, suggestion and so on in the specific domains.

[14]

Our work is a significant extension on the covered type ofconcepts and relations, domains and data resources, and applications in the research community of knowledge base.

References[1] C. Fellbaum, editor. “WordNet: An Electronic Lexical
Database”, MIT Press, 1998.
[2] Wordnet 3.0 in RDF, available at http://semanticweb.cs.
vu.nl/lod/wn30/.
[3] F. M. Suchanek, G. Kasneci, and G. Weikum. “YAGO A Core of Semantic Knowledge”, WWW’07, pp. 697-706,
2007.
[4] J. Hoﬀarta, F. M. Suchanekb, K. Berberich, E.L. Kelham,
G.D. Melo, G. Weikum. “YAGO2: Exploring and Querying World Knowledge in Time, Space, Context, and Many
Languages”, WWW’11, pp. 229-232, 2011.
[5] J. Hoﬀarta, F. M. Suchanekb, K. Berbericha, and G.
Weikum. “YAGO2: A Spatially and Temporally Enhanced
Knowledge Base from Wikipedia”, Artificial Intelligence,Vol. 194, pp. 28-61, Jan. 2013.
[6] S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak,
and Z. Ives. “DBpedia: a Nucleus for a Web of Open Data”,
ISWC’07/ASWC’07, pp. 722-735, 2007.
[7] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Tay-

[15]

[16]

[17]

[18]
[19]
[20]

[21]

[22]

lor. “Freebase: A Collaboratively Created Graph Databasefor Structuring Human Knowledge”, SIGMOD’08, pp. 12471250, 2008.
F. Suchanek, and G. Weikum. “Knowledge Harvesting from
Text and Web Sources”, ICDE’13, pp. 1250-1253, 2013.
F. Suchanek, and G. Weikum. “Knowledge Harvesting in
the Big-Data Era”, SIGMOD’13, pp. 933-938, 2013.
N. Nakashole, M. Theobald, and G. Weikum. “Scalable
Knowledge Harvesting with High Precision and High Recall”, WSDM’11, pp. 227-236, 2011.
N. Nakashole, G. Weikum, and F. Suchanek. “Discoveringand Exploring Relations on the Web”, VLDB Endowment,
Vol. 5, No. 12, pp. 1982-1985, 2012.
N. Nakashole, G. Weikum, and F. Suchanek. “PATTY A
Taxonomy of Relational Patterns with Semantic Types”,
EMNLP-CoNLL’12, pp. 1135-1145, 2012.
N. Nakashole, G. Weikum, and F. Suchanek. “Discovering
Semantic Relations from the Web and Organizing Themwith PATTY”, SIGMOD Record, Vol. 42, No. 2, pp. 29-34,
Jun 2013.
O. Etzioni, M. Cafarella, D. Downey, A.M. Popescu, T.
Shaked, S. Soderland, D.S. Weld, and A. Yates. “Unsupervised Named-Entity Extraction from the Web: an Experimental Study”, Artificial Intelligence, Vol. 165, Iss. 1, pp.
91-134, 2005.
M. Banko, M.J. Cafarella, S. Soderland, M. Broadhead, andO. Etzioni. “Open Information Extraction from the Web”,
IJCAI’07, pp. 2670–2676, 2007.
O. Etzioni, A. Fader, J. Christensen, S. Soderland, and M.
Mausam. “Open Information Extraction: the Second Generation”, IJCAI’11, pp. 3-10, 2011.
G. Weikum, and M. Theobald. “From Information to
Knowledge: Harvesting Entities and Relationships fromWeb Sources”, PODS’10, pp. 65-76, 2010.
L.M. Olsen et al.. “NASA/Global Change Master Directory(GCMD) Earth Science Keywords”, Version 8.0.0.0.0, 2013.
C.C. Chang and C.J. Lin. “LIBSVM : a library for support
vector machines”, ACM TIST, Vol. 2, Iss. 3, No. 27, 2011.
B. Schlkopf, J. Platt, J. Shawe-Taylor, A. J. Smola, and
R. C. Williamson. “Estimating the support of a highdimensional distribution”, Neural Computation, Vol. 13,
Iss. 7, pp. 1443-1471, 2001.
T.S. Chua, J.H. Tang, R.C. Hong, H.J. Li, Z.P. Luo,and Y.T. Zheng. “NUS-WIDE: A Real-World Web Image
Database from National University of Singapore”, CIVR’09,
2009.
M.J. Huiskes, and M.S. Lew. “The MIR Flickr Retrieval
Evaluation”, MIR’08, pp. 39-43, 2008.