DEIM Forum 2023 1b-3-4

Mapping Wikipedia Categories and Lists to DBPedia Ontology Based on
Structural and Semantic Features
Zhenyang ZHANG†

Zhaoyi WANG‡

and

Mizuho IWAIHARA‡†Graduate School of Information, Production and Systems, Waseda University
2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808-0135 Japan
E-mail: †zhangzhenyang@fuji.waseda.jp, ‡wangzy-joey@akane.waseda.jp, ‡iwaihara@waseda.jp
Abstract Ontology mapping plays an important role in the integration of knowledge resources and many down-streamtasks that include entity recognition, such as entity linking, entity typing, table column typing, relation extraction, question
answering, and other knowledge graph-related tasks. With the development of deep leaning and its successful application tovarious domains, the deep learning approach has been applied in ontology mapping. However, existing methods rarely focus
on the hierarchy of the knowledge graph and the syntactic structures of the ontology class names. In this paper, we propose anovel ontology mapping method based on structural and semantic features. It contains two main parts: (i) Type inheritancebased on structural features and (ii) Fine-tuning a classifier based on pre-trained language model BERT for capturing semantic
features. For the first part, we design heuristic rules for assigning and propagating DBPedia types over Wikipedia list andcategory nodes. For the second part, we utilize POS tagging and dependency parsing graphs to find root phrases of Wikipedia
category names, then use the unsupervised SimCSE model to generate embedding of each category name. Then the cosine
similarities between these embeddings are used to fine-tune the BERT model. We also search sibling nodes which are likely toshare the same DBPedia type, for augmenting the training set. Finally, we use Random Forests to select the most specific node
among the results generated by two parts. Experimental results show that our method achieves superior results than baselines.
Keyword Knowledge graph，Ontology mapping，Wikipedia categories and lists，Distant supervision
DBPedia ontology (at the time of 2022 -08-29) currently

1. Introduction
Ontology mapping attempts to match entities fromcontains about 4,828,418 instances.

similar

CaLiGraph [8] is a large semantic knowledge graphproperties. A mapping is a connection between two

with a rich ontology compiled from the DBPedia ontology

matched entities, where each mapping is often categorized

and Wikipedia categories and list pages, as Figure 3 (a)as equivalence and subsumption.

shows. The ontology is enriched with fine-grained value

different

ontologies

that

have

semanticallyWikipedia is the world’s largest and free en cyclopedia

restrictions on its classes that are discovered with themaintaining high-quality trusted information. Most of its

Cat2Ax [9] approach. A large number of CaLiGraph's

entries (i.e., Wikipedia articles) can be considered as

entities is extracted from Wikipedia listings through a(semi-structured) representations of entities. Wikipedia

combination

offers three complementary ways to group related entries

transformer-based extractors.

together: Categories, lists, and navigation templates [21].

of

the

ontologicalinformation

an dMapping Wikipedia categories and lists to DBPedia

each

Ontology plays a critical role in many downstream tasks

Wikipedia article is assigned to at least one category. Lists

that include entity recognition, such as entity linking,provide a means for manual categorization of articles andentity typing, table column typing, relation extraction,

can include entities that do not have a Wiki pedia page yet.

question answering, and other KG-related tasks. For

Lists are more difficult to process automatically due toexample,informal construction.

“1999_science_fiction_novels”, if we assign the DBPedia

Categories

areorganizedinahierarchy

and

for

the

Wikipedia

category

DBPedia [22] is a large knowledge graph which

type “Novel” to this category, then, given a table column

leverages gigantic source of knowledge by extractingof novel names such as “Vector_Prime” as Figure 1 shows,

structured information from Wikipedia. The DBPedia

we can search the entities of the given column under the

ontology is the heart of DBPedia. The ontology (at the

Wikipedia category or search the entity resources in thetime of 2022-08-29) covers 768 classes which form a

CaLiGraph ontology and assign the mapped DBPedia typesubsumption hierarchy as Figure 2 (a) shows and areto the matched entities as the table column typing.

described by around 3000 different properties. The

class name “Opera house in Puerto Rico”, its root word is“house”. Here we do the POS tagging to find root words of
CaLiGraph class names. (3) Find ing root phrases. In thispaper, a root phrase is defined as a root word with a prefixof n-words which have “compound”, “modifier of nominal”
or “appositional modifier” dependency relation s with theroot word, and the part of speech (POS) of the words inroot phrase must be “NOUN”. For example, for a given
CaLiGraph class name “Opera house in Puerto Rico”, itsroot phrase is “Opera house”. A root phrase is a part of a
Figure 1 Illustration of matching column to CaLiGraph(Wikipedia category). [13]Lexicalroot phrase utilizing Level Order Traversal on dependency
forparsing graph. (4) Root path search . In this paper, a root

traditional ontology mapping solutions, which is often

path is defined as the concatenation of the ancestor nodes’

combined with structural matching. This gave rise toroot word which appears in the path from the given

various traditional systems such as Cat2Ax [9]. Their

CaLiGraph class name node to the top of CaLiGraph’s

lexical matching approach, however, only focuses on thehierarchy. For example, for a given CaLiGraph class nametext's surface form, such as overlapping sub -strings and“Opera house in Puerto Rico”, its root path is “Venue

sharing a textual pattern, which is unable to capture word

Theatre House”. Here we search the ancestor nodes in thesemantics. Lexical and structural matchings have recently

hierarchy of CaLiGraph and find their root words tobeen suggested to be replaced by machine learning; forgenerate the root path. (5) Semantic embedding generation

instance, DeepAlignment [11] and OntoEmma [16] use

for computing cosine similarities, where we utilize

word embeddings to represent classes and compute the

SimCSE to generate embeddings for constructing training

similarity of two classes according to the Euclidean

data. (6) Training data augmentation, where we augment

distance between their word vectors. However, these

the training data for BERT fine-tuning by searchingapproaches either require extensive feature engineering

sibling nodes in the hierarchy of CaLiGraph. (7) BERT

that is ad-hoc and relies on great amounts of annotatedfine-tuning, using the training data, where a suitable

examples for training, or they use classic non -contextual

pre-trained BERT model is chosen and fine -tuned. (8)word embedding models like Word2Vec, which only learns

Combining results, where Random Forests is u tilized toaselect the most plausible type.

globalmatching

CaLiGraph class name. Here we extend a root word to aserves(context-free)

as

the

embeddingfoundation

for

each

word.

Contrarily, pre-trained transformer-based language models

We evaluate our method on the CaLiGraph -DBPedia

such as BERT [2] and SimCSE [8] can learn robust

mapping task, and experimental results show that ourcontextualized text embeddings, and often just need amethod achieves superior results than the baseline model

little amount of training resources to be fine -tuned.

Cat2Ax in terms of Macro-Averaged F1 scores.

Although these approaches excel at many NLP tasks,
ontology mapping has not yet received enough research

2. Related Workattention from them.
In this paper, we propose a new ontology mapping
method that utilizes hierarchies of two knowledge graphs

2.1 CaLiGraph and Cat2Ax
CaLiGraph is a large semantic knowledge graph with ato do type inheritance and exploits semantic embeddingsrich ontology compiled from the DBPedia ontology andto construct training data for BERT fine -tuning to perform

Wikipedia categories and list pages [8]. Since the first

type prediction. Specifically, our method includes theversion released in Oct. 14, 2019, the latest (19th) version

following main steps: (1) Type inheritance, where wewas released in Sept. 21, 2021. The ontology is enriched

predict the DBPedia type based on heuristic rules, whichwith fine-grained value restrictions on its classes that areprovides training samples for distantly supervised learning.

discovered

with

the

Cat2Ax(2) Finding root words. In this paper, a root word is acombination

of

the

ontologicalsingle noun word which holds the most basic meaning of atransformer-based extractors, many CaLiGraph instanceslong noun phrase. For example, for a given CaLiGraph

are retrieved from Wikipedia categories and listings.

approach.

Through

informationaandClasses in CaLiGraph are derived from lists and categories

phrase B that follows A. Every token's first embedding

in Wikipedia [13].

encodes its content, place in the sequence, and the phrase

The Cat2Ax [9] approach has four major steps: (1)it belongs to (A or B). The model’s architecture consists of

Identify candidate category sets that share a textual

several sequential layers with the same design. The

pattern. (2) Find characteristic properties and types for

multi-head self-attention block, which is its core part,

candidate sets and combine them to patterns. (3) Applycomputes a contextual hidden representation of each token

patterns to all categories to extract axioms. (4) Applyby taking into account the whole output of the preceding

axioms to their respective categories to extract assertions.

layer's sequence. The embeddings of th e tokens from the

2.2 Distant Supervision

final layer can be used as the input for a customized

Machine learning methods basically need a collection ofdownstream layer. Pretraining is conducted by minimizing

training data. Manually assigning labels to a collection oflosses on two tasks: Next sentence prediction and masked

documents is a standard method for constructing training

language model. Contrary to traditional non -contextual

data. This method is time- and money-consuming, and ifword embedding techniques, which only give each token

the corpus is vast, produced data are not for models to be

one embedding, BERT may identify many instances of the

trained.

same token. In finetuning, customized downstream layers

Another method to generating training data is distant

are fine-tuned based on a pretrained BERT model. Forsupervision. Distant supervision is a learning scheme infinetuning, it normally needs a relatively small numb er ofwhich training samples are labeled automatically based on

epochs and data samples [2].

certain rules, suitable for situations where training data
construction is costly. Distant supervision’s assumption
for relation typing is that any statement that contains two

2.4 Sentence embedding
Sentenceembeddingtechniques

represent

andsemantic

information

entities that are involved in a relationship may refer tosentencesthat relationship [12].

low-dimensional

theirvectors.

This

helps

the

entire

model

as
in

Distant supervision for semantic typing is an extension

understanding the context, intention, and other nuances inof the paradigm used by [18] for utilizing WordNet tothe entire text. Sent2vec [17] utilizes word n -gram

uncover hypernym (is-a) relations between entities, and isfeatures to produce sentence embeddings. Arora et al.

analogous to the application of poorly labeled data inpropose SIF in which sentences are represented as thebioinformatics [19][20], and in relation extraction whichweighted average of the word embeddings.

has no labeled data [12]. In ou r method for mapping

Recently, Siamese networks and contrastive learning

entities to knowledge graph types, we apply distantframework have been proposed for sentence embedding

supervision to discover mapping between CaLiGraphmodels. SimCSE [8] is a simple contrastive learning

ontology classes and DBPedia types, based on manuallyframework that advanced SOTA records on sentence

constructed rules to generate initial mappings, then extend

embeddings.

the

the

supervised approaches. The unsupervised SimCSE takes

CaLiGraph hierarchy, that are predicted to share the samean input sentence and predicts itself in a contrastive

DBPedia type through dense representations. Based on theobjective, with only standard dropout used as noise. In ourobservations on Wikipedia category/list name structure

work,

and the hierarchy of CaLiGraph classes, we generate four

similarities between candidate root phrases (or root path)rules for entity mapping between CaLiGraph nodes andand DBPedia types.

mappings

tosiblings

anddescendants

in

we

SimCSEutilize

employs

SimCSEforunsupervised

calculating

andcosineDBPedia types.

3. Problem definition
2.3 BERT pretraining and finetuning

Our goal is to predict DBPedia types of Wikipedia

BERT is a contextualized pretrained language model

category and list names that are included in CaLigraph.

built on bidirectional transformer encoders [15]. BothWe formulate this task as a multi -class classification task.

pretraining and finetuning are part of its training paradigm.

Given a pair of ontologies of two knowledge graphs 𝐾

In pretraining, the input consists of a sequence thatand 𝐾 ′ , 𝑂 and 𝑂 ′ , whose named class sets are 𝐶 andincludes a special token [CLS], tokens from one sentence

𝐶 ′ , respectively, and the ontology class hierarchies 𝐻 andA, another special token [SEP], tokens from another

𝐻 ′ of two knowledge graphs. Then the ontology mappingproblem is to determine a mapping M from′

𝐶 where M maps each class 𝑐 ∈ 𝐶

𝐶 tobased on fine-tuning the pre-trained model BERT on′

semantic features. The training process mainly consists of

to a target class 𝑐 ∈𝐶 such that 𝑐 ′ represents the minimum concept thatthesubsumes 𝑐. Table 1 shows the symbols used in our

CaLiGraph ontology class names to find root words. (2)method.

ExtendTable 1. Symbol list.

followingfiveeach root

steps:
word

(1)

Do

POS

taggingofto a root phrase utilizing

dependency parsing graph. (3) Search the ancestor nodes
in the hierarchy of CaLiGraph and use ancestor nodes’
root words to generate root paths. (4) Utilize SimCSE to

SymbolDescription

K

Knowledge graphs

O

Ontology ofknowledge graphs

𝐶 = {𝑐1 , 𝑐2 , … , 𝑐𝑚 }H

embed the root phrase, root path, DBPedia type into thesame space. (5) Calculate the cosine similarity between
the root phrase, root path and DBPedia type, respectively,
and select the most similar DBPedia type according tocosine similarities as training data for BERT fine -tuning.
(6) Augment the training data for BERT fine -tuning

The set of named
classes of O

utilizing sibling nodes which share the same root phrase.

Ontology classhierarchy

classifier.

(7) Use the augmented training data t o fine-tune the BERT
Finally, we utilize Random Forests to select the most
specific DBPedia type among two results generated by

The set of DBPedia
types

𝑇 = {𝑡1 , 𝑡2 , … , 𝑡𝑚 }Type Inheritance and BERT classifier as the final
mapping.

4.2. Type Inheritance

4. Methodology

Heuristic rules. We assume there exists a relation that
there is a unique DBPedia specific type for each

4.1. Model Structure

CaLiGraph node, thus this relation is a mapping. Based on

In this paper, we propose a new ontology mapping
method, which can predict the DBPedia type for a given
CaLiGraph

Rule 1: A DBPedia class node in the hierarchy ofsemantic features. Figure 2 shows the framework of our

DBPedia may have an exactly identical class name in themethod.

hierarchy of CaLiGraph.
model

mainly

class

based

containson

two

structural

rules to do type inheritance.

and

Our

ontology

this assumption, we construct the following four heuristic

parts:(i)Type

CaLiGraph uses DBPedia as an

upper-level taxonomy and categorizes these rather general

inheritance based on structural features and (ii) Classifier

Table 1 Symbols of problem.

Figure 2 Overall structure of our proposed model.

types in DBPedia into more specific types. We check an
exactly matching CaLiGraph node in the hierarchy forevery DBPedia node. Figure 3 shows an example of Rule1.

Figure 5. Example of multiple inheritance
Rule 3: The depth of DBPedia node in the hierarchy of
DBPedia reflects the specificity degree of the DBPedia
node. For example, as Figure 6 shows, the depth of
DBPedia node “Politician” is larger than the depth of
Figure 3. Example of Rule 1

DBPedia node “Person”, and we can say the type

Rule 2: Inheritance. If a CaLiGraph node c is mapped to“Politician” is more specific than the type “Person”.

a DBPedia type t, then c’s descendants are also mapped tothe same type t. So, if a DBPedia node has an exactly
matching

CaLiGraph

node,

then

its

descendant

CaLiGraph nodes can also be mapped to the DBPedia node .
For example, as Figure 4 shows,

the CaLiGraph node

“20th-century_American_women_politician”

has

an

ancestor node “Politician”, and this ancestor node has anexactly matching DBPedia node “Politician”, so we canassign this DBPedia type “Politician” to the CaLiGraphnode“20th-century_American_women_politician”.

However, a descendant may be inheriting from multiple
ancestor nodes, so this rule will give candidate types but
not a unique type, as Figure 5 shows.

Figure 6. Example of Rule 3
Rule 4: The root word of a CaLiGraph node hassignificant semantic similarity with a DBPedia type. For
example, as Figure 7 shows, the root word of the
CaLiGraph node “Cuban expatriate sportsperson in the
United States” is “sportsperson” which is semanticallysimilar with its DBPedia type “Athlete”.

Figure 4. Example of Rule 2

Figure 7. Example of Rule 4Type Inheritance, Ranking, Selecting.
Given a CaLiGraph node, we first judge whether there
is a path between the CaLiGraph node and also judge

whether there exists a CaLiGraph node in the hierarchy ofdefines the relationships between "head" words and words,

CaLigraph exactly matching with the DBPedia node. If thewhich modify those heads.

path exists, we add this DBPedia node to the candidate

Sometimes only the root word cannot fully represent thesemantic features of CaLiGraph class names. For example,

type list of the given CaLiGraph node.
Then we define a scoring function for ranking candidate

for the CaLiGraph class name “South Korean football club

types: if the candidate type has a score proportional to t heseason”, the root word is “season”, but it is hard for us to

depth in the CaLiGraph hierarchy, denoted as S_depth, andassign a plausible DBPedia type to the CaLiGraph node

if the candidate type contains the root word of the given

only according to the root word . However, for the root

CaLiGraph node, we give it a score denoted as S_root, andphrase "football club season”, we can easily find a

we rank the candidate types by the combination of twoplausible DBPedia type “SoccerClubSeason”.

scores as follows:

For the above reason, we extend a root word to a rootphrase utilizing Level Order Traversal on the dependency

𝑆𝑐𝑜𝑚 = 𝛼 ∗ 𝑆𝑑𝑒𝑝𝑡ℎ + (1 − 𝛼)𝑆𝑟𝑜𝑜𝑡
Here, 0 ≤ 𝛼 ≤ 1 is a hyperparameter. Finally, we selectthe DBPedia type having the highest score as the result tobe assigned to the given CaLiGraph node.

parsing graph. The process includes the following main
six steps:
Step1: Tokenize the CaLiGraph class name and create
an index for each token.
Step2: Find a root word based on POS Tagging.

4.3. Training data construction
POS tagging. SpaCy is a POS tagging and syntactic
dependency parse

tool

[24].

We utilize

the spaCy

Step3: Draw the dependency parsing graph based on theroot word.

processing pipeline to do POS tagging to find the root

Step4: Do level order traversal on the dependency

word of a CaLiGraph node. For a long noun phrase, theparsing graph and judge if the token satisfies a condition.

root word holds the most basic meaning of the whole

If it satisfies the condition, add this token to thephrase.

candidate_token_list.

Firstly, to generate a Doc object, spaCy tokenizes the

Step5:

Afterthetext when we call nlp on a paragraph of text. Next, the Doc

candidate_token_list

object is processed by a few steps which can be seen as thebefore.

processing pipeline. Utilized by the trained pipelines, thepipelinecommonlycontainsatagger,aparser ,a

Step6:

level

order

accordingConcatenate

thetraversal,

to

the

sorted

sort

index

tokensthecreatedinthecandidate_token_list as the root phrase.

lemmatizer and an entity recognizer. Every component in

Figure 9 Shows an example of Level order traversal

the pipeline returns the processed Doc, which is thenalgorithm using FIFO queue on a dependency analysis

forwarded to the following component. Figure 8 shows thegraph.

example of POS tagging.

Figure 8. Example of POS tagging
Extend the root word to root phrase
Dependency parsing, one of the views of linguistic

Figure 9. Example of dependency analysis graph
Another Root phrase extraction strategy

structure, refers to examining the dependencies between

However, while enriching semantic information, root

the words of a sentence to analyze its grammaticalphrases sometimes bring noise. Therefore, we propose astructure. Based on this, we can extract a dependency of adifferent strategy for root phrase extraction. We extract

sentence that represents its grammatical structure anddifferent

lengths of

a

root

phraseasrootphrasecandidates.

sentence embedding models BERT and SimCSE catch

For example, “American football team in Finland” is aattention because of their good performance.

class in CaLiGraph. After we use dependency parsing, w e

However,

can obtain the dependency relationship in Figure 10.

vectorsusing

directly

non-finetuned
for

BERT

unsupervised

senten ce
semantic

similarity calculation has poor performance. The
similarity of BERT sentence vectors of any two
sentences is quite high, where one of the reasons isthenonlinearity

andsingularity

of

its

vectordistribution.
SimCSE reflects the semantic relatedness between
Figure 10. “American football team in Finland” after

phrases when using similarity measures on thedependency parsing.

corresponding vectors. The unsupervised SimCSE

In the diagram, the “team” forms the head of the above

model is greatly outperforming previous SOTA

sentence. The dependency relationship between any twomodels.

words is represented with arrows. For instance, “American”

By observing CaLiGraph node class names, we

is an adjectival modifier of root word “team”. In this paper,

find that the roo t word of a class name has a highwe start from the root word, then traverses the left andsemantic similarity to its corresponding DBPedia

right

the

type. However, just a root word is not enough, sincedependency tree. The words in the left and right subtrees

subtrees

of

the

root

word

accordingtothe root word only is lacking contextual information.

are added to the root phrase candidate set with the root

So, we search the path from CaLiGraph node to theword. In the above sentence, the left subtrees of the root

top of the hierarchy and do POS tagging for the se“team” are “American” and “football”, and the rightancestor nodes in the path to generate a root path tosubtree is the preposition “in”. We first add the root word

capture contexts. However, some ancestor nodes in“team” in the root phrase candidate set. Then we combine

the top of hierarchy are too general, so that making“American” and “team” and add the phrase “American

them meaningless. We set a maximum token number

team” to the candidate set, as is “foo tball”. Since

to drop meaningless root words. In this paper we set“American” and “football” are directly modif ying “team”,

the maximum token number as 3 as a boundary. If the

we should combine these three words (“American footballlength of a root path is larger than this boundary, weteam”) as a root phrase. For the left subtree, “in” is adrop the root word beyond the boundary. If the

preposition which has no sense, and we should not add it

length of the root path is smaller than the boundary,to the root phrase candidate set, but “in” and the following

keep it as it is. As Figure 11 shows, the root path ofobject “Finland” together form a phrase which modifies

the CaLiGraph path is “Venue Theatre House”, the“team”. Thus, “team in Finland” should be added in thetokens “Thing” and “Building” are dropped because

root phrase candidate set. Finally, for this sentence, thethey are beyond the boundary.

root phrase candidate set contains “team” , “American
team”, “football team”, “American football team” and“team in Finland”. For each CaLiGraph class, we extractroot phrases as described above.
Using SimCSE to generate node representation.
Sentence
relatedness

embeddingvia

themodelscapturedistancessemantic

betweenthecorresponding vector representations within the

Figure 11. Example of root path generating

shared vector space. Because there is a high semantic

In this paper, we utilize the unsupervised SimCSE

similarity

between

root

words

(phrases)and

DBPedia types, we use a sentence embedding model

model to generate vectors for both root phrases and root
paths of CaLiGraph nodes and DBPedia types.

to rank the candidate types by measuring their

Cosine similarity computing. The cosine similarity

distance to the root words (phrases). Recently, twobetween a root phrase of CaLiGraph node embedding
𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 and a DBPedia node embedding 𝑉𝐷𝐵𝑃 iscalculated for ranking, as follows:

𝑆𝑖𝑚(𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 , 𝑉𝐷𝐵𝑃 ) = 𝐶𝑜𝑠(𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 , 𝑉𝐷𝐵𝑃 )=The

cosine4.5. BERT Fine-tuning
Given sets of pairs of CaLiGraph nodes and DBPedia⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗
𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 × ⃗⃗⃗⃗⃗⃗⃗⃗⃗
𝑉𝐷𝐵𝑃⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗|𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 | × |𝑉
𝐷𝐵𝑃 |downstream classifier is finetuned on the objective of the

CaLiGraph node embedding 𝑉𝑟𝑜𝑜𝑡_𝑝𝑎𝑡ℎ and DBPedia nodecross-entropy loss. A pair of a tokenized CaLiGraph node

embedding 𝑉𝐷𝐵𝑃 is calculated in the same way. The cosine

and DBPedia node label with a maximum length of 85 issimilarity between root phrase and DBPedia node isan input sequence for the BERT model. The classifier has

𝑆𝑖𝑚(𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 , 𝑉𝐷𝐵𝑃 ) , the cosine similarity between aa linear layer (with dropout) that receives the embedding

root

as

of the [CLS] token from the outputs of the final layer of𝑆𝑖𝑚(𝑉𝑟𝑜𝑜𝑡_𝑝𝑎𝑡ℎ , 𝑉𝐷𝐵𝑃 ). We calculate the weighted cosine

BERT as input, and before applying the o utput softmax

similarity, with weighting factor 𝛽,layer, and the output is convertd into a 2-dimensional

andbetween

DBPediathesemantic features, a pre-trained BERT model with a
of

path

similarity

type nodes generated by the previous method based on

node

is

root

path

denotedas:𝑆𝑖𝑚𝑤𝑒𝑖𝑔ℎ𝑡𝑒𝑑 = 𝛽 ∗ 𝑆𝑖𝑚(𝑉𝑟𝑜𝑜𝑡_𝑝ℎ𝑟𝑎𝑠𝑒 , 𝑉𝐷𝐵𝑃 ) + (1 − 𝛽)vector. The Adam algorithm is used to perform the∗ 𝑆𝑖𝑚(𝑉𝑟𝑜𝑜𝑡_𝑝𝑎𝑡ℎ , 𝑉𝐷𝐵𝑃 )optimization. The final output is vectors of a probability

We rank DBPedia types according to the weighted
cosine similarity scores and select the DBPedia type which

distribution over the DBPedia types for each CaLiGraphnode.

has the highest weighted cosine similarity to the given
CaLiGraph node.

4.6. Result combination

4.4. Training data augmentation

by Type Inheritance and BERT Classifier introduced

We can obtain two DBPedia type candidates generated
In the hierarchy of CaLiGraph, nodes sharing the same
parent node are regarded as sibling nodes. The sibling

before. We utilize the machine learning method Random
Forests to select an optimum one.

nodes which share the same root phrase are likely to have

Numerous classification trees are grown in Random

the same DBPedia type, therefore we search sibling nodes

Forests. Here we denote a CaLiGraph node embeddingof the CaLiGraph nodes in the tr aining data constructed

representation

before,The

embedding representation by Type Inheritance as B, theaugmentation mainly contains the following three steps: (1)predicted DBPedia type representation embedding by the

Search all sibling nodes of a CaLiGraph node in the

BERT classifier as C. Then We utilize SimCSE to do thehierarchy of CaLiGraph. (2) For each sibling node, judge

concatenation of the embedding [A; B] as 𝐸1 and [A; C] as

whether it contains the root phrase of the given CaLiGraph

𝐸2 , and use 𝐸1 and 𝐸2 as the input vectors. We refer tonode. If so, we assign the same DBPedia type to theeach tree's categorization as a “vote” for that class. The

sibling node. (3) Add the sibling node -DBPedia type pairs

categorization with the highest vote s is selected by theto the training data as data augmentation. Figure 14 shows

forest (over all the trees in the forest). Each tree is grownan example of augmentation: We have assigned theas follows:todotraining

data

augmentation.

as

A,

the

predicted

DBPedia

type

DBPedia type “BasketballPlayer” to the CaLiGraph class(1) If the number of cases in the training set is N,

name “Virginia Tech Hokies women’s basketball player”

sample N cases at random but with replacement, from theutilizing the methods introduced before, then we searchoriginal data. This sample will be the training set for

sibling nodes in the hierarchy and find three sibling nodes

growing the tree.

as shown in Figure 14. The root phrase of sibling node(2) We use the dimension of the embeddings as M

“Penn State Nittany Lions softball player” is “softball

input variables, a number m << M is specified such that at

player” which is different from the root phrase ofeach node, select m variables randomly out of M and the“Virginia Tech Hokies women’s basketball player”, so webest split on these m is utilized to split the nodes.

drop it, and other two sibling nodes share the same root

Throughout the growth of the forest, m is maintained at aphrase “basketball player” with CaLiGraph class namefixed value.

“Virginia Tech Hokies women’s basketball player.” Thus
we assign the same DBPedia type “BasketballPlayer” tothe sibling nodes as augmentation.

(3) Every tree is developed to its full potential.
Pruning is not done.
To construct training data for Random Forests training,we annotate labels to a small subset of CaLiGraph nodes

on the CaLiGraph -DBPedia mapping task. We use

by human annotation. Finally, we enter the training set

Macro-Averaged Precision, Macro -averaged recall,into Random Forests to obtain the final prediction.

and Macro -Averaged F1 as main evaluation metrics.
The formulas for calculating these metrics are as

5. Experiments

follows:

In this section, we first introduce datasets used in ourwork, followed by evaluation metrics. Then, we present
anddiscussourexperimental

resultson

𝑃=|𝑀𝑜𝑢𝑡 ∩ 𝑀𝑡𝑒𝑠𝑡 ||𝑀𝑜𝑢𝑡 |

(1)

𝑅=

|𝑀𝑜𝑢𝑡 ∩ 𝑀𝑡𝑒𝑠𝑡 ||𝑀𝑡𝑒𝑠𝑡 |

(2)

2∗𝑃∗𝑅
𝑃+𝑅

(3)

the

CaLiGraph-DBPedia mapping task.

5.1. Datasets
Dataset construction. For BERT classifier training, we

𝐹1 =first randomly sample a small subset of all CaLiGraphnodes as the test set, and we annotate the golden label to
the CaLiGraph nodes in the test set by human annotation
as the ground truth. Then we randomly sample a subset of
all CaLiGraph nodes as the training dataset, and wechecked whether the CaLiGraph node in the test set appear
in the training dataset. If it appears, we remove it from thetraining dataset to make sure that the CaLiGraph nodes inthe training dataset are disjoint with the CaLiGraph nodes
in the test set. Then we split the training dataset to 80% as
a training set and 20% as a validation set.
For Random Forests training, we use the sam e test set

5.3. Experimental Settings
For the BERT classifier, we choose the pretrained
bert-base-multilingual-cased

the training set for Random Forests training. We also
check whether each CaLiGraph node in the test set appears
in the training set. If the node appears, we remove it fromthe training set.

For the Random Forests, n_estimators (number ofclassification

trees)=

2000,

criterion

as

gini,

utilizingthemax_features = sqrt(n_features).

5.4. Experimental Results and Analysis
Table 3 shows the results of the experiment s.
From

the

results

we

can

see

that

combination of semantic feature and structural feature to
predict DBPedia types for CaLiGraph nodes outperforms
the method which only utilizes the semantic feature tofine-tune the BERT Classifier. The reason may be that weonly the root word cannot accurately represent

the semantic feature of the CaliGraph node and cause

We evaluate the model performanceofthe input max length = 85, learning rate = 1e -5.

node, but

5.2. Evaluation Metrics

Table 2 Statistics of dataset.

For BERT Classifier

CaLiGraph-DBPe
diadetails

use root words to represent meaning of each CaliGraph

Table 2. shows statistics of the dataset .

DatasetThe

training are that epoch = 10, the training batch size = 16,

constructed above as the test set for Random Forests, andwe randomly sample a small subset of CaLiGraph nodes asmodels.

For Random Forests

Number of training
set

Number of validation
set

Number of training
set

142758

35689

5186

Number of testset500Models

Macro-Averaged
Precision

Macro-averaged recall

Macro-Averaged F1

Cat2Ax

0.593

0.607

0.599

0.589

0.608

0.588

0.649

0.621

0.635

0.726

0.708

0.707

0.665

0.633

0.649

0.835

0.826

0.830

0.884

0.891

0.883

SimCSE+BERT(root word)
SimCSE+BERT(root phrase)
SimCSE+BERT(root phrase withdifferent length)
SimCSE+BERT(root phrase and siblingnodes)
Type Inheritance+SimCSE+BERT(root word)Type Inheritance+SimCSE+BERT(root phrase and siblingnodes)

Table 3 Experiment results.
Model SimCSE+BERT (root word) uses the root word of CaLiGraph class names to compute cosine
similarity for training data construction. Model SimCSE+BERT (root phrase) extends the root word toroot phrase to compute cosine similarity for training data construction. Model SimCSE+BERT (rootphrase and sibling nodes) uses the root phrase for training data construction and searches the siblingnodes for training data augmentation. Model Type Inheritance+SimCSE+BERT (root word) combines
the results of type inheritance and BERT classifier utilizing root word. Model Typemisunderstanding
and errors on predicting
the DBPedia
For
example, the resultsfor
thetype CaLiGraph
node
Inheritance+SimCSE+BERT(root phrase
and sibling nodes)
combinesofinheritancetype. and
Therefore,
tocontainmore
usefulinformation,
we“San_Francisco_Deltas_player”,
San
Francisco
Deltasis
BERT classifier utilizing root phrase and sibling nodes for data augmenta tion.
extend root words to root phrases to better represent thean American professional soccer team, but only by classsemantic feature of

CaLigraph nodes. From the result wename we cannot find this information, so the predicted

can see that the root phrase method outperforms the root

result of our model is “AthleticsPlayer,” which is notword method. However, some root phrase s will bringspecific enough. (2) Class name is too short. A number of

meaningless words which can be seen as noise. To reduce

CaLiGraph class names are too short to have enoughthe influence of noise, we further tried different length of

information for prediction. For example, the CaLiGraphroot phrases and select the optimum one.

class name “Molluscicide” is a kind of pesticide, but

We also do training data augmentation to increase

according to semantic similarity, the predicted result given

training data. Although this augmentation can amplify

by our model is “Mollusca”, which is wrong. (3) Part ofnoise in the orignal training data, the result shows a n

speech error. According to the semantic feature of class

improvement on F1 score.

names, we sometimes obtain a semantically related
prediction result but its part of speech is not matching

5.5. Analysis on Errors and Difficulties

with the node. For example, for the CaLiGraph node

After checking the experimental results, we find that“1970s crime”, the most semantically similar DBPedia

our model’s final prediction has about 10% rate of errors.

type is “Criminal”, but the crime is an event, while

We compared predicted DBPedia types with golden labels

criminal is a person, which is obviously

wrong.

of the error CaLiGraph nodes, and find that there are

Due to the above errors, we need to find more

mainly three kinds of CaLiGraph class names which are

information such as Wikipedia category pages which candifficult to predict DBPedia types: (1) Proper noun.

indicate corresponding DBPedia types.

Certain CaLiGraph class names only have proper noun,

6. Conclusion and Future Work

this make us hard to find the corresponding DBPedia type.

In this paper, we proposed a novel ontology mappingmethod based on structural and semantic features. We
construct four heuristic rules to do

Type Inheritancebased on the hierarchies of CaLiGraph and DBPedia. We
use spaCy processing pipeline to do POS tagging to findroot words of CaLiGraph nodes. Then we utilize adependency parsing graph based on root word s and do theLevel Order Traversal by FIFO queue on the dependency
parsing graph to extend the root word to a root phrase. We
use SimCSE to generate embeddings of a root phrase androot path of CaLiGraph nodes and compute the cosine
similarity between them and DBPedia types, and rank the
DBPedia type candidates and select the most similar one
to construct the training data for BERT Classifier
finetuning.

We augment the training data by searching

sibling nodes in the hierarchy of CaLiGraph. We further
combine the predictions of Type Inheritance and BERTClassifier to generate the final result by Random Forests.
Our experiments show that our method is outperforming
Cat2Ax on the CaLiGraph-DBPedia mapping task.
In future work, we intend to apply masked language
model to predict phrases which indicate the DBPedia types
and design a ranking method to rank the candidate
DBPedia

types.

Furthermore,

for

a

small

part

of

CaLiGraph nodes, we cannot predict specific DBPedia
types only by class names,

because class names are

sometimes too short to contain much useful information.
Therefore, we try to find more information besides theclass names, such as looking up useful keyphrase s in
Wikipedia categories and list pages .

References[1] Biswas R, Sofronova R, Sack H, et al. Cat2type:
Wikipedia category embeddings for entity typing inknowledge graphs[C]//Proceedings of the 11th on
Knowledge Capture Conference. 2021: 81 -88.
[2] Devlin J, Chang M W, Lee K, et al. Bert: Pre -trainingof deep bidirectional transformers for language
understanding[J]. arXiv preprint arXiv:1810.04805,
2018.
[3] Gao T, Yao X, Chen D. Simcse: Simple contrastive
learning of sentence embeddings[J]. arXiv preprint
arXiv:2104.08821, 2021.
[4] He Y, Chen J, Antonyrajah D, et al. BERTMap: A
BERT-based
ontologyalignment
system[C]//Proceedings of the AAAI Conference on
Artificial Intelligence. 2022, 36(5): 5684 -5691.
[5] He Y, Chen J, Antonyrajah D, et al. Biomedical
ontology alignment with BERT[J]. 2021.
[6] Heist N, Paulheim H. Entity extraction from
Wikipedia list pages[C]//European Semantic Web
Conference. Springer, Cham, 2020: 327 -342.
[7] Heist N, Paulheim H. Information extraction fromco-occurring similar entities[C]//Proceedings of the
Web Conference 2021. 2021: 3999 -4009.
[8] Heist N, Paulheim H. The CaLiGraph ontology as achallenge for OWL reasoners[J]. arXiv preprint
arXiv:2110.05028, 2021.
[9] Heist N, Paulheim H. Uncovering the semantics of
Wikipedia categories[C]//International semantic web
conference. Springer, Cham, 2019: 219-236.
[10] Ho T K. Random decision forests[C]/ /Proceedings of
3rd international conference on document analysis
and recognition. IEEE, 1995, 1: 278 -282.
[11] Kolyvakis P, Kalousis A, Kiritsis D. Deepalignment:
Unsupervised ontology matching with refined word
vectors[C]//Proceedings of the 2018 Conference o f
the North American Chapter of the Association for
Computational
Linguistics:
Human
LanguageTechnologies, Volume 1 (Long Papers). 2018:787-798.
[12] Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009).
Distant supervision for relation extraction withoutlabeled data.
[13] QIN J, IWAIHARA M. Annotating Column Type
Utilizing BERT and Knowledge Graph Over
Wikipedia Categories and Lists[J].
[14] Svetnik V, Liaw A, Tong C, et al. Random forest: aclassification and regression tool for compoundclassification and QSAR modeling[J]. Journal ofchemical information and computer sciences, 2003,
43(6): 1947-1958.
[15] Vaswani A, Shazeer N, Parmar N, et al. Attention is
all you need[J]. Advances in neural information
processing systems, 2017, 30.
[16] Wang L L, Bhagavatula C, Neumann M, et al.
Ontology alignment in the biomedical domain using
entity definitions and context[J]. arXiv preprint
arXiv:1806.07976, 2018.
[17] Moghadasi, M. N., & Zhuang, Y. (2020, December).
Sent2vec: A new sentence embedding representation
withsentimental
semantic.
In
2020
IEEE
International Conference on Big Data (Big Data) (pp.
4672-4680). IEEE.
[18] Snow, R., Jurafsky, D., & Ng, A. (2004). Learning
syntactic patterns for automatic hypernym discovery.
Advances in neural information processing systems,
17.
[19] Craven, M., & Kumlien, J. (1999, August).
Constructing biological knowledge bases byextracting information from text sources. In ISMB
(Vol. 1999, pp. 77-86).
[20] Morgan, A. A., Hirschman, L., Colosimo, M., Yeh, A.
S., & Colombe, J. B. (2004). Gene nameidentification and normalization using a modelorganism database. Journal of biomedical informatics,
37(6), 396-410.
[21] https://en.wikipedia.org/[22] https://www.DBPedia.org/resources/ontology/[23] http://caligraph.org/statistics.html[24] https://spacy.io/usage/processing -pipelines