DEIM Forum 2023 1b-9-1

Fine-grained Column Type Annotation Using Multiple Knowledge
GraphsShen SHUYANG† and Mizuho IWAIHARA‡† ‡Graduate School of Information, Production and Systems, Waseda University
2–7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808–0135 Japan
E-mail: †shua@akane.waseda.jp, ‡iwaihara@waseda.jp
Abstract Tabular data contain rich semantic information. However, identifying and interpreting tables are stillchallenging. Particularly, we look into the column type annotation task as part of table interpretation. Althoughdeep-learning models based on pretrained language models have been achieving state of the art performance in recent years, these column typing results are still not successful in predicting speciﬁc types, because of the limitation
on re-lying only on pretrained language models, which can be improved by incorporating multiple knowledge graphswith extended ontology. In this paper, we propose a hybrid method which combines the advantage of deep-learningand linguistic feature approaches over knowledge graphs. We utilize large-scale knowledge graph CaLiGraph over
Wikipedia lists and categories to create a ﬁne-grained mapping to the target knowledge graph of DBPedia. Our
results were evaluated on the ﬁne-grained subset of WikiTable dataset, and shows advantages on granularity overthe state-of-art model.
Key words Column Type Annotation, Deep-learning，Knowledge Graph, Fine-grain

1 Introduction

knowledge inventory for a wide range of applications, including web search, natural language understanding, data inte-

Web tables (relational HTML table from the Web) are

gration, and data mining. It is also requested to be as com-a kind of tabular data source which can be accessed overplete, correct, and up-to-date as possible to ﬁnish a mapping

the Internet. Twelve years ago, Caferella et al. [4] estimatedfrom the external contents to the concepts or entities.

that the Web contains around 14.1 billion HTML tables. It

The English version of DBpedia [15] describes 6.2M entities

is convincing that an abundant source of tabular knowledge

using 1.1B triples, including 1.6M persons,800K places, 480K

is currently available on the Semantic Web. To fully under-

works (e.g., ﬁlms, music albums), 267K organizations, 293K

stand tabular data, table interpretation solutions with LOD

species, and 5K diseases. Besides, the DBpedia Ontology is(Linked Open Data) are necessary.

a shallow, cross-domain ontology, which has been manually

Column type annotation [1] [7] [5] [23], as an important task

created based on the most commonly used infoboxes withinof table understanding, is the process of identifying and an-

Wikipedia. The ontology currently covers 685 classes whichnotating the data type of each column in the given tabularform a subsumption hierarchy and are described by 2,795

dataset. It is important in data processing because correctly

diﬀerent properties, which maintains hierarchical structure

identifying the data type of each column is crucial for manyat the semantic level, and has been widely used in a variety

downstream tasks, such as triplet extraction and databaseof tasks and ﬁelds.

update [22]. If a column is mistakenly annotated with an-

CaLiGraph [10] [9] [11] is a knowledge graph that uses theother type, any statistical analysis or machine learning mod-encyclopedic structure of Wikipedia to derive its ﬁne-grained

els that are applied to the data may produce incorrect or bi-

type ontology and extract huge amounts of novel entities.

ased results. Therefore, accurately identifying the data types

Additionally, it provides restrictions for many of the derivedof columns is an important step in preparing and cleaning

types to further reﬁne the contained entities. The ontology

data for further processing.

currently covers 1,061,598 types, which oﬀer way too much

LOD such as DBpedia [15], YAGO [13] creates an ontology by deﬁning a series of categories, properties, and rela-detailed information than DBpedia [15], acting as a reliable
external knowledge graph other than DBpedia.

tions between the concepts, data, and entities to establish a

Column type annotation can be done manually, but it is of-cross-domain knowledge graph, which is used as backgroundten time-consuming and error-prone. As a result, automated

(a)

(b)

Figure 1: Two examples of vertical tables annotated semantic types above each column. (a) is a vertical table with multiple
columns, and (b) is one single column with two diﬀerent types annotation.

approaches have been developed to accurately and eﬃciently

and extend the deep-learning part by a hybrid model and im-annotate column types. These approaches can range fromprove its performance with a contrastive learning approach.It

simple rule-based systems to more complex machine learning

should be noted that our approach is agnostic to pretrained

based methods. Rule-based systems [1] [17] [12] [21] [24] often

language models, so diﬀerent pre-trained language models

utilize information from various knowledge bases and makecan be utilized in the deep learing part and knowledge graphthe ﬁnal ranking and predicting part through ensemble ofpart for ﬁne-tuning and embedding generation, respectively.

features. On the other hand, machine learning based meth-

Our main contributions can be summarized below:

ods [28] [5] [25] [7] [23] focus on extracting semantics with lim-

1. Improved the deep-learning part [23] by adding a con-ited table data (like only the surface contents of tables given).

trastive learning module whose training samples are gener-

Besides, most datasets [7] [29] [6] for column type annota-

ated according to the results on conﬁdent samples in eachtion have overlap among the given types (like Director andepoch, which can separate types having overlapping seman-

Artist).

tics with other types.

We focus on the WikiTable dataset [7], which is a collec-

2.

Use an unsupervised method on embeddings overtion of tables collected from Wikipedia. According to [7],type texts and interpretable linguistic features to accom-

the type and relation labels are from FreeBase [2] and areplish mapping between ontologies of multiple knowledge

obtained by aggregating entity links attached to the originalgraphs. Combine the advantage of deep-learning and knowl-

tables. However, FreeBase has been shut down years ago,edge graphs, building a more interpretable, robust, and ﬁne-and the API would not be supported anymore, which makes

grained method to cope with the challenge of column type

it diﬃcult to access the metadata with FreeBase schema toannotation.

obtain ﬁne-grained typing results. To this end, we propose

3.

Create a ﬁne-grained WikiTables dataset.

Since

to utilize multiple knowledge graphs like DBpedia [15] andTURL WikiTables adopts both ﬁne-grained and coarse-

CaLiGraph [10] [9] [11] to harvest ﬁne-grained types.

grained types, which is a multi-label dataset. To further

This paper attempts to show a hybrid model which is de-evaluate the perform on column type annotation, we manu-

signed to exploit the advantages of deep-learning and knowl-

ally curate part of WikiTable dataset and choose one mostedge graphs to perform column type annotation on tabular

speciﬁc type out of the given multiple types as ﬁne-grained

data. As shown in Figure 1, there is one column in table

WikiTables dataset. Evaluation is performed on both TURL(b) that has values identical to table (a), and we consider

WikiTables and ﬁne-grained WikiTables dataset.

the task to assign the column type of (b) as ﬁne-grained

The remainder of this paper is organized as follows. In Sec-as possible to capture the speciﬁc semantics of the target

tion 2 we introduce the related work we investigated. The

table. It is diﬃcult to improve granularity of column andproblem is formulated in Section 3 and the method proposed

obtain ﬁne-grained semantic types in (a). However, the en-to annotate column type is speciﬁcally explained in Section 4.

tities in cells of each column can bring extended informa-

Then, we evaluate our hybrid model with a subset of TURL

tion from corresponding knowledge graphs, thus leading to

WikiTables in Section 5 and make a conclusion in Section 6.

more ﬁne-grained column type annotation in (b) (in this case,basketball team is more speciﬁc and thus appropriate than

2 RELATED WORK

sports team). The main idea is to retrieve ﬁne-grained types
from multiple knowledge graphs by utilizing deep represen-

Column type annotation is a kind of named-entity recog-

tations of the contents of tables. We choose the backbone ofnition, which is a subtask of table interpretation that aims

DODUO [23], which proposes a shared parameter encoder,to locate or categorize named entities in tabular data into

pre-deﬁned concepts. While column type annotation focusesding model based on Word2Vec [19] is the most commonlyon annotating the header of the table, entity linking is aim-used. But recent work [7] [23] are adopting transformer [27]ing to link all the mentions in columns (except the headers)

based models.TURL [7] is a transformer-based pretrainingof a table to or are instantiated from ground-truth concepts

framework for table interpretation. Pre-training contextu-

in a knowledge graph.

alized representations of table contents are learned in an

Approaches based on Markov random ﬁeld model [17] [1]unsupervised way during pre-training and later applied to[14] are ﬁrst introduced into semantic table interpretation by

diﬀerent tasks during ﬁnetuning phase. Moreover, Suhara

Limaye et al. [17]. In this method, Iterative Classiﬁcation Al-

et al. [23] propose shared parameter model DODUO, which

gorithm (ICA) [8] and knowledge graphs are used with latent

allows diﬀerent training tasks to ﬁne-tune the same encoder

variables and some manually constructed potential functions

together, capturing various levels of semantics and generaliz-based on the structural features of tables. These potentials

ing the model. This framework also allows diﬀerent datasets

could connect the components in the tables， jointly con-

to be applied to ﬁne-tune the same pre-trained language

structing a maximal objective function to describe the tablemodel,extending the range of knowledge.

interpretation problem. Such methods based on the Markov
random ﬁeld model were improved by a method proposed

3 PROBLEM FORMULATION

by C.S. Bhagavatula et al. [1] mainly by introducing more

This section gives mathematical formulation of our task.

potential functions and more prior knowledge. Accordingly,

Assuming a target table gives column names and cell val-

the improved method which exhausts the background knowl-

ues that can be linked to its metadata, but without otheredge is computationally expensive to realize better perfor-

information such as the table title, caption and contexts.

mance. Let alone the memory limit for storing that amountWith this restriction, the goal of the proposed task is to pre-of knowledge. Embedding was later introduced by Takeoka

dict semantic types of the table columns and make them as

et al. [25] to exploit the semantic information with the texts

ﬁne-grained as possible. Here the target semantic types are

and terms, which addressed the issue of computational costs

chosen from the DBPedia ontology. With the construction

by translating the knowledge in knowledge graph into vector

schema of the WikiTable dataset, the problem is therefore

representation instead of searching through the knowledge.

formulated as a classiﬁcation task, annotating one of the pre-Compared with the table interpretation method based on

deﬁned semantic types on the target column.

Markov random ﬁeld, which disambiguates all background

Assume that a table T which contains r rows of content

knowledge on the knowledge graph at once, the methods

cells (excluding the row of table headers) and c columns are

based on traditional feature engineering and queries are more

to be interpreted. Symbol mij is called a mention, whichrobust and applicable. Among them, the most important

denotes the cell content of row i and; column j (suppose one

feature engineering and query methods are the candidate en-cell contains only one mention). Ti is the ith row of table T ,tity search that relies on a database, the context modeling

Tj is the j th column of table T . DBpedia knowledge graphof table contents, and extension of information sources [3].

(KG) in this paper can be described by (E, T , P, F ), where

G. Hignette et al. [12] interprets tables through feature con-

E = {e1 , ..., e|E| } is a set that consists of all the entities in the

struction based on similarity, scoring function and database

KG. T = {t1 , ..., t|T | }is the set of entity types in the KG, andsearch. Several researches [21] [24] [26] emphasize the inﬂu-

the types are connected with the relations to form the ontol-

ence of database background knowledge on table interpre-ogy of KG. P = {p1 , ..., p|P| }is the set of properties (whichtation, trying to interpret tables by exploiting the ontology

is used to describe the entity). F = {f1 , ..., f|F | } contains all

and search methods of LOD. Table augmentation method isthe facts where each fact is denoted by a unique RDF triplet

used in [12], [16] to improve the quality of table interpretationin the KG (includes the triplets deﬁned by ontology). For

for these small web tables. Several researches have improved

every table T in the dataset, column type annotation is tothe processing ﬂow [30], [31], using a part of the contents ofassign a type M(T, Tj ) ∈ T to each column in T .

a table as a supplementary explanation to the rest, making
full use of the characteristics of the disordered structure oftables.

4 METHODS
In this section, we ﬁrst introduce the deep-learning part of

There are also works focused on the improvement of word

our hybrid model, describing in detail how contrastive learn-

embedding methods [18] [5] [7] [23], and a number of novel

ing with hard samples are achieved. Then, the features gen-word embedding processes have been proposed, and state-

erated by multiple knowledge graphs are listed individually,

of-art results have been achieved in the evaluation of Chenfollowed by the feature combination method.

et al.’s work [5]. Among these methods, the word embed-

Figure 2: This ﬁgure shows how the deep-learning part o our model is trained using the same BERT encoder layers. Every
training task would contribute to the ﬁne-tuning of the same encoder. Weight of every task could be contralled by adjusting
number of training data.

4. 1 Deep-learning Prediction

Sequencetable (T ) ::= [CLS] m11 ... mr1 [CLS]

The training part of our model is shown in Figure 2, wherethe ﬁrst two tasks represent the tasks described in Suhara etal.’s [23] work. The last task is contrastive learning task we
add to improve the generality of model. It should be notedthat contrastive learning is one of the training progress as
is shown in the ﬁgure instead of ﬁne-tuning before training.
Training methods and contrastive learning methods will be(2)... [SEP ] m1c ... mrc [SEP ]In this case, the column sequences are connected with
[CLS] instead of [SEP ] to calculate an embedding vector
for every classiﬁcation token based on surrounding tokens,which allows the table-wise semantics to be involved in each
[CLS] token of corresponding column. For the ﬁrst two tasks
in Figure 2, table sequence input is encoded to corresponding embedding h. The embedding which represents [CLS]

introduced speciﬁcally.

tokens in the sequence is extracted for type classiﬁcation.

4. 1. 1 Model Training

Let LM donates the pre-trained language model of the enAs mentioned before, the backbone of the deep-learningpart of our model follows the idea proposed by Suhara etal. [23], where the column serialization is implemented in avery simple way by concatenating the content in the cell ofcoder while |Ctype | is the number of types in the dataset. Let
g be the dense layer of dimension d × |Ctype | for the column
type prediction. The deep-learning model would computecorresponding column in order. That is, suppose a column

sof tmax(LM (Sequencetable (T )) · g)

Tj has values T1j , T2j ,..., Tmj , the column could be serialized

Since the Wikitable dataset is a multi-label dataset, binary(3)

cross-entropy loss is applied instead of cross entropy loss toto the sequence as follows

properly calculate the loss.
4. 1. 2 Contrastive learning
Sequencecol (Tj ) ::= [CLS] m1j ... mrj [SEP ](1)

Where [CLS] (classiﬁcation) and [SEP] (separation) are
special tokens that are used to represent the beginning and end of a sentence, respectively.

They areused to mark the boundary between individual sentences
or document segments in the input text.

For exam-ple, the column in Figure 1(b) would be serialized to[CLS] panathinaikos athens montepaschi siena Zalgiris
kaunas [SEP ]. In this way, the problem is converted into asequence classiﬁcation task. Using training data to ﬁne-tune
the BERT model is therefore straightforward.
When it comes to table-wise serialization, the idea is almost th same. Take the entire table T as the input, column
sequences are concatenated in order, constructing the tablesequence

Contrastive learning [20] is a machine learning techniquethat involves training a model to recognize the diﬀerence between two or more input samples. In contrastive learning,the model is presented with two or more input samples andis trained to maximize the diﬀerence between the representations of these samples in the learned feature space.
In the module we propose, every batch for contrastive
learning is composed of one target column, one positive column, and several negative columns. They are all serialized
to a corresponding column sequence and sent into the encoder to obtain the embedding h. The embedding of thetarget embedding h and the positive embedding h+ shouldbe pulled together. Also, h and negative column embeddings
h− should be pushed apart. Accordingly, the loss functionof contrastive learning loss issemantically overlapped type pair will not be taken into conLcl = − logcos(h,h+ )
τ

e
∑N −

i=1 e

−
cos(h,h )iτ(4)where N − is the number of negative samples.
The generation of negative samples is performed by analyzing confusion matrix which is produced in each epoch
over the validation dataset. Confusion matrix compares thepredicted class labels of the model with the true class labels
to determine the number of true positives, false positives,
false negatives, and true negatives. The precision of eachtype in the confusion matrix could vary diﬀerently and somesideration.
Contrastive learning dataset is organized with respect
to the proportion of corresponding types in the validation
dataset. For example, there are two target types Artist andPLayer, where Artist accounts for 0.1 in the validation set
while Player accounts for 0.4 in the validation set. Let N be
the total number of contrastive learning samples. Then thenumber of samples in a batch whose target is Artist will be
0.2N , and the number of samples in a batch whose target isPlayer will be 0.8N .
4. 2 Knowledge Graph Prediction

types could reach a precision of 1. Those types should not
be considered as the types to be optimized. Let the threshold factor be u. Every type whose precision is smaller thanu will be selected as the target type, And the column withcorresponding ground truth target type will be collected astarget columns. Negative samples are also generated according to row where target types are in the confusion matrix.
Let u− be the threshold for negative sample selection. The
false positive types whose ratio in the row exceeds u− will
be collected as the negative samples for the target type.

We utilize Mmultiple knowledge graphs for ﬁnding ﬁnegrained column types from predicted types by the deeplearning model. Acting as an alternative to FreeBase which
is not under maintenance anymore, which makes the typeidentiﬁcatin very hard. And such problem can be converted
to a mapping or alignment task to other trusted knowledge
graphs, which exists ubiquitously in many ﬁelds.

In ourmodel, we try to create a mapping from ontologies of DBpedia and CaLiGraph to the ontology of FreeBase using ranking
result based on linguistic features, which calculates the conﬁdence of mapping between each type pair. The features weselect to perform the mapping and the feature combination
methods will be shown.
4. 2. 1 Entity Linking
As is shown in Figure 3, we ﬁnd meta URL e (entity page)of every mention in the column according to the hints in thedataset. After a SPARQL query over these entities is executed on DBpedia and CaLiGraph, statistics is collected onthese queried types, recording their count for the following

Figure 3: The features of the surfaces are extracted infeature computation.
Below are the main SPARQL queries used in our model

diﬀerent tasks. Weight of every task could be contralled byadjusting number of training data. m11 , m21 and m31 are

SELECT DISTINCT s t r ( ? mtype ) a s ? mtype

mentions in table T

WHERE {
e n t i t y r d f : type ? mtype .

In TURL WikiTables, for example, one column could
have two ground truth types sport.athlete and basket-}
FILTER ( s t r s t a r t s ( s t r ( ? mtype ) ,’ http :// c a l i g r a p h . org / ontology / ’ ) )ball.basketball player. If sport.athlete is classiﬁed to basketball.basketball player, the positive sample is the column with

SELECT DISTINCT s t r ( ? mtype ) a s ? mtype

type sport.athlete, and the negative sample should include

WHERE {

type basketball.basketball player. Mutual inclusion case wille n t i t y r d f : type ? mtype .

not be further processed because the batch is constructed}

randomly and the quantity of TURL WikiTables is large

FILTER ( s t r s t a r t s ( s t r ( ? mtype ) ,enough to avoid the duplicates. Besides, since the trainingis accomplished on TURL WikiTables, where the misclassiﬁcations mostly exist between non-overlapped types, the’ h t t p : / / dbpedia . o r g / o n t o l o g y / ’ ) )4. 2. 2 Feature SelectionIn order to pass the information from ontologies of CaLiGraph and DBpedia to FreeBase, several features need to

If the number of terms in s is greater than one, the results
will be averaged.

be designed to achieve the mapping. Since DBpedia on-

For every type collected by querying DBpedia and CaLi-

tology well maintains semantic hierarchy and relationship

Graph, levenshtein similarity is performed on all the Free-

among entities in the KB, types in this ontology form a tree

Base types provided by the WikiTable dataset. Let the out-

structure according to the thickness of the granularity. Thisput be ls, which is of dimension |th | by |Ctype |, where |th |means that relying on a pure voting mechanism can not se-

is total number of queried type.

lect the optimal column type. This is because certain types

Embedding Similarity: :are superior in numbers, but their coverage is too wide to de-In our proposed method, the ﬁne-tuned deep-learning

scribe the common characteristics of the content in a column,

model is used to predict column types. At the same time,thus several features need to be constructed manually to de-

we also use other plain pre-trained language models to di-liver information. As Figure 3 shows, we select three features

rectly produce the embeddings of types and compared withto capture the information of a knowledge graph: embedding

all the types in FreeBase using cosine similarity to generate

similarity, levenshtein similarity, and majority voting.

conﬁdence.

Similarity between strings:Given two plain texts s1
cosSim(h1 , h2 ) =and s2 , we deﬁne the string similarity SSim(s1 , s2 ) as follows:h 1 · h2||h1 || · ||h2 ||(7)

For every type collected by querying DBpedia and CaLiGraph, embedding similarity is performed on all the FreeSSim(s1 , s2 ) =LevenshteinDistance(s1 , s2 ))1−
sum {length(s1 ), length(s2 )}(5)Base types provided by the WikiTable dataset. Let the output be es, which is of dimension |th | by |Ctype |.
4. 2. 3 Feature Combination

It should be noted that the Levenshtein distance calculation method here uses the calculation method in pythonlevenshtein（注 1）, which treats the ”replace” edit operation differently than the other operations (i.e. with a cost of 2).
We think such change is more reasonable. For example, theStringSimilarity between “abcd” and “dcba” should be 0.25
instead of 0 (if the traditional Levenshtein Distance method
is adopted). Which could retain the relevance of certain texts
when the string order changes。

The ontology of CaLiGraph is rich but complicated, whichindicates that types with few occurences should be ignored.
Compared to CaLigraph, DBPedia has fewer samples, but
the ontology is more organized. Most of the types in DBpedia have standard format labels which describe their general
information. Therefore, a reasonable weighting strategy isrequired.
Let the count statistics vector produced in Entity Linking

Levenshtein Similarity: In our model, tasks are ac-

be c. The ﬁnal conﬁdence is shown as follows

complished using multiple knowledge graphs. Most of thetypes in DBPedia have standard format labels which describe the general information. We can easily access the typeof each entity on local dump and SPARQL endpoint. However, in FreeBase, the type is organized by the category (intype tume.event, time is the category and event is the type),

sof tmax((es + ls) ·

c
)
|c|

(8)

Where |c| is the total count of queried types. The application of softmax could neglect the type with little count.
4. 3 Column Type Annotation

which makes it hard to perform similarity function on thetype phrase. Hence we deﬁne the similarity between type

Assuming the outputs of deep-learning model and knowledge graph method are D and; K respectively (of dimension

phrases as follows:|Ctype |). The ﬁnal ranking function isLSim(s, a) = max SSim(s, ai )
ai ∈a

(6)

argmaxidx∈1,...,|D| ((αD + βK)idx )(9)where a = {a1 , a2 , ..., an } is the preprocessed type phrasewith the removal of stop words and n = |a| is the length ofwhere α, β ∈ [0, 1] with α + β = 1 are the weight param-

preprocessed type phrase. After removing the stop words,

eters. The type with the highest score will be selected as

we tokenize the type phrase to retain the main information

the output result. Since the range of these scores is betweenof it. In this case, the similarity between phrases can avoid

0 and 1, it can also be regarded as the conﬁdence of corre-

the interference of invalid information as much as possible.

sponding type as the column type.

（注 1）：https://github.com/ztane/python-Levenshtein

DODUOTURL WikiTables

R

P

92.69

92.21

92.45

22.07

22.07

38.90

87.63

TURL

Micro F1 Macro F1

Proposed method (dl only)

R

P

Micro F1 Macro F1

-

90.54

87.23

88.86

-

20.89

26.81

20.05

20.05

13.89

18.97

54.15

77.01

33.43 91.93

49.03

-

R

P

93.12

90.57

Micro F1 Macro F1
92.51

-

23.11 23.11

23.11

28.59

38.82

53.68

74.74

Fine-grained
WikiTables
w/o threshold
Fine-grained
WikiTables

86.96

w threshold

Table 1: Performance on WikiTable Dataset with only deep-learning module of the proposed method.

5 EVALUATION
5. 1 Dataset

ters among diﬀerent tasks to improve the accuracy of column annotation, even when dealing with complex and noisy
datasets.
5. 3 Experimental Setting

As the benchmark dataset used in the evaluation,
TURL WikiTable [7] is a collection of tables collected fromWe use the bert-base-uncased as default encoder, andWikipedia, which consists of 580,171 tables in total. The

AdamW optimizer with learning rate as 5e-5, and train

dataset provides both annotated column types and relations

epochs set to be 20, input max length as 32, and batch sizefor training and evaluation. For column type prediction, theas 16. For the combination part α is set to 0.8, and β isdataset provides 628,254 columns from 397,098 tables anno-correspondingly 0.2. For every training epoch in the sharedtated by 255 column types. For column relations, the dataset|
parameter model, |T
samples is oﬀered by the contrastive
10

provides 62,954 column pairs annotated with 121 relation

learning step.

types from 52,943 tables for training. According to [7], the5. 4 Main Resulttype and relation labels are from FreeBase [2] and are obtained by aggregating entity links attached to the original
fulltables. For both tasks, we used the same train/valid/test

Micro F1 Macro F1

splits as TURL [7]. Each column/column-pair allows to havemore than one annotation, and thus, the task is a multi-label
classiﬁcation task.
During the curation on TURL WikiTables dataset, we
check the header text, table name and section title in order.

DODDUO

20.89

28.61

TURL

13.89

18.97

Proposed Method

41.19

46.17

Table 2: Performance on ﬁne-grained WikiTable Dataset
using top 1 prediction.

For example, given two types ﬁlm.actor and tv.tv actor andthe table name is american television star, thentv.tv actor
will be chosen as the ﬁne-grained type. Our ﬁne-grained

In Table 1, with and without threshold indicates diﬀerent

WikiTables dataset includes 617 tables containing 1350

ways to deal with the conﬁdence generated by each model.

columns. Our ﬁne-grained WikiTables dataset it’s only used

When the evaluation is performed without the threshold,as test dataset in evaluation.

only the type with the highest conﬁdence will be selected

5. 2 Baselines

as the result. When we evaluate with the threshold, for any
type whose conﬁdence is greater than a certain threshold will

TURL [7] is a Transformer-based language model (LM),be select as the result, which makes the task multi-output

in which a pre-trained LM is further pre-trained using tabletask.

As result showed in Table 1, with the contrastive

data, making it suitable for tabular data.TURL proposes alearning module being added to the shared parameter model,

novel approach for table understanding, which involves learn-recall increased and procession decreased. That might be

ing a low-dimensional representation of tabular data using

caused by the additional module improving the generality

deep neural networks.

of the model. The ﬁne-grained WikiTables without thresh-

DODUO [23] proposes a method for automatically anno-

old means we only take the top one type predicted by thetating tabular data columns with semantic types using pre-model as the result type. the macro f1 score is remarkably

trained LMs. The proposed method leverages the contextual

improved. But if we use the evaluation with threshold which

informgtiation captured by encoders which share parame-is designed for multi-label output, Doduo is better, whichtakes top K type as the predicted types where K equals to
the number of types whose logit is greater than a speciﬁc
threshold. In the experiment the threshold is set to log 0.5.
As shown in Table 2, After integrating the knowledge[6]graph module, performance for ﬁne-grained WikiTables
dataset without threshold on micro F1 and macro F1 score
both extensively outperform DODUO, which means plain[7]

surface texts provided by knowledge graphs can show noise
brought by embeddings generated by the pre-trained lan-

[8]

guage model can be alleviated.
[9]6 Conclusion
In this paper, we proposed a hybrid model that makes use[10]of deep-learning and knowledge graph-based approaches forannotating table columns. The key contributions we introduce include 1) Used an unsupervised method on embed-[11]

dings over type texts and interpretable linguistic features toaccomplish mapping between ontologies of multiple knowl-[12]edge graphs; 2) combining the advantage of deep-learningand knowledge graphs, to build a more interpretable, robust
and ﬁne-grained typing results to cope with the challenge of[13]

column type annotation.
In the future, we will pay more attention to the performances on macro F1 score of the model and ﬁnish the ab-[14]

lation experiments for knowledge graph module. Since thesemantic hierarchy and relationship are well maintained in
the ontology of DBPedia (which has a limited number of[15]words and terms), the granularity of the result in task mightalso be improved by adding more interpretable features likethe depth of DBPedia ontology, making the results more ﬁnegrained. With embeddings, we can achieve better accuracy
when it comes to correlations at the semantic level, which[16]

makes contrastive learning worth a try in the future. Furthermore, we will try to develop a real-time system based on[17]

the model proposed in this paper.

References[1] Chandra Sekhar Bhagavatula, Thanapon Noraset, andDoug Downey. Tabel: Entity linking in web tables. In International Semantic Web Conference, pp. 425–441. Springer,
2015.
[2] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge,
and Jamie Taylor. Freebase: a collaboratively created graphdatabase for structuring human knowledge. In Proceedingsof the 2008 ACM SIGMOD international conference on
Management of data, pp. 1247–1250, 2008.
[3] Michael J Cafarella, Alon Halevy, and Nodira Khoussainova. Data integration for the relational web. Proceedings of
the VLDB Endowment, Vol. 2, No. 1, pp. 1090–1101, 2009.
[4] Michael J Cafarella, Alon Halevy, Daisy Zhe Wang, Eugene
Wu, and Yang Zhang. Webtables: exploring the power oftables on the web. Proceedings of the VLDB Endowment,
Vol. 1, No. 1, pp. 538–549, 2008.
[5] Jiaoyan Chen, Ernesto Jiménez-Ruiz, Ian Horrocks, and

[18]

[19]

[20]

[21]

[22]

Charles Sutton. Colnet: Embedding the semantics of web
tables for column type prediction. In Proceedings of the
AAAI Conference on Artiﬁcial Intelligence, Vol. 33, pp.
29–36, 2019.
Vincenzo Cutrona, Federico Bianchi, Ernesto Jiménez-Ruiz,
and Matteo Palmonari. Tough tables: Carefully evaluating
entity linking for tabular data. In International Semantic
Web Conference, pp. 328–343. Springer, 2020.
Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu.
Turl: Table understanding through representation learning.
arXiv preprint arXiv:2006.14806, 2020.
Lise Getoor. Link-based classification. In Advanced methods
for knowledge discovery from complex data, pp. 189–207.
Springer, 2005.
Nicolas Heist and Heiko Paulheim. Uncovering the semantics of wikipedia categories. In International semantic web
conference, pp. 219–236. Springer, 2019.
Nicolas Heist and Heiko Paulheim. Entity extraction fromwikipedia list pages. In European Semantic Web Conference, pp. 327–342. Springer, 2020.
Nicolas Heist and Heiko Paulheim. Information extraction
from co-occurring similar entities. In Proceedings of the Web
Conference 2021, pp. 3999–4009, 2021.
Gaëlle Hignette, Patrice Buche, Juliette Dibie-Barthélemy,
and Ollivier Haemmerlé. Fuzzy annotation of web data tables driven by a domain ontology. In European Semantic
Web Conference, pp. 638–653. Springer, 2009.
Johannes Hoffart, Fabian M Suchanek, Klaus Berberich,and Gerhard Weikum. Yago2: A spatially and temporally
enhanced knowledge base from wikipedia. Artiﬁcial Intelligence, Vol. 194, pp. 28–61, 2013.
Yusra Ibrahim, Mirek Riedewald, and Gerhard Weikum.
Making sense of entities and quantities in web tables. In
Proceedings of the 25th ACM International on Conferenceon Information and Knowledge Management, pp. 1703–
1712, 2016.
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch,
Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer,et al. Dbpedia–a large-scale, multilingual knowledge base
extracted from wikipedia. Semantic web, Vol. 6, No. 2, pp.
167–195, 2015.
Oliver Lehmberg and Christian Bizer. Stitching web tables
for improving matching quality. Proceedings of the VLDB
Endowment, Vol. 10, No. 11, pp. 1502–1513, 2017.
Girija Limaye, Sunita Sarawagi, and Soumen Chakrabarti.
Annotating and searching web tables using entities, types
and relationships. Proceedings of the VLDB Endowment,
Vol. 3, No. 1-2, pp. 1338–1347, 2010.
Xusheng Luo, Kangqi Luo, Xianyang Chen, and Kenny Zhu.
Cross-lingual entity linking for web tables. In Proceedingsof the AAAI Conference on Artiﬁcial Intelligence, Vol. 32,
2018.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado,and Jeff Dean. Distributed representations of words andphrases and their compositionality. In Advances in neural
information processing systems, pp. 3111–3119, 2013.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado,and Jeff Dean. Distributed representations of words andphrases and their compositionality. Advances in neural information processing systems, Vol. 26, , 2013.
Varish Mulwad, Tim Finin, Zareen Syed, and Anupam
Joshi. T2ld: Interpreting and representing tables as linked
data. In 9th International Semantic Web Conference
ISWC, p. 25. Citeseer, 2010.
Dominique Ritze, Oliver Lehmberg, Yaser Oulabi, andChristian Bizer. Profiling the potential of web tables foraugmenting cross-domain knowledge bases. In Proceedings

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

of the 25th international conference on world wide web, pp.
251–261, 2016.
Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang,Çağatay Demiralp, Chen Chen, and Wang-Chiew Tan. Annotating columns with pre-trained language models. In Proceedings of the 2022 International Conference on Management of Data, pp. 1493–1503, 2022.
Zareen Syed, Tim Finin, Varish Mulwad, Anupam Joshi,
et al. Exploiting a web of semantic data for interpreting tables. In Proceedings of the Second Web Science Conference,2010.
Kunihiro Takeoka, Masafumi Oyamada, Shinji Nakadai,
and Takeshi Okadome. Meimei: An efficient probabilistic
approach for semantically annotating tables. In Proceedingsof the AAAI Conference on Artiﬁcial Intelligence, Vol. 33,
pp. 281–288, 2019.
Zareen Syed Varish Mulwad, Tim Finin and Anupam Joshi.
Using linked data to interpret tables. In Proceedings ofthe the First International Workshop on Consuming Linked
Data, November 2010.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia
Polosukhin. Attention is all you need. In Advances in neural
information processing systems, pp. 5998–6008, 2017.
Daheng Wang, Prashant Shiralkar, Colin Lockard, Binxuan
Huang, Xin Luna Dong, and Meng Jiang. Tcn: Table convolutional network for web table interpretation. In Proceedingsof the Web Conference 2021, pp. 4020–4032, 2021.
Dan Zhang, Yoshihiko Suhara, Jinfeng Li, Madelon Hulsebos, Çağatay Demiralp, and Wang-Chiew Tan. Sato: Contextual semantic type detection in tables. arXiv preprint
arXiv:1911.06311, 2019.
Ziqi Zhang. Towards efficient and effective semantic tableinterpretation. In International Semantic Web Conference,
pp. 487–502. Springer, 2014.
Ziqi Zhang. Effective and efficient semantic table interpretation using tableminer+. Semantic Web, Vol. 8, No. 6, pp.
921–957, 2017.