Review on Developmentof Asian WordNet
Thai Computational Linguistics Lab., NICT Asia Research Center,and National Electronics and Computer Technology Center(NECTEC)

Virach Sornlertlamvanich

PROFILEHe received the D.Eng. degree from Tokyo Institute of Technology in 1998. He worked with NEC
Corporation as a sub-project leader for Thai language processing in the Multi-lingual MachineTranslation Project. He later founded the Linguistics and Knowledge Science Laboratory (LINKS) to
conduct the research on Natural Language Processing (NLP) in the National Electronics and Computer
Technology Center (NECTEC) of Thailand in 1992. He initiated a wide range of applied NLP projects,
such as ParSit (a web-based English to Thai machine translation service), LEXiTRON (an online ThaiEnglish corpus-based dictionary), and Sansarn (a probabilistic based Thai-English search engine). He
was awarded by the National Research Council of Thailand as the Most Outstanding Researcher of
the Year 2003. His research interests are machine translation, natural language processing, lexicalacquisition, information retrieval and other related fields. He is currently the Co-Director of Thai
Computational Linguistics Laboratory (TCL), NICT Asia Research Center, and the Assistant Executive
Director of National Electronics and Computer Technology Center (NECTEC), Thailand.

virach@tcllab.org
Thai Computational Linguistics Lab., NICT Asia Research Center,and National Electronics and Computer Technology Center(NECTEC)

Thatsanee Charoenporn

Thai Computational Linguistics Lab., NICT Asia Research Center

Kergrit Robkop

PROFILE

thatsanee@tcllab.org
PROFILEkergrit@tcllab.org
Thai Computational Linguistics Lab., NICT Asia Research Center

Chumpol Mokarat

PROFILEchumpol@tcllab.org
National Institute of Information and Communications Technology

Hitoshi Isahara

PROFILEisahara@nict.go.jp

Abstract. This paper describes the approach we used to create Asain WordNet (AWN) from any existing bi-lingual dictionaries. We
found that most of the bilingual dictionaries of a language are paired with the English language. Based on the English equivalents inthe bi-lingual dictionary we estimate the WordNet synset assignment. In general, a term in a bi-lingual dictionary is provided with very
limited information such as part-of-speech, a set of synonyms, and a set of English equivalents. This type of dictionary is comparatively
reliable and can be found in an electronic form from various publishers. In this paper, we propose an algorithm for applying a set ofcriteria to assign a synset with an appropriate degree of confidence to the existing bi-lingual dictionary. We show the efficiency innominating the synset candidate by using the most common lexical information. The algorithm is evaluated against the implementationof Thai-English, Indonesian-English, and Mongolian-English bi-lingual dictionaries. The experiment also shows the effectiveness of using
the same type of dictionary from different sources. The results are reviewed collaboratively online via http://www.tcllab.org and can beviewed on http://www.asianwordnet.org that connects Asian languages through the Princeton WordNet (PWN).
Keywords: WordNet, Asian language, synset assignment, visualization, collaborative tools

276

寄稿集
機械翻訳技術の向上

1 Introduction

5

a bilingual dictionary, a monolingual explanatory
dictionary, and Hungarian thesaurus in theconstruction [5], etc.

The Princeton WordNet (PWN) [1] is one ofThis paper presents a new method tothe most semantically rich English lexicalfacilitate the WordNet construction by using

databases that are widely used as a lexicalthe existing resources having only English

knowledge resource in many research andequivalents and the lexical synonyms. Ourdevelopment topics. The database is divided

proposed criteria and algorithm for application

by part of speech into noun, verb, adjective andare evaluated by implementing them foradverb, organized in sets of synonyms, called

Asian languages which occupy quite different

synset, each of which represents “meaning”

language phenomena in terms of grammars andof the word entry. PWN is successfully

word unit.

implemented in many applications, e.g., word

To evaluate our criteria and algorithm, wesense disambiguation, information retrieval,

use the PWN version 2.1 containing 207,010

text summarization, text categorization, and sosenses classified into adjective, adverb,

on. Inspired by this success, many languages

verb, and noun. The basic building block is

attempt to develop their own WordNets using

a “synset” which is essentially a context-

PWN as a model, for example , BalkaNet

sensitive grouping of synonyms which are linked(Balkans languages), DanNet (Danish),by various types of relation such as hyponym,

Eurowordnet (European languages such as

hypernymy, meronymy, antonym, attributes,

Spanish, Italian, German, French, English),and modification. Our approach is conducted to

Russnet (Russian), Hindi WordNet, Arabic

assign a synset to a lexical entry by considering

WordNet, Chinese WordNet, Korean WordNet

its English equivalent and lexical synonyms.

and so on.

The degree of reliability of the assignment is1

Though WordNet was already used as adefined in terms of confidence score (CS) based

starting resource for developing many languageon our assumption of the membership of the

WordNets, the constructions of the WordNet

English equivalent in the synset. A dictionary

for languages can be varied according to thefrom a different source is also a reliable source

availability of the language resources. Someto increase the accuracy of the assignment

were developed from scratch, and some were

because it can fulfill the thoroughness ofdeveloped from the combination of various

the list of English equivalent and the lexical

existing lexical resources. Spanish and Catalansynonyms.

Wordnets [2], for instance, are automatically

The rest of this paper is organized as follows:

constructed using hyponym relation, aSection 2 describes our criteria for synset

monolingual dictionary, a bilingual dictionary

assignment. Section 3 provides the results

and taxonomy [3]. Italian WordNet [4] is semi-of the experiments and error analysis on Thai,

automatically constructed from definitions inIndonesian, and Mongolian. Section 4 evaluatesa monolingual dictionary, a bilingual dictionary,

the accuracy of the assignment result, andand WordNet glosses. Hungarian WordNet uses

the effectiveness of the complimentary use of

1 List of wordnets in the world and their information is provided at http://www.globalwordnet.org/gwa/ wordnet_table.htm

277

a dictionary from different sources. Section 5

more than one English equivalent with aexhibits the cross language visualization for

confidence score of 4.

Asian WordNet (AWN), and Section 6 concludes
Fig. 1 simulates that a lexical entry L 0 has

our work.

2

two English equivalents of E 00 and E 01. Both

SynsetAssignment

criterion implies that both E00 and E01 are the

A set of synonyms determines the meaning

set of synonyms in S1. Therefore the relativelyof a concept. Under the situation of limited

high confidence score, CS=4, is assigned forresources on a language, an English equivalent

this synset to the lexical entry.

E00 and E01 are included in a synset of S1. The
synset for L0 which can be defined by a greaterword in a bi-lingual dictionary is a crucial key
to find an appropriate synset for the entry
word in question. The synset assignment
criteria described in this section relies on theinformation of English equivalent and synonymof a lexical entry, which is most commonly
encoded in a bi-lingual dictionary.

Fig. 1. Synset assignment with CS=4

Synset Assignment Criteria
Applying the nature of WordNet which
introduces a set of synonyms to define theconcept, we set up four criteria for assigning a
synset to a lexical entry. The confidence score(CS) is introduced to annotate the likelihood

In the above example, the synset, S 1 , isassigned to the lexical entry, L0, with CS=4.

of the assignment. The highest score, CS=4,
is assigned to the synset that is evident toCase 2: Accept the synset that includes more

include more than one English equivalent ofthan one English equivalent from the synonym of

the lexical entry in question. On the contrary,

the target language with a confidence score of 3.

the lowest score, CS=1, is assigned to any

If Case 1 fails in finding a synset thatsynset that occupies only one of the English

includes more than one English equivalent, theequivalents of the lexical entry in question when

English equivalent of a synonym of the lexical

multiple English equivalents exist.

entry is picked up to investigate. Fig. 2 shows

The details of assignment criteria are: Li

an English equivalent of a lexical entry L0 anddenotes the lexical entry, Ej denotes theits synonym L1 in a synset S1. In this case theEnglish equivalent, Sk denotes the synset, andsynset S1 is assigned to both L0 and L1 with∈ denotes the member of a set.

CS=3. The score in this case is lower than the
one assigned in Case 1 because the synonym

Case 1: Accept the synset that includes

278

of the English equivalent of the lexical entry is

寄稿集
機械翻訳技術の向上
indirectly implied from the English equivalentof the synonym of the lexical entry. The

5

In the above example, the synset, S 0 , isassigned to the lexical entry, L0, with CS=2.

newly retrieved English equivalent may not be
distorted.

Case 4: Accept more than one synset that
includes each of the English equivalents with aconfidence score of 1.
Case 4 is the most relaxed rule to provide
some relation information between the lexical
entry and a synset. Fig. 4 shows the assignment

Fig. 2. Synset assignment with CS=3

of CS=1 to any relations that do not meet theprevious criteria but the synsets include one of
the English equivalents of the lexical entry.

In the above example, the synset, S 1 , isassigned to both lexical entries, L0 and L1, with
CS=3.
Case 3: Accept the only synset that
includes the only one English equivalent with a

Fig. 4. Synset assignment with CS=1

confidence score of 2.

Fig. 3. Synset assignment with CS=2

Fig. 3 shows the assignment of CS-2 when
there is only one English equivalent and there is

In the above example, each synset, S0, S1, andS2 is assigned to lexical entry L0, with CS=1.

of the assignment, at the same time there is

3 Experiment Results

no synonym of the lexical entry to distort theWe applied the synset assignment criteria to a

relation. In this case, the only English equivalentThai-English dictionary (MMT dictionary) [6] withshows an uniqueness in the translation that can

the synset from WordNet 2.1. To compare themaintain a degree of confidence.

ratio of assignment for Thai-English dictionary,no synonym of the lexical entry. Though there is
no English equivalent to increase the reliability

we also investigated the synset assignmentof Indonesian-English and Mongolian-English
dictionaries.

279

In our experiment, there are only 24,457

WordNet (synset)

TE Dict (entry)

total

assigned

total

assignedNoun145,103

18,353(13%)

43,072

11,867(28%)

Verb

24,8841,333(5%)

17,669

2,298
(13%)

Adjective

31,302

4,034
(13%)

18,448

3,722
(20%)

Adverb

5,721

737
(13%)

3,008

1,519
(51%)

total

207,010

24,457(12%)

82,197

19,406(24%)

synsets from 207,010 synsets, which is 12%of the total number of the synsets that can be
assigned to Thai lexical entries. Table 1 shows
the successful rate in assigning synsets to
the Thai-English dictionary. About 24 % ofThai lexical entries are found with the English
equivalents that meet one of our criteria.
Going through the list of unmapped lexical
entries, we can classify the errors into three

Table 1. Synset assignment to Thai-English dictionarygroups:1.

2.

We applied the same algorithm to Indonesia-

CompoundThe English equivalent is assigned in a

English and Mongolian-English [7] dictionaries

compound, especially in cases where there

to investigate how it works with otheris no appropriate translation to represent

languages in terms of the selection of English

exactly the same sense. For example,

equivalents. The difference in unit of concept isL:

E: retail shop

basically understood to affect the assignment

L:

E: pull sharplyof English equivalents in bi-lingual dictionaries.

PhraseIn Table 2, the size of the Indonesian-English

Some particular words culturally useddictionary is about half that of the Thai-English

in one language may not be simply

dictionary. The success rates of assignment totranslated into one single word sense in

the lexical entry are the same, but the rate ofEnglish. In this case, we found it explainedsynset assignment of the Indonesian-English

in a phrase. For example,

dictionary is lower than that of the Thai-English

L:

dictionary. This is because the total number of

E: small pavilion for monks to sit on to chantlexical entries is about in the half that of theL:Thai-English dictionary.

E: bouquet worn over the ear
3.

A Mongolian-English dictionary is also evaluated.

Word form
Inflected forms, i.e., plural, past participle,

WordNet (synset)

IE Dict (entry)

are used to express an appropriate sense

total

assigned

total

assignedNoun145,103

4,955
(3%)

20,839

2,710
(13%)

Verb

24,884

7,841
(32%)

15,214

4,243
(28%)

Adjective

31,302

3,722
(12%)

4,837

2,463
(51%)

Adverb

5,721

381
(7%)

414

285
(69%)

total

207,010

16,899(8%)

41,304

9,701
(24%)

of a lexical entry. This can be found in
non-inflected languages such as Thai andmost Asian languages. For example,
L:

E: grieved

The above English expressions cause an error
in finding an appropriate synset.

Table 2. Synset assignment to Indonesian-English dictionary

280

寄稿集
機械翻訳技術の向上

5algorithm are shown in Table 4.

WordNet (synset)

ME Dict (entry)

total

assigned

total

assigned

Table 5 shows the accuracy of synset

Noun145,103268(0.18%)

168

125(74.40%)assignment by part of speech and CS. A small

Verb

24,884240(0.96%)

193

139(72.02%)

Adjective

31,302211(0.67%)

232

129(55.60%)Adverb

5,721

35(0.61%)

4217(40.48%)

total

207,010754(0.36%)

635

410(64.57%)

Table 3. Synset assignment to Mongolian-English dictionary

set of adverb synsets is 100% correctly
assigned irrelevant to its CS. The total numberof adverbs for the evaluation could be too small.
The algorithm shows a better result of 48.7% inaverage for noun synset assignment and 43.2%
in average for all part of speech.
With the better information of English
equivalents marked with CS=4, the assignment

Table 3 shows the result of synset assignment.

accuracy is as high as 80.0% and decreases

These experiments show the effectivenessaccordingly due to the CS value. This confirmsof using English equivalents and synonym

that the accuracy of synset assignment strongly

information from limited resources in assigning

relies on the number of English equivalents in

WordNet synsets.

the synset. The indirect information of English

4 Evaluations

equivalents of the synonym of the word entry is
also helpful, yielding 60.7% accuracy in synset
assignment for the group of CS=3. Others
are quite low, but the English equivalents are

In the evaluation of our approach for synset

somehow useful to provide the candidates for

assignment, we randomly selected 1,044

expert revision.

synsets from the result of synset assignment
to the Thai-English dictionary (MMT dictionary)
for manually checking. The random set covers

Noun

all types of part-of-speech and degrees ofVerbconfidence score (CS) to confirm the approachin all possible situations. According to the
supposition of our algorithm that the set ofEnglish equivalents of a word entry and its

will be correspondent to the degree of CS.
It took about three years to develop theBalkan WordNet on PWN 2.0 [8], [9]. Therefore,
we randomly picked up some synsets that
resulted from our synset assignment algorithm.
The results were manually checked and thedetails of synsets to be used to evaluate our

CS=3

CS=2

CS=1

7

total

479

64

272

822

44

75

29

148

Adjective

1

25

32

58

Adverb

7

4

4

1

16

total

15

552

143

334

1044

Table 4. Random set of synset assignment

synonyms are significant information to relate
to a synset of WordNet, the result of accuracy

CS=4

CS=4

CS=3

CS=2

CS=1

total

Noun

5
306
34
55
400
(71.4%) (63.9%) (53.1%) (20.2%) (48.7%)

Verb

23
6
4
33
(52.3%) (8.0%) (13.8%) (22.3%)

Adjective2(8.0%)

2(3.4%)

Adverb744116(100%) (100%) (100%) (100%) (100%)

total

12
335
44
60
451
(80.0%) (60.7%) (30.8%) (18%) (43,2%)
Table 5. Accuracy of synset assignment

281

CS=4
Noun

CS=3

2

Verb

2

CS=2

CS=1

total

22

29

53

6

4

12

5

Collaborative Review and
Visualization of Asian WordNet

The results of the synset assignment for each

Adjective

language are stored and indexed under KUIAdverb(Knowledge Unifying Initiator) environment for

total

2

2

28

33

65

Table 6. Additional correct synset assignment by otherdictionary (LEXiTRON)

online collaborative review [11]. Contributors
are registered to participate as a supporter of
the translation by voting for the best translationor posting a better translation for each synset.

To examine the effectiveness of English

From the result of the translation, a table for

equivalent and synonym information from a

mapping between sense id and word entry isdifferent source, we consulted another Thai-

created. When there is a request for a pair ofEnglish dictionary (LEXiTRON) [10]. Table 6

languages WordNet expression, the word entry

shows the improvement of the assignment byof the source language will be used to retrieve

the increased number of correct assignment in

the sense id, and then with the sense id theeach type. We can correct more in nouns andtranslated word entry of the target language willverbs but not adjectives. Verbs and adjectives

be obtained. Since each translated word entry

are ambiguously defined in Thai lexicon, andis accommodated with a vote score, the word

the number of the remaining adjectives is too

entry with the highest score will be selected tofew, therefore, the result should be improved

display the current best translation.

regardless of the type.
CS=4
total

CS=3

CS=2

CS=1

total

14
337
72
93
516
(93.3%) (61.1%) (50.3%) (27.8%) (49.4%)

Table 7. Improved correct synset assignment by additional bilingual dictionary (LEXiTRON)

Table 7 shows the total improvement of
the assignment accuracy when we integrated

Table 8. Result of mapped word entry between Thai andJapanese

English equivalent and synonym informationfrom a different source. The accuracy for

Table 8 shows the result of mapped word entry

synsets marked with CS=4 is improved frombetween Thai and Japanese through the sense id

80.0% to 93.3% and the average accuracy

when making a request for a Thai word (is also significantly improved from 43.2% to).

Fig. 5 shows the result of retrieving the) for Japanese equivalents.

49.4%. All types of synset are significantly

Thai word (improved if a bi-lingual dictionary from different

This service can be found at http://www.

sources is available.

asianwordnet.org/. Currently the based PWN isconverted to version 3.0 for better compatibility

282

寄稿集
機械翻訳技術の向上
with other WordNets.

5

accuracy in the assignment. Applying the same

6 Conclusion

criteria to other Asian languages also yielded asatisfactory result. Following the same processthat we implemented for the Thai language,
we are expecting an acceptable result from

Our synset assignment criteria were effectively

the Indonesian, Mongolian languages andapplied to languages having only English

so on. Resulting from the AWN creation, theequivalents and its lexical synonym. Confidence

visualization of AWN across languages canscores were proven efficiently assigned toefficiently serve the request for any pairs ofdetermine the degree of reliability of thelanguages through the PWN sense id.

assignment which later was a key value inthe revision process. Languages in Asia are

References

significantly different from the English language

1. Fellbaum, C. (ed.).: WordNet: An Electronic

in terms of grammar and lexical word units.

Lexical Database. MIT Press, Cambridge,

The differences prevent us from finding theMass (1998)

target synset by following just the English
equivalent. Synonyms of the lexical entry and2. Spanish and Catalan WordNets, http://www.
lsi.upc.edu/~nlp/

an additional dictionary from different sources

3. Atserias, J., Clement, S., Farreres, X., Rigau,

can be complementarily used to improve theG., Rodr?guez, H.: Combining Multiple

Fig. 5. Screen shot of AWN cross language visualization

283

Methods for the Automatic Constructionof the International Conference on Recent「アジアワードネット AWNの開発の省察」の概要

Advances in Natural Language, Bulgaria.

第一著者ウィラット・ソンラートラムワーニッチ博士

of Multilingual WordNets. In: Proceedings(1997)

は、タイ計算言語学研究所 TCL の Co- リーダであり、

4. Magnini, B., Strapparava, C., Ciravegna, F.,

タイ国立電子コンピュータセンター NECTEC の参事でPianta, E.: A Project for the Constructionある。20 年前、通産省 ( 現経産省 ) 所轄の CICC 近隣

of an Italian Lexical Knowledge Base in

諸国間機械翻訳プロジェクトのタイ国チームの副代表でthe Framework of WordNet. IRST Technical

あった。東京工業大学において博士号を取得した日本語

Report # 9406-15 (1994)の分かる自然言語研究者である。

5. Proszeky, G., Mihaltz, M.: Semi-Automatic
Development of the Hungarian WordNet.
In: Proceedings of the LREC 2002, Spain.
(2002)
6. CICC.: Thai Basic Dictionary. Technical
Report, Japan. (1995)

1 はじめにプ リ ン ス ト ン 大 学 WordNet(PWN) は 英 語 に お ける意味情報を最も豊富に含んだ語彙データベースであ7. Hangin, G., Krueger, J. R., Buell, P.D.,

り、語彙の知識源として広く利用されている。このデー

Rozycki, W.V., Service, R.G.: A modern

タベースは品詞（名詞、動詞、形容詞、副詞）で分類

Mongolian-English dictionary. Indiana

されている。その特徴は、同義語 ( シノニム ) をまと

University, Research Institute for Inner

めて語彙の意味分類（“Synset”と呼ばれる）を与Asian Studies (1986)

えたことである。PWN は多くの自然言語応用に利用

8. Tufis, D. (ed.).: Special Issue on the BalkaNet

され成功している。例えば、多義の解消、情報参照、

Project, Romanian Journal of Information

文書要約、文書分類等である。この成功によって各国

Science and Technology, vol. 7, no. 1-2.

語の WN が開発されている。例えば、バルカン言語(2004)

BalkaNet、デンマーク語 DanNet、西・伊・仏・独・

9. Barbu, E., Mititelu, V. B.: Automatic Building

英 語 の EurowordNet、 ロ シ ア 語 RussNet、Hindi

of Wordnets. In: Proceedings of RANLP,

WordNet、Arabic WordNet、Chinese WordNet、

Bulgaria (2005)

Korean WorddNet 等 々 で あ る。（ 訳 注： 今 年 の 2

10. NECTEC. LEXiTRON: Thai-English Dictionary,
http://lexitron.nectec.or.th/

月に独立行政法人情報通信研究機構 NICT から日本語
WordNet が公開された。）

11. Sornlertlamvanich, V., Charoenporn, T., Robkop,

各国語の WordNet は PWN を初期言語資源としてK., and Isahara, H.: KUI: Self-organizing

開発されるが、利用する言語資源によって構築方法が異Multi-lingual WordNet Construction Tool. In:なる。あるものは人手で、また、あるものは種々の言

Proceedings of the Fourth Global WordNet

語資源を用いて開発された。スペイン語とカタール語

Conference (GWC2008), Szeged, Hungary.

の WordNet は全自動であり、上位下位関係、単言語辞(2008)

書、対訳辞書、語彙分類を利用している。イタリア語のWordNet は半自動である。単言語辞書の語義文、対訳
辞書、WordNet の語釈を利用している。ハンガリー語の WordNet は対訳辞書と説明つき単言語辞書、ハンガ

284

寄稿集
機械翻訳技術の向上
リー語のシソーラスを用いている。

5

信度４では 80% の正解率で漸次低下して行く。新たに

この論文では、新たな WordNet 構築法を示す。利別のタイ英対訳辞書情報を追加して割り振ったところ、用する言語資源は、英語対訳辞書と単言語の同義語辞書

確信度４の正解率は 93% になった。（表 7）つまり、である。同じ手法でアジア言語の中で言語的に異質なタ追加情報があれば、品質が向上することが分かった。

イ語、インドネシア語、モンゴル語の WordNet を構築し、評価したのでここに報告する。

WordNet の
5 アジア
共同開発と可視化

2 Synset の割り振り（Knowledge Unifying Initiator）の下でインデック

タイ語の単語見出しに対して、上記の言語資源によス化され、共同利用が進んでいる。図 5 はタイ語と日り Synset を対応させ、4 段階の確信度 Confidence

本語の Synset を介したクロス言語可視化の一画面でScore を与えた。

ある。

基準 1：確信度４( 高い )、1 見出し語に対し複数の英

アジア言語の WordNet は、知識統合支援システムの Synset を共有しているとき。

6 結論

基準３：確信度２、唯一の英語対訳の場合、その英語対

英語対訳辞書と同義語辞書のみによる Synset 割り

語対訳が Synset を共有しているとき。
基準２：確信度３、ターゲット言語の同義語と英語対訳

訳の Synset を割り振る。

振り方式が効果的であることが分かった。さらに確信度

基 準 ４： 確 信 度 １、 複 数 の 対 訳 が そ れ ぞ れ 異 な るを導入することで信頼性が数値化できることが分かっ

Synset を持っているときは全ての Synset を割り振た。

る。

3 実験結果AWN の開発により、PWN の Synset ID を入力す
ると、アジア言語の単語の相互参照と表示が効率的にサービスできるようになった。
（作成：Japio 特許情報研究所）

Synset の総数は 207K であるが、タイ英対訳の見出語数 82K、インドネシア英対訳の見出語数 41K、
モ ン ゴ ル 英 対 訳 の 見 出 語 数 635 に 対 し、 実 際 にSynset が割り振られたのはそれぞれ、19K(24%)、
9K(24%)、410(64%) であった。

4 評価
割り振りのアルゴリズムを評価するために、1,044
語をランダムに抽出して目視チェックで評価した。先ず、サンプリングした語彙の品詞ｘ確信度の表で分類し、その要素に正しかった個数を記入した。（表 5）確285