SHINRA2020-ML

Categorizing 30 language Wikipedia into Extended Named Entity categories

Overview ofSHINRA2020-ML
Motivation
Explainable NLP applications need structured Knowledge BaseThis Task
Categorize entities in 30 language Wikipedia into ENE (name ontology)
using the categorized Japanese Wikipedia and the language links

Resource by Collaborative Contribution
The KB will be created using the outputs of the share-task

Structured KB
Toronto (CITY)
country

Canada…
airportPearson Inter. Airport(AIRPORT)
Location

Pearson Inter.
AirportOntario…
NameOrigin

Lester B Pearson(PERSON)birthday

23 April 1897…
Lester B.
PearsonCareerAmbassador toUS, One of thefounders of UN

Existing KB are very noisy
2019/9/30

NTCIR15 Kick-off

2

Design and Contents• Top-down design• Use a name ontology well designed• Bottom-up population• Populate by Crowdsourcing or by AI

2019/9/30

NTCIR15 Kick-off

3Extended Named Entity(name ontology)

riesogeta219 c
2019/9/30

NTCIR15 Kick-off

4

The TASK
German Wikipedia

Japanese Wikipedia
Family NameCity Name
720K pages categorized
The task is to categorize
all pages without linkfrom JP

2019/9/30?

NTCIR15 Kick-off

316K pages categorized by link

1,946K pages have no link from JP5

The TASK
German Wikipedia

Japanese Wikipedia
Family NameCity NameTEXT CATEGO

720K pages categorized

RIZATION TASK

The task is to categorize
all pages without linkfrom JP

2019/9/30

316K pages categorized by link?

NTCIR15 Kick-off

1,946K pages have no link from JP
6

Target Languages
Language

Number ofUsers

Number ofpages

EnglishSpanishFrenchGermanChineseRussian
Portuguese
ItalianArabicJapaneseIndonesianTurkishDutch
PolishPersianSwedish

35,464,188
5,289,422
3,334,739
3,101,292
2,663,839
2,451,838
2,199,869
1,770,376
1,611,381
1,432,174
1,027,019
1,021,218
970,607934,491
795,312652,290

5,790,377
1,500,013
2,074,648
2,262,582
1,041,039
1,523,013
1,014,832
1,496,975
661,205
1,136,222
451,336
321,937
1,955,483
1,316,130
660,4873,759,167

Pages
with links
from jp510,840
283,539359,783
316,652
290,631
280,565
238,065
304,174
75,773
121,598
118,107223,354248,229
181,710
200,555Link
ratio
8.8
18.9
17.3
14.0
27.9
18.4
23.5
20.3
11.5
26.9
36.7
11.4
18.9
27.5
5.3

Language

Number of
Users

Number ofpages

Vietnamese
KoreanHebrew
Romanian
NorwegianCzech
Ukrainian
Hindi
FinnishHungarianDanishThai
Catalan
Greek
Bulgarian

643,871
549,017484,630
461,670
450,588
440,040
437,029
425,415406,339
403,368
343,249
343,054
312,980
265,153
245,986

1,200,157
439,577
236,984
391,231501,475
420,195
881,572
129,141
450,537
443,060
242,523
129,294
601,473
157,566
248,913

Pages
with links
from jp123,745
210,271
103,137
98,897
144,751
137,144
181,122
31,828156,445
128,71291,81162,441150,829
63,42793,434

Linkratio
10.3
47.8
43.5
25.3
28.9
32.6
20.5
24.6
34.7
29.1
37.9
48.3
25.1
40.3
37.5

Data to bedistributed• Japanese Wikipedia categorized by Extend Named Entity [JSON]• excluding list articles, disambiguation pages, minor pages (less than 5 inter-link)• Language links for 31 language Wikipedias [SQL]• Wikipedia contents in 31 languages• Wikipedia Dump [XML]• Cirrus Search Dump [JSON]• Extend Named Entity Definition (English/Japanese) [JSON]※ The time stamp of All Wikipedia related data is January 20, 2019

Sample Data
Japanese Wikipedia entry [JSON]{"pageid" : 347190,"title" : "バッハ家","ENE_id" : "1.4.3",<- ”Family name”"ENE_score" : 1,
"ENE_annotation" : "HAND.AIP_201904”}

Language links [JSON]{"source" : { "pageid" : 347190, "lang" : "ja", "title" : "バッハ家" }, "destination" : { "lang" : "en", "title" : "Bach family", "pageid" : 713599 } }{"source" : { "pageid" : 347190, "lang" : "ja", "title" : "バッハ家" }, "destination" : { "lang" : "es", "title" : "Familia Bach", "pageid" : 436582 } }{"source" : { "pageid" : 347190, "lang" : "ja", "title" : "バッハ家" }, "destination" : { "lang" : "fa", "title" : " ‫“ﺧﺎﻧدان ﺑﺎخ‬, “pageid" : 4515101 } }{"source" : { "pageid" : 347190, "lang" : "ja", "title" : "バッハ家" }, "destination" : { "lang" : "fr", "title" : "Famille Bach", "pageid" : 813257 } }{"source" : { "pageid" : 347190, "lang" : "ja", "title" : "バッハ家" }, "destination" : { "lang" : "it", "title" : "Bach (famiglia)", "pageid" :2767443 } }Schedule
2019• October:
2020• April:• August:• December:

Dataset distribution(at the final meeting of SHINRA2019-JP)
Registration deadline
Evaluation
ConferenceEvaluation andCommunication• Prepare around 500-1,000 data each below for all languages that
don’t have a link from Japanese Wikipedia• Test Data• Leaderboard Data

• We will prepare …

• Leaderboard• Slack among the participants and the organizers• NTCIR15 page

• http://research.nii.ac.jp/ntcir/ntcir-15/index.html (English)