Resource of W ikipedias in 31 Languages Categorized into Fine-Grained Named Entities (2022)

Abstract: This paper describes a resource of Wikipedias in 31 languages categorized into Extended Named Entity (ENE), which has 219 fine-grained NE categories. We first categorized 920 K Japanese Wikipedia pages according to the ENE scheme using machine learning, followed by manual validation. We then organized a shared task of Wikipedia categorization into 30 languages. The training data were provided by Japanese categorization and the language links, and the task was to categorize the Wikipedia pages into 30 languages, with no language links from Japanese Wikipedia (20M pages in total). Thirteen groups with 24 systems participated in the 2020 and 2021 tasks, sharing their outputs for resource-building. The Japanese categorization accuracy was 98.5%, and the best performance among the 30 languages ranges from 80 to 93 in F-measure. Using ensemble learning, we created outputs with an average F-measure of 86.8, which is 1.7 better than the best single systems. The total size of the resource is 32.5M pages, including the training data. We call this resource creation scheme “Resource by Collaborative Contribution (RbCC)”. We also constructed structuring tasks (attribute extraction and link prediction) using RbCC under our ongoing project, “SHINRA.”

## introduction wikipedia consists of a large volume of entities that are significant resources for the knowledge base (kb) used in many natural language processing (nlp) applications, including question answering, information extraction, and so on. to maximize the use of such a kb, the information in wikipedia has to be categorized and structured in a consistent manner for machines to perform inference, reasoning, and many other purposes. the current categorization and structure of wikipedia and other kb derived from it, such as dbpedia, yago, and wikidata, are extremely noisy for nlp applications. this noise is inherent to wikipedia categories and structures because they are created by multiple independent crowd workers using a bottom-up approach. there are no consistent rules exists for most parts of wikipedia. as a result, instead of the cumbersome wikipedia categories and structures, we must rely on a well-defined ontology. extended named entity (ene) (ene homepage) is one such ontology for named entities @xcite . ene version 8 has 219 hierarchical categories, and a set of attributes is defined for each category. our final goal is to transform wikipedia information into the structure of ene so that machines can use the rich information in wikipedia. this study reports a resource of wikipedias in 31 languages categorized into fine-grained named entity (ene) categories. categorization is the first task in creating a structured kb. after developing the techniques of automatic categorization are developed, we can structure the contents of wikipedia pages in each category. we worked on 31 languages, rather than a few major languages. we created resources for practical nlp applications in various languages, instead of platforming a few major languages. for a top-down designed ontology for named entities, we employed the extended named entity, ene (ene-homepage). ene is a cleanly-designed named entity classification hierarchy that includes the attribute definition for each category @xcite @xcite . it includes 219 fine-grained categories of named entities in a hierarchy of up to four layers. it contains not only the fine-grained categories of the typical ne categories, such as "city" and "lake" for "location", and "company" and "political party" for "organization", but also new named entity types such as "products", "event", and "natural object". these categories cover various entities that are often mentioned in encyclopedias and many other resources. figure 1 shows ene version 8.0. the category "concept" is used for anything that doesn't fit into ene categories; typically common nouns. "ignore" is for wikipedia-specific titles such as "redirect", "disambiguation", and "meta information". attributes were also designed for each category based on the investigation of the sample entities. for example, the attributes for the "airport" category include the following: "reading", "iata code", "icao code", "nickname", "name origin", "number of users per year", "number of runaways", and so on. please refer to the ene homepage for the complete definitions. ## categorized japanese wikipedia we categorized 920k japanese wikipedia pages into one or more of 219 ene categories. for the categorization process, we excluded less popular entities, that is, those having fewer than five incoming links (approximately 151k entities) and nonentity pages (approximately 53k pages), such as common nouns and simple numbers (concept), and wikipeida's meta-information pages and forward pages (ignored). this categorization was done using the machine learning method with a training data manually created @xcite followed by manual validation. the accuracy of the categorization was confirmed as 98.5% by senior annotators on 1,000 sample data. the remaining 1.5% covered ambiguous and difficult ones, even for human annotators. we conducted wikipedia page categorization in 30 languages shared-tasks in 2020 and 2021. we refer to them as shinra2020-ml and shinra2021-ml, respectively. we tackled the problem of categorizing wikipedia entities in 30 languages in fine-grained categories of 219 categories, as defined in ene (ver. 8.0). we selected 30 languages based on the number of active users from wikipedia statistics. the languages are listed in ## related work ontologies and structured kbs have been considered among the most important knowledge resources in nlp. previously, several major projects intended to construct kbs have been undertaken. cyc was one of the earliest projects, followed by more recent wikipedia-based projects, such as dbpedia, yago, freebase, and wikidata. moreover, there are shared tasks aimed at building techniques for knowledge base structuring, such as kbp and conll. here, we introduce these resources and projects and describe the points considered as issues to be solved in these projects. cyc ontology is a large kb constructed of commonsense knowledge @xcite . it was one of the largest ai projects between 1980 and 1990, which mainly used human labor to construct a kb. constructing and maintaining handmade kbs for the general domain is extremely costly. the kb suffered from a "knowledge acquisition bottleneck", which included coverage and consistency problems. dbpedia is a more recent project that constructs structured information from semi-structured data on wikipedia, such as infoboxes and categories @xcite . however, dbpedia is challenged by inaccuracy, low coverage, and lack of coherence. like cyc, infobox and categories in wikipedia are also created by humans; however, they are non-experts in the ontology. the categories are extremely noisy. for example, in the japanese wikipedia, "shinjuku station which is a railway station, has a category, "odakyu electric railway", which is a major railway company using the station. however, a station is not an instance of a railway company; therefore, this is not an appropriate category. this type of category definition (i.e., a topic rather than a hypernym relation) is allowed in wikipedia. there are multiple instances of this in dbpedia, which heavily relies on wikipedia. additionally, several inconsistencies were observed in the category structure. it is not even a hierarchy and there is a loop in the category structure. yago is an ontology constructed by mapping wikipedia articles to wordnet synsets @xcite . similar to dbpedia, yago adopts attribute information extracted from infoboxes because no attribute is defined in the wordnet synsets. freebase is a project that constructs a structured knowledge base using crowdsourcing, similar to wikipedia @xcite . however, when using the crowdsourcing approach, freebase lacks a well-organized ontology. the resource is noisy and lacks coherence because it is created using unorganized crowds. the freebase project was terminated and integrated into wikidata. wikidata aims to be a structured knowledge base based on a crowdsourcing scheme @xcite . it has noise and lacks coherence because it was constructed using a bottom-up approach, similar to wikipedia. fine grained entity recognition (figer) is a project that identifies 112 finely defined named entity classes, similar to ene @xcite . the category in figer is biased and does not have attribute definitions for each category. ## conclusion we reported a resource of wikipedias in 31 languages categorized as enes. japanese wikipedia pages (920k pages) were categorized with an accuracy of 98.5% and 30 languages wikipedia pages (32.5m pages) categorized with 87 f-measure. it was created through a shared-task of wikipedia categorization in 30 languages. we call this resource creation scheme rbcc". we have also been conducting structuring tasks (attribute extraction and link prediction) using rbcc under our ongoing project, "shinra".