UZWORDNET : A Lexical-Semantic Database for the U zbek Language (2021)

Abstract: The results reported in this paper aim to increase the presence of the Uzbek language in the Internet and its usability within IT applications. We describe the initial development of a “word-net” for the Uzbek language compatible to Princeton WordNet. We called it UZWORDNET. In the current version, UZWORDNET contains 28140 synsets, 64389 sense and 20683 words; its estimated accuracy is 75.98%. To the best of our knowledge, it is the largest wordnet for Uzbek existing to date, and the second wordnet developed overall.

## introduction by living in the world, we-human 'agents'-and machines as well do not just make meanings up from language independently of the world. this is the language problem @xcite @xcite , and it is crucial for it applications worldwide @xcite . unfortunately, computer scientists and engineers are still learning how to efficiently solve the language problem in their theories or applications, and understand how language-based technologies called "universal language models" work. they are often surprised by the mistakes that new ai tools are making. 1 in short, new technologies proliferate, and language-based biases appear increasingly almost anywhere in applications. a problem of current technologies is that if a language is endangered, it is possible it will never have a life within them-and in the internet on-line. far from infinite, usable technology seems only as big as the language(s) we speak as users. language is just as important for building human connections online as it is offline: it forms the basis of how users identify with each other, the lines on which exclusion and inclusion are often drawn, and the boundaries within which communities grow around common interests. as a consequence, the relationship between language diversity and the internet is a growing area of policy interest and academic study. 2 the story emerging is one where language profoundly affects our experience of the internet. it is a matter of fact, for instance, that google searching for "restaurants" in english may bring us back 10+ times the results of doing so in another language. for "another language", we focus on the northern uzbek language, a turkic language officially recognized as the state language of the republic of uzbekistan. in particular, in this paper we advance and discuss initial results on the ongoing development of uzwordnet (uzw in short), a prototypical version of a wordnet for the uzbek language compatible to princeton wordnet @xcite . 3 our long-term objective is to motivate, support and increase the study of computational aspects of uzbek and, more generally, the usability of uzbek within it applications and the internet. as a consequence, uzword-net is added to the wordnets in the world 4 and provided open source under a license and format compatible with the open multilingual wordnet @xcite . 5 this paper is structured as follows. below are some elements of the uzbek language, followed by a brief excursus on word-nets (section 3). in section 4, we focus on the few previous attempts towards the construction of a wordnet for uzbek. in section 5 we advance and discuss the work that produced uzwordnet. we validate and analyse the results in section 7 and 8. we conclude with a summary and future work (section 9). ## elements of uzbek language unless otherwise stated, in this paper by "uzbek language" (native: o'zbek tili) we refer to the northern uzbek language. in fact, there is another uzbek language-the southern uzbek-statutory language of provincial identity in afganistan, spoken by about 6.5 million people wordwide @xcite . the (northern) uzbek language is a statutory national language in uzbekistan. 6 it is a turkic language and spoken by approximately 26.8 million people around the world @xcite , remarkably by a large group of ethnic uzbeks residing abroad in afghanistan, kyrgyzstan, kazakhstan, turkmenistan, tajikistan, russia, turkey, and xinjiang (china), making it the second-most widely spoken turkic language after turkish (ethnologue, 2020b). figure 1 provides the rough geographical distribution of the northern (majority) and southern (minority) uzbek languages. 6 in spite of its status (1995, official language law, amended, 3561-xi, art.1), the uzbek languange has been experimenting a number of issues for the disclosure of its full potentialities; see for instance cabar.asia/en/uzbekistan-whyuzbek-language-has-not-become-a-language-of-politics-andscience (accessed 12 oct 2020). ## word-nets in mid 1980s, several linguists and psychologists at princeton univesity started to model and develop a lexical-semantic database now referred to as princeton wordnet @xcite . the basic idea behind pwn has been to provide an aid in searching dictionaries conceptually, rather than merely alphabetically. pwn divides the lexicon into four categories: nouns, verbs, adjectives, and adverbs. they are organized into synonym sets, each representing one underlying lexical concept @xcite . pwn is based on synonyms and hyponyms of nouns and verbs, as well as antonyms of adjec-tives. in addition, it includes troponyms for verbs and hyponyms for nouns. princeton wordnet and a vast majority of wordnets (see for instance @xcite for surveys and @xcite for extensions to open multilingual wordnet) were formed by expanding the semantic structure of pwn according to the extend model foot_4 @xcite , which assumes that lemmas of the new language are created by translating english synsets of pwn. there is also possibility for creating semantic network by directly adding words and their definitions for the language under consideration. however, few wordnets have been created by using this method (merge model), due to the high cost of human expertise. ## a wordnet for uzbek language computational linguistics appeared as a field of research in uzbekistan since the late 2000s; see for instance @xcite @xcite . since then, there have been few attempts to resolve lexical ambiguity in uzbek by creating a sematic network, and none of them produced a word-net as we generally mean the term today after the pioneristic work at princeton, see section 3. the very first wordnet for uzbek-and, to the best of our knowledge, the only existing up to the present work on uzwordnet-is due to @xcite as part of the extended open multilingual wordnet project. the resulting wordnet (code uzb, accessible online foot_5 ) is minimal in terms of available synsets and coverage of "core concepts" @xcite : synsets words senses core concepts 889 1,115 1,157 8% it also seems not clear what of two uzbek languages the wordnet was built for. moreover, we could not find a specific report on it (apart from the aforementioned numbers) and the estimated specific accuracy. 9 in @xcite , the authors focused on modeling a wordnet-like thesaurus for uzbek, and tried to come up with a way to create rules for converting paper-based dictionary thesauruses into e-version using prolog. to develop a formal model of thesaurus, they built a dictionary's meta-language and defined its systematic properties. as a result, they obtained a model, not a wordnet as a computer system. @xcite surveys the main features of pwn towards its translation to uzbek. the list of works that explicitely target uzbek (northern or southern) for the purpose of building a wordnet ends here. however, there have been numerous projects for other turkic languages, for instance the development of a turkish wordnet @xcite . the project, started at sabanci university of istanbul as part of the balkanet project, uses a combination of the expansion and merge approaches. another wordnet for turkish is kenet @xcite . kenet is not based on pwn and is the most comprehensive wordnet for turkish built from scratch using a bottom-up method. the wordnet was created by using the contemporary dictionary of turkish (cdt) as lexical resource. ## our approach in this section we describe our approach to the construction of uzwordnet. we divide it into three parts. first, the choice and pre-processing of the lexical resource. second, the automatic construction of the pwn-like structure for (northern) uzbek, i.e., of uzwordnet. third, the expert human validation of the automatic construction. ## automatic construction the main procedure for building up uzwordnet is an automatic translation-called "connectivity restoration algorithm" in reason of the most significant part of it (cra; algorithm 1)-of pwn (version 3.0) into uzbek provided by the lexical resource (subsection 5.1) preprocessed into tabular format and files for each part of speech (subsection 5.2). the algorithm exploits the expansion method, as we accept the temporary assumption (see future work; section 9) that the semantic structure of pwn is similar to the semantic structure of target language, the northern uzbek for us. ## expert validation two native uzbek speakers and one expert linguist-three co-authors of this paper-were asked to independently validate a sample of uzbek synsets ("target lemmas") produced automatically. for each part of speech (nouns, verbs, adverbs, and adjectives), an excel file with 70 synsets ("lemmas") from pwn randomly selected was provided to the three "validators". for each file, the guidelines were the following: some explanatory notes were also provided. in particular, this important one: 2.1 if english lemma l is translated into more than one word and only some of those words are the correct translation of l according to l's definition, but some other words are not, write 0 (i.e., "translation incorrect"). ## results we run the cra on pwn and the entries from the lexical resource as preprocessed (cf. subsections 5.2 and 5.2.1; also input in algorithm 1). the resulting semantic network, that is, uz-wordnet, contains 28140 synsets, 64389 sense and 20683 words, positioning uzwordnet at the 18th place in the list of wordnets ranked by number of synsets, see ## conclusion and future work in this paper, we have advanced and discussed the results on the initial development of uzword-net, a lexical-semantic database, or a "word-net", for the northern uzbek language compatible by expansion (extend/expansion method) to princeton wordnet. uzwordnet contains 28140 synsets, 64389 senses and 20683 words and is the output of an automatic process whose central procedure is an algorithm of connectivity run on princeton wordnet' semantic network and an external lexical resource. evaluation by three validators of uzwordnet's accuracy in the translation, run over 280 sample entries, 70 for each pos (nouns, verbs, adjectives, adverbs), resulted in an estimated accuracy of 71.79% minimum and 75.98% maximum according to the methodology of validation; 74.64% to 76.79% if considering only the evaluation by an expert linguist.