A bui W ordnet: Using a Toolbox Dictionary to develop a wordnet for a low-resource language (2022)

Abstract: This paper describes a procedure to link a Toolbox dictionary of a low-resource language to correct synsets, generating a new wordnet. We introduce a bootstrapping technique utilising the information in the gloss fields (English, national, and regional) to generate sense candidates using a naive algorithm based on multilingual sense intersection. We show that this technique is quite effective when glosses are available in more than one language. Our technique complements the previous work by Rosman et al. (2014) which linked the SIL Semantic Domains to wordnet senses. Through this work we have created a small, fully hand-checked wordnet for Abui, containing over 1,400 concepts and 3,600 senses.

## introduction this paper describes the development of a wordnet for abui, one of more than twenty timor-alor-pantar languages of eastern indonesia. the timor-alor-pantar (tap) languages are a western outlier among other papuan languages, the bulk of which are spoken in and around the island of new guinea. while the tap languages constitute a coherent family @xcite , their relationship to other papuan families of new guinea has not been demonstrated @xcite . within the tap language family dictionaries exist for only a handful of languages, listed here in alphabetical order: abui @xcite , blagar @xcite , kamang @xcite , sawila @xcite , teiwa @xcite , and western pantar @xcite . these dictionaries exist in printed form and have been also distributed in the speech community. for the remaining languages a number of wordlists exist: from 1930s the holle lists @xcite , stokhof lists @xcite , and various wordlists produced by the indonesian language development and fostering agency (pusat bahasa indonesia). all available wordlists are consolidated in the lexirumah online database @xcite which contains at least two hundred words per language. none of the above listed tap dictionaries contain more than 4,000 words although each of them took several years to create. beyond the basic vocabulary, which is also included in the lexirumah wordlists, the dictionary coverage is determined by the collected texts and the preferences of the compilers. as a result each dictionary inevitably contains random gaps. the lexicographic workflow in language description is slow, over-reliant on a single author or a small team; it does not produce lexicographic materials suitable for language revitalisation or natural language processing applications. there is generally little concern for "open" data and shared formats. ## wordnets and low-resource languages there are two main methods to build wordnets @xcite . the first is known as the 'expansion approach', where the semantic hierarchy of another wordnet is used as pivot. in this approach, the required work is essentially a translation effort -conserving the structure of the pivot wordnet and translating individual nodes of the hierarchy, which can be done incrementally (i.e. usually starting by a subset of frequent concepts) but can take in principle infinitely long until all language specific senses are identified. the princeton wordnet @xcite ) is, by far, the most frequently used pivot for projects that employ the 'expansion approach'. the second method is known as the 'merge approach'. and while this approach is perhaps more principled, in theory, it is both slow and it also requires more resources. in the 'merge approach' no pivot structure is assumed. as such, this method can ensure higher degrees of freedom while modeling the structure of the wordnet without depending on pre-assumed semantic relations. one of the immediate benefits of this approach is the ability to freely add new concepts that are not part of the pivot language -a problem many wordnet projects that followed the 'expansion' approach have struggled with. the major drawback of this approach, however, is its inability to immediately benefit from the parallel translations available from all other projects that used the same pivot. ## lexicographic resources for abui abui (iso 639-3: abz, abui1241) is a timor-alor-pantar (tap) language spoken by about 17 thousand speakers in an area stretching from the northern to the southern coast in central alor. abui is classified by @xcite to the central alor branch of tap. the work reported here focusses on the variety spoken in the village of takalelang at the northern coast. the earliest lexicographic work on abui comes from the pen of two anthropologists who conducted their research in late 1930s in the abui village of atengmelang. cora du bois, who published a monograph on the abui culture @xcite , left behind extensive lexical and grammatical notes (part of the cora du bois personal papers at the tozzer library, harvard university, @xcite ). martha maria nicolspeyer appended to her phd thesis an abui-dutch wordlist @xcite . this work served as a base for w. a. l. stokhof, who worked on abui in late 1970s and 1980s and published the du bois wordlists and provided an abui text with a grammatical commentary @xcite . since 2003 the abui language has been subject to more intensive study which resulted in a full grammatical description @xcite and a dictionary primer @xcite [kd2008]. the dictionary is derived from a toolbox corpus and contains only words which are attested in texts that were recorded during the documentation. the dictionary was revised and expanded in its second edition @xcite [kd2014], available online and counting over 400 pages. it includes abui-english, abui-indonesian and reverses, as well as a semantic ontology based on the sil semantic domains @xcite . between 2013 and 2016, three rapid words workshops were conducted, during which about 17 thousand words [rw2016] were collected using a crowd-sourcing approach designed by the summer institute of linguistics @xcite . currently, these words are being digitised and equiped with their english, indonesian and malay glosses before the method described here can be applied. building and maintaining a wordnet is extremely time-consuming, especially when this is done manually. for this reason, the large majority of wordnets are built by bootstrapping their development using one or more existing wordnets, referred to as the "pivot languages", as we discussed in section 2. in this section we discuss the methods to build the abui wordnet. ## release and licensing a summary of the size and part-of-speech coverage of the first release of the abui wordnet is given in note the low number of adjectives (which include quantifiers) and adverbs in one key motivation for this project was to inspire other field linguistics to follow on our footsteps and release their data using open licenses. field linguists have a responsibility towards the communities they work with, and should embrace an open-shared ownership of the work that is developed with the help of these communities. we want to encourage other field linguistics to use and replicate our work, while working towards the maintenance and preservation of abui and its community. for this reason, the abui wordnet is released under a creative commons attribution 4.0 international license (cc by 4.0) foot_11 . we have produced omw tsv files, which can also be used in the python natural language toolkit @xcite . in addition, and keeping up with the recent requirements to belong to the omw, we will also release this data using the wn-lmf format 13 . the abui wordnet data will be made available on github at https://github.com/ fanacek/abuiwn . ## discussion and future work we have sketched a procedure that facilitates the transfer of the toolbox mdf-formatted data into a wordnet. and we have also shown that it is possible to generate very high quality data through a naive algorithm based on sense intersection. we believe our results could be improved further by improving our sense intersection algorithm to include, for example, semantic domain information, 14 or by attempting to exploit other available information often used in the task of word sense disambiguation such as wordnets' semantic hierarchy, glosses and definitions. in addition, we would like to work towards including pronunciation, grammatical information (aspectual class, valency, etymology and borrowings) and example sentences, all of which we track in the abui toolbox dictionary, and which can be accommodated by the wordnet lexical markup framework (wn-lmf, p. @xcite . in the near future we also expect to have to deal with many specific features particular to abui: (i) concepts unique to abui or the region; (ii) extensive specific taxonomy for animals and plants foot_14 ; (iii) many non-lexicalised cili concepts in abui (especially linked to technology and modernity). finally, another challenge we would like to work on relates to the fact that there is no official abui orthography and many writing conventions exist which reflect dialectal and idiolectal variation as well as individual preferences regarding the spelling of vowel length, velars and uvulars, tone, and clitics. we are taking an aggregating approach and register all examples of alternative spelling and link them to the respective lemma. in the future we would like to use the full extent of the wn-lmf to make this information available in our wordnet. ## conclusion this paper shows the viability of the intersection method in rapid building of wordnets for lowresource languages using data collected in field linguistics. applying a similar method as @xcite we have reached a overall accuracy of 99% when the sense is defined by the intersection of three languages. the accuracy however does drop steeply when fewer than three languages are available.