Building O kinawan Lexicon Resource for Language Reclamation/Revitalization and Natural Language Processing Tasks such as U niversal D ependencies Treebanking (2023)

Abstract: The Open Multilingual Online Lexicon of Okinawan (OMOLO) project aims to create an accessible, user-friendly digital lexicon for the endangered Okinawan language using digital humanities tools and methodologies. The multilingual web application, available in Japanese, English, Portuguese, and Spanish, will benefit language learners, researchers, and the Okinawan community in Japan and diaspora countries such as the U.S., Brazil, and Peru. The project also lays the foundation for an Okinawan UD Treebank, which will support computational analysis and the development of language technology tools such as parsers, machine translation systems, and speech recognition software. The OMOLO project demonstrates the potential of computational linguistics in preserving and revitalizing endangered languages and can serve as a blueprint for similar initiatives.

## introduction this study introduces our ongoing project to create a learner-friendly dictionary of okinawan, a ryukyuan language, based on an existing printed dictionary. 1 we created two versions of the online dictionary, which are useful for the learning and revitalization of okinawan. okinawan (uchinaaguchi) is one of the indigenous ryukyuan languages spoken in and around okinawa island in the ryukyu archipelago and across okinawan diasporas worldwide. however, since mainly elderly speakers can speak it but not many younger speakers, according to (un-esco, 2010), this language is endangered. however, various lexicographical works on the okinawan language have been done so far. among others, an okinawan shuri dialect speaker, seibin shimabukuro, created a 1,856-page manuscript of the okinawago jiten (okinawan dictionary) in 1951, with the headwords written in classical orthography with a katakana syllabary which did not faithfully represent the actual pronunciation. the national institute for japanese language and linguistics (ninjal) extensively revised the headwords and other example sentences using a latin alphabet supplemented with diacritical marks and some ipa faithful to the pronunciations in a unique way original to this dictionary. it published a revised version in 1963 @xcite . its ninth version was digitized in xslx format and published in ninjal's repository under a cc by 4.0 license in 2001. 2 although helpful for researchers, it poses difficulties for language learners due to its use of alphabetic phonological notation and special supplementary symbols, requiring familiarity with both the language and the international phonetic alphabet (ipa). in japan, using ipa-based letters and diacritical marks on the latin alphabet is unknown to ordinary users usually. this study utilizes the digital okinawago jiten dataset to create a userfriendly multilingual web application, open multilingual online lexicon of okinawan (omolo), in japanese, english, portuguese, and spanish for language learners and contributes to language revitalization and okinawans' "language reclamation" @xcite by collaborating with learners from okinawan communities, including okinawan diaspora communities in countries outside of japan such as the united states of america, brazil, and peru. 2 orthographical and multi-lingual challenges in tei lex-0 and its visualization language revitalization has recently been flourishing in the ryukyus. although the ryukyuan languages have been unwritten for a long time until recently, except for okinawan, which has its classical literature and writing system, contemporary writing systems that are more conforming to the actual pronunciation have been established, most of which employ only phonetic kana characters. however, as a result of our survey of existing dictionaries, it was clarified that many okinawan speakers prefer writing systems employing both phonetic kana and ideographic characters, i.e., chinese (kanji) characters, and orthographies using only phonetic characters are not in line with speaker demand. thus, there is a demand amongst learners for a kanji-hiragana combination in addition to hiragana-only text. if okinawago jiten, which until now has been written only in phonetic characters, were to be expressed in kanji-kana script, it could meet the demands of a larger number of learners. therefore, we transcribed the headwords and example sentences in the okinawago jiten with our provisional orthography consisting of both hiragana-only and kanjihiragana texts, which are designed to enable easy input with the default settings on ordinary computers or smartphones. the headwords and example sentences in the ninjal's spreadsheet version of the okinawago jiten are written in alphanumeric characters in the ascii range. in this study, we conducted a survey of the existing okinawan orthographies in eleven textbooks and dictionaries, such as okinawa prefectural shimakutuba orthography council's (shimakutuba seishohō kentō iinkai, 2022), nishioka et al. (2006), uchima and nohara (2006), hanazono et al. (2020), nakamatsu (1999), fija (2015), miyara (2021), carlino (2022 ), an orthography for the shuri dialect of okinawan and another for the tsuken dialect by @xcite , and partial katakana renditions seen in the introductory chapter of @xcite . we created a database of these existing okinawan orthographies and a python program converting one orthography to another (see @xcite . 14,549 headwords were converted into major orthographies. we chose the hiragana rendition of @xcite 's orthography as the standard for the headword but put the other orthographies in sub-layers including our original kanjihiragana notation, in which the hiragana part is based on @xcite . using xslt, we converted the data into tei lex-0, foot_3 a tei xml subset for dictionary data (fig. 1 ). tei xml is a de facto standard of text mark-up in digital humanities. currently, we are also translating the meanings of each word and example sentence written in japanese into portuguese, spanish, and english for okinawan diaspora communities in countries such as the united states of america, brazil, and peru. each language and writing system is written according to bcp47 foot_4 in the xml:lang attribute. from this xml file, we created the prototype website using xslt and hugo, a static site generator developed in the go language, with the theme hugo curious foot_5 , which is searchable and has easyto-read headwords on each page (fig. 2 ). 3 omeka s and linked open data in addition to the prototype website created using xslt and hugo, we are working on an alternative platform for presenting the open multilingual online lexicon of okinawan (omolo) using omeka s, foot_6 an open-source web publishing platform designed for sharing digital collections and creating media-rich online exhibits (fig. 3 ). omeka s is well-suited for digital humanities projects, offering a user-friendly interface and advanced features for organizing and presenting collections of various digital assets. figure 3 : omeka s visualization of the data by utilizing the flexibility and extensibility of omeka s, we plan to create a visually engaging and interactive experience for users to explore omolo. we import the tei lex-0 xml data into omeka s, converting it into compatible metadata for items and item sets within the platform, providing json-ld data (fig. 4 ) as linked open data (lod)/resource description format (rdf). this enables us to present the lexicon entries in a more structured and organized manner, facilitating easy navigation and browsing for language learners and providing the data to other external services easily, following the standard of lod/rdf. figure 4 : json-ld output file of a lexicon entry following the lod/rdf standard furthermore, omeka s allows the incorporation of multimedia assets, such as audio recordings and images, which can be attached to individual lexicon entries. this feature will significantly enhance the learning experience, providing users access to native-speaker pronunciations and visual aids to support language acquisition. ## application of the data to okinawan universal dependencies treebank the data generated from the omolo project is to build an okinawan universal dependencies treebank (fig. 5 ), which will be a valuable resource for researchers such as descriptive linguists and computational linguists working with the okinawan language. universal dependencies (ud) is a framework for cross-linguistically consistent treebank annotation that aims to create a comprehensive and multilingual resource for natural language processing and linguistic research foot_7 . it was created by integrating three dependency grammar projects: (universal) stanford dependencies (de @xcite @xcite , google universal part-of-speech tags @xcite , and the interset interlingua for morphosyntactic tagsets @xcite . currently, the latest version of ud is ver. 2.11, including 243 treebanks and 138 languages. more minority and endangered language treebanks such as amazonian indigenous languages and australian aboriginal lan-guages @xcite . so far, there is one japonic language, namely standard japanese, which has treebanks in ud. okinawan ud @xcite , using omolo's example sentences and other text corpora @xcite @xcite , will contribute to the diversity of ud so that we can execute more diverse typological research using ud. figure 5 : visualization of pmmiimee @xmath0, including the headwords, example sentences, and translations. 8 these data points are annotated in the conll-u format following the ud guidelines to create dependency trees that capture the okinawan syntactic structure. after this phase, using this data as the training data, we train models included in the hugging-face transformers library foot_9 , such as bert @xcite , roberta @xcite , and t5 @xcite , with this training data, and create a model to parse okinawan text into conll-u with dependency relation (deprel) and universal parts-of-speech (upos) tags automatically. thus, by developing the okinawan ud treebank, omolo, as its lexicon data and supplier of sample texts, supports the computational analysis of okinawan, which can help develop language technology tools, such as parsers, machine translation systems, and speech recognition software. these tools can contribute to revitalizing the okinawan language by making it more accessible to a broader audience and promoting its use in digital communication. ## conclusion in conclusion, the omolo project aims to provide a user-friendly, multilingual, and accessible resource for language learners and researchers interested in okinawan. by utilizing nlp and digital humanities methodologies, we have created a digital lexicon that can be used for language reclamation and revitalization efforts. future developments will include creating an omeka s version for a more visually engaging presentation with the output function of lod/rdf and constructing an okinawan ud treebank to support computational analysis and language technology tools. the foundation laid for the okinawan ud treebank is an essential aspect of this project, as it provides computational linguists with a valuable resource for working with okinawan. the treebank facilitates the development of language technology tools, such as parsers, machine translation systems, and speech recognition software, which can significantly increase the accessibility and use of the okinawan language in digital communication and contribute to its revitalization. at present, there are no large-language models (llms), such as gpt-4 @xcite , that can effectively handle the okinawan language. this is primarily due to the limited availability of highquality text corpora essential for training these models. however, the methodologies presented in this paper can create more comprehensive and accessible online resources for okinawan. as these okinawan text corpora grow in size and quality, llms will be better equipped to learn and understand the language. with sufficient training data, future iterations of llms, like gpt-4, may be able to process and generate okinawan text effectively, thereby contributing to the language's revitalization and making it more accessible to a broader audience. additionally, the availability of high-quality okinawan resources can help facilitate the development of advanced language technology tools, such as machine translation systems, parsers, and speech recognition software, further promoting the use and preservation of the okinawan language in the digital age. in summary, the omolo project showcases the immense potential of computational linguistics in preserving and revitalizing endangered languages. the methodologies and approaches employed in this project can serve as a blueprint for other similar initiatives, ultimately fostering linguistic diversity and preserving cultural heritage through the innovative use of digital technology.