GUIDE : Creating Semantic Domain Dictionaries for Low-Resource Languages (2024)

Abstract: Over 7,000 of the world’s 7,168 living languages are still low-resourced. This paper aims to narrow the language documentation gap by creating multiparallel dictionaries, clustered by SIL’s semantic domains. This task is new for machine learning and has previously been done manually by native speakers. We propose GUIDE, a language-agnostic tool that uses a GNN to create and populate semantic domain dictionaries, using seed dictionaries and Bible translations as a parallel text corpus. Our work sets a new benchmark, achieving an exemplary average precision of 60% in eight zero-shot evaluation languages and predicting an average of 2,400 dictionary entries. We share the code, model, multilingual evaluation data, and new dictionaries with the research community: https://github.com/janetzki/GUIDE

## background before describing the task in more detail, we introduce key resources and terms. ## related work existing approaches to creating dictionaries address different sets of languages and map words to ontologies or words of other languages. the universal wordnet (uwn) is a graphstructured knowledge base for more than 200 languages that de melo and weikum (2009) automatically generated. they used several data sources, especially existing bilingual dictionaries and to a limited extent also parallel corpora. as a scaffold, they used princeton wordnet @xcite , which provides a semantic hierarchy of english terms, and they enriched it with more than 1.5 million new semantic links for more than 800,000 words. our work has a similar goal, as we investigate how to create and enrich another linguistic resource automatically. we use the semantic domains as a scaffold and focus on low-resource languages, for which we assume only a small amount of parallel text. @xcite show how to fnd new translations of words in three endangered uralic languages. their key idea is to construct a graph of words in these and other languages, with known translations as edges. an advantage of their approach is that it does not require parallel texts or word alignment. by predicting missing links in this graph, they built new bilingual dictionaries that help preserve these endangered languages. similarly, we build a graph in which words in different languages are separate nodes. but there are two important differences: based on the reviewed related work on dictionary creation, we can summarize the research gap as follows: there is a need to create highly multiparallel dictionaries for low-resource languages without labeled data. we address this gap by using existing parallel text. ## dictionary creation with guide we now describe the guide technique to induce dictionary entries for semantic domains based on a graph neural network. ## experimental setup this section provides details about the environment in which we executed guide and how we evaluated it. ## evaluation this section evaluates guide's performance, shows the results of an ablation study, and discusses the fndings.