L ex C -Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons (2024)

Abstract: Data scarcity in low-resource languages can be addressed with word-to-word translations from labeled task data in high-resource languages using bilingual lexicons. However, bilingual lexicons often have limited lexical overlap with task data, which results in poor translation coverage and lexicon utilization. We propose lexicon-conditioned data generation LexC-Gen, a method that generates low-resource-language classification task data at scale. Specifically, LexC-Gen first uses high-resource-language words from bilingual lexicons to generate lexicon-compatible task data, and then it translates them into low-resource languages with bilingual lexicons via word translation. Across 17 extremely low-resource languages, LexC-Gen generated data is competitive with expert-translated gold data, and yields on average 5.6 and 8.9 points improvement over existing lexicon-based word translation methods on sentiment analysis and topic classification tasks respectively. Through ablation study, we show that conditioning on bilingual lexicons is the key component of LexC-Gen. LexC-Gen serves as a potential solution to close the performance gap between open-source multilingual models, such as BLOOMZ and Aya-101, and state-of-the-art commercial models like GPT-4o on low-resource-language tasks.

## introduction extremely low-resource languages do not have any labeled data and are thereby considered the "left-behinds" in nlp language technology development @xcite @xcite . nonetheless, many of them have bilingual lexicons resources, which are usually the first product of language documentation @xcite @xcite . bilingual lexicons are dictionaries that map words from one language to their translations in another languages, and they cover more existing task data high lexical overlap low lexical overlap bilingual lexicon lexc-gen generated task data (a) intuition of lexc-gen. word translation coverage lexicon utilization rate 0 20 40 60 80 100 percentage (%) existing data lexc-gen generated data (b) lexical overlap between lexicons and data. figure 1 : we observe data-lexicon mismatch (i.e., low lexical overlap) between existing task data and bilingual lexicons (figure 1a ). lexc-gen addresses the issue by generating data using words from lexicons so the data will have more words translated (i.e., higher word translation coverage) and higher lexicon utilization rate (figure 1b ). in this work, we introduce lexc-gen, 1 which is a lexicon-conditioned data generation method, to task classes @xcite (1) sample (words, class) (4) word-to-word translation julia and i put a stop to our love story. julia sareng kaulâ nyaba' settong ambu ka our senneng careta. ## lexc-gen we aim to generate data for classification tasks in a low-resource language l, given access to ( 1 ) labeled task data t h with c classes in a highresource language h, (2) a bilingual lexicon d l h that maps words from h to l, and (3) an llm supporting h. lexc-gen uses these inputs to generate labeled task data t l in low-resource language. our key idea is to prompt the llm to generate task data using high-resource-language words from bilingual lexicons in order to create task data that have a higher lexical overlap with those bilingual lexicons (figure 1a ), and thus can be more effectively translated into l. in the following, we describe the steps to obtain t l . for readability, we refer to d l h as d. ## experimental setup we compare lexc-gen against baselines and gold translations on sentiment analysis and topic classification tasks. we describe the task datasets in section 4.1, how we instantiate lexc-gen in section 4.2, and our baselines as well as gold translations in section 4.3. ## discussion application in resource-scarce scenarios lexc-gen addresses the resource-scarce scenarios faced by extremely low-resource languages that lack labeled data. we show that we can leverage their existing-yet-scarce lexical resources such as gatitos @xcite , which only contains around a few thousand of lexicon entries for common words or phrases, to generate labeled task data. furthermore, lexc-gen is a practical solution to data-lexicon mismatch problem, as it does not require linguists to build task-specific lexicons, such as multilingual sentiment lexicons @xcite , or practitioners to collect labeled task data in low-resource languages. open-source models current open-source models like bloomz @xcite and aya-101 @xcite fail to close the performance gap against gpt-4o and gold translation baseline. our work bridges the gap-we show that using them to generate training data improves performance over direct zero-shot or few-shot prompting and can match training classifiers on humanlabeled data. furthermore, due to the permissive nature of the models, their generated data can be used for proprietary or public research for broader multilingual applications. ## conclusion we propose lexc-gen to generate low-resourcelanguage task data for sentiment analysis and topic classification tasks. finetuning on our generated data can match gold data that are difficult to collect. given that lexc-gen is a practical solution, we hope it alleviates the severe data scarcity problem of low-resource languages and accelerates nlp progress in these long-tail languages.