LexC-Gen: Generating Data for Extremely Low-Resource Languages with
  Large Language Models and Bilingual Lexicons
Authors: name: Zheng-Xin Yong; name: Cristina Menghini; name: Stephen H. Bach
Published: 2024-02-21 19:20:06

Abstract: Data scarcity in low-resource languages can be addressed with word-to-word
translations from labeled task data in high-resource languages using bilingual
lexicons. However, bilingual lexicons often have limited lexical overlap with
task data, which results in poor translation coverage and lexicon utilization.
We propose lexicon-conditioned data generation (LexC-Gen), a method that
generates low-resource-language classification task data at scale.
Specifically, LexC-Gen first uses high-resource-language words from bilingual
lexicons to generate lexicon-compatible task data, and then it translates them
into low-resource languages with bilingual lexicons via word translation.
Across 17 extremely low-resource languages, LexC-Gen generated data is
competitive with expert-translated gold data, and yields on average 5.6 and 8.9
points improvement over existing lexicon-based word translation methods on
sentiment analysis and topic classification tasks respectively. We show that
conditioning on bilingual lexicons is the key component of LexC-Gen. LexC-Gen
is also practical -- it only needs a single GPU to generate data at scale. It
works well with open-access LLMs, and its cost is one-fifth of the cost of
GPT4-based multilingual data generation.