Construction-Based Reduction of Translationese for Low-Resource Languages: A Pilot Study on B avarian (2025)

Abstract: When translating into a low-resource language, a language model can have a tendency to produce translations that are close to the source (e.g., word-by-word translations) due to a lack of rich low-resource training data in pretraining. Thus, the output often is translationese that differs considerably from what native speakers would produce naturally. To remedy this, we synthetically create a training set in which the frequency of a construction unique to the low-resource language is artificially inflated. For the case of Bavarian, we show that, after training, the language model has learned the unique construction and that native speakers judge its output as more natural. Our pilot study suggests that construction-based mitigation of translationese is a promising approach. Code and artifacts are available at https://github.com/cisnlp/BayernGPT .

## introduction the multilingual capabilities of large language models (llms) are impressive for medium-and high-resource languages, but they are still poor for low-resource languages for which the size of the available text corpus is small. while llms have recently improved their performance on low-resource comprehension tasks, little progress has been made on generation since the training demands for effective generation are much higher than those for comprehension. bavarian is a low-resource language that instantiates this state of affair: some large state-of-the-art models' performance is decent for comprehension of bavarian, but this does not carry over to generation. our hypothesis is that there are at least two different problems with limited generation capabilities of llms: lack of knowledge and translationese behavior. ## related work multilingual language models have emerged as the dominant paradigm for supporting low-resource languages. these models range from smaller architectures such as multilingual bert @xcite , xlm-r @xcite , mbart @xcite , mt5 @xcite , and glot500 @xcite , to large-scale models including bloom @xcite , aya @xcite , mala500 @xcite , emma500 @xcite , and llama 3 @xcite . trained jointly on data from a wide range of languages, these models demonstrate strong cross-lingual transfer and generalization capabilities, offering a promising foundation for low-resource language applications. despite this progress, generative performance on low-resource languages remains limited -particularly for tasks such as machine translation (mt), which are highly sensitive to the quantity and quality of available training data. mt has thus become a central benchmark for evaluating the generative abilities of multilingual models. in the context of large language models, recent efforts have explored two major directions: supervised fine-tuning on parallel corpora @xcite @xcite @xcite , and incontext learning methods that incorporate external linguistic resources -such as grammar books and bilingual dictionaries -without modifying model weights @xcite @xcite . these challenges are particularly pronounced for extremely low-resource languages such as bavarian. due to very limited annotated data, bavarian remains largely excluded from multilingual pretraining. @xcite introduced one of the first bavarian-german mt systems, demonstrating that translation between closely related language varieties can yield relatively strong bleu scores. to further enhance translation quality while minimizing artifacts such as translationese, they employed back-translation @xcite to generate a compact but effective set of synthetic training examples. however, their approach depends solely on wikimatrix @xcite , a parallel corpus known to be noisy and dominated by simplistic sentence structures, which limits its ability to robustly capture more nuanced translation characteristics. one such characteristic is translationese -a linguistic phenomenon that arises when translated text retains unnatural or non-native structures. this artifact is especially problematic for low-resource languages @xcite 2024 ) introduced a storyboardbased data collection method, in which native speakers generate descriptions from visual prompts without access to the source text-resulting in outputs that are more fluent and natural. however, these methods still fall short of enabling large language models to directly produce fluent, translationese-free output for truly low-resource languages, such as bavarian. ## evaluation to assess the effectiveness of our article reduplication augmentation strategy, we conducted both sentence-level and noun phrase-level (np) evaluations using human judgments from two native bavarian speakers (two of the authors of this paper). ## conclusion we propose a method to remedy the problem of translationese when translating to low-resource languages and apply it to bavarian. our approach synthetically creates a training set in which the frequency of a construction unique to the low-resource language is artificially inflated. we show that a model trained with this synthetic data produces output with this construction and that it is perceived as being more natural than the baseline.