Machine Translation of Folktales: small-data-driven and LLM -based approaches (2023)

Abstract: Can Large Language Models translate texts with rich cultural elements? How “cultured” are they? This paper provides an overview of an experiment in Machine Translation of Ukrainian folktales using Large Language Models (Open AI), Google Cloud Translation API, and Opus MT. After benchmarking their performance, we have fine-tuned an Opus MT model on a domain-specific small dataset specially created to translate folktales from Ukrainian to English. We have also tested various prompt engineering techniques on the new Open AI models to generate translations of our test dataset (folktale ‘The Mitten’) and have observed promising results. This research explores the importance of both small data and Large Language Models in Machine Learning, specifically in Machine Translation of literary texts, on the example of Ukrainian folktales.

## introduction in addition to human translation, machine translation has undeniable potential in connecting people and cultures. therefore, improving accuracy and accessibility to high-quality machine translation tools is very important. we chose ukrainian folktales for this experiment due to their unique nature and rich linguistical ecosystem. folktales are usually passed on from one generation to another, going back hundreds and sometimes thousands of years, creating immense depth of knowledge and layers of cultural relevance. the ukrainian language has an extensive collection of myths, legends, proverbs, songs, and folktales. even though these texts have literary translations available, many of them are rather transcreations, meaning that stories are retold and adapted to the target language and culture. this experiment uses a recently created corpus of domain-specific curated parallel training data: ukrainian-to-english folktale corpus (burda-lassen, 2022). we wanted to use this curated corpus for finetuning machine translation models and see the impact on the accuracy and quality of translation. ## conclusion more research is necessary to increase the size of the ukrainian-to-english folktale corpus to include a broader range of cultural terms, which will help further explore the preferable size of small data to make a more noticeable impact on accuracy score. since we have noticed an increase in translation accuracy at the word level after fine-tuning an opus mt model, it would be valuable to explore the depth and volume of cultural terms needed to increase the accuracy score even further. another area of research could be prompt engineering and fine-tuning llms, while exploring their added benefit of creating machine translation tailored to specific literary styles. contrary to human translation of folklore, machine translation techniques must be more literal and descriptive. therefore, a significant difference exists between human and machine translation techniques for folktales. that's where using a more informational translation style could be very valuable. we believe that this type of research would be important for other language pairs as well. the domain of literary translation, specifically the translation of folklore and other culturally specific texts, is a vibrant environment full of fascinating challenges and great potential.