Quantifying the Ethical Dilemma of Using Culturally Toxic Training Data in AI Tools for Indigenous Languages (2024)

Abstract: This paper tries to quantify the ethical dilemma of using culturally toxic training data to improve the performance of AI tools for ultra low-resource languages such as Indigenous languages. Our case study explores the use of Bible data which is both a commonly available source of training pairs for translators of Indigenous languages and a text which has a trail of physical and cultural violence for many Indigenous communities. In the context of fine-tuning a WMT19 German-to-English model into a Guarani Mbya-to-English translator, we first show, with two commonly-used Machine Translation metrics, that using only Bible data is not enough to create successful translators for everyday sentences gathered from a dictionary. Indeed, even fine-tuning with only 3,000 pairs of data from the dictionary produces significant increases in accuracy compared to Bible-only models. We then show that simultaneously fine-tuning with dictionary and Bible data achieves a substantial increase over the accuracy of a dictionary-only trained translator, and similarly happens when using two-step methods of fine-tuning. However, we also observed some, measurable, contaminated text from the Bible into the outputs of the best translator, creating concerns about its release to an Indigenous community. We end by discussing mechanisms to mitigate the negative impacts of this contamination.

## 73 when he arrived at his house, he said to his wife, "can there be anyone who can hunt like me?" when he came to the tomb, he said to his mother, how can i not know where i am. ## 30 when he got home, he said to his wife, "could it be that i'm from here?" 37 38 when he got home, he said to his wife, "couldn 't i find the ring?" 32 39 when he fell, he hit his back on the ground and died then and there. so then, how much more will the earth bear down on him than the earth will bear down on him. ## 43 but the one who comes after him will eat the bread, and the bread will come out of his mouth. ## 9 when evening came, the birds were singing and singing, but the indian was still stuck. but the spirit of the spirit is in the spirit, and the spirit is in the spirit. ## 17 and then it was the turn to eat the birds, both of which were indians. ## 35 bibles test set, the performance of the models only trained with bible data significantly decreased after they are fine-tuned with dictionary data.