Ethical Considerations for Low-resourced Machine Translation (2022)

Abstract: This paper considers some ethical implications of machine translation for low-resourced languages. I use Armenian as a case study and investigate specific needs for and concerns arising from the creation and deployment of improved machine translation between English and Armenian. To do this, I conduct stakeholder interviews and construct Value Scenarios (Nathan et al., 2007) from the themes that emerge. These scenarios illustrate some of the potential harms that low-resourced language communities may face due to the deployment of improved machine translation systems. Based on these scenarios, I recommend 1) collaborating with stakeholders in order to create more useful and reliable machine translation tools, and 2) determining which other forms of language technology should be developed alongside efforts to improve machine translation in order to mitigate harms rendered to vulnerable language communities. Both of these goals require treating low-resourced machine translation as a language-specific, rather than language-agnostic, task.

## introduction the challenge of building machine translation systems for low-resourced languages is often seen merely a problem of data scarcity, but such a framing obscures the systemic differences between lowresourced languages and high-resourced languages, as well as between their corresponding speaker communities. before building machine translation systems for low-resourced languages, it is important to consider the real impact that these systems can have on their intended users. this paper investigates the ethical implications for improvements to machine translation for low-resourced languages using a value scenarios @xcite framework to identify potential harms and construct recommendations to mitigate them. the remainder of this paper uses the armenian language as a case study and considers the specific circumstances of speakers in the armenian diaspora. i conducted interviews with four stakeholders (armenian speakers who use machine translation systems) and analyzed their responses to identify common desires and concerns for machine translation. i then used these analyses to construct value scenarios, which illustrate potential unintended consequences of improving the quality of machine translation between english and armenian to the level of current machine translation between english and other high-resourced languages. in examining machine translation for armenian, i hope to provide examples of the kinds of harms that may be caused to speakers of low-resourced languages in the development of machine translation tools. this paper is not meant as an exhaustive exploration of all possible harms; instead, it provides a starting point for considering how the specific circumstances of a language community can inform the creation of ethically-produced machine translation tools. my findings show that machine translation for low-resourced languages should not be undertaken as an aggregated language-agnostic task, but should instead be approached in a language-specific way that contextualizes speakers' needs and other facets of existing language technology. such an approach would allow nlp researchers to move away from a data-first paradigm that privileges high-resourced languages and varieties towards one that can take into account the particular circumstances of vulnerable groups in order to ensure that we build machine translation tools that are genuinely useful and reliable. the rest of this paper is organized as follows. section 2 provides background information on machine translation for low-resourced languages and describes value scenarios. following that, section 3 gives an overview of considerations for armenian. section 4 describes the methodology i followed to conduct stakeholder interviews, the results of which are analyzed in section 5. based on interviewees' responses, two value scenarios are presented and analyzed in section 6, and recommendations based on these scenarios are given in section 7. section 8 describes ethical considerations for this project, and section 9 provides a conclusion. ## considerations for armenian in this section, i present two important challenges specific to the improvement of machine translation for armenian: language variation and orthography. while the details of these challenges pertain to the particular situation of the armenian language and of armenian speakers, it is likely that improvements to other low-resourced languages would require similar considerations. ## methodology this section describes the process by which i conducted stakeholder interviews. this study was approved by the institutional review board (irb) at the university of washington. ## results below is an overview of significant themes that emerged from my stakeholder interviews. ## value scenarios in this section, i present two value scenarios that i have constructed based on the above findings. these value scenarios are intended to illustrate potential unwanted consequences of improving machine translation for low-resourced languages. they are not meant to be predictions of real events; rather, they are deliberately dark imaginings of the impacts that new technology could have @xcite . the purpose of creating these value scenarios is to uncover considerations that may need to be made before developing improvements to machine translation for low-resourced languages. while the two scenarios below are presented as separate outcomes, it should be noted they could occur simultaneously. the distinction between them is merely for the purpose of more easily illustrating different possible consequences. ## recommendations the potential benefits of improved machine translation for low-resourced languages are enormous. the stakeholders i interviewed all named specific uses they would have for better translation tools, ranging from improving their literacy skills to strengthening their connections to their families and communities. the potential harms are enormous as well, as the above scenarios illustrate. different speaker communities will have other uses for and concerns about machine translation @xcite . ensuring that improved machine translation tools maximize the benefits and mitigate the harms requires the nlp community to take explicit steps to collaborate with and support low-resourced language communities. first, it is necessary to examine the particular wants and needs that language communities have during the planning stage of a project. this paper demonstrates the efficacy of a value-sensitive design approach in surfacing a particular community's needs and anticipating potential harms before technology is built. the interviews described in section 5 and the resulting value scenarios illuminate concerns that otherwise might only be apparent to armenian speakers. similar efforts can be undertaken with speakers of low-resourced languages to uncover other community-specific considerations. value-sensitive design provides a number of other practical techniques for collaborating with stakeholders @xcite , which may be useful in future efforts. second, we must consider what other facets of language technology should be developed alongside improvements to machine translation. the deployment of robust machine translation allows for the generation of large volumes of text in a low-resourced language, which can have negative impacts for language communities. these impacts are likely impossible to prevent without actions taken by entities outside of nlp; for instance, preventing the outcome described in value scenario 2 requires social media platforms to implement stronger content moderation policies in lowresourced languages. nlp researchers can, however, work to expand the capabilities of other facets of language technology (in this case, hate speech detection for armenian) that can mitigate potential harms caused by improved machine translation. fulfilling these goals requires disaggregating the task of machine translation; rather than creating translation tools for many languages at once, each language should be considered separately. doing this would undoubtedly be more resource-intensive than a language-agnostic approach, but it is a necessary step towards prioritizing the needs of lowresourced language speakers. the scenarios in section 6 illustrate just a couple of the ways that speakers of low-resourced languages may have very different circumstances than speakers of highresourced languages, both linguistically and geopolitically, that need to be taken into account when machine translation applications are deployed. in both scenarios, harms fall unduly on groups that are already marginalized: in the first scenario, minoritized armenian speakers bear the brunt of these harms, while in the second, armenians in general are impacted negatively. treating machine translation as an abstract language-agnostic task, divorced from the specific needs of distinct groups of users, obscures harms like these. worse, it risks exacerbating inequitable conditions. taking a language-specific, stakeholder-focused approach does more than prevent potential harms; it also builds better, more reliable technology. when researchers assemble datasets for languages they are not familiar with, they are often unable to verify the validity of a data source and may not be able to find an existing high-quality data source @xcite . this is illustrated by the difference in reliability that interviewees reported for google translate and nayiri: while google translate has more useful features, nayiri is more trustworthy because it is built by a team with deep knowledge of the language and communities it serves, using carefully curated resources that may be inaccessible to outsiders. the current paradigm of building nmt systems that rely on vast quantities of unlabelled data, whose size prevents careful curation @xcite , makes it difficult to build systems that can account for language variation and serve users that speak minoritized varieties. as a result, machine translation systems cannot produce reliable and useful output for speakers whose varieties do not have substantial bodies of data. building better and more equitable systems requires moving away from data-first approaches and investing in holistic methods that take into consideration the state of existing language technology and external circumstances of the communities in question, as well as developing higher-quality data sources @xcite . this process does not need to begin from scratch; as with the example of nayiri, low-resourced language communities may already have ongoing intra-community projects that would be fruitful sites for investment from and collaboration with nlp practitioners. ## ethical considerations as described in section 4.3, the methodology used in this paper has a number of limitations that affect how these results may be generalized. most prominently, the stakeholder group that i interviewed was small and represented only a small subsection of the perspectives of armenian speakers. additionally, the group of participants described in this paper comprises speakers of only one low-resourced language; speakers of other lowresourced languages would likely have very different needs and concerns. this case study is meant only to provide examples of the concerns of speakers of a particular low-resourced language. it is important to avoid generalizing low-resourced languages and their speakers. this paper does not cover all of the potential harms of machine translation; further efforts are needed to uncover other concerns for individual language communities. if only the harms i described in this paper were taken into consideration in the development of a machine translation system, it is certain that other important concerns would be missed, which could cause substantial harms to speaker populations. ## conclusion using value scenarios, this paper illustrates some potential harms that a general-purpose machine translation system could have for speakers of a lowresourced language. avoiding these harms requires direct collaboration with stakeholders before the creation of a machine translation system intended for low-resourced languages. to do so, machine translation for low-resourced languages should be undertaken as a language-specific task.