Not always about you: Prioritizing community needs when developing endangered language technology (2022)

Abstract: Languages are classified as low-resource when they lack the quantity of data necessary for training statistical and machine learning tools and models. Causes of resource scarcity vary but can include poor access to technology for developing these resources, a relatively small population of speakers, or a lack of urgency for collecting such resources in bilingual populations where the second language is high-resource. As a result, the languages described as low-resource in the literature are as different as Finnish on the one hand, with millions of speakers using it in every imaginable domain, and Seneca, with only a small-handful of fluent speakers using the language primarily in a restricted domain. While issues stemming from the lack of resources necessary to train models unite this disparate group of languages, many other issues cut across the divide between widely-spoken low-resource languages and endangered languages. In this position paper, we discuss the unique technological, cultural, practical, and ethical challenges that researchers and indigenous speech community members face when working together to develop language technology to support endangered language documentation and revitalization. We report the perspectives of language teachers, Master Speakers and elders from indigenous communities, as well as the point of view of academics. We describe an ongoing fruitful collaboration and make recommendations for future partnerships between academic researchers and language community stakeholders.

## a thought experiment say that we have three speech communities, which we will refer to as elephant, ocelot, and coyote. each community has their own language, which is commonly characterized by language technology researchers as "low-resource" or "underresourced". elephant's language is spoken by * equal contribution. elephant wants to develop spoken language technology primarily to support the use of speechenabled applications and smartphone features. ocelot and coyote also need spoken language technology but their focus is often on using these technologies to support use of the language by the community through language preservation, documentation, and instruction. creating chatbots or enabling hands-free cellphone use, while appealing, might not currently be a priority for ocelot and coyote. ## academic perspectives as illustrated above, the current research field has coarsely referred both to widely spoken languages lacking an established tradition of natural language processing (nlp) and to endangered indigenous languages as "low-resource" or "under-resourced", without acknowledging or mentioning the drastically different conditions of their data availability. recent high-profile work on diversity in language technology included, in one case, just a few sentences encouraging researchers to prioritize endangered languages @xcite , and in another case, no discussion of endangered languages at all @xcite . while the language taxonomy based on resource availability proposed in @xcite is quite impressive, the authors simply grouped all languages that are currently lacking resources into the same category. for instance, the mixtec language (with different varieties), spoken in mexico, has around half a million l1 speakers, while the juruna language in brazil has less than 300 l1 speakers; yet both were categorized together as "still ignored in the aspect of language technology". roughly 1,050 of the nearly 30,000 abstracts available in the acl anthology bibliography contain the token phrases "low resource", "under resourced", or "resource constrained language". of these abstracts, about half name a specific language that has been assigned an iso-639 code. excluding obviously high-resource languages that are mentioned in these abstracts as a point of comparison, as a source language for transfer learning, or as a source or target for machine translation, we are left with 163 unique languages (considering their dialectal variations) that have been characterized in at least one acl abstract as low-resource. there is great variability both in the degree and in the nature of the challenges that arise when developing language technologies for languages with scarce training resources. it is crucial that nlp researchers actively acknowledge this variability and avoid giving the impression that models or architectures developed for elephant will be suitable for ocelot or coyote, or vice versa. we can begin by distinguishing languages classified as "endangered" from those that are not, and provide case-bycase detailed descriptions of the speaker population size and language data availability for the language being investigated. contingent on that, it is only recently that the academic community has started holding workshops devoted to endangered and indigenous languages, such as computel @xcite , held four times since 2014, and amer-icasnlp @xcite , which took place for the first time in 2021. work published in these and other venues has included research on several nlp tasks that pertain to language documentation and reclamation for endangered languages, from morphological segmentation @xcite , finite-state morphological analyzers @xcite , to machine translation @xcite and asr @xcite @xcite . that being said, most of the work has focused on technology development, with relatively little regard for the ways in which the development of language technology for endangered languages might be different from that for languages with few existing resources but a much larger numbers of speakers. discussions of whether a proposed language technology would be useful for the workflow of the community's own language documentation efforts, or how it would be combined with the community's revitalization and instructional activities are also noticeably lacking, with a few notable exceptions such as the the verb conjugator, kawennón:nis, developed for the ohsweken dialect of kanyen'kéha foot_0 @xcite ; the online dictionary developed for hupa foot_1 , which is currently used for language-related activities in the community; and the indigenous languages technology project at nrc canada @xcite . the nlp community has formally recognized the importance of developing technologies for endangered languages, and we have the tools to support work in this area. now we must try to answer this question: what priorities and considerations should researchers take into account when developing technology for endangered languages? often nlp technologies presume that a language has a standardized written form that may act as source or target for various computational tasks (e.g., asr, machine translation, named-entity recognition). while standardization is typical of languages in most of the w.e.i.r.d societies @xcite , this is atypical for much of the rest of the world. for many endangered language contexts, the tradition of literacy is very recent, and writing is far less privileged than the oral medium. sociolinguistic research on small languages has identified significant variation, both dialectal and ideolectal @xcite , in these contexts as well. standardization, frequently considered a first task in language documentation and the development of language technology, often tends to run counter to the goals of the speech communities working towards revitalization @xcite . for these communities, linguistic variation is not a problem to be solved but an important element of a vital language ecology. a major goal in language revitalization involves facilitating the usage of the endangered language in a wider set of contexts and situations than it is being used in currently. this entails careful forethought into how language tools can assist in this broadening of usage. providing state-of-the-art language technology to a community that is critically involved in training new speakers and developing new contexts for usage may not be the most efficient use of time and resources. tools that assist in classroom education and in developing new usage situations are likely to be of much more immediate value to groups involved in revitalization. the number of speakers of a language also impacts the maximum rate at which new data can be collected to add to the resources available for a language. even for a language with just thousands of speakers, documentation projects can accumulate resources at a pace impossible for a highlyendangered language. in yoloxóchitl mixtec, for instance, with around 5000 speakers, there exists a speech corpus of over 100 hours of running speech @xcite . for languages with few speakers capable and comfortable of speaking their language, who are mostly elders and also very occupied with other activities related to the revitalization of the language, the goal of collecting longduration recordings in the language does not seem feasible or even reasonable. additionally, while through the decades certain linguistic academic scholars have responsibly and sensitively built trustworthy and collaborative bonds with indigenous communities @xcite (e.g., dr. ken hale working with warlpiri @xcite and @xcite ; dr. david rood working with lakhota (rood and taylor, 1996); dr. r. m. w. dixon working with australian aboriginal languages @xcite ), for many endangered language communities in north america, the attitudes of earlier european-american "settler" scholars towards indigenous communities and their languages have engendered distrust in the motives and biases of outside "experts" @xcite . earlier fieldwork often involved outsider linguists paying speaker consultants to participate in research that was designed and conducted solely by the researcher in what is now referred to as the linguist-focused model @xcite . a more recent trend in linguistics is the movement toward community-based language research, in which community members collaborate with outsider linguists on the research which they themselves help design. in the development of language technology, providing the speech communities a central role in the design and implementation of language tools may improve the likelihood of the tools' success. ## endangered language teachers perspectives while language technology is certainly of significant value from an academic's point of view, is it actually useful to stakeholders in endangered language communities? to learn from community voices, we designed an informal survey ( in the survey, we asked for language teachers' thoughts on whether they would consider writing to be a technology, and additionally, whether they think writing, morphological segmentation @xcite , asr @xcite , video processing, and pedagogical learning applications @xcite , which are all common in the research field of nlp, could be useful for them. overall, the majority of community language teachers think all of the five technological applications would be helpful but to different extents. for example, most think that having written documentation would be valuable for aspects of language teaching, reclamation, and intergenerational transmission of cultural and linguistic knowledge. in the words of one respondent, "written documentation is useful because a lot of it is old. it captures the question yes no do you think writing could be a technology? 91.30% 8.70% can you read written form(s) of the language? 95.65% 4.35% do you use written documentation when working on language revitalization? 100% -do you think morphological segmentation could be a useful technology? 86.96% 13.04% do you think automatic speech recognition could be a useful technology? 82.61% 17.39% do you think video parsing could be a useful technology? 82.61% 17.39% do you think pedagogical learning applications are a useful technology? 95.65% 4.35% in the case of asr, some language teachers thought that it would be "an interesting idea"; they could see themselves "reviewing the transcripts" and the technology would help with "sound recognition and pronunciation."; these transcripts, however, would not be as effective as "listening to the audio". others expressed strong feelings against asr, saying that "to me this is just a way for linguists to secure funding for themselves and their tech project, which takes money and resources away from speech communities. this kind of thing is not language revitalization, as it doesn't create 4 we acknowledge that different indigenous speech communities have different perspectives on these matters. writing alone is extremely useful for speech communities with no master speakers left. we note particularly the breath of life (bol) speech communities that reclaim their languages from written documentation; the written documentation helps bol community researchers become the next generation of master speakers. in other words, writing is not the end goal in the language learning process for these indigenous communities. new speakers. it generates new texts so they can sit on the shelf of some archive. not helpful". almost all language teachers favor automatic processing of video materials. they mentioned this technology is of great value because "(it) captures the authenticity of the speaker/apprentice"; others said they would use videos "to watch the elders' mouths when they speak as well as their facial expressions. intonations and body expression really add to conversations. those things can be lost when just reading, writing or even listening. seeing the facial expressions and body language are important to understand the contexts of how specific words are being used". at last, regarding pedagogical applications, community members suggested that they could be beneficial given that the applications could "provide repetition, drill and practice. this would increase learner confidence in their own adequate exposure to the language (in a non-threatening manner). it would allow the learner to understand how the language works, so that new speech beyond the app could be developed by the learner". they indicated, however, that for the applications to work, people would have to actually "use them". the observations from this informal survey might be surprising to researchers, who typically consider language technology to be broadly useful and beneficial to all people. an awareness that is lacking in the research field, however, is that the purpose of language technologies and their development process might vary significantly when applied to indigenous and endangered languages. for many of these languages, the priorities of the speech communities are how to more effectively document, teach, and reclaim their language; how to save the cultural heritage passed down from the elders; and how to let their language have a voice among other widely-spoken or dominant languages. ## elder perspectives to linguists, the karuk speech community is a critically endangered language community of north california. to speech community members, the karuk language is a vital language with approximately 25 speakers, including five master speakers and other language teachers, archivists and activists. the karuk language community is one which is thriving, though surviving through grass roots revitalization with very little infrastructure at the tribal language program level. when it comes to modern language revitalization, data cannot and should not be separated from the master speakers. their experience of government policies aimed at linguicide @xcite and their sense of loss of their mother tongue, as l1 speakers become more and more scarce, are realities that new speakers and field linguists need to acknowledge. for speakers of the karuk language, these issues come through in the community's internal documentation of their language. for instance, one master speaker of the language explained the loss of speakers when he said: "(t)hose were all my friends. that's what i was telling [the nurse]. i said i got a lot on my mind. i said, i sit here all by myself and i'm thinking about all the people that left me. i said, it's kinda, you don't feel good i mean, you know, when you think about them. you're not supposed to, they're gone. xâatik, let it go. but i just can't help it. i think about all the funny things we did together, laughing and talking" foot_3 . his words reflect the culture of language loss as it occurs on the human level. sometimes this reality stops potential master speakers from working with linguists and young speakers eager to reclaim their language/identity. it is clear that when working with elderly master speakers, methodologies must include space for the elders to vent their grief. only after this is attended to, can the language be learned by young language workers striving to create a future where the language is worth more than the losses felt. the young people and master speakers who attend to this re-envisioning of an indigenous future with endangered languages in perpetuity, are the cornerstone of hope and healing @xcite @xcite . one speaker discusses her view of indigenous second language acquisition (sla): "reading outta them books is ok. you can probably gather a lot. yeah, you guys work hard at it. i've been trying to get all these people together. at least once a month all the speakers should work together. you can just gather one day, all the language speakers, you know? and start talking the language. a lot of [speakers] get enough [language] so they can teach... but they're not really getting all of it. if we're all together, maybe you had something that you wanted to say and you didn't know the words, you could ask somebody. somebody would know. in other words, help one another". here we come to understand that linguistic documentation is useful for language reclamation, and writing "them books" is a fruitful task @xcite . but one theme that emerges from documentation of elderly master speakers of the karuk language is that writing cannot replace the value of speech communities coming together and speaking their language, continuing the oral tradition. written data doesn't save a language; it safeguards knowledge. the ultimate goal of endangered language communities is to someday house that same knowledge in the hearts and minds of their members. ## a case study in bridging the indigenous and academic communities what does a healthy and trustworthy workflow look like when bringing together an endangered indigenous language community with the (academic) research community? after gaining perspectives from academics as well as language teachers and elders from indigenous endangered language communities, here we describe an ongoing workflow devoted to developing a morphological parser for the cayuga language. with approximately 50 l1 elder speakers and an ever-growing number of l2 speakers, the cayuga language fits the description of a highly or critically endangered language. community-based revitalization projects include an immersion language preschool, adult language courses, as well as teacher-training programs at the local polytechnic. two authors of this paper are participants in this project, and both are linguists in academia. specifically, one of them has years of connections with the cayuga speech community and advanced research knowledge in the language, while the other has extensive training in computational linguistics but no initial connections with the speech community. this project started in late september, 2021, and is ongoing. the overall workflow is simple and straightforward. first, the author known by the cayuga speech community introduced the other author to the community. they described the general idea for the project and mentioned that if they were to carry out the project, they would begin with words already found in the published grammar. while morphological segmentation is of interest to linguists, and morphological supervision has potential utilities for certain nlp tasks such as dependency parsing @xcite and bilingual word alignment @xcite , in this case, the main goal was to ask whether community members would find morphological segmentation useful for their own language teaching and documentation. community members mentioned that explicitly teaching students various inflectional elements of complex verbs, segmenting them, and in some instances color-coding morphemes have been useful for students to learn verbal arguments. second, after securing the go-ahead from community members, the two authors have been meeting almost every week for an hour to discuss progress. the author with extensive research knowledge of the language manually performs morphological segmentation of around 50 words every week. in particular, he provides annotations of both surface segmentation and canonical segmentation @xcite . the former is to be later incorporated into the workflow for building asr systems for the language using recordings already collected; the latter has the objective of gaining a better understanding of the language from a linguistic perspective. the key difference between these two types of segmentation is that for surface segmentation, the concatenation of the individual morphemes stays true to the initial (orthographic) representation of the word (e.g., onadowa:do ˛@xmath0:d-o ˛h; the word means they are hunting), whereas for canonical segmentation, decomposing a word into its component morphemes involves the addition and/or deletion of characters (or phonemes) in order to outline the orthographic or phonological changes during the word formation process (e.g., onadowa:do ˛@xmath1. with the new words annotated every week, the author with a computational background trains segmentation models in an iterative fashion, by combining the words of the current week with those from previous weeks to construct a data set for model training and evaluation. model performance, indexed by f1 score, is recorded weekly. as of now we have annotations for 262 words. the f1 scores for both surface and canonical segmentation approximate 50%. our follow-up step is to train models using all these words, then apply them to new data that has not yet been annotated to enhance and accelerate manual annotation. once the f1 scores reach around 75%, we plan to report back to the community, inform them about where things are, and discuss details of incorporating our research output with their own language work. ## recommendations for ethical collaborations considering academic output on endangered languages more holistically, we conclude that there are not enough narratives about the process of working with the community. academic (nlp) researchers working on indigenous languages, particularly endangered languages like karuk and cayuga that have historically been suppressed, should take the following steps when planning projects, describing their research, and collaborating with stakeholders in the speech communities: lastly, each perspective and motivation for indigenous language documentation has value and is worth recording. we hope our work will encourage academics to focus on prioritizing the needs and preferences of endangered speech communities when working with them to develop technologies for their languages. in particular, academics must keep in mind the relationship many endangered language communities have with their languages. one master speaker of the karuk language captured this nature of this relationship when he said "(t)he karuk language is a canoe. it holds all of our baskets, our regalia, our materials, our food. the canoe holds all our practices, songs and stories. it holds all our people and all the karuk people yet to be born. the canoe carries us all; without it, we can't get anywhere" @xcite .