The Zeno’s Paradox of ‘Low-Resource’ Languages (2024)

Abstract: The disparity in the languages commonly studied in Natural Language Processing (NLP) is typically reflected by referring to languages as low vs high-resourced. However, there is limited consensus on what exactly qualifies as a ‘low-resource language.’ To understand how NLP papers define and study ‘low resource’ languages, we qualitatively analyzed 150 papers from the ACL Anthology and popular speech-processing conferences that mention the keyword ‘low-resource.’ Based on our analysis, we show how several interacting axes contribute to ‘low-resourcedness’ of a language and why that makes it difficult to track progress for each individual language. We hope our work (1) elicits explicit definitions of the terminology when it is used in papers and (2) provides grounding for the different axes to consider when connoting a language as low-resource.

## introduction if the fleet-footed achilles and a slowmoving tortoise are in a race, achilles will never catch the tortoise if the tortoise has a head start. regardless of how fast achilles runs, he first has to reach a point the tortoise already passed, by which point the tortoise will have moved ahead. -zeno's achilles paradox 1 the majority of research in the nlp community has focused on only a handful of the world's languages @xcite . particularly, languages spoken by communities in the global south have largely been neglected @xcite . languages understudied by the nlp community are usually referred to as 'low-resource', while those well-studied are referred to as 'high-resource.' this framing of high vs low-resource languages resembles zeno's achilles paradox: 'high-resourced languages' are the tortoise, that have been given a head start in the research community and continue to receive much of the attention, and 'low-resource languages' are achilles. in reality, achilles can always outrun the tortoise foot_0 . however, the face value interpretation of the paradox can serve as an analogy for how the current trajectory of the nlp research community to include majority of the worlds languages in the path already forged for 'high-resourced' languages leaves 'low-resource languages' constantly trying to catch up to a goalpost that is always moving. the disparity in research and performance of language technologies across languages can be a double-edged sword. on the one hand, understudied and underserved languages may be at a higher risk of language loss and have speakers exposed to direct downstream harm due to failures of language technologies @xcite . on the other hand, the drive to include these languages in research without proper consideration of community needs (1) may lead to aggressive-and at times exploitative-data collection and (2) result in technologies that do not meet the needs of the communities who speak those languages @xcite @xcite . recently, we have seen efforts to increase the representation of 'low-resource languages' in nlp research (e.g. nllb, 2024; @xcite . yet, the exact definition of the term 'low-resource' remains elusive 3 . a common criterion to connote languages as 'low' vs 'high' resourced is data. however, using data as the only criterion oversim-plifies the context of the language itself. languages dubbed as 'low-resource' may vary depending on factors like their number of speakers, non-digital archives, or language experts @xcite . the lack of consensus in what qualifies a language as 'low-resource' makes it challenging to (1) track progress in research and development for 'low-resource languages' in general, (2) determine what interventions are effected towards a language, (3) pinpoint when a language stops being 'lowresource', and (4) discern if technologies built for these languages truly address the needs of the communities who speak them or if they are built simply on the premise that the same technology exists for a 'higher resourced language. @xcite in this work, we survey papers that study languages coined as 'low-resource'. we qualitatively analyzed 150 papers that include the keywords 'low-resource' and 'under-resource.' we used qualitative methods to unravel (1) how such papers define the term 'low-resource' or 'under-resource', (2) what languages are studied as 'low-resource', and ( 3 ) what criteria is used to classify a language as 'low-resource.' our analysis reveals four separate but interacting aspects of 'resourcedness' that are used to connote a language as 'low-resource' (see section 3 & section 4). in section 5, we use real-world examples to demonstrate how each of the aspects interact and how those interactions impact what interventions are designed and implemented for a language. finally, we use our analysis to ground recommendations for different stakeholders (see section 6). ## methodology data we collected data for papers published at *cl venues foot_2 from the acl anthology foot_3 and at the following speech processing conferences: inter-speech and international conference on acoustics, speech, and signal processing (icassp) using the semantic scholar @xcite api . we used a keyword search to identify papers that include the terms 'low-resource' or 'under-resource' in their titles or abstracts. our final corpus included 868 unique papers. ## why does it matter? in the previous section, we describe four overarching aspects that determine if a language is 'lowresource': socio-political aspects, human and digital resources, artifacts, and agency of community members. in figure 4 , we present language profiles for 6 languages. we choose the six languages from the bottom three classes in @xcite : 'the left behinds' with limited labeled and unlabeled data, 'the scraping-bys' with some amount of unlabeled data, and 'the hopefuls' with some labeled data. we use literature about these languages and their communities to demonstrate why it matters that we are specific in the terminology we use. languages in the same class of data availability might differ in other aspects. from 'the left behinds', we present profiles for numma-guhooni foot_6 and warlpiri. numma-guhooni is spoken in kenya where the official federal languages are kiswahili foot_7 and english. warlpiri is spoken by the warlpiri people of australia, where the most dominant language is english. while both languages fall into the same class, the number of speakers for warlpiri is 4 times that of numma-guhooni. ethnologue classifies warlpiri as a stable language, while nummaguhooni is endangered. in terms of digital resource availability, ethnologue classifies numma-guhooni as still meaning, there is no sign of digital support for the language, while warlpiri is labeled emerging with some digital content available. warlpiri also has some nlp tools available, for instance, kirrkirr is a dictionary visualization tool for the warlpiri language @xcite . from 'the scraping-bys', we look at cherokee and kalaallisut. cherokee, spoken by around 2,000 out of the 300,000 cherokee people of the cherokee nation in the united states of america, is labeled as endangered by ethnologue. on the other hand, kalaallisut, which is spoken by about 50,000 people and is the official federal language of greenland, is labeled as institutional by ethnoluge. however, cherokee has a higher ranking for digital language support, dubbed vital while kalaallisut is ascending. for 'the hopefuls', we look at isizulu and konkani. we observe the two languages are somewhat similar in terms of human and digital resources, with both being institutional in vitality and vital in digital access. however, we see the languages vary by their number of speakers with isizulu having about 6 times the number of speakers as konkani. additionally, isizulu is the most common language spoken as a first language in south africa, while konkani has shown a decline in number of speakers, with speakers outside of its primary province declaring other, dominant languages as their native language @xcite . both languages have nlp tools available for tasks like machine translation and speech processing as well as pre-processing tools. overall, we observe that within a given class based on data availability, there are drastic differences in what other resources are available for a figure 4 : language profiles for six languages across three classes based on data availability. the first row in each profile deals with socio-political issues, the second row resources, and the last row with artifacts (see figure 1 ). we observe drastic differences between languages of the same class. see appendix a for details on the labels. language. we observe that the variance decreases as we move up the classes, which can partially be explained by the stark 88.38% of the world's languages belonging to 'the left-behinds', compared to 5.49% in 'the scraping-bys' and 0.36% in 'the hopefuls' @xcite . however, as we demonstrate, the realities of each of the languages within each class are very different. the different aspects that determine 'lowresourcedness' have causal links. the four aspects we discuss in section 3 interact with each other in constraining what is available. sociopolitical issues constrain what resources are available for a given language, which in turn impact what artifacts are produced for that language. for instance, while there are no official languages in the usa or australia, federal policies in the us up to 1948 forced indigenous children to assimilate into western culture, punishing students for speaking their languages @xcite . similarly, colonization destroyed several languages of indigenous populations in australia (laura stocker and rooney, 2016). as a result, both cherokee and warlpiri, along with the numerous other indigenous languages of the americas, australia, and canada are endangered, i.e lack human resources. assimilation is not limited to the languages of the colonizer. post-independence from colonial rule of britain, kenya adopted the educational and language policies of britain, with english declared the official language in formal sectors and kiswahili the national language of the country. as a result, the majority of data available in digital and electronic media as well as in public settings are in english or kiswahili @xcite . hence, speakers of languages like numma-guhooni are largely assimilated with larger ethnic groups and kiswahili is predominantly spoken and learned by the new generation @xcite . while in 2010, the kenya constitution shifted towards centering the preservation of native languages, there were not enough funds allocated to carry this through @xcite . though at a different scale, this is similar to the case of konkani, which is in 'the hopefuls' class, losing native speakers to more dominant local languages @xcite . constraints of human and digital resources restrict the creation of artifacts for languages. as discussed in section 3.2, the minimal digital presence results in limited available data, especially at the scale needed for training sota models. links among the different aspects are not necessarily linear; socio-political issues also directly constrain what languages are taught in schools, impacting linguistic knowledge produced for a language. ad- aspect sub-division terminology definition sociopolitical economic low-affluence (hammarström, 2009) based on gross language product (glp) (product of the number of native speakers in any country and the country's per capita gross national product.) political politically-disadvantaged languages not used in mainstream media and governmental communications due to political forces resources native speakers* extinct; critically endangered; severely endangered; definitively endangered; unsafe; safe (brenzinger et al., 2003) 6 point scale based on number of speakers of the language online presence low-web resource(patil et al., 2022) limited online corpus or web presence language experts expert-constrained limited number of linguistic experts or researchers artifacts linguistic knowledge * oral languages; non-native orthography; native orthography based on the availability and type of orthography a language has. undocumented; inadequate; fragmentory; fair; good; superlative (brenzinger et al., 2003) 6 point scale based on the amount and quality of documentation available for a language. data* class 0; class 1; class 2; class 3; class 4; class 5 (joshi et al., 2020) 6 classes based on the availability of labeled and unlabeled data technology* still; emerging; ascending; vital; thriving (simons et al., 2022) 5-level classification based on digital language support available in a given language. knowing which aspect a language is lacking in allows for targeted interventions. one of the main factors that determine the survival of a language is inter-generational transmission @xcite . for instance, while cherokee and kalaallisut are both in the same class, cherokee is endangered while kalaallisut is institutional. hence, interventions-both in socio-political and artifact aspects-are best targeted toward reviving and preserving the cherokee language. on the other hand, digital access for kalaalisut is ascending, hence there might be more efforts towards increasing the availability of digital data. since kalaallisut is institutional, financial resources for preserving and growing the language are available at a federal level. additionally, it is used as the language of instruction in the education system of the country, aiding in the inter-generational transfer of the language. across classes, we observe similarities in numma-guhooni and konkani, of native speakers assimilating to other dominant but local languages. hence, interventions for these languages may be more effective in language learning apps that focus on learning the less-dominant language and translation systems between dominant local languages and the target language. communities are actively resisting exploitation and sustaining their languages; our tools should support them. despite the several layers of constraints, it is important to note that communities are not in idle state of deficit. across classes, we observe a similarity between warlpiri and cherokee, in that there are community-based initiatives to preserve and grow the languages (e.g. the warlpiri education and training trust (wett) 9 and the cherokee immersion school foot_9 ). by centering community values in our designs and research, we can collectively forge new paths for each language, conditioned on its unique circumstances. using specific terminology or having explicit definitions allows us to measure progress more precisely. the specific resource a language is deemed 'low' in directly impacts what interventions are effected towards it. for instance, programs aimed at increasing language representation in human language technologies (hlts) have several selection criteria @xcite . such programs use different terminologies and definitions, where "each term encodes differences in traditions, goals, and approaches" @xcite . as a result, what languages are included and served by such programs differ, even if languages have the same amount of data. while cherokee is tagged as having vital digital resources, it is also an endangered language. collecting more data in the language from the limited number of speakers or including it in large language models may not exactly alleviate its lowresourcedness. we argue for more explicit declarations of which aspects of resources are being referred to when the term low-resource is used. in recommendations for stakeholders: based on our findings, we give recommendations for different stakeholders involved in the effort to increase language representation in nlp research. individual researchers can (1) engage with community members and speakers of the languages they work on, (2) articulate how their work is limited in relation to the characteristics of the languages they work on, and (3) be explicit about what criteria they use to denote a language as 'low-resource.' community members can also form grassroots organizations such as masakhane 11 , which allow researchers who speak diverse languages to build language technologies together and learn from each other's experiences. additionally, such organizations can prioritize engaging with native speakers who may not be in the nlp research field, allowing for diverse perspectives when deciding what tools should be built for what language. work-11 https://www.masakhane.io/ shops such as americasnlp foot_10 and africanlp foot_11 continue to serve as spaces for fostering research and collaboration for languages that are mostly ignored in mainstream nlp research. however, main (*)cl conferences can increase the representation of these languages by (1) offering alternative tracks for papers, (2) easing the cost of attendance and registration for researchers from these communities, and (3) diversifying conference venues. academic institutions can aid researchers who speak these languages by promoting interdisciplinary collaboration and partner with local and international organizations to document and preserve marginalized languages. industry players interested in language diversity of their products can play a role by offering financial and technical support; for example, subsidizing resources for communities working on low-resource languages. companies could also prioritize making their products accessible to the communities (e.g. @xcite . government bodies can play a role in preserving languages through policies, funding, and digital inclusion. funding agencies can support language diversity and enforce building technologies that are relevant to the specific linguistic community by setting research priorities and prioritizing grants to underrepresented researchers. ## conclusion in this paper, we present 4 aspects of 'resourcedness' used to classify a language as 'low-resource' based on a qualitative survey of 150 papers. based on our analysis, we give recommendations for terminology that explicitly calls out which resource we are referring to when we say a language is 'lowresource.' a language may lack in several aspects, making the use of individual terminology difficulte.g. in multilingual settings. however, the difficulty does not absolve us from the responsibility to provide detailed documentation. at the very least, clear statements on what exactly is meant by low-resource when referring to a language would allow us to more clearly articulate the problems a particular technology resolves for a particular language.