NLP for Language Documentation: Two Reasons for the Gap between Theory and Practice (2024)

Abstract: Both NLP researchers and linguists have expressed a desire to use language technologies in language documentation, but most documentary work still proceeds without them, presenting a lost opportunity to hasten the preservation of the world’s endangered languages, such as those spoken in Latin America. In this work, we empirically measure two factors that have previously been identified as explanations of this low utilization: curricular offerings in graduate programs, and rates of interdisciplinary collaboration in publications related to NLP in language documentation. Our findings verify the claim that interdisciplinary training and collaborations are scarce and support the view that interdisciplinary curricular offerings facilitate interdisciplinary collaborations.

## introduction in 2019, 5 out of 68 indigenous languages from colombia were about to become extinct: one of them, tinigua, had only a single speaker left; foot_0 for the others, the situation looked only marginally better. globally, approximately half of humanity's roughly 7,000 languages are considered endangered (bromham et al., 2022). while many people in latin america and other places around the world want their languages to be preserved, language documentation -the process of producing grammars and texts to record a language -is very labor-intensive. demand for individuals who can perform language documentation far outstrips supply worldwide, and there is little reason to expect this will change any time soon. in the past 20 years, foot_1 the computational linguistics (cl) and natural language processing (nlp) communities have responded with systems which can automate some of the labor required in language documentation (ld). for example, elpis (foley et al., 2018) can transcribe audio into text even under the challenging conditions endemic to the ld process, such as low data volumes. despite the considerable number of computational systems which have been proposed and described over this period, they have seen little practical use (see, e.g., good et al. 2014; flavelle and lachler 2023). it is puzzling, prima facie, that systems with proven potential to facilitate ld have not been integrated into ld projects, and several explanations of this have been offered: gessler (2022) cites lack of interoperability between nlp systems and ld apps. flavelle and lachler (2023) cite an array of organizational barriers that linguists, nlp researchers, and community members face in their collaborations, including conflicting professional incentives and a lack of understanding of the other party's conceptual frameworks. they further observe that "coursework in computational linguistics is rarely required (or even available) to students training to be documentary linguists, and vice-versa", with the consequence that they "miss out on the opportunity to learn even the basic concepts of each other's fields, they also miss out on the opportunity to build connections with others who may go on to specialize in those areas". we expect that there is plenty of room for many explanations to be correct, as this issue is multifaceted. in this work, we aim to quantify two potential reasons for the lack of usage of nlp systems in real-world ld projects and to compare situations across countries. specifically, we ask the following research questions (rqs): (1a) how many top-25 universities offer graduate programs in which students can learn about both nlp and ld? (1b) how does the answer to the aforementioned question differ across countries? (2a) what percentage of papers on nlp for ld are the result of truly interdisciplinary collaborations between nlp re-searchers and documentary linguists? (2b) how does the answer to the aforementioned question differ across countries? (3) finally, is there a connection between the answers to ( 1a ) and ( 2a )? to answer our rqs, we use publicly available data from two sources: graduate program curricula and academic publications. we treat each country as as an individual unit: quantities we gather are aggregated per country before we proceed with analysis. aggregation at any smaller unit (e.g., at the university or individual level) would make data collection impractical, and while it is true that countries are not monolithic with respect to curricular offerings or publishing cultures, we observe that these differences are in sum much more pronounced between rather than within countries. ## university curricula we examine five countries: the united states, germany, brazil, mexico, and colombia. we choose the united states and germany because of their prevalence in ai publication venues and because their academic cultures are quite distinct. we additionally choose brazil, mexico, and colombia, as these countries, like much of latin america, have many indigenous languages. for each country, we consider the 25 top-ranked universities according to qs world university rankings 2024. @xcite for each university, we then determine whether it offers a graduate program in computer science (cs) or linguistics (ling). we define a "graduate program" as anything that is at least partially beyond the scope of a united states bachelor's degree: any ms or phd program would qualify, though some degree programs such as licenciaturas vary in whether they include graduatelevel training, and we examine their curricula on an individual basis. as for whether a program qualifies as "computer science" or "linguistics", we would like to capture the programs that have the highest densities of nlp researchers and documentary linguists. to this end, we define a "computer science" program as any program that has "natural language processing" or "computational linguistics" in its name, or has a graduate course in algorithms; and we define a "linguistics" program as any program that offers at least one graduate course in theoretical linguistics. @xcite 25 17 17 12 0 12 9 germany 25 20 20 1 0 6 0 mexico 12 3 2 0 0 1 0 brazil 23 12 12 1 0 2 1 colombia 8 4 3 0 0 1 0 ## publications we collect a large number of publications, each with the following annotations: ## conclusion we have presented what is to our knowledge the first evidence that provides an empirical understanding of two factors in the adoption of language technologies in ld: university curricula and collaboration trends between nlp researchers and documentary linguists. our data confirms previous claims that rates of interdisciplinary training and collaboration are low, even for work that cites application in language documentation as a motivation. moreover, while the scale of our data precludes a firm conclusion, it is consistent with the claim that interdisciplinary coursework is a partial determinant of collaboration rates, as the higher rates of interdisciplinary course offerings in the united states (relative to germany) are mirrored by higher rates of interdisciplinary publishing by authors working in the united states. this broadly supports the view that interdisciplinary graduate coursework is important for supporting the incorporation of human language technologies into ld practice. more evidence is needed, however, in order to investigate other possible factors: perhaps other influences, such as nation-level grant programs or academic cultures, are directly affecting both curricula and rates of interdisciplinary collaborations. we therefore join flavelle and lachler (2023) in identifying interdisciplinary curricular offerings as an important way for the nlp and linguistics communities to work towards the ultimate goal of aiding ld with language technologies. additionally, we observe that many of the same benefits could be gained from interdisciplinary workshops such as the ltldr workshop (neubig et al., 2020), which gathered documentary linguists, nlp researchers, and community members for the explicit purpose of fostering interdisciplinary collaborations.