Assessing Digital Language Support on a Global Scale (2022)

Abstract: The users of endangered languages struggle to thrive in a digitally-mediated world. We have developed an automated method for assessing how well every language recognized by ISO 639 is faring in terms of digital language support. The assessment is based on scraping the names of supported languages from the websites of 143 digital tools selected to represent a full range of ways that digital technology can support languages. The method uses Mokken scale analysis to produce an explainable model for quantifying digital language support and monitoring it on a global scale.

## introduction the users of endangered languages struggle to thrive in a digitally-mediated world. the opportunities afforded by digital technology differ drastically depending on the language being used. this has been dubbed the "digital language divide" @xcite @xcite . as digital modes of communicating and accessing information become increasingly necessary in daily life, lack of digital language support (dls) for a language means that its speakers must use other languages to participate in the global information society or be left out. linguists have been writing for decades about the role digital technology could play in language revitalization @xcite @xcite @xcite . language technologists are recognizing the inequities facing the vast majority of the world's languages @xcite and are embracing the challenges of bringing greater equity in dls @xcite @xcite . however, in a world where most people are multilingual and each language fits into its functional niche within an ecology of languages @xcite , full digital support for every language is not a realistic goal nor what those multilingual individuals are necessarily looking for @xcite . the goal of our research is to develop a method for measuring dls in every language, so that it will be possible to provide an empirical view of the digital state of the world's languages and to observe the progress as so-called low-resource languages move toward crossing the digital language divide. ## related work our primary inspiration has been the seminal work by @xcite on developing a method for assessing the digital vitality of any language. he proposes a four-way classification of languages as digitally thriving, vital, heritage, or still, "roughly corresponding to the amount of digital communication that takes place in the language." his method harvests data from the web, then uses supervised classification to automatically label all known languages. in practice, he adds a fifth level, borderline, to represent languages that show signs of crossing the gap from still to vital. he and his colleagues have applied this method to the languages of india @xcite , the former soviet union @xcite , and the uralic family @xcite . in reviewing kornai's method, @xcite focused on the huge gap between still and vital. he argues that two additional levels are needed to fill this gap: one for when the needed elements (like a keyboarding solution) are in place for potential digital language use, and another for when digital language use is indeed taking off. we follow gibson's lead in adding two levels, but use names that achieve better congruence with the geometry of the s-curve model that emerges from our method (see figure 3 ). digital vitality should be orthogonal to nondigital vitality. we exclude features like population and language vitality from the feature data. kornai notes that the egids level as reported in ethnologue @xcite is "the best predictor of digital status." but digital vitality is distinct from non-digital vitality. for instance, our method reports the "dead" language latin to be the 80 th most digitally vital language in the world. by contrast, aimaq with nearly two million speakers is found to be digitally still. the assessments should be explainable. a standard critique of machine learning models based on black-box methods is that the models cannot explain why they produce the answers they do @xcite . @xcite bases his results on the majority outcome from 100 runs of a black-box model that yields a slightly different result each time. users will be more likely to trust results if they are deterministic and explainable. the assessment scale should measure a single underlying trait. the data features used by @xcite covered a variety of digital uses. some had to do with quantifying the extent to which the language has been documented in digital archives by researchers. others, like the sizes of wikipedias, had to do with quantifying the extent of digital language use by the language community itself. still others looked at specific software products and recorded which languages they support. these strike us as three distinct traits, each of which should be assessed in its own right: digital language preservation, digital language use (dlu), and digital language support (dls). of these, the latter two are what speak to monitoring the digital vitality of a language as it moves toward crossing the digital language divide. dlu and dls are distinct traits that should be assessed separately-speakers of unsupported languages may nevertheless use it digitally (for instance, making do in texting; see @xcite ), while speakers of supported languages may choose to use digital resources in another language they know. we have chosen to focus on dls since the data for monitoring that phenomenon are openly accessible-the developers of digital tools are usually keen to advertise all of the languages they support. by contrast, data on actual digital use is typically not shared on a language-by-language basis by the vendors concerned. a comparable effort to assess dlu on a global scale is much needed, though we anticipate that it will be significantly harder to acquire the needed data. ## methodology the method we have adopted for building an explainable model of dls is mokken scale analysis @xcite ). mokken's method is a generalization of the more widely known guttman scaling @xcite . in the latter, the items in a scale form a strict hierarchy. if a subject has an item on the scale, then all lower items also apply. a subject's score on the scale is thus the highest item that is true for the subject. intuitively, dls has these properties. if a language has a good virtual assistant (like siri), then we can infer that it also has good machine translation-but having good machine translation does not imply having a good virtual assistant. similarly, if a language has good machine translation, we can guess that it must also have good spell checking, though we cannot assume that the reverse would hold. in a guttman scale, an exception to the hierarchical ordering is considered an error, but in an arena like dls we can expect there to be exceptions. mokken scaling is a method for placing the items of a supposed hierarchical scale into their optimal order, while providing metrics that allow one to evaluate how well the hierarchical model fits. ## conclusion we have presented a method that produces an explainable model for quantifying dls. we are currently working with ethnologue to add reporting on dls in its description of languages, beginning with the next edition. regularly updating the assessments should serve to document the digital trajectory of every known language.