Endangered Languages meet M odern NLP (2020)

Abstract: This tutorial will focus on NLP for endangered languages documentation and revitalization. First, we will acquaint the attendees with the process and the challenges of language documentation, showing how the needs of the language communities and the documentary linguists map to specific NLP tasks. We will then present the state-of-the-art in NLP applied in this particularly challenging setting (extremely low-resource datasets, noisy transcriptions, limited annotations, non-standard orthographies). In doing so, we will also analyze the challenges of working in this domain and expand on both the capabilities and the limitations of current NLP approaches. Our ultimate goal is to motivate more NLP practitioners to work towards this very important direction, and also provide them with the tools and understanding of the limitations/challenges, both of which are needed in order to have an impact.

## description computational linguistics and natural language processing (nlp) have taken immense strides, spearheaded by neural methods and large data collections. the result is ubiquitous language technology and vast amounts of research on new tasks and products. however, the vast majority of the world's languages have been mostly ignored, including the most vulnerable among them: endangered languages. the lack of communication between the nlp community and the documentary linguistics community is partly to blame @xcite . even though field and documentary linguists produce resources and use nlp methods, this is done in isolation, as computational methods are seen as a means towards the final goal, which typically is language description. the extreme pace of language loss and the urgent needs for language revitalization, however, require that we utilize documentations and go beyond language description: enter 21st century nlp. himmelmann's 20-year-old radical vision @xcite for a data-centric approach to language documentation (which sparked the creation of modern documentary linguistics) has slowly begun to materialize @xcite . for example, the workshops on the use of computational methods in the study of endangered languages (comput-el) @xcite @xcite have provided a small forum for the much-needed discussion between nlp practitioners and documentary and field linguists. meanwhile, increasingly more focus is dedicated on nlp research and bringing modern technologies to endangered languages. for example, mobile applications have been developed for data collection @xcite and are actively used in documentation projects @xcite ; automatic speech recognition models have been created to aid with automatic phonetic or orthographic transcriptions focusing in indigenous australian @xcite or tonal languages from china and the americas @xcite ; machine translation for under-represented languages have been presented as new corpora have been collected @xcite ; cross-lingual transfer has been successfully applied for tagging, morphological analysis and inflection @xcite ; multitask and active learning are being used for learning from continuous annotations on multiple tasks @xcite @xcite ; approaches dedicated to indigenous polysynthetic languages have been developed @xcite ; and computational methods have been used to study or discover typological features from large collections of text @xcite . in this tutorial, we will outline the language documentation process and revitalization efforts, while also mapping them to concrete computational tasks. we will then focus on the machine learning approaches tailored to tackle these tasks under this very data-constrained setting. an overview of many of those nlp methods, as applied for language documentation, can be found in co-proposer anastasopoulos' phd dissertation @xcite . other surveys focus on the state of language technologies within specific geographic areas, such as co-proposer cox's overview of canadian languages @xcite or the one by @xcite , focusing on indigenous american languages. the goal of our tutorial will be two-fold. on one hand, we will aim to acquaint the audience with the needs of the documentary linguistics community, and cover the already existing computational research in the field. on the other hand, we will discuss the capabilities and limitations of current computational approaches, so that the participants will know when and how to apply nlp methods, as well as how they could collect data and create corpora that can be used by nlp methods to aid both documentation and computational work. ideally, by the end of our tutorial, the attendees will be familiar with the current research and the state-of-the-art in nlp for endangered language documentation and revitalization and be aware of the many standing challenges that lie ahead. first, we will introduce the challenges posed by language documentation and revitalization, such as the transcription bottleneck, and how machine learning methods can fit into the pipeline. unlike what is considered a typical nlp setting, working with endangered language data has intricate nuances: the lack of standard orthography, or even complete absence of a writing system; the extremely limited amount of data; language typologies widely different than anything used/tested in prior work; and even the lack of established benchmarks. we will discuss these nuances in depth, and how they relate or can be remedied with existing nlp research. we will also provide example code for many of these methods, and show how standard nlp pipelines need to be modified in order to account for these nuances. finally, we will close our tutorial by discussing open problems and challenges. relevance to linguistics community this tutorial will bring together two linguistics communities, documentary/field linguists and nlp practitioners. as a result, we hope, the tutorial will build enough capacity of computational researchers that will be not only interested in nlp for endangered languages, but also aware of the current approaches and challenges. elevating endangered languages nlp research is necessary towards bringing these under-represented communities to the spotlight, as speakers of such endangered languages frequently lack the skills to build nlp tools themselves. tutorial type the proposed tutorial combines the introductory and cutting-edge tutorial types. the acquaintance of computer scientists with the language documentation process will, by necessity, be at an introductory level. at the same time, though, the tutorial will cover cutting-edge nlp methods and their application to the endangered languages domain. ## tutorial structure the tutorial will be structured in order to be informative for both linguists (documentary, computational, or otherwise) and for computer scientists who are interested in performing computational work for language documentation and revitalization. we aim for a three-hour tutorial that covers a reasonable range of all important aspects of this area. times for the proposed structure are approximate, and they might be adjusted as we refine the tutorial content. ## diversity considerations this tutorial has been constructed with a focus on encouraging diversity in all aspects: graham neubig is an assistant professor at carnegie mellon university specializing in natural language processing and machine learning. one of his major research interests is methods for low-resource language processing, and specifically for aiding the documentation of endangered languages. he has previously given well-attended tutorials at nlp conferences (emnlp and naacl) and the lisbon and cifar machine learning summer schools. he has won a number of best papers at nlp venues (e.g. emnlp2016, eacl2017, naacl2019) and given a number of invited talks on the proposal topic of low-resource language processing, including at google, umass amherst, and new york university. website: http://www.phontron.com/ 7 details breadth we aim to provide the first wide coverage overview of nlp approaches for endangered languages. out of the 25 referenced works (which is not an exhaustive list of the works we will cover) only 7 are co-authored by the presenters, and we expect only about 30% of the tutorial to be based on the presenters' prior work. similarly, 60% of the suggested reading has not been co-authored by the presenters. audience size estimate the first three iterations of the comput-el workshops (which are very relevant to the theme of our tutorial) have been steadily growing in size, while a regional meet-up organized in pittsburgh by co-presenters anastasopoulos and neubig drew about 40 participants. similar tutorials on the use of nlp in the international conference on language documentation and conservation (icldc) also draw large crowds. given this interest, we expect a healthy audience of at least 60 participants. to our knowledge, no similar tutorial has been given before at any nlp conference.