S ch A man: Spell-Checking Resources and Benchmark for Endangered Languages from Amazonia (2022)

Abstract: Spell-checkers are core applications in language learning and normalisation, which may enormously contribute to language revitalisation and language teaching in the context of indigenous communities. Spell-checking as a generation task, however, requires large amount of data, which is not feasible for endangered languages, such as the languages spoken in Peruvian Amazonia. We propose here augmentation methods for various misspelling types as a strategy to train neural spell-checking models and we create an evaluation resource for four indigenous languages of Peru: Shipibo-Konibo, Asháninka, Yánesha, Yine. We focus on special errors that are significant for learning these languages, such as phoneme-to-grapheme ambiguity, grammatical errors (gender, tense, number, among others), accentuation, punctuation and normalisation in contexts where two or more writing traditions co-exist. We found that an ensemble model, trained with augmented data from various types of error achieves overall better scores in most of the error types and languages. Finally, we released our spell-checkers as a web service to be used by indigenous communities and organisations to develop future language materials.

## introduction in natural language processing (nlp), the normalisation of a language is closely related to automatic spell checking, a process in which a computer program identifies a misspelling and suggests correct or standardised alternatives to the user. spell-checking, an important step towards grammar checking, can be addressed as a sequenceto-sequence problem with deep neural networks (junczys- @xcite . a common problem with this approach, however, is the large amount of data required. one possible way to deal with this is the generation of synthetic data @xcite , since many of these errors are random, or typographical errors due to close keys. for low-resource and endangered languages, developing a speller or normalisation tool is an important step for supporting further language revitalisation and documentation efforts, as well as indigenous education programs. this is particularly important in regions like amazonia, where linguistic diversity is in serious risk @xcite . although there are rule-based spellcheckers for some languages spoken in peruvian amazonia, such as shipibo-konibo @xcite and pan-ashaninka @xcite , their vocabulary coverage is limited and they are not context-sensitive. these are issues that can be assessed by subword and neural-based generation models for sequences of words. in this study, we propose the implementation of neural spell-checkers for four indigenous languages spoken in peruvian amazonia: one pano language, shipibo-konibo (shp), and three arawak languages, ashaninka (cni), yanesha (ame) and yine (pib). for this purpose, we introduce error augmentation methods to take advantage of the scarce monolingual corpus available ( §4), and we create an evaluation resource with a diverse typology of errors: phoneme-to-grapheme ambiguity, grammatical errors (gender, tense, number), accentuation, punctuation and normalisation ( §5). we present an initial neural benchmark with a model trained with different types of augmented data ( §6), and finally, we release our spell-checkers as a web service ( §7), which is ready to deploy and use. finally, for the languages spoken in amazonia, there are only spell-checkers for shipibo-konibo @xcite and ashaninka @xcite . the former is a rule-driven approach with graphs and syllabic information, whereas the latter is a finite-state-transducer or fst. however, they work at word-level, meaning that they lack context and are at disadvantage when words are joined or split by mistake. ## languages and data the four languages in the focus of this paper are highly agglutinating and synthetic, meaning that they can compress a large amount of information in a single word composed of several bound morphemes, often with more or less clear-cut morphological boundaries. in addition, they do not have language # sentences |v| shipibo-konibo (shp) 22,032 22,904 asháninka (cni) 12,629 23,721 yanesha (ame) 13,241 23,626 yine (pib) 7,658 14,142 ## error augmentation approaches we create different augmented training sets (same size) with each type of error described as follows. ## evaluation corpora with the support of language teachers, we defined an error typology of the most common mistakes of their students: phoneme ambiguity, grammar mistakes (gender, tense, number), punctuation, accentuation, syntactic, semantic and normalisation. after that, we provide an annotation protocol to create a parallel corpus of corrected written sentences aligned with misspelled ones, with an annotation of the type of errors included in each sentence (it could be more than one): ## web service we implement an api and a web service that includes all the models presented in the previous section. the web interface includes the following features: (1) the user can select the language and model of preference, (2) the system highlights which words are updated, and what is the modification, (3) the user can modify the output and provide feedback. figure 1 shows an example. finally, we open-source our demo code in: https://github.com/iapucp/schaman-demo . ## how to scale up to new languages for new languages from amazonia, the first step is to obtain a monolingual corpus as seed text. according to the results, it is more significant to augment training data with the denorm and p2gamb approaches, which require a short involvement of an expert or the study of language grammars. this is less expensive than to develop an fst-based tool for spell-checking foot_5 . the creation of the evaluation resource is the most costly (in terms of expert hours), however, our methodology can be reproduced easily. ## conclusions and future work we develop spell-checking resources (for training and evaluation) and define an initial benchmark for four endangered languages of the amazonia region of peru. experiments showed that denorm and ensemble models achieve overall better results in most error types and languages, and they have a positive impact when dealing with new vocabulary. the spell-checking models are available as an api and web service, and it was made available to language teachers and students. as future work, we plan to develop multilingual models (three of the four targeted languages are from the same language family), and to deploy a more explainable spelling application (e.g. indicating which type of error has been corrected).