A Concise Survey of OCR for Low-Resource Languages (2024)

Abstract: Modern natural language processing (NLP) techniques increasingly require substantial amounts of data to train robust algorithms. Building such technologies for low-resource languages requires focusing on data creation efforts and data-efficient algorithms. For a large number of low-resource languages, especially Indigenous languages of the Americas, this data exists in image-based non-machine-readable documents. This includes scanned copies of comprehensive dictionaries, linguistic field notes, children’s stories, and other textual material. To digitize these resources, Optical Character Recognition (OCR) has played a major role but it comes with certain challenges in low-resource settings. In this paper, we share the first survey of OCR techniques specific to low-resource data creation settings and outline several open challenges, with a special focus on Indigenous Languages of the Americas. Based on experiences and results from previous research, we conclude with recommendations on utilizing and improving OCR for the benefit of computational researchers, linguists, and language communities.

## introduction latin america is home to a linguistically diverse set of hundreds of indigenous languages. many of these are low-resource in terms of text and audio resources, and generally lack basic natural language applications such as spell checkers, part of speech (pos) taggers, etc. however, these languages have a large number of digital resources (not machine-readable) in the form of recordings, plays, stories, and dictionaries. one major repository of such materials is the archive of the indigenous languages of latin america (ailla). 1 of the documents in ailla's collection, particularly interesting to nlp researchers are linguistic materials such as grammars, dictionaries, ethnographies, 1 a joint effort of the llilas benson latin american studies and collections and ut austin. and field notes, that can serve as training data for nlp applications and optical character recognition (ocr). releasing digitized versions of such a repository of hundreds of datasets can preserve invaluable linguistic materials and accelerate research in nlp. modern ocr can extract text from such documents, but this requires accurate layout detection and post-processing to make the extracted text usable for downstream nlp tasks @xcite . ocr is a well-established field, with its advances mostly drawing from innovations in computer vision. more recently, ocr has been increasingly used for resource-creation for lowresource languages in nlp contexts @xcite . there are also several excellent surveys and tutorials @xcite @xcite on building and using ocr for broad applications, however, there is a dearth of specialized surveys for low-resource language ocr. therefore, the aim of this paper is to fill this gap and acquaint researchers and language ## undigitized data over the past decade, many researchers, linguists, and consortiums have worked closely with native speakers and language communities to create datasets including digitized text, audio, transcriptions, translations, stories, etc. some of these resources may not be machine-readable but include extremely valuable resources from an nlp perspective, such as multilingual lexicons, pronunication guides, plain text from a wide-variety of domains such as stories, essays, plays, news, linguistics etc. a comprehensive guide with resources in all lowresource languages (over 6000+) would be valuable but is out of the scope of this paper, so we focus on highlighting relevant resources in 10 american indigenous languages. the ailla collection contains several textual corpora in non-machinereadable image format for the selected languages in a large majority of ocr work for low-resource settings includes preservation and digitization of historical data, early printed books @xcite , palm-leaf manuscripts @xcite @xcite etc. preexisting repositories (such as pubmed or arxiv) are also widely used for training ocr systems @xcite , however, note that this approach is not scalable to lowresource settings which often lack such ready-touse datasets. for the americas, due to widespread adoption of extended latin alphabets in writing, texts from the last couple of centuries are often typed, but several collections include partially or completely written handwritten documents and annotations. historically used typing fonts may be challenging to decipher or out of use due to orthographic reforms @xcite @xcite , and handwriting varies widely across individuals, making extraction challenging @xcite @xcite . over time, language communities may even adopt new orthographies, which might require researchers to build new keyboards and transcription systems to make the digitized corpora readable by community members @xcite . digitizing these resources can allow for more accessible linguistic research, training language models, translation systems, pos taggers, etc. the ailla collection constitutes of a healthy mix of both typed and handwritten text. as evident from note that in our concise survey paper, our aim is not to digitize these specific books -that would warrant separate carefully designed studies as each resource is bound to have unique challenges and is connected to language communities with possibly different language technology needs. this paper highlights, for researchers unfamiliar with these languages and domain, different resources available for experimenting with ocr modeling approaches and recommended workflows for achieving such digitization. ## a concise survey of ocr now that we've seen the data resources available for our 10 selected indigenous languages ( §2), we will highlight useful and practical ocr adaptations and innovations necessary for digitization of such low-resource language data. we cover techniques in four major parts of the digitization pipeline: preparation of the data and model, active training, decoding or generation, and post-processing. to ground the following discussion, we will define an example dataset c, with k pages, where p i represents separate pages. l represents the paired labels for each p @xmath0. ## open problems layout preservation one of the most pressing issues remaining largely unsolved in ocr literature is that of structure preservation. ocr tools, especially those off-the-shelf may not good preserve the layout of the page in the output ocr text accurately and might require manual post-ocr alignment @xcite . the detected bounding boxes may not follow a logical layout as would be expected by human inspection. this means that researchers need to perform some level of alignment after getting the ocr outputs @xcite , before applying ocr models @xcite , or cropping each image into separate line-level images (may be financially impractical if using commercial systems). from a resource-creation perspective for indigenous languages, preserving structure in the final output is extremely important, so we recommend that researchers think about how to design their experiments early on to address this issue. to the best of our knowledge, while previous work has focused on layout detection as a first-step @xcite , it has not been explored as a post-processing step, primarily due to a lack of ground-truth structural data. previously, two major studies @xcite have used existing large-scale corpora like arxiv to extract large-scale ground truth (source-code); but, this approach is not scalable to resource-creation efforts involving low-resource languages. to build such a structure post-correction model, annotators would be required to not only correct the text in the ocr but also structurally correct the first-pass ocr outputs in some kind of graphical user inter-face (as shown in figure 2 ). this would involve scaling, translating, merging, or splitting bounding boxes, while keeping the text within faithful to the each box's new coordinates. such a task could be framed as follows: for every text-corrected page q i , we output a corrected page r @xmath1, c 1 ), (y 2 , c 2 ), .., (y m i , c m i )] where m i denotes the number of new bounding boxes after post-correction (may be different from n i ). we consider human-corrected r i as the groundtruth text and layout. note that while this step mainly transforms the structure, it would also involve transferring the initially corrected text (b i , b i+1 , etc) from the first-pass boxes that now make up the corrected box y i , and therefore, the texts are labeled as c i . however, since such structural post-correction ground-truth data may be expensive to obtain, researchers may also consider getting this ground-truth from a dedicated layout detection model automatically, and post-correcting output from the best first-pass ocr system to adapt to this automatically-extracted desired layout. atypical characters, fonts, and words modeling historical orthographic variations with modernday lms, trained on current spelling conventions, can prove challenging during the decoding step @xcite . work on better text extraction from historical documents from the printing press era resulted in the development of the popularly used unsupervised ocular model @xcite . synthetic data has been successfully used before to offset the effect of atypical characters and typefaces @xcite , and unsupervised techniques have been used to automatically learn the font style of a document in the context of historical document recognition and ocr @xcite . however, research is still limited in the low-resource domain and researchers would need to ensure that their fonts and character sets are supported by their chosen ocr model (if training one from scratch) or are reconstructed using recent work in visual representation learning @xcite . for off-the-shelf systems (interfaced through apis), it is not possible to directly include support for unique characters and researchers will need to add a post-correction step that includes a mix of post-processing scripts for easily resolvable errors and dedicated trained supervised models to correct first-pass output. linguistic diversity for researchers working with several low-resource or indigenous languages at the same time, it can be desirable to train one model that is capable of handling different writing systems, diacritics, differing image qualities, and unique document formatting @xcite . while one approach may be to develop 'languageagnostic' methods, previous work has shown that in practice such models are far from languageagnostic @xcite and tend to have high-performance only for a handful of languages. high ocr accuracy is usually desirable for all the low-resource languages under consideration, and in such a scenario, it may be best to train separate ocr or post-correction models. ## workflow recommendations in this section, we share recommendations based on the most successful strategies followed in the surveyed papers. this can serve as a starting point for computational researchers, linguists, and students new to the low-resource domain. we acknowledge that these recommendations, while grounded in our survey, are still subjective and researchers may need to modify some elements to suit their specific cases. language and document selection to anchor our survey, we selected 10 languages that have permissive licenses, use the latin alphabet, whose special diacritics were available on the english keyboard, and which had typed documents in common fonts. similarly, when selecting documents for their languages of interest, researchers should consider licensing, need for special keyboards, quality of the document, and layout diversity. evaluation techniques for evaluation of the ocr quality, we recommend simply looking at the final output i.e. text. provided there is a reference or gold text, these predictions can be compared to them and a cer/wer can be obtained. there is no gold standard target cer/wer so researchers will have to inspect the quality of the output and decide, with feedback from language community members, the cer/wer they would like to target in any potential modeling. note that a line-aligned version of the extracted text will be required and this may either be obtained by only ocring at the line-level (after cropping) or by aligning the predicted text with the reference text using metrics such as levenshtein distance. preliminary experiments for preliminary experiments, we recommend that 1-2 lead researchers manually annotate and audit a modest sample of the dataset themselves. this can help ensure that the researchers and language community members are familiar with the annotation workflow and can better guide any future annotators. conducting some annotation before running ocr experiments is crucial because there needs to be some standard set for evaluation of all the models we will now experiment with. once a few pages have been annotated, researchers can begin the ocr process using a common off-the-shelf ocr method like google vision or ocular. data annotation now that the researchers have an idea of the quality of off-the-shelf ocr systems, we recommend that they recruit annotators to annotate a larger sample of the data if the ocr quality was found to be low. annotators don't need to be speakers of the indigenous languages selected; however, they should have basic pattern recognition, data annotation, and typing skills. previous work has shown that annotators without knowledge of the indigenous language are fairly adept at performing ocr corrections, if they can read the language's script and distinguish between any new diacritics @xcite . annotators should be trained to use the annotation platform using standardized guidelines, and a manual audit should be conducted by lead researchers to ensure compliance. post-correction if the performance from preliminary experiments is satisfactory, we recommend post-correcting to further improve the results. a post-correction model should ideally help reduce character-level errors down to less than 5% @xcite @xcite . as discussed in §3, we recommend that researchers use a combination of copy mechanism, coverage, diagonal/positional attention, and active learning to improve performance. @xcite implements most of the necessary post-correction features and their code can be used directly to train post-correction models for low-resource settings. training from scratch on the other hand, if the preliminary experiments reveal that the error rates are quite high, researchers can consider simply training a custom ocr system from scratch. this will require a sizeable amount of annotated pages for training in addition to computational expertise in settling on the best hyperparameters and setting up the training pipeline. human-annotations collected in the data annotation phase can be used to train ocr models from scratch and first-pass outputs can be used to train further with postcorrection models. we recommend using opensource tools like tesseract @xcite or ocular @xcite to train custom models due to their efficiency, optimizations, and active user community. more advanced researchers may also consider writing their own architecture and training pipeline from scratch. however, note that training systems from scratch is not straightforward, and researchers are bound to run into challenges. for instance, tesseract has a high setup time and learning curve, doesn't have any graphical user interface, and requires high-quality images which may not be available for certain low-languages or image collections. deployment and improvements for in-house use, the final trained model can be used directly to digitize the entire corpus and any other collections in that language. we recommend that computational researchers and language community members stay in touch throughout the training, annotation, and deployment process, and flag any issues with ocr quality and modeling. in some cases, if sufficient ocr quality is not being achieved despite trying the aforementioned techniques, some concessions and further data selection and annotation may be required. for instance, the quality of the data itself might need improvement (redoing scanning of the original source text), another phase of annotation may need to be conducted for substantially more data, or some unique algorithmic techniques may need to be developed to achieve quality ocr for the particular documents. we recommend using labelstudio @xcite , which is an open-source labeling and annotation platform. the user interface is highquality, user-friendly, and quite simple to setup and share with collaborators and annotators. there is also an active labelstudio slack where issues get resolved relatively quickly. and can even be used in real-time due to test-time efficiency @xcite @xcite . ocr can involve extracting characters, words, paragraphs, and even preserving the layout of text on a page or in an image. ocr is widely used in the digital humanities @xcite @xcite and in businesses since it is a necessary step for digitization of rare manuscripts, books, linguistic field notes, invoices, business documents etc. it is also an invaluable technique in creating new data for low-resource languages for downstream nlp tasks and applications @xcite . in the last two decades, several excellent surveys from the computer vision community have been published covering ocr developments @xcite @xcite . in the low-resource domain, hedderich et al. (2021)'s survey covers broad nlp advances but it does not cover optical character recognition. to the best of our knowledge, other than ours, no previous work has surveyed ocr for low-resource languages. resource creation text or image-based datasets and corpora are most commonly created by scraping or crawling the web; however, we would like to highlight a few additional ocr-created datasets, especially those that work with american indigenous languages other than those reported in ## conclusion in this paper, we have presented a concise survey of optical character recognition (ocr) techniques shown to be most applicable to low-resource languages in the ocr literature. the survey is focused and similar work has not been published before due to the small community of ocr re-searchers working with low-resource and indigenous language communities. we also highlight undigitized datasets in 10 central and south american indigenous languages, mostly from the ailla collection, that can be extremely valuable to digitize for downstream nlp applications. based on our own experiences and on findings from our literature review, we conclude with recommendations on utilizing and improving ocr for the benefit of computational researchers, linguists, and language communities. we hope that our paper can be used as a starting point for researchers or language community members wishing to digitize their resources but unaware of what ocr adaptations have become absolutely necessary to move towards a high-quality ocr output as well as what the open challenges in the field are.