VOLTAGE : A Versatile Contrastive Learning based OCR Methodology for ultra low-resource scripts through Auto Glyph Feature Extraction (2024)

Abstract: UNESCO has classified 2500 out of 7000 languages spoken worldwide as endangered. Attrition of a language leads to loss of traditional wisdom, folk literature, and the essence of the community that uses it. It is therefore imperative to bring digital inclusion to these languages and avoid its extinction. Low resource languages are at a greater risk of extinction. Lack of unsupervised Optical Character Recognition(OCR) methodologies for low resource languages is one of the reasons impeding their digital inclusion. We propose VOLTAGE - a contrastive learning based OCR methodology, leveraging auto-glyph feature recommendation for cluster-based labelling. We augment the labelled data for diversity and volume using image transformations and Generative Adversarial Networks. Voltage has been designed using Takri - a family of scripts used in 16th to 20th century in the Himalayan regions of India. We present results for Takri along with other Indic scripts (both low and high resource) to substantiate the universal behavior of the methodology. An accuracy of 95% for machine printed and 87% for handwritten samples on Takri script has been achieved. We conduct baseline and ablation studies along with building downstream use cases for Takri, demonstrating the usefulness of our work.

## introduction the unesco "atlas of the world's languages in danger" (unesco, 2021) is considered as a benchmark for the comprehensive list of the world's endangered languages. this study unveils more than 2500 languages and dialects as endangered out of which 200 come from indian demography. optical character recognition (ocr) is used for digitizing historical archives, helping language con-servation. there are plenty commercial and opensource ocr engines available for contemporary documents. however, very low resource scripts (lrs) differ in their requirements mainly because of non-availability of large volume of data and limited users. the two most popular unsupervised (or semi-supervised) ocr methods available include ocular @xcite and anyocr @xcite . both methods are designed for large datasets and hence cannot be applied to lrs effectively. another alternative is to apply pretrained models for a high resource language as a foundation model and apply few shot learning to customize it and get the desired results. our experiments conducted for "takri" using this approach do not result in good accuracy. we have discussed this in detail in results section. we develop an automated versatile unsupervised ocr methodology (voltage) for very low resource scripts to address the gap. we use takri as an example to develop our methodology due to @xcite no available labelled data and scanty user base (b) extremely low digital unlabelled resources. we further evaluate the proposed methodology on four other languages to validate the universal behavior. as illustrated in figure 1 , voltage comprises of four steps, (a) extraction: segmentation of available data into pages, lines, words, characters and symbols; (b) annotation: feature extraction and recommendation followed by cluster based labelling; (c) re-enforcement: augmentation of dataset using image transformation and generative ai (gans); (d) identification: contrastive learning based classification for character identification. the novelty of the methodology is that the manual intervention including human oracles is bareminimum. the proposed glyph pattern-based feature recommender system can be applied to any script to recommend apropos feature set. we empirically discuss our results in detail for takri and also evaluate on other indic scripts to validate its generalization capabilities. we also conduct baseline and ablation studies to substantiate our results. the contribution summary of our work is three fold: ## voltage: the proposed methodology versatile contrastive learning based ocr methodology for ultra low-resource scripts through auto glyph feature extraction (voltage) follows the pipeline of tasks including, pre-processing and segmentation, automated feature engineering and unsupervised labelling, data augmentation and classification, post processing and evaluation (see figure 1 ). we validate our results for takri on end to end errors and character/word error rates. we further validate voltage for modi, ol chiki, gujarati and wancho to establish the universal effectiveness of our work. we use python 3.8.12 with conda, opencv for image processing along with ml libraries (keras, numpy, transformers) for our experiment. ## results and discussion quantification of errors in ocr pipeline, specially for very low resource scripts is ambiguous unless properly defined @xcite . the document page needs to be segmented into lines, words, characters and symbols for ocr engine. any error which occurs in this step would fall under layout segmentation error. most ocr literature do not include segmentation errors as part of ocr errors. ocr systems generally consider errors either for the entire end to end (e2e) pipeline, at word level or character levels. e2e includes errors at various stages of pipeline such as pre-processing, segmentation, classification, and post processing. word recognition accuracy or word error rate (wer) is the average percentage of mis-recognised words. character errors rate (cer) is the ratio of misrecognised symbols within accurately segmented symbols. the concept for character/symbol is also ambiguous unless defined clearly, due to the linguistic peculiarity for each script. symbols are monolithic for alphabetic languages (like english) which uses each symbol in same form, however for abugida scripts (like indic scripts) symbols are of multiple types, namely (a) root symbols: like ("ka") and most ocr studies, considers errors within (0-2%), (2-10%) and (>10%) as good, average and poor @xcite , respectively. however for indic scripts with limited training data and unknown vocabulary along with heterogeneous handwritten forms, a cer value as high as around (10-20)% is @xcite . we compute many error metrics (see ## conclusion and future directions the paper presents a comprehensive unsupervised ocr methodology, voltage, which includes a novel glyph feature recommendation system (gfrs) for effective symbol labelling. we developed voltage using a very low resource language, takri, and validated its effectiveness and generalization on various indian scripts. we achieve accuracy at par with sota of respective test scripts. we also build use cases for takri to demonstrate the usefulness of the work. our work can facilitate the digitization of ultra low resource scripts thereby save them from extension. as part of our future work, we shall use our method to build more comprehensive datasets along with building the vocabulary for indic languages to help in error correction during post processing. we also plan to use the method as described to digitize more languages within india partnering with local governments.