# Improving Text Recognition Accuracy for Serbian Legal Documents Using BERT  
[PERSON]  
1 Faculty of Electronic Engineering, University of Nis, 18000 Nis, Serbia; [EMAIL_ADDRESS] (M.F.G. ); [EMAIL_ADDRESS] (J.K.); [EMAIL_ADDRESS] (L.S.) 1  
[PERSON]  
1 Faculty of Electronic Engineering, University of Nis, 18000 Nis, Serbia; [EMAIL_ADDRESS] (M.F.G. ); [EMAIL_ADDRESS] (J.K.); [EMAIL_ADDRESS] (L.S.) 1  
[PERSON]  
1 Faculty of Electronic Engineering, University of Nis, 18000 Nis, Serbia; [EMAIL_ADDRESS] (M.F.G. ); [EMAIL_ADDRESS] (J.K.); [EMAIL_ADDRESS] (L.S.) 1  
[PERSON]  
1 Faculty of Electronic Engineering, University of Nis, 18000 Nis, Serbia; [EMAIL_ADDRESS] (M.F.G. ); [EMAIL_ADDRESS] (J.K.); [EMAIL_ADDRESS] (L.S.) 1  
Footnote 1: [[https://doi.org/10.3390/app15020615](https://doi.org/10.3390/app15020615)]([https://doi.org/10.3390/app15020615](https://doi.org/10.3390/app15020615))  
Footnote 2: https://www. task becomes even harder in cases where the language used within the documents is in the position of being low-resourced in the field of digital infrastructure and digitized language resources, such as the Serbian language [3]. The production of new corpora for a single domain starts with a repository constituting a single domain which often contains documents varying in size, but also in structure and layout. Nevertheless, domain experts are capable of extracting the desired information, but this process tends to be time-consuming and error-prone. The domain of legislation is an example of such a domain. Due to the nature of the legal domain, legal documents are often elaborated in terms of structure from a general document and can be too long to analyze and understand. Furthermore, they appear in different forms, such as legal contracts, law commission reports, tribunals, judgments, different acts, etc. Thus, information extraction from legal documents is highly desired [4; 5; 6], and it is foreseen to be significant for the development of AI tools in this domain. Deep neural networks have proven their potential in a range of document-related tasks such as semantic search and document retrieval, question answering, and document generation. The quality and performance of models built using deep neural networks depend on the representation and the quality of the text used for training. If a neural network model is built for a particular field and purpose, it will become more efficient and accurate if it is provided with high-quality text during its training process. Thus, the existence of high-quality text corpora is a prerequisite for building precise and efficient models in any field, including the legal domain [7]. Although it has plenty of resources, the Serbian language is considered to be low-resourced in the field of digitized language resources. Our focus in this research was the development of a BERT-augmented system named LexBERT capable of generating high-quality text corpora by improving text recognition accuracy for scanned documents belonging to Serbian legal heritage. We consider the improved text recognition procedure that we present in this paper to be a fundamental prerequisite to the development of reliable AI tools in the Serbian language legal domain. LexBERT focuses on a problem previously described within the scientific community [8; 9] and emerged from a general approach we define and describe in this paper. To demonstrate the usability of our approach, it was implemented for a specific domain--the legal domain. Thus, LexBERT emerged as a combination of three major components, two of which were specially developed for this system:  
* SrBERTa v2--BERT-based language model for Serbian legal texts written in Cyrillic;
* Optical character recognition engine--Tesseract OCR;
* Word-level similarity measure for Serbian Cyrillic used to compare OCR results and SrBERTa v2 suggestions in cases of lower OCR accuracy. The major contribution of this paper is within the approach, its components, and its overall capability.