# Improving Text Recognition Accuracy for Serbian Legal Documents Using BERT
## 5 Conclusions  
The results presented in this paper confirm that producing new corpora in any field of digital humanities is a big challenge. Even with the help of tooling and approaches such as the one presented in this paper, producing a new high-quality corpus for low-resourced languages comes with complexities, labor, and obstacles. Nevertheless, we are convinced that the approach we presented in this paper offers a step forward in terms of efficiency and quality when generating a new corpus out of legacy documents written in the Serbian language. Figure 8: Example of a scanned paragraph, year 1983. Figure 9: Merged OCR output for scanned paragraph in Figure 8.  
Starting with an image, through text extraction using OCR and word reconstruction augmented with a specially designed BERT-base deep neural network, our approach has shown very promising results. Although it is still quite dependent on OCR text extraction quality and capability, our approach is currently capable of correcting up to 88.3% of incorrectly extracted words in a single image. Furthermore, the evaluation we have conducted proves that there are many situations that we can use to improve accuracy and provide more accurate results. For example, we have determined that of all tests in which the approach did not provide an appropriate correction proposal, in over 41% of these tests, the model we developed suggested a synonym to the expected word, a correct word expressed in a different form or a different case, or the expected word consisting of multiple tokens. As previously stated, the evaluation we performed offers possibilities for improving our approach. We would like to provide an outlook on the first step we will implement to improve the quality of word reconstruction. The most obvious approach is an improvement in the quality of suggestions given by the SrBERTa model that we developed for these purposes. Currently, SrBERTa efficiency can be improved through the usage of higher-quality datasets originating from different epochs of Serbian language usage. Our model would benefit from different datasets during the initial training phase, but also from the usage of a larger dataset containing Serbian legislation. Moreover, case sensitivity is one of the aspects that should be improved during dataset cleaning and the SrBERTa training process. Furthermore, the SrBERTa architecture is sensitive to the size of the paragraph used during the word reconstruction process, since the paragraph plays the role of providing context for the SrBERTa model. This is a consequence of the RoBERTa architecture we used as a foundation, the masked language modeling problem definition, and the fact that OCR paragraph detection varies in terms of quality and size. By augmenting the size and the quality of the context given to the model, combined with previously stated improvements, we expect the quality of suggestions to improve, giving us the ability to improve reconstruction precision. M.B. : Conceptualization, Methodology, Software, Writing; M.F.G. : Software, Data Curation, Methodology, Writing; J.K.: Software, Writing; L.S. : Supervision. All authors have read and agreed to the published version of the manuscript. This work was supported by the Ministry of Science, Technological Development and Innovation of the Republic of Serbia [grant number: 451-03-66/2024-03/200102]. Not applicable. The model presented in this study is openly available at [[https://huggingface.co/[PERSON]/SRberta](https://huggingface.co/[PERSON]/SRberta)]([https://huggingface.co/[PERSON]/SRberta](https://huggingface.co/[PERSON]/SRberta)) (accessed on 26 December 2024). The Corpus of Legislation texts of Republic of Serbia 1.0 used within this study is available at [[https://www.clarin.si/repository/xmlui/handle/11356/1754](https://www.clarin.si/repository/xmlui/handle/11356/1754)]([https://www.clarin.si/repository/xmlui/handle/11356/1754](https://www.clarin.si/repository/xmlui/handle/11356/1754)) (accessed on 26 December 2024).