G r E ma: an HTR model for automated transcriptions of the G irifalco asylum’s medical records (2025)

Abstract: This paper deals with the digitization and transcription of medical records from the historical archive of the former psychiatric hospital of Girifalco (Catanzaro, Italy). The digitization is carried out in the premises where the asylum once stood and the historical archive is stored. Using the ScanSnap SV600 flatbed scanner, a copy compliant with the original for each document contained within the medical records is returned. Subsequently the different training phases of a Handwritten Text Recognition model with the Transkribus tool are presented. The transcription aims to obtain texts in an interoperable format, and it was applied exclusively to the clinical documents, such as the informative form, the nosological table and the clinical diary. This paper describes the training phases of a customized model for medical record transcription, named GrEma, presenting its benefits, limitations and possible future applications. This work was carried out ensuring compliance with current legislation on the protection of personal data. It also highlights the importance of digitization and transcription for the recovery and preservation of historical archives from former psychiatric institutions, ensuring these valuable documents remain accessible for future research and potential users.

## introduction the historical archives of former psychiatric hospitals represent a cultural written heritage of inestimable value for different research areas. they are multidisciplinary resources, providing a comprehensive insight into the history of psychiatry, the role of asylums in mental illness treatment, and the evolving cultural and social perception of mental illness over time. following the entering into force of the basaglia law in 1978, which mandated the definitive closure of asylums in italy, there has been a growing recognition of the importance of studying these archives; among them the historical archive of the former psychiatric hospital of girifalco @xcite attracted the interest of many scholars. in southern italy, before the official opening of the girifalco asylum, patients were systematically hospitalized in the royal asylum of aversa (caserta) until, in july 1877, new admissions were barred due to a lack of space. consequently, most patients in the province of catanzaro were kept in private homes under the custody of family members and friends. initially, the provincial deputation attempted to distribute the hospitalizations across the national territory, but due to considerable logistical and administrative difficulties, it was necessary to establish a new asylum in the calabria region. as a result, in 1878, the provincial deputation approved the city of girifalco as the site for new asylum and the decree for its opening was issued in 1880 @xcite . the girifalco asylum officially opened in 1881, and it became a point of reference for patients suffering from mental disorders, not only coming from the region. in almost a century of activity, 22,415 hospitalizations were recorded. however, today, the historical archive contains 15,794 medical records. the discrepancy between the number of hospitalizations and the number of medical records in the historical archive is due to multiple admissions of the same patient, as well as the loss and deterioration of some documents before their current arrangement @xcite . the medical records stored in the historic building of the former psychiatric hospital of girifalco contain various documents, produced not only for clinical purposes but also for administrative and socio-demographic reasons, effectively representing patients' personal files. all these documents are handwritten texts in italian dating back to the 19th and 20th centuries. users seeking access to the content of a medical record in this archive must first obtain authorization from archival superintendence and subsequently visit the archive in person to consult the medical record. nonetheless the documents could be not always fully understandable, as the legibility of the medical records may potentially be affected by their preservation status and the handwriting style. based on the access requests received so far, possible stakeholders and users interested in accessing the medical records are primarily scholars seeking to analyze their contents for various research purposes. there have been genealogical studies on the incidence of neurodegenerative diseases @xcite @xcite , examinations of how the lexicon and language used by clinicians and patients have evolved over time @xcite , reconstructions of the psychiatric history and the activity of this mental institution @xcite , as well as various anthropological studies @xcite . additionally, some users may be interested in exploring their clinical family history or understanding the reasons why one of their ancestors or relative was institutionalized. however, paper-based medical records can be difficult to consult because they require an on-site visit or because handwriting may be illegible. moreover, frequent handling can accelerate their deterioration. therefore, to make accessible the knowledge conveyed in these documents, digitization and transcription are two crucial processes that can improve the ability to analyze and interpret this cultural written heritage. in the archival domain, a good way to overcome barriers and issues related to document consultation and their accessibility is the adoption of innovative technologies for information retrieval in a digital environment. digitization and transcription can be key processes to increase the accessibility and usability of documents @xcite . the aim of this work is therefore to facilitate and promote the accessibility of this historical archive by providing authorized users with a digital and machine-readable format of the medical records. this paper describes the digitization and transcription processes carried out on the medical records of the former psychiatric hospital in girifalco, with a focus on the training phase of a handwritten text recognition (htr) model using transkribus foot_0 , a tool which allows automated text recognition and transcription. the main aim is to present an automatic text recognition model that could improve the intelligibility and the interpretability of these historical documents. in detail, this paper is structured as follows: section ii provides an overview of the types of data and documents considered. section iii outlines the techniques generally employed to preserve and protect archival heritage, focusing then on the digitization and transcription processes implemented to recover the historical archive of the girifalco asylum. finally, section iv details the training phases of grema, our htr model for medical records, and presents the achieved results. ## data and documents in italy, the growing interest in the history of psychiatric institutions has led to several initiatives aimed at improving access to their historical archive. many projects have been carried out to enhance the access to their written cultural heritage @xcite @xcite ; among these, the general directorate for archives of the italian ministry of culture promoted the project carte da legare foot_1 , which proposes an organic vision for the protection of the archival heritage of former psychiatric hospitals by surveying, reorganizing, and enhancing their archives @xcite . the girifalco asylum's archive is partially inventoried in carte da legare thanks to the metadata reconciliation carried out by the institute of informatics and telematics of the national research council (iit-cnr), through the project alpha (easy intelligent service platform for healthy ageing). it involved the digitization of over 5,000 medical records, covering the period 1881-1931, and the manual transcriptionsthrough a dictation software -of the first 540 in chronological order. for the purposes of this work, clinical documents were taken into consideration, particularly the nosological table, the informative form and the clinical diary. the nosological table, compiled by the psychiatrist, includes personal, sociodemographic, and clinical information -such as etiology, diagnosis, hospitalization outcome, and medical observations. the informative form instead collects the patient's and family's health history at the time of the admission. finally, the clinical diary consists of a set of notes relating to the patient's conditions during the hospital stay. all medical records are handwritten in italian and present syntactic and lexical characteristics, related to the psychiatric domain and the influence of the local dialect, that could make their interpretation more challenging. ## archive accessibility the italian code for cultural heritage and landscape, article 6 (1), states that the enhancement of cultural heritage consists in the exercise of functions and activities aimed at promoting its knowledge and ensuring the best conditions for its use and public enjoyment (code of cultural assets -legislative decree january 22, 2004, no. 42). enhancing accessibility is particularly crucial for historical archives of former psychiatric hospitals, as it is essential for preserving and valuing this collective memory while also fostering research on the documents. the digitization of the historical archive of the girifalco asylum is therefore necessary to allow easier access without the need to directly consult the original documents. digitization involves converting analog archival materials into digital format using specialized acquisition technologies. however, a proper digitization process of an archival document requires more than just a photographic acquisition. it is essential to provide an accurate transcription of every word to ensure the content is available in an interoperable format suitable for analysis. ## grema: a transcription model for medical records this paragraph aims to present the work phases carried out to train an htr model for the medical records of the former psychiatric hospital of girifalco, using transkribus. the first phase involved the construction of the ground truth. considering the different types of documents present within the medical records, it was decided to transcribe only clinical documents, as they represent a valuable source of knowledge for countless future research. specifically, the transcription was focused on the following clinical documents: i) the nosological table, ii) the informative form, iii) the clinical diary, and iv) the patient correspondence. considering the sensitive nature of the data processed, particular attention has been paid to the protection of personal data contained within these documents, in order to ensure compliance with current legislation. in fact, despite transkribus privacy policies stating that documents are stored on read coop servers in compliance with the european general data protection regulation (gdpr), it was decided to avoid uploading to the platform all administrative documents, cover pages of the medical records and documents containing personal data capable of making a patient or one of his/her family members identifiable. being aware of the efforts that would be required to train a customized model, it was at first decided to test one of the already available public models on transkribus, in order to evaluate its performance on girifalco's medical records. in particular, an attempt was made with one of the most developed public models for the italian language, which is transkribus italian handwriting m1. this model is specifically designed for handwritten italian text from the 16th to the 19th century and has a cer of 6.70% foot_3 . while model m1 is designed for documents from a period close to that of the medical records used in this work, it did not perform optimally on these specific documents. the factors that likely compromised its effectiveness were the peculiar handwriting styles, the specialized vocabulary related to the psychiatric domain, and the page layout. consequently, it was decided to train a specific model for the medical records. we decided to call it grema from the conjunction of the names of the researchers who developed it. generally, carrying out an effectively handwritten text recognition process requires around 15,000 transcribed words (approximately 75 pages), while printed text requires around 5,000 words (about 25 pages). for this reason, it was initially necessary to ensure that we had a ground truth composed of a representative number of images of the medical records and their corresponding transcriptions. the first training session was conducted with a total of 20,776 words (approximately 130 pages), while the final dataset used to train the model consists of 94,624 words. grema was trained using pylaia as its engine, which is based on pytorch. a 10% portion of the original dataset was selected as validation set, as this was considered the most suitable option given the relatively limited number of transcribed pages. grema used the public model m1 as its preexisting base to leverage already available data for improved transcription performance. in fact, transkribus offers a feature that allows users, during the training of their own custom model, to build upon existing public models. this functionality enhances the learning process by leveraging previously trained data, leading to improved performance. after the first training session, the cer was 19.90%. it decreased to 16.92% in the second training phase with a dataset of approximately 34,500 words. further training reduced the cer to 14.70 %. grema was subsequently trained, achieving additional improvements in the cer value, which settled at 14.04%. in this case, the word error rate (wer) was 39.03%. this metric indicates the percentage of words in the automatic transcription that do not match those in the ground truth transcription. however, wer tends to be higher than the cer, because a word is considered incorrect even if it differs from the reference by just a single character. as a result, wer may not always provide a fully reliable or representative measure of the model's performance, as such discrepancies do not necessarily make the transcription unintelligible, although manual correction is still required to ensure full accuracy. all the training phases and the related results obtained are summarized in when training a model with transkribus, once the model achieves a relatively low cer, further reducing errors become increasingly challenging. this is because the model has already learned most of the patterns from the data, and additional improvements require a significantly larger amount of training data. in figure 2 , it is shown how the cer is also influenced by the progressive increase in the number of training epochs: as the number of epochs increases, the cer decreases. another important aspect to take into consideration is that when a model is used to make a transcription, the output is not a direct transcription but rather a confidence matrix that assigns probabilities to the presence of each character in a specific position within the text. consequently, the output is not influenced by the grammatical structure or syntax of the target language. the model may assign high probability to characters that visually resemble the original handwriting but may not form meaningful words or follow the rules of the language. this is because the model focuses on character recognition rather than understanding the context or meaning of the text. the character-based probability approach also explains why grema often struggles with visually similar characters. for instance, the confusion between u and n, o and a, or s and r is common. these errors occur because the model relies solely on visual patterns rather than contextual understanding. such mistakes are not always present but sometimes tend to appear in context where even a human reader might have difficulty in distinguishing the characters. as shown in fig. 3 , grema produced a good transcription of the medical records, but still containing some kinds of errors. fig. 3 : an example of transcription made with grema, with cer at 14.04%. in this example, the word "emotivo" is incorrectly transcribed as "emativo"; "addietro" appears as "addietso"; and "d'animo" is transcribed as "d'ammo". in particular, the last two errors are unlikely to occur with human transcribers, whose syntactic knowledge of the language would allow them to infer the correct word even when not every letter is clearly legible. in addition, human transcribers would be aware of which sequences of letters are grammatically acceptable in the target language. therefore, additional post-processing is often necessary to validate the model's output and produce more accurate and readable transcriptions. in the future, this step could be crucial for fixing character misrecognitions and refining the text, especially when the model struggles with complex handwriting styles. however, identifying errors in certain transcription contexts opens up the possibility of automating the correction process by defining fixed rules, for instance with python. using regular expressions, dictionary-based checks, and language models might be useful to create automated scripts that detect common misrecognitions and apply predefined corrections, reducing the need for manual intervention and improving the overall efficiency and accuracy of the transcription process. however, recently we trained our model again, but despite using a training set composed of 130,715 words, the cer did not decrease as expected but instead rose to 18.23%. an explanation for this increased value can be attributed to the inclusion of new handwriting styles in the analyzed dataset. the introduction of new handwriting styles remains a primary adaptation challenge for a model. nonetheless, as medical records have been written by different hands over the years, it is not uncommon to find multiple handwriting styles even within the same document, as often happens in the clinical diary, because it was used to document the patient's stay in the hospital, and the physician on duty was responsible for making the necessary entries. for handwritten texts, optimal performance is achieved when the texts are written by the same hand. however, within the same historical period, similarities can be observed in the way certain characters are shaped. consequently, even when different handwriting styles are present, these similarities can help the model recognize a character or a sequence of characters with high probability. for instance, at the end of the 19th century, it was common practice to use a single uppercase s to indicate two s in the middle of a word, and we found it often recurring into the girifalco's medical records too. however, if the validation set includes a handwriting style that is underrepresented in the training dataset, the cer will inevitably be higher, even though the model may still provide highly effective transcriptions. this highlights how, in some cases, the cer value may not accurately represent the model's final performance. in addition to the challenges posed by the continuous integration of new handwritings into the model, another significant limitation of this work arises from the need to manually transcribe specific pages of medical records, in order to protect sensitive and personal data. this manual transcription is necessary to safeguard sensitive information, ensuring that no private data is inadvertently shown. at the same time, this process introduces additional human efforts, hindering the ability to benefit from a fully automated transcription process. despite these limitations, the use of automatic htr systems remains promising, as it helps reduce transcription times (as transkribus allows to upload on the platform several documents to be transcribed simultaneously) and aids in recovering document content. in fact, one significant advantage of automatic transcription is its ability to recognize words that could be challenging for a human interpreter to decipher. this capability allows for the retrieval and transcription of text portions that would otherwise be lost with traditional methods, especially when dealing with handwriting styles difficult to read. ## conclusion this paper presents the digitization and the transcription activities realized to preserve the medical records of the former psychiatric hospital of girifalco. in particular, it describes the training phases of a customized model for transcribing clinical documents, named grema, outlining the steps taken to build it and the results achieved. the digitization of the historical archive of the former psychiatric hospital of girifalco required particular attention to ensure that the documents were digitized, preserving their integrity. initially, careful planning of the digitization process was required, as it was necessary to physically reach the location where the archive is stored in order to go on with the digitization. furthermore, the scanning equipment could not remain in the premises of the archive, meaning that it had to be disassembled and reassembled each day, with the equipment being transported back and forth. to address these needs, the equipment was carefully selected for its portability, leading to the decision to use a portable scanner. considering the large volume of documents to be digitized, it was crucial to carefully evaluate how to manage the long-term preservation and the storage space, particularly in terms of selecting the most suitable file format. consequently, the pdf format was chosen, as it can be optimized to reduce size while maintaining high quality. pdfs are ideal for archiving due to their data integrity, security features, and the ability to ensure long-term preservation and accessibility. as the digitization, the transcription presented some challenges, including the data protection aspects and the adaptation of the htr model to the documents' peculiarities. to ensure gdpr compliance, it was decided not to process administrative documents or others containing sensitive information. as concerns the documents characteristics, grema was trained with progressively larger datasets to improve its accuracy. however, while the cer decreased over multiple training phases, ultimately reaching 14.04%, integrating new handwriting styles led to unexpected increases in errors. obstacles such as confusion between visually similar characters persisted, highlighting the need for post-processing to refine the results. despite these challenges, automatic transcription offers significant advantages. it reduces the time required for the transcription process, enhances data accessibility, and enables the recovery of text that might be difficult for human readers to decipher. although transcribed texts are in natural and unstructured language, different natural language processing techniques could be applied to analyze them in the future, exploring their contents, increasing inferences, and creating new research opportunities, for example, in the fields of linguistics, medical history, and neurodegenerative diseases. in fact, the digitization and transcription of these medical records not only allow to preserve important historical data but have the potential to transform them into dynamic resources for multidisciplinary research, improving both their accessibility and usability providing new opportunities for research. a key future direction of this work involves the development of a digital platform inspired by the model of the cambridge digital collection platform (cdcp) adopted by the cambridge university. the idea is to make available on this platform all digitized medical records, according to the iiif (international image interoperability framework) standard, and their corresponding transcriptions, encoded in xml-tei. this would allow authorized users to engage with enriched versions of the medical records, accessing both a faithful copy of the original documents and its structured textual transcription. the adoption of xml-tei encoding would further enable users to navigate the internal structure of the documents and their contents according to their specific research needs. in order to guarantee proper archival treatment and contextualization of each document, it will be necessary to integrate international archival standards such as ead (encoded archival description) and isad (g) (general international standard archival description) into the platform's architecture. additionally, it is also essential to continue the transferring of the medical records metadata in accordance with the carte da legare project guidelines, which promote standardized cataloging of medical records and ensure the creation of a coherent and aggregated dataset that will allow for broader statistical analysis and cross-institutional research based on harmonized data. the difficulties faced during this work underscore the complexity of transcription for historical documents, where factors like handwriting variation, specialized vocabulary, and document degradation pose significant barriers to accurate recognition. nevertheless, it is essential to continue these efforts in order to preserve and enhance the medical records from the girifalco asylum, fully exploiting and discovering the knowledge contained within them.