Automatic Transformation of Clinical Narratives into Structured Format (2021)

Abstract: Vast amounts of data in healthcare are available in unstructured text format, usually in the local language of the countries. These documents contain valuable information. Secondary use of clinical narratives and information extraction of key facts and relations from them about the patient disease history can foster preventive medicine and improve healthcare. In this paper, we propose a hybrid method for the automatic transformation of clinical text into a structured format. The documents are automatically sectioned into the following parts: diagnosis, patient history, patient status, lab results. For the “Diagnosis” section a deep learning text-based encoding into ICD-10 codes is applied using MBG-ClinicalBERT - a fine-tuned ClinicalBERT model for Bulgarian medical text. From the “Patient History” section, we identify patient symptoms using a rule-based approach enhanced with similarity search based on MBG-ClinicalBERT word embeddings. We also identify symptom relations like negation. For the “Patient Status” description, binary classification is used to determine the status of each anatomic organ. In this paper, we demonstrate different methods for adapting NLP tools for English and other languages to a low resource language like Bulgarian.

## introduction the analysis of medical data can lead to the discovery of knowledge that has a significant effect on healthcare and the effectiveness of treatment. unfortunately, large amounts of health data are available in an unstructured text format, usually in the local language of the countries. these documents contain valuable information, but it cannot be extracted directly from them. transformation of the clinical record from unstructured text format into a structured presentation that includes key facts and relations will enable further analysis, which can help to obtain valuable information about the patient's medical history and in this way to support preventive medicine and improve healthcare. @xcite use huge manually annotated corpora for patients with depression with expertselected terms of interest. they apply statistical models and active learning for structured information extraction with an average f1-score of 74% for temporal characteristics, diagnosis, disease history, symptoms, cognitive scores, medications, response, and adverse effects. @xcite also apply statistical methods for structuring information from radiology reports. @xcite presents a method seremed, based on semantic transformation rules that achieves precision 93% and recall 88% for chest x-ray reports. @xcite investigate in their systematic review the clinical nlp methods. still, the rule-based approaches predominate. @xcite apply different word embedding techniques for automatic prediction of icd-10 codes for free-text clinical notes for 8 cardio-vascular diseases. the reported au-roc rating is in the range from 94% up to 99%. @xcite apply hybrid methods combining keyword-based method and deep learning methods (word2vec and sentence2vec), achieving f1-score 83% and 88% correspondingly for both methods. there is limited research for languages other than english about ner from clinical text @xcite . @xcite present nlp techniques for name entity recognition (ner) for clinical text in chinese. the authors explore the application of different classical machine learning techniques like conditional random fields (crf), support vector machines (svm), maximum entropy (me), and structural svm (ssvm) over about 400 manually annotated admission notes and discharge letters. the best score is 93% for the former and 90% for the latter datatype. @xcite propose an entropy-based model for temporal information extraction for medical facts and medical events from chinese clinical text. some preliminary work was reported for structuring clinical information in bulgarian -for extraction of various numerical values related to lab test by @xcite , medication extraction by @xcite , and annotation with icd-10 codes by @xcite . the automatic generation of diabetes register from unstructured clinical texts using rule-based nlp techniques is presented in @xcite . therefore, we aim to develop a natural language processing (nlp) pipeline for automatically extracting information from clinical texts and presenting it in a structured format. this task is even more challenging for low-resource languages like bulgarian. the developed approach employs both classical methods such as those based on dictionaries and rules and modern language models based on deep neural networks transformers such as bert (bidirectional encoder representations from transformers by @xcite ) and can be easily adapted to other languages. ## conclusion and further work in this paper, we presented a method for automatic extraction of data from clinical text and displaying it in a structured format for the bulgarian language. the developed approach employs both classical methods such as those based on dictionaries and rules and modern language models based on deep neural networks transformers such as bert. as the targeted language is a low resource one we also presented various methods for adapting the nlp tools for english and other widely spread languages. we also present results from conducted experiments to evaluate the trained ml models, which show competitive performance, making them applicable for the considered tasks. despite the presented methods are developed for a particular language they are general and can be easily adapted to other languages. further improvements of the presented approach can include extending the dictionaries and investigating other deep learning language models and using a semi-supervised approach to train them on the named entity recognition and text classification tasks.