Automatic Coding at Scale: Design and Deployment of a Nationwide System for Normalizing Referrals in the C hilean Public Healthcare System (2023)

Abstract: The disease coding task involves assigning a unique identifier from a controlled vocabulary to each disease mentioned in a clinical document. This task is relevant since it allows information extraction from unstructured data to perform, for example, epidemiological studies about the incidence and prevalence of diseases in a determined context. However, the manual coding process is subject to errors as it requires medical personnel to be competent in coding rules and terminology. In addition, this process consumes a lot of time and energy, which could be allocated to more clinically relevant tasks. These difficulties can be addressed by developing computational systems that automatically assign codes to diseases. In this way, we propose a two-step system for automatically coding diseases in referrals from the Chilean public healthcare system. Specifically, our model uses a state-of-the-art NER model for recognizing disease mentions and a search engine system based on Elasticsearch for assigning the most relevant codes associated with these disease mentions. The system’s performance was evaluated on referrals manually coded by clinical experts. Our system obtained a MAP score of 0.63 for the subcategory level and 0.83 for the category level, close to the best-performing models in the literature. This system could be a support tool for health professionals, optimizing the coding and management process. Finally, to guarantee reproducibility, we publicly release the code of our models and experiments.

## introduction the clinical text represents a significant proportion of patient's health records, commonly found in a non-structured format. these texts have particular challenges due to the extensive use of abbreviations, the variability of clinical language across medical specialties, and its restricted availability for privacy reasons @xcite . due to the complexity of its analysis, this data is commonly discarded in projects that seek to support clinical decisionmaking @xcite . clinical coding involves mapping medical texts into codes using a controlled vocabulary consistent across different departments, hospitals, or even countries @xcite . the world health organization maintains an open, controlled vocabulary called the international classification of diseases (icd), which is used in almost every country. currently, the most widely used revision is the tenth (icd-10) @xcite , and they are developing its eleventh revision, which will include not only diseases @xcite . regarding the chilean public health system, the icd-10 terminology is used for coding hospital discharges (morbidity coding by each healthcare provider) and deaths (mortality coding by the ministry of health). having patients' data normalized using these controlled vocabularies enables the ability to summarize information automatically and not deal with the noisiness of free-text data. the already-digested information from the normalized data empowers data analysts who are not experts in nlp to add more complex information into their workflows. the waiting time management system (sigte, in spanish) contains electronic records of referrals from the chilean waiting list, which is the system that manages the high demand existent for consultation by specialists (ministerio de salud de @xcite . this data provided by 29 health services contain information about the medical diagnoses of patients but is not standardized @xcite . as of november 2022, sigte recorded 25,374,491 waiting list referrals, of which 18,716,629 correspond to "new specialty referrals" and are associated with patient pathologies. of these referrals, approximately 5,760,750 (30.7 %) have an icd-10 code. this calculation was performed by searching for a regular expression formatted as an icd-10 code in the free-text diagnosis fields. clinical experts perform the disease coding task manually, which is not optimal for several reasons. firstly, since this process is subject to errors, medical personnel must have significant competence in coding rules and a thorough knowledge of specialized terminologies, such as icd, which also get updated frequently. in other words, expert coding staff must be familiar with the clinical field, analytical and focused, and have fundamental skills for inspecting and analyzing highly specialized texts. in addition, manual coding is time-consuming @xcite , which could be optimized by a support system, and this time could be used for other tasks relevant to clinical decision-making. these difficulties can be efficiently addressed using computational systems capable of automatically performing the coding task using nlp. currently, most automatic coding systems are based on an end-to-end architecture based on deep learning techniques. although these systems have boosted the performance of several coding tasks, they cannot incorporate context-specific rules, such as code priority, medical assumptions, code definition, and synonyms. in this work, we developed an automated disease coding system, thus being able to code the entire historical waiting list in chile, identifying a total of 18,716,629 referrals. our system is based on two steps; first, the automatic extraction of diseases is addressed using a state-of-the-art ner model, and then, using a search engine, the most probable code for each disease found is identified. finally, we explored the potential applications derived from this system and studied in more depth the most frequent diseases in the country today. ## related work the disease coding task involves transforming clinical texts, commonly written by physicians in a non-structured format, into codes following medical terminologies. this is not an easy task since a medical ontology such as icd in spanish has 14,668 codes, an example of extreme multi-label classification @xcite . we have identified two major groups of computational methods proposed to solve this task; rule-based coding and neural network-based coding. ## data and methods the chilean waiting list is characteristic of the the public healthcare system. this list arises due to the high demand for medical care and the limited capacity of the public health system to meet it. entry on the waiting list begins when a patient goes to primary care or secondary care physician to treat pathology. the patient has two possible paths: if the pathology is included in the "garantías explícitas en salud" (ges) program, the patient enters a process where his or her health problem is assured a maximum waiting time for medical attention. if the ges program does not cover the pathology, the referral is classified in one of these five options: new specialty consultations (cne), follow-up consultations (cce), diagnostic procedures (proc), surgical intervention (iq) and complex surgical intervention (iqc). in any of these alternatives, the patient is placed on a waiting list and must wait a variable amount of time to receive medical attention from a specialist. the chilean waiting list comprises 25,374,491 referrals, divided into five categories: 18,716,629 correspond to cne type referrals, 4,391,257 to proc type referrals, 2,222,545 to iq type referrals, 39,266 to cce type referrals, and finally, 4,794 to iqc type referrals. in particular, this work will focus on cne-type referrals. within the chilean waiting database, 73 attributes are separated into two main types of sets. the first set corresponds to the attributes associated with the person (date of birth, sex, national identifier). in contrast, the second set corresponds to the administrative information associated with the referral given to the person (date of admission, date of discharge, the benefit provided, specialty, diagnostic suspicion, and diagnostic confirmation). for the analysis of the diagnoses present in the referrals, two free-text attributes representing medical diagnoses are considered: diagnostic suspicion and diagnostic confirmation. ## proposed system to code the narratives, we first used a ner model to automatically recognize sequences of words in the text associated with diseases. then, each disease found is associated with its most likely icd-10 code through a search engine. figure 1 shows an overview of our proposed system. ## error analysis ## model deployment and use cases due to internal regulations, we could not send patients' data to third-party systems such as cloud providers or academic supercomputing clusters (ministerio secretaría regional de la presidencia, 1999). for this reason, we deployed the whole coding system on-premise on a bare metal machine with a gpu compute module (nvidia rtx a4000 7 ) to process the coding requests from the whole department efficiently. the complete automatic coding system was deployed as a pair of microservices running inside containers to ease portability. one container hosts 7 the compute module has 16 gb of gpu memory and 6.144 cuda cores. more information at https://www.nvidia.com/en-us/ design-visualization/rtx-a4000/ the ner module and exposes an api as a web service listening to disease-mention detection requests. the other container consists of the recommended implementation of the elasticsearch software, which also exposes its api as a web service listening to mention-coding requests. to code the waiting list and schedule recurrent coding when new data arrives, we used the kn-ime foot_6 software, a visual-programming data mining platform. we chose this software because of its ease of use for non-expert developers. the workflow starts with the raw waiting list, which is first passed through the ner module to detect disease mentions, and then each mention is sent to the coding module to assign the most relevant code. the automatic coding result from the workflow mentioned above is persisted on a table inside a database that stores each disease mention for each referral along with the predicted code from the system. ## conclusions in this work, we created a nationwide system to improve the management of the chilean public healthcare system. specifically, we addressed the challenge of creating an automated system to code the diseases present in the chilean waiting list referrals. we developed and validated a model based on two steps: a ner model to recognize disease mentions and a search engine based on elasticsearch to assign the codes to each disease. this mapping system was enriched with several terminology resources used in real life by manual coders to assign codes, thus partially simulating the pipeline followed by these professionals when solving this task. the system allowed us to assign codes to 18,716,629 referrals, thus demonstrating its efficiency and effectiveness. the performance obtained in our experiments was 0.83 according to the map score, which is close to the most advanced systems currently in the coding task. the model was deployed into production in the department of health statistics and information systems of the ministry of health of chile. the use of this system could be an important support for the management of waiting lists. in addition, since 75% of the chilean population is in the public healthcare system, the analysis of the new specialty consultations can be used for epidemiological studies, such as the one done on the incidence of psoriasis @xcite .