The Anatomy of Evidence: An Investigation Into Explainable ICD Coding (2025)

Abstract: Automatic medical coding has the potential to ease documentation and billing processes. For this task, transparency plays an important role for medical coders and regulatory bodies, which can be achieved using explainability methods. However, the evaluation of these approaches has been mostly limited to short text and binary settings due to a scarcity of annotated data. Recent efforts by Cheng et al. (2023) have introduced the MDACE dataset, which provides a valuable resource containing code evidence in clinical records. In this work, we conduct an in-depth analysis of the MDACE dataset and perform plausibility evaluation of current explainable medical coding systems from an applied perspective. With this, we contribute to a deeper understanding of automatic medical coding and evidence extraction. Our findings reveal that ground truth evidence aligns with code descriptions to a certain degree. An investigation into state-of-the-art approaches shows a high overlap with ground truth evidence. We propose match measures and highlight success and failure cases. Based on our findings, we provide recommendations for developing and evaluating explainable medical coding systems.

## introduction explaining model predictions is a key component in automatic medical coding systems. healthcare systems worldwide employ coding systems to document diagnoses and procedures, and to process medical bills. codes are often structured in hierarchies which contain up to hundreds of thousands of terms to cover the complexity of medical care and the multitude of synonymous medical expressions. medical codes are used for administrative purposes @xcite , for prediction of length of stay and mortality @xcite , adverse event detection, and clinical decision support @xcite . the international classi- the patient was admitted with severe hypertension i10 . beta-blockers and ace inhibitors were administered. the patient reported a history of smoking f17 and high cholesterol levels. fication of diseases (icd) foot_0 is the most prominent system in hospital settings. a patient's set of icd codes together with the main diagnosis dictates the level of reimbursement. assigning appropriate icd codes manually requires extensive expertise and is guided by detailed, country-specific rule books. to reduce manual efforts and increase efficiency, automatic coding methods have been developed. with the adoption of deep learning, especially the transformer architecture @xcite , automatic coding systems have achieved practical usage @xcite . state-of-the-art approaches treat code prediction as a multi-label classification task and rely on pre-trained language models @xcite . however, these models with billions of parameters lack transparency -which diminishes acceptance by coding staff and presents a barrier during regulatory assessments by external authorities. the goal is to develop medical coding systems which are able to provide text evidence for a predicted code. this objective can be achieved with the use of explainability methods. in particular, feature attribution methods identify which input features are decisive for the model output @xcite @xcite assigning scores to each token of a medical document. however, explainability research has been mostly constrained to short text and binary classification on social media data or product reviews @xcite . in the medical domain, the scarcity of data and the need for domain expertise make word level annotations challenging. @xcite carried out a re-annotation effort of a mimic-iii subset @xcite and created annotation spans as evidence for each medical code. this dataset is the first to include textual evidence for assigned medical codes and offers new research opportunities in information extraction and explainability. building on this, recent work investigated different training strategies and evidence extraction methods showing promising results using gradientbased explanations @xcite . 2 the evaluation focused on two key properties of explanations, faithfulness and plausibility @xcite . however, prior work was technique-centric, lacking depth in data analysis and application-focused evaluation. hence, what is still missing is an investigation from a practical perspective on (a) how to utilize the dataset for research purposes and (b) how well current explainability approaches, incl. evaluation, work for clinical adoption. this work aims to deepen the understanding of icd coding, and improve evidence extraction and evaluation. 3 the focus is on data understanding and plausibility evaluation. our contributions can be summarized as follows: ## dataset overview mdace is a medical dataset in the english language containing electronic health records (ehrs) along with associated icd codes @xcite . the ehrs included in mdace are 2 throughout the paper, evidence and explanation are used interchangeably. amount and length across all admissions, documents and codes, the dataset contains 9,499 evidence spans, 5,563 in profee and 3,936 in inpatient. the average labels per document is 11.3 for inpatient and 31.4 for profee. hence, there is roughly three times more evidence in the complete labeling scheme. figure 2 shows the average number of evidence per document type for inpatient and profee. evidence length for inpatient and profee is roughly the same, 2.18 and 1.96 tokens. ## analysis of human-annotated evidence to determine the characteristics of medical code evidence, we perform an in-depth analysis of the mdace dataset. compared to other data resources, the documents in mdace are relatively long with an average of around 2,000 words @xcite . given the considerable length, we explore whether focusing on relevant segments could be a driver for computational efficiency. to the same effect, we seek efficient methods to detect relevant labels. we analyze to what extent evidence and medical codes can already be captured with existing knowledge, i.e., code descriptions. prior work did not discuss the relation of the common documents in inpatient and profee which we address in this work. in a qualitative analysis, we shed light on annotation dependencies and annotation diversity. these points give rise to the following research questions: for the position analysis, we examine whether focusing on specific text segments, e.g., beginning of a document, is beneficial for solving the classification and evidence extraction tasks. we concentrate on the two most frequent and commonly used document types: discharge summaries and physician notes. the underlying data basis is the common subset of documents annotated by both, profee and inpatient coders. figure 3 shows the distribution of evidence position throughout the discharge summaries and physician notes for inpatient and profee. evidence counts are normalized per plot and a relative position of 0.5 means that evidence appears halfway through the document. for inpatient, evidence occurs mostly at the beginning and end of documents. in comparison, the relevant information in physician notes seems to occur more towards the end of documents. for profee, a valley is only perceivable for the physician notes, there seems to be no such pattern in the discharge summaries. does sufficient evidence occur earlier in the document? since text is typically scanned from top to bottom, we assumed that sufficient evidence is annotated when it first occurs in the document. that leads to the hypothesis that sufficient evidence occurs more in the beginning of documents in comparison to complete evidence. the position information shows that this is not the case: the evidence counts in inpatient are not higher for lower positions. contrary to our assumption, sufficient evidence is not located more in the beginning of the documents. ## analysis of model explanations for the modeling approaches, we investigate to what extent current approaches for explainable icd coding are able to extract plausible evidence that matches with the ground truth. in a qualitative analysis, we highlight success and failure cases. we were interested in finding out whether model correctness relates with explanation characteristics, in particular explanation length. taken together, we consider the following research questions for the analysis of model evidence: ## discussion the analysis of code descriptions together with the qualitative analysis of codes with low diversity and high exact match count suggests that parts of the icd coding task can be performed in a simple and efficient manner, for example with rule-based systems. this opens up research potential for hybrid approaches. we have shown that match counts that take proximity into account are able to capture nuances of extracted evidence that would otherwise be missed with other metrics. the findings, that models are more likely to be wrong when fewer words are extracted and that duplicates are frequent, are counterintuitive (occam's razor). as we would have expected more variance across models, further research is needed to investigate the precise role of robustness strategies on feature sparsity and associated model performance. considering the probabilities, there was no pattern that exact matches have higher output probabilities. a promising avenue may be to investigate which effect model calibration has on match types. ## conclusion in this work, we studied the task of explainable icd coding based on medical records. we investigated the mdace dataset which contains medical codes with corresponding evidence spans. our analysis revealed a certain overlap of evidence with code descriptions. we examined the performance of current approaches, comparing unsupervised and supervised methods, and found that overlap with the ground truth is relatively high. the supervised approach is better in extracting exact matches. based on our findings, we proposed recommendations for extracting and evaluating evidence for icd coding including the intended use of evidence and reporting of false positives. going forward, there are several open research avenues in improving evaluation of semantic similarity, investigating the relation of expanation characteristics and performance, and a formalization of sufficient and complete evidence.