Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework (2025)

Abstract: Clinical named entity recognition (NER) aims to retrieve important entities within clinical narratives. Recent works have demonstrated that large language models (LLMs) can achieve strong performance in this task. While previous works focus on proprietary LLMs, we investigate how open NER LLMs, trained specifically for entity recognition, perform in clinical NER. Our initial experiment reveals significant contrast in performance for some clinical entities and how a simple exploitment on entity types can alleviate this issue. In this paper, we introduce a novel framework, entity decomposition with filtering, or EDF. Our key idea is to decompose the entity recognition task into several retrievals of entity sub-types and then filter them. Our experimental results demonstrate the efficacies of our framework and the improvements across all metrics, models, datasets, and entity types. Our analysis also reveals substantial improvement in recognizing previously missed entities using entity decomposition. We further provide a comprehensive evaluation of our framework and an in-depth error analysis to pave future works.

## introduction clinical narratives hold immense value for clinical experts @xcite @xcite , largely due to their wealth of information often inaccessible in the structured data of the electronic health records (ehr) @xcite @xcite @xcite . their free format, however, causes significant challenges for healthcare systems to utilize. the richness of information trapped within the narratives, followed by its significance, has spurred a plethora of works in tackling the clinical information extraction problem within the clinical nlp community @xcite . one key building block in clinical information extraction is named entity recognition (ner), focusing on identifying clinical concepts within these narratives. prior methods @xcite rely on either traditional natural language processing (nlp) techniques or supervised learning methods. nevertheless, the former approach can be fragile, while the latter requires significant effort to annotate. in addition, supervised methods cannot simply scale for the large number of concepts available in the clinical domain @xcite . in light of this, large language models (llms), with their strong capabilities for zero-and fewshot learning @xcite @xcite , serve as promising solutions for clinical ner. while previous works focus on llms trained on general tasks @xcite @xcite , here we focus on llms specifically trained for entity recognition, or open ner llms @xcite . inspired by their results in clinical domain @xcite , outperforming even proprietary llms @xcite , we conduct deeper investigations in this study. surprisingly, our preliminary experiment (section 5.1) suggests a stark performance gap between retrieving different clinical entities (figure 2 ). for instance, an open ner llm called universalner @xcite performs significantly better at extracting medications rather than clinical treatments (85.88% vs 53.81% exact match f1-scores). upon closer inspection, we find these unidentified treatment entities can be effectively recognized by exploiting simpler, albeit specific, entity types. for example, by explicitly specifying "medication" rather than "treatment" as input, the model can capture a substantial portion of the previously unidentified medication-related treatment entities. building upon this insight, we present a novel framework, entity decomposition with filtering, or edf, aimed at tackling clinical ner. to the best of our knowledge, we are the first to explore strategies to effectively use open ner llms in the clinical domain without using any samples. we draw inspiration from the divide-and-conquer paradigm @xcite , which breaks down a complex problem into simpler sub-problems. concretely, we posit that a direct retrieval of entities may be too complex for the model and instead propose to disentangle it into a series of retrievals through entity decomposition. unlike the previous approach @xcite , entity decomposition breaks down the task by identifying through their entity sub-types, which, ideally, are easier to retrieve. nonetheless, entity decomposition alone is insufficient since some entity sub-types do not form strict subsets (further discussed in section 3.2.3). to address this, we introduce a filtering mechanism in our framework to further improve performance. we illustrate them in figure 1 . our work introduces a cost-effective approach to improve clinical entity recognition using opensource large language models. overall, we observe improvements across models, metrics, datasets and entity types. our ablation study also reveals the robustness of our framework, allowing users to adjust the components based on their performance and cost. ## experimental setup we provide the experimental setup here and leave the details in the appendix. all of the base models are available in huggingface * . ## experimental results throughout this section, we abbreviate the entity types in the tables as follows to save space: tr for treatment, pr for problem, te for test, cd for clinical department, dd for disease/disorder, ad for adverse drug, and ade for adverse drug event. given the space limit, we include more results in the appendix, including few-shot (appendix e), performance on a bert-based model (appendix f), and error analysis on cd performance drop (appendix g). ## conclusion and future work in this work, we propose a novel edf framework to tackle the clinical named entity recognition task. our comprehensive experiments demonstrate the strength of our framework across different dimensions. we also thoroughly investigate each framework component and provide several key insights. in future works, we hope to explore how to address the limitations of our work described in section 7.