INSIGHTBUDDY - AI : Medication Extraction and Entity Linking using Pre-Trained Language Models and Ensemble Learning (2025)

Abstract: This paper presents our system, InsightBuddy-AI, designed for extracting medication mentions and their associated attributes, and for linking these entities to established clinical terminology resources, including SNOMED-CT, the British National Formulary (BNF), ICD, and the Dictionary of Medicines and Devices (dm+d).To perform medication extraction, we investigated various ensemble learning approaches, including stacked and voting ensembles (using first, average, and max voting methods) built upon eight pre-trained language models (PLMs). These models include general-domain PLMs—BERT, RoBERTa, and RoBERTa-Large—as well as domain-specific models such as BioBERT, BioClinicalBERT, BioMedRoBERTa, ClinicalBERT, and PubMedBERT.The system targets the extraction of drug-related attributes such as adverse drug effects (ADEs), dosage, duration, form, frequency, reason, route, and strength.Experiments conducted on the n2c2-2018 shared task dataset demonstrate that ensemble learning methods outperformed individually fine-tuned models, with notable improvements of 2.43% in Precision and 1.35% in F1-score.We have also developed cross-platform desktop applications for both entity recognition and entity linking, available for Windows and macOS.The InsightBuddy-AI application is freely accessible for research use at https://github.com/HECTA-UoM/InsightBuddy-AI .

## introduction extracting information about medications and their associated attributes is a crucial task in natural language processing (nlp) for the clinical domain, particularly to enhance digital healthcare solutions. traditionally, clinicians and healthcare professionals have manually performed clinical coding to translate medical events-such as diseases, medications, and treatments-into standardised terminologies like icd and snomed. this manual process is often labour-intensive and prone to human error, potentially compromising accuracy. automating the extraction of medicationrelated information paves the way for automatic mapping of these terms to existing medical terminologies, enabling automated clinical coding. given the potential of this approach, numerous nlp models have been applied in recent years to tasks such as medication mining and clinical coding-though typically in isolation. in this study, we unify these tasks by 1) developing a pipeline that integrates medication and attribute extraction (including dosage, route, strength, adverse effects, frequency, duration, form, and reason) with automated clinical coding. furthermore, 2) we explore ensemble learning techniques-specifically stacking and voting-across a diverse set of nlp models fine-tuned for named entity recognition (ner). these include general-domain models like bert, roberta, and roberta-l, as well as clinicaldomain models such as biobert, bioclinical-bert, biomedroberta, clinicalbert, and pub-medbert. our approach allows practitioners to bypass the challenge of selecting individual models for clinical ner tasks; instead, they can incorporate newer models into the ensemble framework to evaluate their effectiveness. despite these advancements, single-model solutions still encounter obstacles due to the inherent variability and complexity in clinical language, as demonstrated in the comparative evaluation in @xcite , which tested models including bert, clinicalbert, biobert, and customtrained transformers. to mitigate these limitations, ensemble techniques have gained traction. successfully applied in other areas such as computer vision @xcite , ensemble methods combine multiple models to exploit their complementary strengths and reduce their individual shortcomings. in the ner domain, ensembling has led to improved outcomes, as evidenced by @xcite , who demonstrated significant performance gains by applying ensemble strategies to health and life sciences corpora. @xcite employed max voting across models for word-level data in biology, chemistry, and medicine. however, their work focused on french for the clini-cal/medical ner domain using the deft benchmark, while english data were only utilised for the biology and chemistry domains. among ensemble methods, two of the most widely adopted are voting and stacked ensembles: 1) maximum voting, where each model has equal influence on the final decision-as used in @xcite -selects the label with the most votes. 2) stacking, a more advanced method introduced by wolpert (1992), involves training a meta-model on the outputs of base models to learn complex relationships between predictions. for instance, @xcite showed that stacking, when implemented with a support vector machine (svm), improved sentiment analysis performance. in our work, we opt for a simple feed-forward network that maps the ensemble outputs to final predictions. additional examples of stacking can be found in @xcite . while ensemble strategies have shown promise across various ner applications, their applicability to clinical ner-especially with complex datasets like n2c2 2018 @xcite -has yet to be thoroughly explored. this study seeks to bridge that gap by examining whether ensemble approaches, particularly stacking and voting, can enhance ner performance on clinical texts and help overcome the challenges associated with individual model limitations. ## methodologies the overall architecture of insightbuddy is illustrated in figure 1 , which outlines the base models used from both general and clinical domains. from the general domain, we included 1) bert @xcite , roberta @xcite , and roberta-large; and from biomedical/clinical domains, 2) biobert @xcite , bioclinical-bert @xcite , biomedroberta @xcite , clinicalbert @xcite , and pubmedbert @xcite . all eight models were fine-tuned using the same hyperparameters and training set from the n2c2-2018 shared task, following data pre-processing. the performance of each model was first evaluated individually using the n2c2-2018 test set, providing a baseline comparison. subsequently, ensemble learning was applied to the outputs of all models. we then introduced an entity linking component to map the extracted medical entities into standardised clinical terminologies. initially, we the base models are drawn from two domains: general and biomedical. data pre-processing involves splitting the input sequence either at the first full stop (".") occurring after the 100th word or, if none is found, truncating at 128 words. fine-tuning is carried out using identical hyperparameter settings across all eight models. ensembling is performed using various strategies, which are detailed in figure 3 . entity linking connects extracted entities to clinical knowledge bases (kbs), specifically bnf and snomed ct. for pre-processing, the input text was segmented into chunks of up to 128 tokens. if a full stop (".") appeared between the 100th and 128th word, the chunk was cut at that punctuation mark. to explain our ensemble-learning approach, we present the insightbuddy ensemble diagram in figure 3 . the initial outputs from each of the eight fine-tuned ner models are in sub-word format, as per their tokenisation strategy. for example, the word "paracetamol" may be tokenised as "para ##ce ##tam ##ol". therefore, our first step is to reconstruct words from sub-word tokens for practical usage and voting. however, since each sub-word receives a potentially different label, discrepancies often occur within the same word. to resolve this, we implemented three grouping strategies: first-token voting, max-token voting, and average voting. in the first-token voting method, the label of the first sub-word is applied to the entire word. for instance, if "para" is labelled as "b-drug", then "paracetamol" will be assigned the same label, regardless of labels on subsequent sub-words. in the maxtoken voting method, the label with the highest logit score among the sub-words is assigned to the word-reflecting the model's highest confidence in that prediction. the average voting approach computes the mean of logits across all sub-words, from which the label for the full word is derived. regarding word-level ensemble learning, we explore a classical voting approach with two specific strategies: the ">=4 or o" strategy assigns the majority label if at least four models agree. if no majority exists, the label "o" (non-entity) is used by default to signify context words. the max-voting strategy selects the most frequently predicted label, regardless of how many models it came from (e.g. 2, 3, or 4 votes). in cases of a tie (e.g. two labels each receiving three votes from six models), we resolve it either alphabetically or randomly. we also depict the stacked-ensemble approach in figure 2 . during training, the data is split into 80% for training and 20% for testing the ensemble model. output data from base models is only used when at least two models assign a label other than "o"; otherwise, "o" is kept and the token is excluded from stacked training data. for the stacked model's input, we convert each model's output logits into one-hot encoded vectors, then concatenate them alongside the true label of each token. as we use eight models, the training input consists of eight one-hot vectors and one label. each vector is of length 19 (representing 19 possible labels), containing a single '1' at the predicted label's index and '0' elsewhere. as a result, each training sample contains 8 vectors × 19 @xmath0, with exactly eight '1's and the remaining 144 being '0's. we choose to use ## experimental evaluations we employed the dataset from the n2c2-2018 shared task, which focuses on named entity recognition (ner) of adverse drug events and associated medical attributes @xcite . the data includes annotated labels such as ade, dosage, drug, duration, form, frequency, reason, route, and strength in bio tagging format, resulting in a total of 19 possible tags: 2 (b/i) for each of the 9 classes, plus 1 (o). the original dataset comprises 303 training letters and 202 testing letters. following the data split approach by @xcite , we divided the training set into a 9:1 ratio for training and validation purposes. we evaluate the models using precision, recall, and f1-score under both "macro" and "weighted" averaging schemes, along with overall accuracy. the "macro" average gives equal importance to each class, regardless of how often it appears in the dataset, whereas the "weighted" average scales scores according to label frequency. we begin by reporting the results from individual fine-tuned models (sub-word level), followed by evaluations of ensemble models using various strategies (word level). ## entity linking: bnf and snomed to integrate the recognised named entities with a clinical knowledge base, we utilised the existing mapping resources provided by the british national formulary (bnf), which establish links between snomed-ct, bnf, dm+d, and icd codes (available at https: //www.nhsbsa.nhs.uk/prescription-data/ understanding-our-data/ bnf-snomed-mapping). we began by reducing the full set of 377,834 snomed codes to 10,804 entries through pre-processing, eliminating duplicate mappings between snomed and bnf. additionally, we filtered out non-drug terms found in the text. this included removing items that contained words such as @xcite , as these typically refer to medical equipment rather than pharmaceuticals. for mapping to snomed ct, we applied a fuzzy string-matching technique on the refined list, using drug names as search queries. when a match was found, the associated snomed ct code was appended and used to generate a direct link to the snomed ct online portal. in contrast, the bnf mapping process relied on a keyword-based search to retrieve matching entries from the bnf website. this approach was necessary due to differences in how the bnf site handles search queries compared to the snomed ct platform. depending on their needs or preferences, users can choose to utilise either of these two clinical knowledge bases (kbs), as illustrated in figure 9 . ## discussion and conclusion this paper presented a pilot investigation into the application of stacked and voting ensemble techniques for medical named entity recognition, utilising eight pre-trained language models (plms) drawn from both general-purpose and biomedical/clinical domains. our experimental results demonstrate that the best-performing finetuned individual models surpassed the state-ofthe-art results on the standard n2c2-2018 shared task dataset. moreover, by incorporating ensemble approaches-specifically using output logits and one-hot encoded vectors-we achieved further performance gains, with a 2.43% improvement in precision and a 1.35% increase in f1score. in addition, we developed a desktop tool and user interface for our fine-tuned models, which includes an entity linking and normalisation feature that maps recognised entities to the bnf and snomed ct clinical knowledge bases. this tool, named insightbuddy-ai, is publicly accessible at https://github.com/hecta-uom/ insightbuddy-ai .