JEBS : A Fine-grained Biomedical Lexical Simplification Task (2025)

Abstract: Though online medical literature has made health information more available than ever, the barrier of complex medical jargon prevents the general public from understanding it. Though parallel and comparable corpora for Biomedical Text Simplification have been introduced, these conflate the many syntactic and lexical operations involved in simplification. To enable more targeted development and evaluation, we present a fine-grained lexical simplification task and dataset, Jargon Explanations for Biomedical Simplification (JEBS). The JEBS task involves identifying complex terms, classifying how to replace them, and generating replacement text. The JEBS dataset contains 21,595 replacements for 10,314 terms across 400 biomedical abstracts and their manually simplified versions. Additionally, we provide baseline results for a variety of rule-based and transformer-based systems for the three subtasks. The JEBS task, data, and baseline results pave the way for development and rigorous evaluation of systems for replacing or explaining complex biomedical terms.

## introduction understanding medical concepts is critical when making informed healthcare decisions @xcite . patients that lack this understanding are at a disadvantage when making health-related choices, which can negatively affect health outcomes @xcite . websites such as pubmed @xcite ) make the latest biomedical knowledge available to everyone. however, because this information is not written for a general audience, attempting to read it without the relevant expertise may cause more harm than good @xcite . manually curated resources, such as medline-plus @xcite or uptodate patient education @xcite , aim to rewrite biomedical knowledge for the public, thus providing a consumer-friendly alternative to resources such as pubmed. however, these resources require massive cost and effort to keep updated with the latest research and are limited in the scope of their topics. for example, uptodate has more than ten times as many articles written for healthcare practitioners than it has in its patient education section. advances in artificial intelligence could help solve this bottleneck by automatically 'translating' the latest medical research into simpler language or by providing real-time explanations as a reading aid. given the high stakes of the biomedical domain, however, rigorous evaluation of such systems is crucial. existing datasets proposed for training and evaluating biomedical text simplification systems take the form of parallel or comparable corpora @xcite @xcite @xcite . by not explicitly providing term replacements, these datasets restrict the development to end-to-end text simplification systems. the lack of explicit term replacements also restricts automatic evaluation to coarse n-gram or similaritybased metrics, which conflate the many distinct types of word-and sentence-level operations involved in text simplification and can thus lead to misleading results @xcite . in this work, we take a step toward more targeted training and evaluation of biomedical text simplification by introducing a manually annotated, fine-grained dataset of multiple lexical simplification operations. we first break the task of lexical simplification into three sub-tasks: (1) identification of complex terms, (2) classification of how best to replace the terms in context, and (3) generation of replacements. further, for the classification sub-task, we review the literature on lexical simplification to create a taxonomy of five term replacement types: substitution, explanation, generaliza-purpose: t o find predictive markers for the visual potential in optical coherence tomography (oct) one month after surgical repair of maculainvolving rhegmatogenous retinal detachment (mird) with and without internal limiting membrane (ilm) peeling. methods: this retrospective single-center, singlesurgeon cohort study included 74 patients who underwent pars plana vitrectomy (ppv) for primary mird between january 2013 and august 2020 with follow-up examinations for at least 6 months. patients developing recurrent detachments, media opacities, or with an axial length over 27 mm were excluded from the analysis. this study aimed to find out if vision can be predicted by diagnostic imaging one month after surgery to repair common retinal detachment (the damage to light-sensitive layer at the back of the eye (retina) when it is pulled away from its normal position). in some surgeries, the tissues next to retina were pulled to repair the detachment. this study looked at past cases at a single health care center, and that had surgery done by a single surgeon. patients who had other retina detachments in the past or media opacities (such as cataracts) were not included in the study. ## dataset creation the jebs dataset is derived from 400 abstracts and their associated adaptations, as found in the plaba dataset @xcite . abstracts were aligned at the sentence level with their corresponding adaptations, then annotated by two authors us-ing the brat rapid annotation tool foot_0 @xcite , which involved selecting expert terms and linking them with their respective simplifications, as found in the plaba adaptations. in total, the jebs dataset contains 10,314 expert terms (25.79 terms per abstract) and 21,595 simplifications. the annotations exhibit a moderate interannotator agreement for both the identification task (0.5203 f1) and the classification task (0.4577 f1). figure 2 shows an example of the brat interface during annotation. while performing annotations, the annotators confirmed that there was no information naming or uniquely identifying individual persons in the plaba dataset, nor was there any offensive content. the jebs dataset therefore does not include any such information. ## baseline systems expert term identification, term classification, and the five forms of simplification were divided into separate sets of language models. all fine-tuned transformer approaches used hugging face and pytorch for fine-tuning. each of those models underwent 3 epochs of fine-tuning. all fine-tuning and evaluations of those models was performed on a single nvidia a100 80gb gpu. prior to training our baseline models, the union of both annotator's annotations were preprocessed into a json file, where each expert term in each abstract was linked with its associated simplifications. each simplification takes the form of a tuple storing both its type (substitution, explanation, generalization, etc) and the contents of that simplification. all data was split into train and evaluation sets according to a 1:3 ratio. the split was performed at the question-level. that is, data from abstracts answering the same question within the plaba dataset were kept together in either the train or evaluation sets. furthermore, neural models designed for the non-identification sub-tasks require the context in which terms were used to function. this data was obtained by splitting plaba abstracts into individual sentences. in the following subsections, we summarize the baseline models for each sub-task, as well as the data preprocessing requirements for each model. ## future work there remains ample space for improving performance in all of the sub-tasks and methods defined in this paper. for example, it remains to be seen if decoder-only llms can effectively perform the identification. while llama3-8b was unable to out-perform most of the encoder-based models, more specific prompt engineering may unlock greater levels of performance. in the simplification classification sub-task, there exist multiple unexplored directions from which one could improve upon our baselines. for example, this task could be framed as a sequence-tosequence problem for generative models to attempt. the issue of class imbalance in the data for this task (wherein the majority of expert terms can be simplified using substitution) must also be addressed, whether that be via class weights, oversampling, or using generative ai to synthesize additional example data. while off-the-shelf language models performed well on the substitution, explanation, generalization, and exemplification simplification methods, they do not make full use of the jebs dataset. to fully utilize our dataset, future approaches should explore fine-tuning language models for each simplification method to achieve stronger performance on the generation sub-task. ## conclusion in this work, we introduced a new task of finegrained biomedical lexical simplification and a corresponding dataset called jebs (jargon explanations for biomedical simplification). the jebs task involves identifying expert terms, classifying how best to replace them, and generating replacement text. unlike existing parallel or comparable corpora for biomedical text simplification, jebs allows targeted development and evaluation of systems to directly provide replacement terms. the jebs dataset contains 21,595 replacements for 10,314 terms. these terms appear in the context of 400 biomedical abstracts and their corresponding manually written plain language adaptations from the plaba dataset. we have introduced a suite of baseline models for identifying expert terms in biomedical texts, classifying them for simplification, and generating consumer-friendly simplifications for those terms. using an array of methods built atop the jebs dataset, we achieved promising results in all of our defined tasks. finally, we proposed avenues for future improvement of our models. we imagine that our work will bridge the gap between medical experts and patients, providing consumers with new tools to aid in healthcare decision making.