When Is a Name Sensitive? Eponyms in Clinical Text and Implications for De-Identification (2024)

Abstract: Clinical data, in the form of electronic health records, are rich resources that can be tapped using natural language processing. At the same time, they contain very sensitive information that must be protected. One strategy is to remove or obscure data using automatic de-identification. However, the detection of sensitive data can yield false positives. This is especially true for tokens that are similar in form to sensitive entities, such as eponyms. These names tend to refer to medical procedures or diagnoses rather than specific persons. Previous research has shown that automatic de-identification systems often misclassify eponyms as names, leading to a loss of valuable medical information. In this study, we estimate the prevalence of eponyms in a real Swedish clinical corpus. Furthermore, we demonstrate that modern transformer-based de-identification systems are more accurate in distinguishing between names and eponyms than previous approaches.

## introduction de-identification of data invariably reduces information content by either removing, concealing, or replacing sensitive text with pseudonyms. pseudonymization of data based on automatic identification and replacement of personally identifiable information (pii) may also introduce misleading information if tokens or text spans are erroneously misclassified as pii. tokens are more likely to be misclassified as pii if they share common features with pii of a certain class. such situations often arise in clinical texts, which often contain eponyms (kucharz, 2020). these medical terms are named after a researcher or clinician, typically somebody involved in the discovery or invention or discovery of the phenomenon bearing their name. sometimes, it can also be the name of a patient affected by a disorder. since eponyms refer to medical phenomena rather than persons, they should not be considered sensitive. it is believed that there are over 8,000 medical eponyms. as discussed by kucharz (2020), eponyms can refer not only to diseases but to a wide range of categories including tests, surgical procedures and anatomical structures. these eponyms can cause difficulty when trying to automatically detect pii. in one study, it was shown that while only 0.81% of clinical entities were misclassified as pii, this was substantially higher for eponyms, where between 10 and 49% of eponyms were misclassified as pii @xcite . the following example highlights the problem: dr. sjögren suspects the patient has sjögren's syndrome. in this example, sjögren's syndrome is an eponymous disorder which is being treated by a physician who happens to have the same name. when de-identifying the sentence, sjögren should be concealed in dr. sjögren but not in sjögren's syndrome. concealing the eponymous name of the syndrome removes clinical information which could potentially be very important for the intended users of the data. however, it is not clear how prevalent eponyms are in clinical text and to what extent transformer-based named entity recognition (ner) systems trained to identify pii can distinguish between eponyms and sensitive names. in this study, we estimate the prevalence of eponyms in a large corpus of swedish clinical text. we also create a manually annotated corpus of clinical notes containing one or more eponyms and use this corpus to study the extent to which classifications of names overlap with eponyms. to that end, we employ a ner system trained to detect sensitive entities (e.g., names). in other words, we seek to understand how eponyms affect these models' ability to distinguish between actual names and eponyms. the main contributions of this study are summarized below: ## related research research looking specifically at eponyms is scarce. the studies that are available often focus on the intersection of the de-identification of clinical texts and the detection of disorders. berg et al. (2020) performed de-identification experiments and observed that rare eponyms in the training data tended to be misclassified as last or first names to a very high degree, but there were also cases where eponyms in the training data were misclassified as last or first names. meystre et al. (2014) compare five de-identification systems and their flaws in erroneously detecting eponyms as protected health information (phi) foot_0 in american clinical text. three systems (mit, mist and hide) misclassify approximately 10% of all eponyms as phi, and the other two systems (hms and meds) misclassify as many as 40% of all eponyms as phi. berg et al. ( 2020 ) created an eponym lexicon by using a ner model for clinical entities, i.e., a system to identify findings, disorder, body parts and drugs in a swedish clinical text. then, they investigated whether these were based on the name of a person, in which case it was marked up as an eponym and added to the eponym lexicon. finally, the created lexicon was manually reviewed to ensure correctness. the resulting eponym lexicon contains 275 eponyms. several studies have examined the impact of de-identification on data utility for machine learning. results are highly contingent on an appropriate sanitization algorithm and a sufficiently strong ner model for detecting sensitive data @xcite . however, there are several examples of studies showing that data utility can be maintained for both fine-tuning, pretraining, and combined scenarios @xcite @xcite . these studies examine the impact of deidentification by evaluating models trained to perform downstream tasks. a shared limitation is that these studies study the impact on downstream task performance overall. as such, these studies cannot conclusively rule out that there may be other scenarios where de-identification could still have a disparate impact on data utility. for instance, misclassifying and removing information contained in eponyms could be harmful in many scenarios, and examining this specific risk provides deeper insights into possible pitfalls in de-identifying clinical data. ## data and experiments in this study, we estimate the prevalence of eponyms in a large sample of swedish clinical texts by using an eponym lexicon to automatically identify mentions of eponyms in clinical notes. we use this to create an eponym corpus by randomly sampling 1,000 clinical notes with at least one detected eponym mention. these notes are then manually reviewed and corrected while we also calculate inter-annotator agreement among four annotators. finally, we fine-tune a swedish clinical bert model to identify pii and calculate to what extent eponyms are misclassified. ## limitations while three of the four annotators had prior experience working with clinical text, none were trained medical professionals but computer scientists. some eponyms may have been missed during the annotation process, and others may have been erroneously annotated. in cases where the annotators needed clarification, they searched online for sources indicating whether or not a name was an eponym. the high iaa indicates that the annota-tions are reliable, but the lack of medical expertise limits the extent to which the annotations can be trusted. a related issue is that the eponym corpus is not a random sample of the entire health bank. instead, it is a consciously chosen subset that was deemed highly likely to contain eponyms based on the matching algorithm described in section 3.1. starting from a purely random subset of the health bank could have led to more robust results and would have allowed us to calculate the recall for the matching algorithm. this was not deemed feasible due to the very low prevalence of eponyms in the overall corpus. starting from a random sample would have required far more annotators than were available for this project. the risk of misclassifying eponyms was only examined for the swedeclin-bert model. it is possible that other architectures and models trained on other datasets may perform better or worse. further research could benefit from including a more diverse range of models, including generative models. nevertheless, the results of this study show that transformer-based models can be less affected by the misclassification risks than models described in earlier studies. determining the mechanism behind this greater resilience is an interesting topic for future research.