Beyond De-Identification: A Structured Approach for Defining and Detecting Indirect Identifiers in Medical Texts (2025)

Abstract: Sharing sensitive texts for scientific purposes requires appropriate techniques to protect the privacy of patients and healthcare personnel. Anonymizing textual data is particularly challenging due to the presence of diverse unstructured direct and indirect identifiers. To mitigate the risk of re-identification, this work introduces a schema of nine categories of indirect identifiers designed to account for different potential adversaries, including acquaintances, family members and medical staff. Using this schema, we annotate 100 MIMIC-III discharge summaries and propose baseline models for identifying indirect identifiers. We will release the annotation guidelines, annotation spans (6,199 annotations in total) and the corresponding MIMIC-III document IDs to support further research in this area.

## introduction access to data remains a major bottleneck in developing machine learning models for healthcare. since data contains sensitive details about individuals, it cannot be shared readily outside hospitals. interactions with legal departments and data security can be cumbersome, and regulations are somewhat unclear, particularly where text is concerned. however, the concept of de-identification is well-defined: according to hipaa, foot_1 it requires the removal of a list of direct identifiers, known as protected health information (phi), 3 including names and addresses. classical de-identification of text data has been explored for many years with various approaches @xcite @xcite and state-of-the-art deidentification systems achieve an f 1 -@xmath0. however, additional manual effort is needed to remove remaining phis, and more importantly, unstructured text often contains additional information beyond phis that can reveal an individual's identity @xcite , making the manual inspection process even more complex. the concept of anonymization goes further: it is defined as an irreversible procedure that is applied to the data such that no information can be linked to any specific individual anymore @xcite . while the terms de-identification and anonymization are often used interchangeably, they refer to distinct concepts @xcite . de-identification focuses solely on removing direct identifiers, whereas anonymization must also address indirect identifiers. indirect identifiers are pieces of information that are potentially publicly known about an individual but do not lead to reidentification when considered alone. however, in combination with other background or external knowledge, they can be used to uniquely identify an individual @xcite . figure 1 shows a synthetic discharge summary with highlighted information (beyond direct identifiers) that may help reveal a person's identity. despite the importance of anonymization, relatively few studies have systematically addressed text anonymization beyond traditional phi detection. @xcite developed a system for extracting and suppressing sensitive information other than phis, but it was limited to diagnoses. @xcite created a dataset with phis and added more categories, namely medical units, relatives and typists. @xcite annotated a set of demographic traits in clinical notes and proposed a framework for detecting sentences that include such traits. @xcite presented a benchmark dataset comprising annotations of court cases and evaluation metrics to assess the performance of anonymization methods. the annotations cover categories such as names and quantities, and annotators mark each of the entities as a direct or indirect identifier. @xcite proposed a framework for text anonymization based on large language models (llms). this framework measures anonymization success simply by checking whether an adversarial llm can guess the name of the person to whom the text belongs. building on prior work, our study defines and identifies information beyond traditional personal health identifiers within a controlled framework. we introduce a schema of indirect personal identifiers (ipis) optimized for a medical context and apply it to annotate relevant spans in discharge summaries from the medical information mart for intensive care (mimic-iii) dataset @xcite . we define the problem of structurally identifying ipis as a span classification problem, rather than a sentence classification problem as in @xcite , to avoid removing whole sentences (which might include other medical information) and to reduce information loss during anonymization. finally, we evaluate the performance of various models in detecting the annotated identifiers. ## indirect personal identifiers (ipi) the type of information that may lead to reidentification in a given text is domain-dependent and requires unique analysis @xcite . in the following, we introduce a schema covering aspects of indirect personal identifiers (ipi) and use it to annotate spans in discharge summaries from mimic-iii. to construct our dataset, we randomly sampled 100 summaries with lengths ranging from 500 to 2,500 words. foot_3 ## experiments to provide a first baseline for the automatic detection of the proposed set of indirect identifiers in medical texts, we experimented with bert @xcite as well as open-source llms. we split the data into training (60%), development (15%) and test (25%) sets, and used the dev set for hyperparameter optimization. we fine-tuned a bert model for span classification using the huggingface library @xcite . for the llm experiments, we used llama-3.1-8b-instruct, mistral-7b-instruct-v0.3 and qwen2.5-14b-instruct in both zero-shot and three-shot settings leveraging declarative selfimproving python (dspy) @xcite to automatically refine and optimize the prompt and pydantic foot_5 to obtain structured and type-validated output from the llms. an example prompt is shown in appendix e. we implemented an llm agent for each category and provided dspy with the description of each category as defined in the annotation guidelines. model performance was assessed using relaxed precision, recall and f 1 -score. further details on data preprocessing, model finetuning and evaluation can be found in appendix c. ## conclusion in this work, we introduced a dataset along with an annotation schema designed to capture a wide range of indirect identifiers in medical texts. the schema is inspired by medical records, but is adaptable to other domains and text genres with minimal modifications. we evaluated the performance of bert and llms in detecting the proposed categories. the overall performance of the models highlights the inherent difficulty of this task, particularly in identifying less frequent and diverse indirect identifiers. however, our work provides a foundation for further exploration and adaptation, with an eye to improving privacy through structural information detection. in future work, we aim to develop a framework that (k-)anonymizes the proposed indirect identifiers and study the utility of the anonymized texts on downstream tasks.