Does BERT Pretrained on Clinical Notes Reveal Sensitive Data? (2021)

Abstract: Large Transformers pretrained over clinical notes from Electronic Health Records (EHR) have afforded substantial gains in performance on predictive clinical tasks. The cost of training such models (and the necessity of data access to do so) coupled with their utility motivates parameter sharing, i.e., the release of pretrained models such as ClinicalBERT. While most efforts have used deidentified EHR, many researchers have access to large sets of sensitive, non-deidentified EHR with which they might train a BERT model (or similar). Would it be safe to release the weights of such a model if they did? In this work, we design a battery of approaches intended to recover Personal Health Information (PHI) from a trained BERT. Specifically, we attempt to recover patient names and conditions with which they are associated. We find that simple probing methods are not able to meaningfully extract sensitive information from BERT trained over the MIMIC-III corpus of EHR. However, more sophisticated “attacks” may succeed in doing so: To facilitate such research, we make our experimental setup and baseline probing models available at https://github.com/elehman16/exposing_patient_data_release .

## introduction pretraining large (masked) language models such as bert @xcite over domain specific corpora has yielded consistent performance gains across a broad range of tasks. in biomedical nlp, this has often meant pretraining models over collections of electronic health records (ehrs) @xcite . for example, @xcite showed that pretraining models over ehr data improves performance on clinical predictive tasks. given their empirical utility, and the fact that pretraining large networks requires a nontrivial amount of compute, there is a natural desire to equal contribution. 1 https://github.com/elehman16/ exposing_patient_data_release . however, in the context of pretraining models over patient ehr, this poses unique potential privacy concerns: might the parameters of trained models leak sensitive patient information? in the united states, the health insurance portability and accountability act (hipaa) prohibits the sharing of such text if it contains any reference to protected health information (phi). if one removes all reference to phi, the data is considered "deidentified", and is therefore legal to share. while researchers may not directly share nondeidentified text, foot_0 it is unclear to what extent models pretrained on non-deidentified data pose privacy risks. further, recent work has shown that general purpose large language models are prone to memorizing sensitive information which can subsequently be extracted @xcite . in the context of biomedical nlp, such concerns have been cited as reasons for withholding direct publication of trained model weights @xcite . these uncertainties will continue to hamper dissemination of trained models among the broader biomedical nlp research community, motivating a need to investigate the susceptibility of such models to adversarial attacks. this work is a first step towards exploring the potential privacy implications of sharing model weights induced over non-deidentified ehr text. we propose and run a battery of experiments intended to evaluate the degree to which transformers (here, bert) pretrained via standard masked language modeling objectives over notes in ehr might reveal sensitive information (figure 1 ). we find that simple methods are able to recover associations between patients and conditions at rates better than chance, but not with performance beyond that achievable using baseline condition frequencies. this holds even when we enrich clinical notes by explicitly inserting patient names into every sentence. our results using a recently proposed, more sophisticated attack based on generating text @xcite are mixed, and constitute a promising direction for future work. ## related work unintended memorization by machine learning models has significant privacy implications, especially where models are trained over nondeidentified data. @xcite was recently able to extract memorized content from gpt-2 with up to 67% precision. this raises questions about the risks of sharing parameters of models trained over non-deidentified data. while one may mitigate concerns by attempting to remove phi from datasets, no approach will be perfect @xcite . further, deidentifying ehr data is a laborious step that one may be inclined to skip for models intended for internal use. an important practical question arises in such situations: is it safe to share the trained model parameters? while prior work has investigated issues at the intersection of neural networks and privacy @xcite @xcite , we are unaware of work that specifically focuses on attacking the modern transformer encoders widely used in nlp (e.g., bert) trained on ehr notes, an increasingly popular approach in the biomedical nlp community. in a related effort, abdalla et al. ( 2020 ) explored the risks of using imperfect deidentification algorithms together with static word embeddings, finding that such embeddings do reveal sensitive information to at least some degree. however, it is not clear to what extent this finding holds for the contextualized embeddings induced by large transformer architectures. prior efforts have also applied template and probe-based methods @xcite @xcite @xcite to extract relational knowledge from large pretrained models; we draw upon these techniques in this work. however, these works focus on general domain knowledge extraction, rather than clinical tasks which pose unique privacy concerns. ## dataset we use the medical information mart for intensive care iii (mimic-iii) english dataset to conduct our experiments @xcite . we follow prior work @xcite and remove all notes except for those categorized as 'physician', 'nursing', 'nursing/others', or 'discharge summary' note types. the mimic-iii database was deidentified using a combination of regular expressions and human oversight, successfully removing almost all forms of phi @xcite . all patient first and last names were replaced with [known first name ...] and [known last name ...] pseudo-tokens respectively. we are interested in quantifying the risks of releasing contextualized embedding weights trained on non-deidentified text (to which one working at hospitals would readily have access). to simulate the existence of phi in the mimic-iii set, we randomly select new names for all patients @xcite . 4 specifically, we replaced [known first name] and [known last name] with names sampled from us census data, randomly sampling first names (that appear at least 10 times in census data) and last names (that appear at least 400 times). 5 this procedure resulted in 11.5% and 100% of patients being assigned unique first and last names, respectively. while there are many forms of phi, we are primarily interested in recovering name and condition pairs, as the ability to infer with some certainty the specific conditions that a patient has is a key privacy concern. this is also consistent with prior work on static word embeddings learned from ehr @xcite . notes in mimic-iii do not consistently explicitly reference patient names. first or last names are mentioned in at least one note for only 27,906 (out of 46,520) unique patients. 6 given that we cannot reasonably hope to recover information regarding tokens that the model has not observed, in this work we only consider records corresponding to these 27,906 patients. despite comprising 61.3% of the total number of patients, these 27,906 patients are associated with the majority (82.6%) of all notes (1,247,291 in total). further, only 10.2% of these notes contain at least one mention of a patient's first or last name. of the 1,247,291 notes considered, 17,044 include first name mentions, and 220,782 feature last name mentions. interestingly, for records corresponding to the 27,906 patients, there are an additional 18,345 false positive last name mentions and 29,739 false positive first name mentions; in these cases the name is also an english word (e.g., 'young'). as the frequency with which patient names are mentioned explicitly in notes may vary by hospital conventions, we also present semisynthetic results in which we insert names into notes such that they occur more frequently. ## enumerating conditions as a first attempt to evaluate the risk of bert leaking sensitive information, we define the following task: given a patient name that appears in the set of ehr used for pretraining, query the model for the conditions associated with this patient. operationally this requires defining a set of conditions against which we can test each patient. we consider two general ways of enumerating conditions: (1) using international classification of diseases, revision 9 (icd-9) codes attached to records, and (2) extracting condition strings from the free-text within records. 7 specifically, we experiment with the following variants. once we have defined a set of conditions to use for an experiment, we assign binary labels to patients indicating whether or not they are associated with each condition. we then aim to recover the conditions associated with individual patients. ## methods and results we first test the degree to which we are able to retrieve conditions associated with a patient, given their name. (we later also consider a simpler task: querying the model as to whether or not it observed a particular patient name during training.) all results presented are derived over the set of 27,906 patients described in section 4. the following methods output scalars indicating the likelihood of a condition, given a patient name and learned bert weights. we compute metrics with these scores for each patient, measuring our ability to recover patient/condition associations. we aggregate metrics by averaging over all patients. we report aucs and accuracy at 10 (a@10), i.e., the fraction of the top-10 scoring conditions that the patient indeed has (according to the reference set of conditions for said patient). ## limitations this work has important limitations. we have considered only relatively simple "attacks", based on token in-filling and probing. our preliminary results using the more advanced generation approach (inspired by @xcite ) is a promising future direction, although the quality of generation from bert -which is not naturally a language model -may mitigate this. this highlights a second limitation: we have only considered bert, as it is currently the most common choice of pretrained transformer in the bionlp community. auto-regressive models such as gpt-2 may be more prone to memorization. larger models (e.g., t5 @xcite or gpt-3 @xcite ) are also likely to heighten the risk of data leakage if trained over ehr. another limitation is that we have only considered the mimic-iii corpus here, and the style in which notes are written in this dataset -names appear very infrequently -likely renders it particularly difficult for bert to recover implicit as-sociations between patient names and conditions. we attempted to address this issue with the semisynthetic name insertion variant, where we artificially inserted patient names into every sentence; this did not yield qualitatively different results for most experiments. nonetheless, it is possible that experiments on ehr datasets from other hospitals (with different distributions over tokens and names) would change the degree to which one is able to recover phi. finally, these results for bert may change under different masking strategies -for example, dynamic masking @xcite or choice of tokenizer. both of these may affect memorization and extraction method performance. ## conclusions we have performed an initial investigation into the degree to which large transformers pretrained over ehr data might reveal sensitive personal health information (phi). we ran a battery of experiments in which we attempted to recover such information from bert model weights estimated over the mimic-iii dataset (into which we artificially reintroduced patient names, as mimic is deidentified). across these experiments, we found that we were mostly unable to meaningfully expose phi using simple methods. moreover, even when we constructed a variant of data in which we prepended patient names to every sentence prior to pretraining bert, we were still unable to recover sensitive information reliably. our initial results using more advanced techniques based on generation @xcite ; our results certainly do not rule out the possibility that more advanced methods might reveal phi. but, these findings do at least suggest that doing so is not trivial. to facilitate further research, we make our experimental setup and baseline probing models available: https://github.com/ elehman16/exposing_patient_data_release .