Using Membership Inference Attacks to Evaluate Privacy-Preserving Language Modeling Fails for Pseudonymizing Data (2023)

Abstract: Large pre-trained language models dominate the current state-of-the-art for many natural language processing applications, including the field of clinical NLP. Several studies have found that these can be susceptible to privacy attacks that are unacceptable in the clinical domain where personally identifiable information (PII) must not be exposed. However, there is no consensus regarding how to quantify the privacy risks of different models. One prominent suggestion is to quantify these risks using membership inference attacks. In this study, we show that a state-of-the-art membership inference attack on a clinical BERT model fails to detect the privacy benefits from pseudonymizing data. This suggests that such attacks may be inadequate for evaluating token-level privacy preservation of PIIs.

## introduction state-of-the-art results in natural language processing typically rely on large pre-trained language models (plms) such as bert @xcite or models in the gpt family @xcite . multiple studies have found that their large number of parameters can cause plms to unintentionally memorize information in their training data, making them vulnerable to privacy attacks @xcite . at the same time, other studies have shown that training plms using domain-specific data yields better results on domain-specific tasks @xcite . in the clinical domain, these combined findings pose a significant challenge: training plms with clinical data is necessary to achieve state-of-the-art results. however, plms can be vulnerable to privacy attacks that are especially dangerous when training with clinical data. broadly speaking, these attacks can be divided into two classes: training data extraction attacks and membership inference attacks. ## methods and data this study closely mirrors the experimental setup used by @xcite in order to minimize discrepancies stemming from differences in implementation details. the datasets and models are based on resources introduced by @xcite . the experiments aim to examine whether or not membership inference attacks can distinguish between a model trained using real or pseudonymized data. ## results three different attacks are performed using three different datasets as the in-data. the accuracy, precision, and recall values of each attack are listed in experiments (a) and (b) represent cases where we have not performed any pseudonymization of the training data. that is, the in-data are used to train the bert model without employing any privacy-preserving techniques. experiment (c) is the result of the simulated scenario where perfect pseudonymization is employed to preserve the privacy of the data. in other words, the model is not exposed to any real names during training. the privacy gains from using this technique are not reflected by the metrics in ## discussion and conclusions this study focuses specifically on protecting names. future research would benefit from analyzing additional categories of pii. however, the data and models created by @xcite focus specifically on names. this class of pii is used in this study to facilitate comparisons with earlier studies. the results from the three experiments in it remains to be shown which of the datapoint's characteristics are used to separate members from non-members. the results of our experiments suggest that using this mia does not accurately quantify the privacy gains from using pseudonymization, which instead operates on the token level. while the scope of this short paper was limited to evaluating a state-of-the-art mia for bert models, future research should also evaluate other mias and a wider range of privacy-preserving techniques.