Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks (2022)

Abstract: The wide adoption and application of Masked language models (MLMs) on sensitive data (from legal to medical) necessitates a thorough quantitative investigation into their privacy vulnerabilities. Prior attempts at measuring leakage of MLMs via membership inference attacks have been inconclusive, implying potential robustness of MLMs to privacy attacks.In this work, we posit that prior attempts were inconclusive because they based their attack solely on the MLM’s model score. We devise a stronger membership inference attack based on likelihood ratio hypothesis testing that involves an additional reference MLM to more accurately quantify the privacy risks of memorization in MLMs. We show that masked language models are indeed susceptible to likelihood ratio membership inference attacks: Our empirical results, on models trained on medical notes, show that our attack improves the AUC of prior membership inference attacks from 0.66 to an alarmingly high 0.90 level.

## introduction bert-based encoders with masked language modeling (mlm) objectives @xcite have become models of choice for use as pre-trained models for various natural language processing (nlp) classification tasks @xcite @xcite and have been applied to diverse domains such as disease diagnosis, insurance analysis on financial data, sentiment analysis for improved user experience, etc @xcite @xcite . given the sensitivity of the data used to train these models, it is crucial to conceive a framework to systematically evaluate the leakage of training data from these models @xcite @xcite , and limit the leakage. the conventional way to measure the leakage of training data from machine learning models is by performing mem-bership inference attacks @xcite , in which the attacker tries to determine whether a given sample was part of the training data of the target model or not. these attacks expose the extent of memorization by the model at the level of individual samples. prior attempts at performing membership inference and reconstruction attacks on masked language models have either been inconclusive @xcite , or have (wrongly) concluded that memorization of sensitive data in mlms is very limited and these models are more private than their generative counterparts (e.g., autoregressive language models) @xcite @xcite . we hypothesize that prior mlm attacks have been inconclusive because they rely solely on the target model's (model under attack) loss on each individual sample as a proxy for how well the model has memorized that sample. if the loss is lower than a threshold, the sample is predicted to be a member of the training set. however, the target model's loss includes confounding factors of variation like the intrinsic complexity of the sample -and thus provides a limited discriminative signal for membership prediction. this scheme has either a high false-negative rate (with a conservative threshold) -classifying many hard-to-fit samples from the training set as non-members, or a high false-positive rate (with a generous threshold) -failing to identify easy-to-fit samples that are not in the training set. reference-based likelihood ratio attacks, on the other hand, when applied to certain probabilistic graphical models and classifiers, have been shown to alleviate this problem and more accurately distinguish members from non-members @xcite . in such attacks, instead of the loss of the model under attack, we look at the ratio of the likelihood of the sample under the target model and a reference model trained on samples from the underlying population distribution that generates the training data for the target model. this ratio recalibrates the test statistic to explain away spurious variation in model's loss for different samples due to the intrinsic complexity of the samples. unlike most other models (e.g., generative models), however, computing the likelihood of mlms is not straightforward. in this paper, we propose a principled framework for measuring information leakage of mlms through likelihood ratio-based membership inference attacks and perform an extensive analysis of memorization in such models. to compute the likelihood ratio of the samples under the target and the reference mlms, we view the mlms as energy-based probabilistic models @xcite over the sequences. this enables us to perform powerful inference attacks on conventionally non-probabilistic models like masked language models. we evaluate our proposed attack on a suite of masked clinical language models, following @xcite . we compare our attack with the baseline from the prior work that relies solely on the loss of the target model @xcite @xcite . we empirically show that our attack improves the auc from 0.66 to 0.90 on the clinicalbert-base model, and achieves a true positive rate (recall) of 79.2% (for a false positive rate of 10%), which is a substantial improvement over the baseline with 15.6% recall. this shows that, contrary to prior results, masked language models are significantly susceptible to attacks exploiting the leakage of their training data. in low error regions (at 1% false positive rate) our attack is 51× more powerful than the prior work. we also present analyses of the effect of the size of the model, the length of the samples, and the choice of the reference model on the success of the attack. finally, we attempt to identify features of samples that are more exposed (attack is more successful on), and observe that samples with multiple non-alphanumeric symbols (like punctuation) are more prone to being memorized. we provide instructions on how to request access to the data and code in appendix a.2.1. ## membership inference attacks in this section, we first formally describe the membership inference attack, how it can be conducted using likelihood ratio tests and how we apply the test for masked language models (mlms) which do not explicitly offer an easy-to-compute probability distribution over sequences. finally, we describe all the steps in our attack, as summarized in figure 1 . ## experimental setup we conduct our experiments using the preprocessed data, and pre-trained models provided by @xcite . we use this medical-based setup as medical notes are sensitive and leakage of models trained on notes can cause privacy breaches. in this section, we briefly explain the details of our experimental setup. appendix a.2 provides more details. ## results in this section, we discuss our experimental results and main observations. first, we explore the overall performance improvement of our approach over baselines. later, we analyze the effectiveness of our approach across several factors of variation that have an effect on the leakage of the model. (e.g. length of samples, model size, including names etc.) finally, we explore correlations between samples that are deemed to be exposed by our approach. we provide further studies and ablations on choosing a lower false-positive rate of 1%, using different energy formulation and changing target sequence lengths in appendix sections a. ## related work prior work on measuring memorization and leakage in machine learning models can be classified into two main categories: (1) membership inference attacks and (2) training data extraction attacks. membership inference. membership inference attacks (mia) try to determine whether or not a target sample was used in training a target model @xcite . these attacks can be seen as privacy risk analysis tools @xcite @xcite , which help reveal how much the model has memorized the individual samples in its training set, and what the risk of individual users is @xcite @xcite @xcite . a group of these attacks rely on behavior of shadow models to determine the membership of given samples @xcite . song and shmatikov mounts such an attack on lstm-based text-generation models, mahloujifar et al. mounts one on word embedding, hisamoto et al. applies it to machine translation and more recently, shejwalkar et al. mounts it on transformerbased nlp classification models. mounting such attacks is usually costly, as their success relies upon training multiple shadow models on different partitionings of shadow data, and access to adequate shadow data for training such models. another group of mias relies solely on the loss value of the target sample, under the target model, and thresholds this loss to determine membership @xcite . song and raghunathan mount such an attack on word embedding, where they try to infer if given samples were used in training different embedding models. @xcite , which is the work closest to ours, uses a thresholding loss-based attack to infer membership on mlms. our approach instead incorporates a reference model by using an energy-based formulation to mount a likelihood ratio based attack and achieves higher auc as shown in the results. training data extraction. training data extraction quantifies the risk of extracting training data by probing a trained language model @xcite @xcite @xcite . one such prominent attacks on nlp models is that of @xcite , where they take more than half a million samples from different gpt-2 models, sift through the samples using a membership inference method to find samples that are most likely to have been memorized. @xcite mount the same data extraction attack on mlms, but their results are inconclusive as to how much mlms memorize samples. they also mount other types of attacks, where they try to extract a person's name given their disease, or disease given name, but in all their attacks, they only use signals from the target model and consistently find that a frequency-based baseline (i.e. one that would always guess the most frequent name/disease) is more successful. ## conclusions in this paper, we introduce a principled membership inference attack based on likelihood ratio testing to measure the training data leakage of masked language models (mlms). in contrast to prior work on mlms, we rely on signals from both the model under attack and a reference model to decide the membership of a sample. this enables performing successful membership inference attacks on data points that are hard to fit, and therefore cannot be detected using the prior work. we also perform an analysis of why these models leak, and which data points are more susceptible to memorization. our attack shows that mlms are significantly prone to memorization. this work calls for designing robust privacy mitigation algorithms for such language models.