Order of Magnitude Speedups for LLM Membership Inference (2024)

Abstract: Large Language Models (LLMs) have the promise to revolutionize computing broadly, but their complexity and extensive training data also expose significant privacy vulnerabilities. One of the simplest privacy risks associated with LLMs is their susceptibility to membership inference attacks (MIAs), wherein an adversary aims to determine whether a specific data point was part of the model’s training set. Although this is a known risk, state of the art methodologies for MIAs rely on training multiple computationally costly ‘shadow models’, making risk evaluation prohibitive for large models. Here we adapt a recent line of work which uses quantile regression to mount membership inference attacks; we extend this work by proposing a low-cost MIA that leverages an ensemble of small quantile regression models to determine if a document belongs to the model’s training set or not. We demonstrate the effectiveness of this approach on fine-tuned LLMs of varying families (OPT, Pythia, Llama) and across multiple datasets. Across all scenarios we obtain comparable or improved accuracy compared to state of the art ‘shadow model’ approaches, with as little as 6% of their computation budget. We demonstrate increased effectiveness across multi-epoch trained target models, and architecture miss-specification robustness, that is, we can mount an effective attack against a model using a different tokenizer and architecture, without requiring knowledge on the target model.

## introduction membership inference attacks (mias) and reconstruction attacks pose significant risks to the privacy of data used to fine-tune large language models (llms). when general purpose llms are used in specific applications such as automated customer support, they often require fine-tuning on proprietary, domain-specific datasets to improve their performance and relevance. this process, however, can inadvertently expose sensitive information. mias witness this vulnerability by reliably determining whether a specific data point was part of the training dataset or not, thereby potentially revealing personal or proprietary information. membership inference attacks are used to audit the privacy of trained models, and successful external attacks can lead to breaches of confidentiality, financial loss, and erosion of user trust. finetuning can amplify these risks, as models trained on smaller, specialized datasets are more susceptible to memorizing and revealing specific data points, and specialized datasets not found on the open internet can contain sensitive user information. recent studies have found that credible privacy attacks can be mounted against modern llms @xcite @xcite . however, it is relatively uncommon to routinely assess fine-tuned llms for mia risk, as current state of the art mias require the training of several 'shadow models'models that are, ideally, identical in nature to the model under attack in terms of architecture, training data distribution, and hyperparameters @xcite @xcite . the result is that mounting such an attack -the building block of an audit -is substantially more expensive than training the llm in the first place. to circumvent the high computational costs of mounting a shadow-model-based attack, recent works such as @xcite has attempted to directly reduce the cost of hypothesis-testing style membership inference attacks by replacing shadow models with a quantile regression step that directly estimates the featureconditional quantile of a score function from data known not to have been used in model training, where the target quantile of the score distribution directly corresponds to the false positive rate of the attack. the quantile regression approach is computationally attractive because it only requires training a single model (rather than multiple shadow models), and the architecture of the quantile regression model need not be related to (and indeed can be much simpler than) the architecture of the model under attack. prior work has only demonstrated the effectiveness of this approach in relatively simple classification settings @xcite and for small diffusion models @xcite . in this paper we extend this line of quantile-regression based attacks to large language models. we briefly summarize our contributions: to exemplify this, we use pythia-160m @xcite and opt-125m @xcite architectures to attack pythia, llama @xcite , and opt models up to 7b parameters. ## method here we provide a detailed description of our attack, starting with the general idea of how scorebased membership inference attacks are designed, and followed with a technical description of how we design our low-cost ensemble attack. we follow a standard setup of membership inference attacks in which the adversary has query access to an llm f trained on an unknown dataset d priv in terms of log likelihoods per-token for an arbitrary input sequence, sampled from a document distribution d. each sample @xmath0, usually split into tokens @xmath1. the model f outputs a probability distribution of the next token x i conditioned on the preceding token sequence because of this, training samples are potentially memorized, or are more likely under the model's distribution than other, similar samples that might be equally likely under the sampling distribution. a membership inference attack is a hypothesis test that exploits this tendency by using a test statistic (score) derived from queries to f that aims to determine whether a document x is a member of the training set d priv or not. we cast this as distinguishing between a null and alternative hypothesis: we restrict our attention to membership inference attacks that define a test statistic (score) s(x; f ). these attacks determine if this input-score pair (s(x; f ), x) is likely under the null hypothesis, and accuse a document of being part of the private dataset if this test fails (i.e. rejects the null). we'll denote s(x; f ) = s(x) = s when clear from context. a score is any function computable given access to the model f and target point x. the intention is to choose a score that takes systematically higher values for @xmath2. examples of such scores are discussed in 2.2; for scores that are computed per token, we take the per document score to be the token-averaged score. the adversary's goal is to learn an attack function a f : @xmath3, 1} that implements the hypothesis test described above. the works discussed here follow a common thread by implementing the adversary as ## results here we discuss our experimental results and main observations. we first show an overall performance evaluation of our method along with the baselines. we then specifically discuss scalability and crossfamily performance, and compare against lira in this scenario. we further explore how ensemble ## broader impact advancements in membership inference attacks (mias) for large language models (llms) are important for improving privacy auditing and compliance. by improving the efficiency of mias, our work helps auditors more routinely evaluate deployed models for privacy properties (or lack thereof). by making privacy leakage more easily measurable, we hope our work encourages privacy to become a first-order design desiderata in largescale machine learning. improved mias of course also increase the risk of external attacks. thus in the short run, work on privacy attacks (including ours) can increase the privacy risk of deployed models. nevertheless, we believe that in the long run exposing privacy risk is an essential step to mitigating it.