Title: Exploring the limits of strong membership inference attacks on large language models
Venue: NeurIPS 2025
Accepted: yes
Keywords: membership inference

memorization
Abstract: State-of-the-art membership inference attacks (MIAs) typically require training many reference models, making it difficult to scale these attacks to large pre-trained language models (LLMs). As a result, prior research has either relied on weaker attacks that avoid training references (e.g., fine-tuning attacks), or on stronger attacks applied to small models and datasets. However, weaker attacks have been shown to be brittle and insights from strong attacks in simplified settings do not translate to today's LLMs. These challenges prompt an important question: are the limitations observed in prior work due to attack design choices, or are MIAs fundamentally ineffective on LLMs? We address this question by scaling LiRA--one of the strongest MIAs--to GPT-2 architectures ranging from 10M to 1B parameters, training references on over 20B tokens from the C4 dataset. Our results advance the understanding of MIAs on LLMs in four key ways. While (1) strong MIAs can succeed on pre-trained LLMs, (2) their effectiveness, remains limited (e.g., AUC<0.7) in practical settings. (3) Even when strong MIAs achieve better-than-random AUC, aggregate metrics can conceal substantial per-sample MIA decision instability: due to training randomness, many decisions are so unstable that they are statistically indistinguishable from a coin flip. Finally, (4) the relationship between MIA success and related LLM privacy metrics is not as straightforward as prior work has suggested.


Peer reviews:
comments: SUMMARY:
This paper studies the effectiveness of strong Membership Inference Attacks (MIAs) on pre-trained Large Language Models (LLMs), such as GPT-2-like models. While previous research has primarily focused on smaller models or weaker attacks, this work scales one of the strongest MIAs, the Likelihood Ratio Attack (LiRA), to large-scale LLMs trained on massive datasets.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The authors conduct experiments on a larger scale than most prior research by training over 4,000 models of varying sizes.

2. The paper covers multiple dimensions of MIAs, such as the number of reference models, model size, training data size, and the impact of different hyperparameters on MIA success.

Weaknesses:

1. The focus on only two MIA methods (LiRA and RMIA) may not provide a full picture of the various possible membership inference strategies. Including more attack methods might have been useful to give a broader view of potential vulnerabilities.

2. While the paper demonstrates that strong MIAs can succeed in a controlled environment, the practical impact remains limited due to the low AUC values (<0.7).

QUESTIONS:
What potential improvements could be made in the MIA attack methodologies to push AUC values beyond 0.7, particularly for large models?

LIMITATIONS:
The experiments are based on GPT-2-like models and may not fully generalize to other types of LLMs (e.g., BERT-based or newer architectures like GPT-4). Further work would be needed to determine how these attacks scale across diverse architectures.

The technical contribution is limited, although this work provides some insights based on experiments.

Updated rating after the rebuttal.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-26

comments: SUMMARY:
This work aims to investigate whether the limitations observed in prior studies stem from the design choices of the attacks themselves or whether MIAs are fundamentally ineffective on large language models. To this end, the authors propose scaling LiRA to GPT-2 architectures ranging from 10 million to 1 billion parameters, training reference models on over 20 billion tokens from the C4 dataset.

STRENGTHS AND WEAKNESSES:
**Strengths**

1. The motivation of the manuscript is novel.

2. The manuscript offers a comprehensive experimental analysis.

**Weaknesses**

1. In principle, the main text should be self-contained; however, too many details placed in the Appendix.

2. The reason for setting 4,000 GPT-2-based reference models is not clearly explained.

3. The transitional claim regarding LLMs is somewhat misleading, as the models used in the manuscript are relatively small and do not truly qualify as large language models.

4. Can the manuscript provide feasible recommendations for MIAs that are limited to LLMs?

QUESTIONS:
Please refer to Weaknesses.

LIMITATIONS:
Please refer to Weaknesses.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies multiple phenomena for membership attacks that were little known before when it comes to larger models—with enormous numbers of parameters in the era of LLMs and huge training corpus. The paper states three claims: 1- The success rate does not exhibit any meaningful relationship with the model size. I.e., it is not true that larger models are easier to break. 2- The success rate is not related to the training size either. 3- Samples later seen at training are more susceptible to the membership attacks, yet the connection with data extraction is not clear.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \nThe paper is well written, with the messages clear in each section and sections well-partitioned which follow a natural flow. The paper has several important messages (mentioned above) that were not fully investigated before, and have a clear impact on the literature. This will motivate the community for further studies. \n\nWeaknesses: \n1- The novelty of the paper is limited as it is grounded on the previously known LiRA attack. The major contribution of the paper is scaling all the models, the number of reference models, and the training data. \n2- One of the surprising results is that AUC never surpasses 0.7 when increasing the number of reference models. Even though the paper claims that they study the fundamental limits of membership attacks (for instance, lines 9 and 10), the low success that is reported only based on LiRA questions the generality and universality of their results. \n3- Having said that, there is more room for explaining many of the results. E.g., why does the saturation for the number of reference models happen so quickly? \n4- In Figure 4a and 4b, where the relation with more training epochs is studied, the authors resort to discussing AUC only. Whereas the plots keep toggling for different values of FPR and it looks like the mentioned message does not apply everywhere. In other words, the statement that membership attacks are more successful when increasing the number of training epochs seems to oversimplify what’s actually happening in the experiments.\n\nQUESTIONS:\nAddress weaknesses above\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper scales strong MIAs to large language models, addressing limitations of prior work that used weaker or small-scale attacks. By applying the LiRA attack to thousands of GPT-2–like models, the authors show that strong MIAs can succeed, but their effectiveness is limited in practical settings. They also find that vulnerability does not consistently increase with model size and that MIAs and data extraction capture different privacy risks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Insightful results (e.g., data used in later training is more vulnerable; no clear relationship between MIA and extraction attacks) — these findings are meaningful and could influence future studies.\n\n2. Large-scale experiments.\n\n3. Demonstrates that strong MIAs on LLMs are feasible.\n\nWeaknesses:\n\n1. For the most part, this paper uses AUC as the primary evaluation metric. However, the LiRA paper highlights that TPR at low FPR may be a more appropriate measure for evaluating MIAs. It would be helpful if the authors could also present findings using these alternative metrics, which may yield additional insights.\n\n2. The attack methods are not very novel, but applying existing SOTA MIAs in the LLM setting.\n\nThank you for your submission. I enjoyed reading this paper and learned new things from it—especially the insightful findings about which data is more vulnerable to MIAs and the relationship between MIA and extraction attacks (where many prior works suggested a connection, but this paper presents a different perspective). That said, I do have some concerns, as mentioned above.\n\nQUESTIONS:\n1. Why does the paper mostly focus on the AUC evaluation metric and not on TPR at low FPR? Can you provide more insight from this perspective?\n\n2. Can we provide more insights into MIA attack designs or suggest directions for improving MIA attack design for LLMs?\n\n3. This paper involves large-scale experiments that require significant computational resources. Can we also provide more insights from a cost perspective? Also, the possibilities of further scaling this to larger LLMs?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-07-03"}