Title: Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Membership Inference Attack

Transfer Learning

Few-shot Learning

Image Classification
Abstract: Membership inference attacks (MIAs) are used to test practical privacy of machine learning models. MIAs complement formal guarantees from differential privacy (DP) under a more realistic adversary model. We analyse MIA vulnerability of fine-tuned neural networks both empirically and theoretically, the latter using a simplified model of fine-tuning. We show that the vulnerability of non-DP models when measured as the attacker advantage at a fixed false positive rate reduces according to a simple power law as the number of examples per class increases. A similar power-law applies even for the most vulnerable points, but the dataset size needed for adequate protection of the most vulnerable points is very large.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper both theoretically and empirically establishes a quantitative relationship between black-box membership inference attack (MIA) success and the number of examples per class for fine-tuned image classification models. Leveraging this relationship, the authors propose a method to predict the MIA vulnerability of datasets under transfer learning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \n- Theoretical analyses: The authors establish a power law relationship between per-example vulnerability and examples per class for a simplified model of membership inference. They then generalize it for average vulnerability. \n- Thorough empirical validation: The empirical findings confirm the theoretical relationship, demonstrating that average MIA vulnerability can be predicted using the number of classes and examples per class. The authors show that this prediction generalizes to different target model and fine-tuning methods, though it tends to underestimate vulnerability when training from scratch. \n- Discussion of how this observation connects to DP guarantees, concluding that the number of examples per class required for protecting the most vulnerable examples is impractically large \n- clearly written, well organized \n\nWeaknesses:  \n- only dataset properties discussed are the number of classes and examples per class (see questions) \n- would benefit from more discussion around the underlying data assumptions (see limitations) \n- As stated, has limited practical relevance (but I still believe the work provides an interesting perspective that emphasizes the necessity of noise addition in transfer learning to mitigate privacy risks)\n\nQUESTIONS:\nWhat about other data properties, e.g., variability within or between classes?\n\nI suggest adding a short explanation of how RMIA improves performance even with a limited number of shadow models (Line 122). Do you know why RMIA is more unstable than LiRA?\n\nLIMITATIONS:\nYes, but I would like to see a more detailed discussion on the assumptions made on the underlying distributions and how these assumptions influence the conclusions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores how class size affects data points' vulnerability to MIA in a transfer learning setting. Building on an experimentally observed power law relationship between class size and MIA vulnerability, the authors theoretically derive and experimentally verify this relationship, specifically for fine-tuned vision transformers and ResNet. The paper works to connect MIA vulnerability under these conditions to DP guarantees.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- Well developed theoretical explanations of how dataset properties relate to MIA vulnerability. This is an important contribution as the community actively works to understand what makes certain points more vulnerable to MIAs. \n- Thorough experimental exploration of power law results in the transfer learning setting with ViTs and ResNet. \n\nWeaknesses: \n- Claims about DP in abstract are confusing/unnecessary given that the majority of the paper proceeds to ignore DP. \n- Unclear how this result generalizes. The power law demonstrated in Figure 1 is derived in a very particular setting (e.g. fine-tuning on a ViT), and it's unclear if this result generalizes to other architectures and/or learning paradigms such as learning from scratch. Theory suggests it will, but it would be nice to see this backed up with experiments and/or argument. \n- The run-on sentence at the end of the abstract was very hard to parse. There are several other run on-sentences in the paper (e.g. last sentence of 1st intro para). Please fix for readability. \n- Figure 1 needs more explanation. What is C? Are all the different colors representing datasets? Why do you use the term \"shots\"? This may all be explained later in the paper, but the first figure should be interpretable on its own. Add content to the legend accordingly.\n\nQUESTIONS:\nWhy do you think your prediction model ``underestimates the vulnerability of the from-scratch trained target models'' (lines 295-296)? What could be modified in your theory to better account for this, given that from-scratch training remains an important learning paradigm. \n\nHow is the regime explored here \"between DP and typical MIA evaluations\"? This claim is made in intro, but practically it just seems like this work derives theory for and experiments with non-DP fine-tuned classifiers. DP is only mentioned in lines 316-327 and some results shown in Figure 1. Given the strong claims in the intro about this work bridging MIA and DP, I would have expected to see much more substantial discussion of the interplay between these two methods. \n\nMore work is needed to distinguish the contribution of this paper from other works that have previously identified small class size as a risk factor for MIA (e.g. Kulynych et al, Chang and Shokri, etc.) Can the authors clarify where others' work ends and theirs begins?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a theoretical analysis of the impact of dataset size and numbers of classes on the performance of MIA attacks. At a high level, it derives relationships between the number of classes, datapoints per class, and the MIA vulnerability of a fine-tuned model when subject to a LIRA or RIMA attacks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n+ This is a large area with lots of empirical results - although the general intuitions exist that 1) larger the dataset (i.e. more examples per class) makes MIA attacks harder and 2) the more complex it is in terms of number of classes the MIA performance improves; this paper formalises this with a theoretical analysis to give a more principled grounding and better predictive ability for pre-deployment useage of models.\n\n+ There is a good selection of different models and datasets (albeit, in the image only domain) showing results across a wide cross section.\n\nWeakness\n\n+ The theoretical analysis holds for fine-tuning, although diverges significantly for models trained from scratch in Fig. 5b, with from scratch models being significantly more vulnerable. Tentative explanations are given, though they are not very rigorous. \n\n+ In particular, this can point to the pre-training dataset having a strong influence on the results - which deviate from the theory. If the theory only follows if the last layer, and last layer only, is fine tuned then this further limits the scope of the results.\n\n+ The analysis focuses heavily on two attacks. The proposed theory may not hold in a general sense to future (potentially stronger) attacks that rely on a different attack approach or principle. It is however, worth noting that the paper highlights that a LiRA based approach is the optimal strategy to carry out MIA via Neyman–Pearson lemma, so a strong attack class is considered here.\n\nOverall\n\nOn the whole, even if just relevant to fine-tuning, it is is a very common deployment scenario and, for now, the analysed attacks are some of the strongest MIA attacks to date. Thus, adding theory to practical MIA effectiveness is a useful contribution.\n\nQUESTIONS:\nSee Weaknesses for areas which could be improved.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

comments: SUMMARY:
This paper explores membership inference attacks (MIAs) with respect to dataset properties under the non-differential privacy (non-DP) setting for fine-tuned models. Specifically, it establishes a power-law relationship indicating reduced MIA vulnerability as the number of examples per class increases, based on a simplified regression model. The employed MIA methods include LiRA and RMIA. This is a technically solid paper, supported by multiple experiments that demonstrate the effectiveness of the findings.

STRENGTHS AND WEAKNESSES:
**Strengths**
1.  The methodology is technically sound, and the presentation is clear and well-structured.

2. The employed MIA methods, LiRA and RMIA, are among the current state-of-the-art.

3.  The paper studies both average and individual-level vulnerabilities, providing a more complete picture.



**Weaknesses**

1. The motivation for focusing on fine-tuned models is unclear. Please refer to Question 1.

2. The theoretical power law is derived from a simplified model and does not directly apply to deep learning models, where the relationship between training data and MIA vulnerability has already been extensively studied. This limits the overall contribution.

QUESTIONS:
1. Regarding the scope of the research, what is the rationale for studying the power law of MIA on fine-tuned models? Would this relationship be more difficult to observe if the model were trained from scratch?
2. For LiRA, does the offline version still exhibit the observed power-law behavior?
3. In Figure 3, could the authors explain why the power-law trend for RMIA appears less stable compared to that of LiRA?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03