Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning
Authors: Marlon Tobaben
Hibiki Ito
Joonas Jälkö
Yuan He
Antti Honkela
Venue: NeurIPS 2025 poster ()
Keywords: Membership Inference Attack
Transfer Learning
Few-shot Learning
Image Classification
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We derive a power-law relationship between the number of examples per class and per-example vulnerability to membership inference and support it with extensive experiments.

Abstract: Membership inference attacks (MIAs) are used to test practical privacy of machine learning models. MIAs complement formal guarantees from differential privacy (DP) under a more realistic adversary model. We analyse MIA vulnerability of fine-tuned neural networks both empirically and theoretically, the latter using a simplified model of fine-tuning. We show that the vulnerability of non-DP models when measured as the attacker advantage at a fixed false positive rate reduces according to a simple power law as the number of examples per class increases. A similar power-law applies even for the most vulnerable points, but the dataset size needed for adequate protection of the most vulnerable points is very large.

Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning
Marlon Tobaben
Hibiki 2+1 Joonas Jälkö Yuan He
Antti Honkela
Department of Computer Science, University of Helsinki, Finland
School of Informatics, Kyoto University, Japan {marlon. tobaben, joonas jalko, , yuan he antti antta.tooneeaa honkell)@heelsinks ito hibiki. 77n0st w
Abstract
Membership inference attacks (MIAs) are used to test practical privacy of machine learning models. MIAs complement formal guarantees from differential privacy (DP) under a more realistic adversary model. We analyze MIA vulnerability fine-tuned neural networks both empirically and theoretically, the latter using a simplified model of fine-tuning. We show that the vulnerability of non-DP DD models when measured as the attacker advantage according to simple power law as the number of examples per class increases. A similar power-law applies even for the most vulnerable points, but the dataset size needed for adequate protection of the most vulnerable points is very large.
fixed false positive rate reduces
Introduction
Membership inference attacks (MIAs; Shokri et al., 2017; Carlini et al., 2022) and differen- tial privacy (DP: Dwork et al., 2006) provide complementary means of deriving lower and up- per bounds for the privacy loss of machine learning algorithm. Yet, the two operate under slightly different threat models. DP implicitly assumes very powerful adversary with access to all training data except the target point and provides guarantees against every target point.
(TPP) (TPR-FFM
at FPR==0.001
0.1
0.01
0.001
0.0001
16
64
256
1024
4096
16384 65536
Examples per class
- Patch Camelyon (C=2) - Pets (C'=37)
CIFAR (C=11)
Resics45 (CO45)
MIAS assume often more realistic adversary model with access to just the data distribution and the unknown training data becoming latent variables that introduce stochasticity into the attack. However, the practical evaluation is sta- tistical and cannot provide universal guarantees.
EuroSAT (C=10)
(C=100)
We observe a power-law relation be- tween MIA vulnerability and examples per class (denoted as or shots) when attacking a fine-tuned VIT-B Head using LiRA. Fach colored line denotes a different fine-tuning dataset where C' specifies the number of classes. The solid line is median and the error bars the min/max bounds for the Clopper- Pearson CIs over six seeds.
Figure 1:
In this paper, seek to explore MIA vulner- ability to extrapolate this gap. Inspired by an empirical finding that average MIA vulnerability of neural network fine-tuning strongly reduces the number of samples the target class increases (see Figure 1), we develop theory of optimal MIA against simple model of neural network fine-tuning and reproduce the decrease in vulnerability.
'These authors contributed equally. Work performed in part while at the University of Helsinki.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Furthermore, the theoretical model predicts that the vulnerability of all individual samples should reduce as the number of samples increases, which we are able to verify empirically.
To achieve our goal, we theoretically analyze and systematically apply two state-of-the-ar black- box MIAs, LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024), to help understand practical privacy risks when fine-tuning deep-learning-
classifiers without DP protections. For lemma (Neyman and Pearson, 1933). Under the black-box threat model, in which the adversary does not have access to the model parameters, LiRA and RMIA have been shown to empirically outperform other attacks, especially when the number of shadow models is sufficiently large.
the theoretical model that we analyze, LiRA is the optimal attack by the Neyman-Pearsee
We focus on transfer learning using fine-tuning because this is increasingly used for all practical applications of deep learning and especially important when labeled examples are limited, which is often the case in privacy-sensitive applications. Our case study focuses understanding and quantifying factors that influence the vulnerability of non-DP deep transfer learning models to MIA. In particular, we theoretically study the relationship between the number of examples per class, which we denote as shots (S), and MIA vulnerability (true positive rate TPR at fixed false positive rate FPR) for a simplified model of fine-tuning, and derive a power-law relationship (Figure 1) in the form
log(TPR- FPR) = -Bg log(S) - Bo.
We complement the theoretical analysis with extensive experiments over many datasets with varying sizes, in the transfer learning setting for image classification tasks, and observe the same power-law. Based on extrapolation from our results, the number of examples per each class that are needed for adequate protection of the most vulnerable samples appears very high.
Related work There has been evidence that classification models with more classes are more vulnerable to MIA (Shokri al., 2017). models trained on fewer samples can be more vulnerable (Chen et al., 2020; Németh et al., 2025), and classes with less examples tend to be more vulnerable (Chang and Shokri, 2021; Kulynych et al., 2022; Tonni al., 2020). A larger generalization error, which is related to dataset size, has also been shown to be sufficient for MIA success (Song and Mittal, 2021), though not necessary (Yeom et al., 2018). Similarly, minority subgroups tend to be more affected by DP (Suriyakumar et al., 2021; Bagdasaryan et al., 2019). Feldman and Zhang (2020) showed that neural networks trained from scratch are required to memorize a significant fraction of their training data to obtain high utility, while the memorization is greatly reduced for fine-tuning. Additionally, Tobaben et al. (2023) reported how the MIA vulnerability of few-shot image classification is affected by the number of shots. Yu et al. (2023) studied the relationship between the MIA vulnerability and individual privacy parameters for different classes. Recently, worst-case MIA vulnerability has gained more attention (Guépin et al., 2024; Meeus et al., 2024; Azize and Basu, 2025). Nonetheless, the prior works do not consider the rate of change in the vulnerability evaluated at low FPR, as dataset properties change. Our work significantly expands on these works by a) explicitly identifying a quantitative relationship between dataset properties and MIA vulnerability, i.e., the power-law in Equation (I), and b) focusing on the worst-case vulnerability, both evaluated at low FPR. This in turn allows us to extrapolate MIA vulnerability to DP guarantée.
List of contributions We analyze the MIA vulnerability of deep transfer learning using two state-of- the-art score-based MIAs, LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024), which are strong realistic threat model. We first analytically derive the power-law relationship in Equation (1) for both MIAS by introducing a simplified model of the optimal membership inference (Section 3). We support our theoretical findings by an extensive empirical study on the MIA vulnerability of deep learning models by focusing on a lransie-learning setting for image classification task, where a large pre-trained neural network is fine-tuned on a sensitive dataset.
1. Power-law in a simplified model of the optimal MIA: We formulate a simplified model of MIA to quantitatively relate dataset properties and MIA vulnerability. In this model LiRA is the optimal attack. For this model, we prove a power-law relationship for both average and worst-case between the LiRA as well RMIA vulnerability and the number of examples per class (See Section 3.4).
2. MIA experiments on the average case vulnerability: We conduct a comprehensive study of MIA vulnerability (TPR at a fixed low FPR) in the transfer learning setting for image classification tasks with target models trained using many different datasets with varying sizes and confirm the theoretical law between the number of examples per class and the vulnerability to MIA (see Figure I and Section 4.2). We fit regression model which follows the functional form of the

---

theoretically derived pawer-law. We show both a very good fit on the training data as well as good prediction quality on unseen data from different feature extractor and when fine-tuning other parameterizations (see Section 4.3).
MIA experiments on the worst-case vulnerability: We extend the experiments to worst-case in- dividual sample vulnerabilities and observe similar decrease in vulnerability for quantiles of vulnerable data points and a slower decrease for the maximum individual vulnerability (Sec- tion 4.4). By extrapolation we find that an adequate protection of the most vulnerable samples would require an extremely large dataset (Section 4.5).
Background
Notation for the properties of the training dataset D: (i) C for the number of classes (ii) S for shots (examples per class) (iii) [D] for training dataset size ([D) = CS). We denote the number of MIA shadow models with M.
Membership inference attacks (MIAs) aim to infer whether a particular sample was part of the training of the targeted model (Shokri al., 2017). Thus, they can be used to determine lower bounds on the privacy leakage of models to complement the theoretical upper bounds obtained through DP.
Likelihood Ratio attack (LiRA; Carlini et al., 2022) While many different MIAs have been pro- posed (Hu et al., 2022), in this work consider the Likelihood Ratio Attack (LiRA). LiRA is strong attack that assumes an attacker that has black-box access to the attacked model, knows the training data distribution, the training set size, the model architecture, hyperparameters and training algorithm. Based on this information, the attacker can train so-called shadow models (Shokri et al., 2017) which imitate the model under attack but for which the attacker knows the training dataset.
LiRA exploits the observation that the loss function value used to train a model is often lower for the examples that were part of the training set compared to those that were not. For a target tuple (a. v). where y is a label, LiRA trains the shadow models: (i) with (x, y) as a part of the training set ((x,)) y) E D) and (ii) without æ in the training set ((x, y) D). After training the shadow models, (x, y) is passed through the shadow models, and based on the losses (or predictions) two Gaussian distributions are formed: one for the losses of (æ. y) E DD shadow models, and one for the y) & D. Finally, the attacker computes the loss for the point I using the model under attack and determines using likelihood ratio test on the distributions built from the shadow models whether it is more likely that (z. y) E D or (x, y) & 'D. When the true distributions of the shadow models are Gaussians, LiRA is the optimal attack provided by the Neyman-Pearson lemma (Neyman and Pearson, 1933). We use an optimization by Carlini et al. (2022) for performing LiRA for multiple models and points without training a computationally infeasible number of shadow models. It relies on sampling the shadow datasets in a way that each sample is in expectation half of the time included in the training dataset of shadow model and half of the time not. At attack time each model will be attacked once using all other models as shadow models.
Robust Membership Inference Attack (RMIA; Zarifzadeh et al., 2024) RMIA is new MIA algorithm, which aims to improve performance when the number of shadow models is limited. We show both theoretically and empirically that the power-law also holds for RMIA.
Measuring MIA vulnerability Using the chosen MIA score of our attack, we can build binary classifier to predict whether a sample belongs to the training data or not. The accuracy profile of such classifier can be used to measure the success of the MIA. More specifically, throughout the rest of the paper, we will use the true positive rate (TPR) at specific false positive rate (FPR) as measure for the vulnerability. Identifying even a small number of examples with high confidence is considered harmful (Carlini et al., 2022) and thus we focus on the regions of small FPR.
Measuring the uncertainty for TPR The TPR values from the LiRA-based classifier can be seen as maximum iikelitood-estimatoos estimators for the probability of producing true positives among the positive samples. Since we have a finite number of samples for our estimation, is important to estimate the uncertainty in these estimators. Therefore, when we report the TPR values for a single repeat of the learning algorithm, we estimate the stochasticity of the TPR estimate by using Clopper- intervals (Clopper and Pearson, 1934). Given TP true positives among P positives, the 1-a a confidence
3

---

Clopper-Pearson interval for the TPR is given as
B(a/2; P- - TP 1) < TPR < B(1 - a/2; P, 4.P-TP). TP),
(2)
where (gg a, b) is the gth-quantile of Beta(a, b) distribution.
Theoretical analysis
In this section, seek to theoretically understand the impact of the dataset properties the MIA vulnerability. is known that different data points exhibit different levels of MIA vulnerability depending the underlying distribution (e.g. Aemi al., 2024; Leemann et al., 2024). Therefore, we start with analysing per-example MIA vulnerabilities. In order to quantitatively relate dataset properties these vulnerabilities, a simplified model is formulated. Within this model, we prove power-law between the per-example vulnerability and the number S of examples per class. Finally, the per-example power-law is analytically extended average-case MIA vulnerability, for which provide empirical evidence in Section 4. We primarily focus on the analysis of (online) LiRA, since is the optimal attack in our simplified model. We show that similar theoretical results also hold for RMIA in Appendix and offline LiRA Appendix A.4.
3.1 Preliminaries
First, let restate the MIA score from (online) LiRA defined by Carlini et al. (2022). Denoting the logit a target model M applied on a target data point (æ, y) as E(M(x)) y). LiRA computes the MIA score as the likelihood ratio
p(((M((z).)) v) (x, y)) LR(x) p(e(M((x)))) v) Qx,((x,)))
(3)
where the Qin/out denote the hypotheses that (a. y) was or was not the training set of M. Carlini et al. (2022) approximate the IN/OUT hypotheses normal distributions. Denoting to = ((M(x)) y). the score becomes
N(ta; Piu (x), (x)))
LR(x) N(tzi Pout (x), out (z))²²
where the Miiioui(1) (z)) and (x)) are the means and standard deviations for the IN/OUT shadow model losses for (x., y). Larger values of LR(x) suggest that (x. y) is more likely in the training set and vice versa. Now, to build classifier from this score, the LiRA tests if LR(x) > for some threshold т. Note that the attacker only has a finite set of shadow models to estimate IN/OUT parameters. Therefore, the MIA scores become random variables over the true population level IN/OUT distributions.
3.2 Computing the TPR for LiRA
Using the LiRA formulation of Equation (4), the TPR for the target point (x, 8) for LiRA is defined as
Pr TPRL:RA(*) Drarge DIPL MM
(LR(x) 2 (xx v) E Dearget).
(5)
where 7 threshold that defines rejection region of the likelihood ratio test, and GM denotes the randomness in shadow set sampling and shadow model training (see Appendix A. for derivation).
We define the average-case TPR for LiRA by taking the expectation over the data distribution:
TPRIRA = ((l..))D/(PRR... (z))
(6)
3.3 Per-example MIA vulnerability
Although LiRA models to by a normal distribution, we consider more general case where the true distribution of the is of the location-scall family. That is.
Min (x: + Jin (x)
if (x, y) Dtarget
(7)
Pout (x) Jout (x)
if (x) y) & Dtarget:

---

where Z has the standard location and unit scale, and (in (2r) and Jin iin (x), Fout (z) are the locations and scales of IN/OUT distributions of tz. We assume that the target and shadow datasets have a sufficient number of examples. This allows us to also assume that (a:) oi (æ)) oot (a)) and ar (a:) = iin (az) = out (z), where (xx) is the standard deviation of iz estimated from shadow models and (()) is the true scale parameter of le- (See Appendix A.2 for the validity of these assumptions). The following result reduces the LiRA vulnerability to the location and scale parameters of tx.
Lemma 1 (Per-example LiRA vulnerabilityy. Suppose that the true distribution of tw is of location-scale family with locations pin (xr) Mout and scale oxx), and that LiRA models to by N(jin (x), )x))) and (iont (x), ( (x)²). Assume that attacker has access to the underlying distribution D. Then for a large enough number of examples per class and infinitely many shadow models, the LiRA vulnerability of a fixed larget example is
Fz F -PRLLIA((:)) FF
Min Pont (x))
if jii (2)) Mont (8)
IPRLIRA(E)
[
Fz (FPRLIRA (2))
[in(a) -out
if jiin (x)) < Pou (x))
(
where Fz is the edf of Z with the standard location and unit scale, assuming that the inverse of Fz exists.
Proof. See Appendix A.3.
Here we assume that an attacker trains shadow models with the true underlying distribution. However, in real-world settings the precise underlying distribution may not be available for an attacker. We relax this assumption in Appendix A.5 so that the attacker only needs an approximated underlying distribution for the optimal LiRA as in Lemma 1.
3.4 A simplified model of the optimal membership inference
Now we construct simplified model of membership inference that streamlines the data generation and shadow model training.
We sample vectors high-dimensional unit sphere and classify them based inner product with estimated class mean. This model is easier to analyze theoretically than real-world deep learning examples. We generate the data and form the classifiers (which
target models) as follows:
1. For each class, we first sample true class mean m. on a high dimensional unit sphere that is orthogonal all other true class means (Vi, E f1..... (1.....C).i j m; 1 m,))
2. We sample 25 vectors ae for each class. We assume that they are Gaussian distributed around the the true class mean
N(m,,, Σ) where the Σ is the in-class covariance.
3. For each "target model" we randomly choose subset of size CS from all generated vectors and compute per-class means m...
The computed mean is used to classify sample æ by computing the inner product (æe, m.,) as a metric of similarity.
The attacker has to infer which vectors have been used for training the classifier. Instead of utilising the logits (like in many image classification tasks), the attacker can the inner products of point with the cluster means. Since the inner product score follows a normal distribution, LiRA with infinitely many shadow models is the optimal attack by the Neyman Pearson lemma (Neyman and Pearson, 1933), which states that the likelihood ratio test is the most powerful test for a given FPR.
This simplified model resembles a linear (Head) classifier often used in transfer learning when adapting to a new dataset. We also focus on the linear (Head) classifier in our empirical evaluation in Section 4. In the linear classifier, we find matrix W and biases b, to optimize the rrossentropy between the labels and logits We + b, where denotes the feature space representation of the data. In the simplified model, the rows of W replaced by the cluster means and do not include the bias term in the classification.
Now, applying Lemma to the simplified model yields the following result.
5

---

Theorem 2 (Per-example LiRA power-law), Fix targer example (x. y). For the simplified model with arbitrary and infinitely many shadow models, the per-example LiRA vulnerability is given as
(og(TPRLARAA (x) ( FPRLIRA( (a:)
(z, x mn
log S - ((FPLIIAA))) + loo
+ log(1 + E(S)),
(9)
2
Vx /xv2
where my is the true mean of class y and {(S) 0(1/v5). For large S we have
log S - ((P(((PRAA))))) log
Kx, x mz)
log(TPRLIRA(E) (2) (PRLIRA(())
(10)
1x''axx 2n
Proof. See Appendix A.6.
An immediate upper bound obtained from Theorem by the Cauchy-Schwarz inequality:
x m
log(TPRLIRA (a) FPRLPRA (x)) < log _00 1 ((P(LERIAAAA + log
V x ExvV2
+ log(1+ +(()))
(11)
This implies that if | m is bounded, then the worst-case vulnerability is also bounded. Hence we can significantly reduce MIA vulnerability of all examples in this non-DP setting by simply increasing the number of examples per class.
Remark 3. By Lemma and the proof of Theorem 2, a necessary condition for the power-law is that (Win (x) - Mout (x))/((x) converges zero rate (1//5*) with a > In our simplified model, this holds with a = 1/2. However, Jin (x) Mout (x) + 0 might not always be the case for larger neural networks trained from scratch. Furthermore, even if (Jin - pout ( (x)) converges to zero, it is not clear what the convergence rate would be.
Now the following corollary extends the power-law to the average-case MIA vulnerability. We will also empirically validate this result in Section 4.
Corollary 4 (Average-case LiRA power-law). For the simplified model with arbitrary C, sufficiently large S and infinitely many shadow models, we have
1
[1xr, x - m=a) >|
log( TPRLIRA FPRLIRA) = log S "((FPRLIRA)) + log (B(
Va Va V 2n
(12)
Proof. See Appendix A.7.
4 Empirical evaluation of MIA vulnerability and dataset properties
In this section, we investigate how different properties of datasets (shots S and number of classes C) affect the MIA vulnerability. Based on our observations, we propose method to predict the vulnerability to MIA using these properties.
4.1 Experimental setup
We focus on a image classification setting where we fine-tune pre-trained models on sensitive downstream datasets and assess the MIA vulnerability using LiRA and RMIA with M = 256 shadow/reference models. We base our experiments on a subset of the few-shot benchmark VTAB (Zhai al., 2019) that achieves test classification accuracy 80% (see Table A2).
We report results for fine-tuning a last layer classifier (Head) trained on top of a Vision Transformer ViT-Base-16 (ViT-B; Dosovitskiy et al.. 2021). pre-trained on ImageNet-2 (Russakovsky et al., 2015). The results for using 50 (R-50; Kolesnikov et al., 2020) as a backbone can found in Appendix D.1. We optimize the hyperparameters (batch size, learning rate and number of epochs) using the Optuna library (Akiba et al., 2019) with the Tree-structured Parzen Estimator (TPE; Bergstra et al., 201 I) sampler with 20 iterations (more details in Appendix C.2). We provide the the code for reproducing the experiments in an open repository
"https://gitae wttp://aaaee////pppeeeceooo dataset- propeettrrorrrreieeeeeeeeeeeeeee wwwpertees--I-vaeneeeeaeee mi-munorrabiltyyyde....

---

4.2 Experimental results
Using the setting described above, we study how the number of classes and the number of shots affect the vulnerability (TPR at FPR as described in Section 2) using LiRA. We make the follow- ing observations:
(TPR-FPR)
atFPR=0.00
0.01
0.001
larger number of S (shots) decrease the vul- nerability in a power law relation as demon- strated in Figure 1. We provide tabular data and experiments using ResNet-50 in the Ap- pendix (Figure A.I and Tables A3 and A4).
2
16
32
(classes)
Patch Camelyon (C=2)
Pets (C=37)
CIFARIO (C=10)
- Resics45 (C=45)
EuroSAT (C=10)
CIFAR (C=100)
Contrary, a larger number of C' (classes) in-
creases the vulnerability as demonstrated in
Small effect of number of classes C (classes) on MIA vulnerability when attacking fine-tuned ViT-B Head. The solid line is median and the error bars the min/max bounds for the Clopper-Pearson CIs over 12 seeds (S 32).
Figure 2:
Figure 2 with tabular data and experiments using ResNet-50 the Appendix (Figure A.2 and Tables A5 and A6). However, the trend w.r.. not as clear as with
RMIA In Figure 3 compare the vulnerability of the models to LiRA and RMIA as a function
of the number of S (shots) FPR 0.1. We observe the power-law for both attacks, but the RMIA is more unstable than LiRA (especially for lower FPR). More results for RMIA are in Figures A.5 A.7 the Appendix.
(TPR-FPR) (RR-FPR) @FPR=
0.1
0.01
0.001
256
1024
4096
16384
65536
64
256
1024
4096
16384
65536
S (shots)
S' (shots)
Patch Camelyon (C=2)
ET (C=10)
Resics45 (C=45)
LiRA
(C=101
- Pets (C=37)
- CIFAR100 (C=100)
--- RMIA
Figure 3: LiRA and RMIA vulnerability ((TPR FPR) at FPR 0.1) function of shots (S) when attacking ViT-B Head fine-tuned on different datasets. For better visibility, we split the datasets into two panels. We observe the power-law for both attacks, but the RMIA is more unstable than LiRA.
The lines display the median over six seeds.
4.3 Model to predict dataset vulnerability
The trends seen in Figure 1 suggest the same power law relationship that we derived for the sim- plified model of membership inference Section 3. We fit linear regression model to predict log(TPR - FPR) for each FPR 10-* k = I...., 5 separately using the log C' and log S as covari- ates with statsmodels (Seabold and Perktold, 2010). The general form of the model can be found in Equation (13), where 3s, 3C and 8o are the learnable regression parameters.
logio (TPR- FPR) 33 logio Be (ogio(() + ,00
(13)
In Appendix D.2, we propose a variation of the regression model that predicts log login(TRR) of log (TPR- FPR) but this alternative model performs worse TPR < FPR in the tail when very large.
(TPR) instead empirical data and predicts
We utilise MIA results of VIT-B (Head) (see Table A3) as the training data. Based the R² (coefficient of determination) score (R² = 0.930 for the model trained on FPR 0.001 data), our model fits the data extremely well. We provide further evidence for other FPR in Figure A.3 and
7

---

As
_c
30
coefficient value
-0.4
0.4
0.50
0.3
0.25
-0.5
0.2
0.00
0.66
0.1
00.5
-0.7
0.0
-0.50
0.001
0.01
0.1
0.001
0.01
0.1
0.001
0.01
0.1
FPR
FPR
FPR
Figure 4: Coefficient values for different FPR when fitting regression model based on Equation (13) fitted on data from ViT-B (Head) with LiRA (Table A3). The error bars display the 95% confidence intervals based Student's t-distribution. Theoretical values in the simplified model is shown by pink dotted lines (3g = 0.5 and 8C = 0).
FiLM
From Seratch
Observed TPR
(TPR- FPR)
@FPR=
0.75
0.01
0.50
X
0.25
0.00
16384 65536
0.00
0.25
0.50
0.75 1.00 0.00 0.50 0.75 1.00
8 (shots)
Predicted TPR
Predicted TPR.
Fitted Line (C=2)
VIT-B (Train)
Camelyon
(C-37)
FPR-0.1
Fined Line (C=10)
R-50 (Test)
CIFAR10
Resies45 (C=45)
FPR-00l
Finted Line (C=100))
EumSAT (C-10)
CC-1R100
PPR-00001
(a) The dots show the median TPR for the train set (ViT-B; Table A3) and the test set (R-50: Table A4) over six seeds (datasets: Patch Camelyon, AT and CIFARI00). The linear model is robust to chang- ing the feature extractor from ViT-B to R-50.
(b) Regression model robust to changing the fine- tuning method from Head to FiLM, but from scratch training
vulnerable than predicted. (i) left: fine-tuner with FiLM (sce Table A7) (ii) right: trained from scratch. Data is from Carlini et al. (2022).
to
Figure 5: Performance of the regression model based Equation (13) fitted data from Table A3.
Table A8 in the Appendix. Figure 4 shows the parameters of the prediction model fitted to the training data. For larger FPR, the coefficient 3s around -0.5, as our theoretical analysis predicts.
Prediction quality other MIA target models We analyze how the regression model trained on the ViT-B (Head) data generalizes to other target models. The main points are: R-50 (Head):: Figure 5a shows that the regression model is robust to a change of the feature extractor, is able predict the TPR for R-50 (Head) (test R 0.790).
R-50 (FiLM): Figure 5b shows that the prediction quality is good for R-50 (FiLM) models. These models are fine-tuned with parameter-effiiient FiLM (Perez et al., 2018 8) layers (See Appendix C.1). Tobaben et al. (2023) demonstrated that FiLM layers are a competitive alternative to training all parameters. We supplement the MIA results of Tobaben et al. (2023) with own FiLM training runs. Refer to Table A7 in the Appendix.
Carlini al. (2022) provide limited results from-seratch-traaiingg To the best of our knowledge these are the only published LiRA results on image classification models. Figure 5b displays that our prediction model underestimates the vulnerability of the from-scratch trained target models. We have identified two potential explanations for this: (i) In from- scratch-training all weights of the model need to be trained from the sensitive data and thus potentially
could be more vulnerable than fine-tuning. attack in Carlini al. (2022) uses data augmentations using this optimization. Additionally, noted in Remark 3, our theoretical analysis is based on a simple model and the power-law might not oceur all for larger neural networks trained from scratch.
(ii) The strongest improve the performance. We are not
8

---

Table 1: The minimum S with C=2 predicted by the models in Sections 4.3 and 4.4 to empiri- cally match the DP bounds (6=10-1 calculated through (Kairouz et al., 2015) in terms of TPR at FPR.
TPR-FPR at FPR=0.1
Max
0.999 Quantile
0.99 Quantile
0.01
0.95 Quantilo
Averago Case
min average case
min worst-case
0.001
PPR=0.1
FPR=0.01
FPR=0.001
FPR=0.1
16384
32768
65536
0.25
54400
69 000
320
5.5 x 10
S (shots)
0.50
1100
160000
000
2.6 x 10
0.75
360
5900
38 000
3.5 x 107
Figure 6: Individual vulnerability for ViT-B
1.00
160
700
19
7.0 x 10
(Head) when fine-tuning on Patch Camelyon. We observe a similar power-law relationship for indi- viduals when looking the quantiles but the decreases slower. The Average Case shows the average LiRA vulnerability over the individuals also illustrated in the right panel of Figure 3.
(TPR F-RFPPP)) (PR))FFP=0.000
Average-Cas Case Vulnerability
(TPR (TPR-TPPM TPR @FPR-00.1
DP Round = 1)
0.1
DP Bound (c = 0.25)
0.01
0.1
Worst-Case Valocrability
0.001
Bound 1)
DP Bound (e 0.25)
0.01
10"
10a
10
105
10"
10
10a
10a
10a
10a
10"
10a0
(shots)
8 (shots)
Figure 7: Illustration of the extrapolation for the average-case FPR 10 (left) and worst-case
FPR = 10 (right).
4.4 Individual MIA vulnerability
the experiment with 257 models with each of them once acting as a target and as shadow model otherwise, compute the TPR at FPR for every sample separately. In Figure for Patch Camelyon and compare with corresponding average case vulnerability from Figure 3. The plot shows the maximal vulnerability all samples and different quantiles over six seeds. The solid line is the median and the errorbars display the min and max over seeds. The quantiles are more robust extreme outliers and show decreasing trends already at much lower S than the maximum vulnerability.
In order to assess the per-sample MIA vulnerability,
display the individual vulnerability a function of S (shots)
When fitting the model in Equation (13) observe that the coefficients 33 that model the relationship between vulnerability and examples per class for the quantiles are -0.6633, -0. -0.5688 and -0.4796 which close the theoretical value of -0.5 derived in the theoretical analysis in Section 3.4. However, the maximum vulnerability decreases with lower slope of -0.2695 which considerably smaller. With larger S the slope of the max vulnerability increases, e.g., with S > 32768 the coefficient is -0.3478 suggesting that higher $ is required for the most vulnerable points.
4.5 Comparison between empirical models and universal DP bounds
While the practical evaluation through MIAs is statistical and does not provide universal formal guarantees like DP, the power-law can aid understanding about how the practical vulnerability to MIA behaves in
realistic threat model when the examples per class increase.
Using the translation between the (TPR, FPR) to (e,8)-DP guarantees proposed by Kairouz al. (2015), compute the minimum S predicted by empirical models such that the predicted TPR

---

matches the theoretical bound at target DP level. We illustrate this in Figure 7. (See Appendix E for a more detailed description.) Table shows the resulting lower bounds of S for various DP levels and values of FPR. While our empirical observations do not provide any formal guarantees, the comparison serves as an illustration to better understand the different requirements of the average and individual vulnerability. We can see that both for the average and the worst case, obtaining low FPR would require a large amount of samples per class in order to match the TPR for meaningfully strong DP bounds.
5 Discussion
Trying to bridge empirical MIA vulnerability and formal DP guarantees is not an easy task because of different threat models and different nature of bounds (statistical vs. universal). While we were able to show both theoretically (Section 3) and empirically (Section 4) that having more examples per class provides protection against MIA in fine-tuned neural networks, the numbers required for significant protection (Section 4.5) limit the practical utility of this observation.
Using a level of formal DP bounds that provide meaningful protection as a yardstick, at least tens of thousands of examples per class are needed for every class even at FPR 0.001. The e values used here are formal upper bounds and must not be compared to actual DP deep learning with comparable privacy budgets, as the latter would be far less vulnerable. At this number of samples per class and good pre-trained model, the impact of DP training is often negligible. This stresses the importance of formal guarantees like DP for privacy protection.
Our formal analysis focuses on a setting that can be linked to fine-tuning. As shown in Figure 5b, from-scratch training likely has higher vulnerability. Formally analyzing more models is an interesting area for future research.
Since our MIA evaluation assumes that only target point is known to the adversary and the rest of the dataset random, this stochasticity likely introduces some protection. Hence the power-law may be invalidated under stronger MIA settings where the adversary has access to other points in the private dataset (Bai et al., 2025).
Our experiments show that there is difference between classes in terms of vulnerability and an interesting direction for future work is to understand the properties of classes that influence this vulnerability, e.g., variability within or between classes or their separability. Another direction for future work understanding the impact of pre-training data on the vulnerability of fine-tuning data.
Broader Impact Our work systematically studies factors influencing privacy risk of trained MI. models. This has significant positive impact on users training MI. models personal data by allowing them to understand and limit the risks.
Limitations We mostly consider LiRA in paper which is optimal for our simplified model of membership inference (Section 3.4), but for the transfer-learning experiments (Section 4) there might be stronger attacks in the future. Furthermore, our simplified model assumes well vell-behaved underlying distributions, meaning that the data is normally distributed around the class centers. We leave the analysis of other data distributions (c.g., heavy-tailed distributions) to future work. Formal bounds on MIA vulnerability would require something like DP. In addition, both our theoretical and empirical analysis focus deep transfer learning using fine-tuning. Models trained from scratch are likely to be more vulnerable. Concurrently, a related paper (Hayes et al., 2025) has studied the relation between LiRA vulnerability and dataset size when attacking LLMs during the pre-training phase. Extending this analysis is promising but computationally very expensive.
We provide the code in an open repository All used pre-trained models and datasets are publicly available.
Acknowledgments
This work was supported by the Research Council of Finland (Flagship programme: Finnish Center for Artificial Intelligence, FCAI, Grant 356499 and Grant 359111), the Strategic Research Council
10

---

at the Research Council of Finland (Grant 358247), the European Union (Project 101070617) as well as JSPS KAKENHI Grant Number 25KJ1515. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the Huropean Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them. he authors wish to acknowledge CSC IT Center for Science, Finland, for computational resources. We thank Mikko A. Heikkilä and Ossi Räisä for helpful comments and suggestions and John F. Bronskill for helpful discussions regarding few-shot learning.
References
M. Aerni, J. Zhang, and F. Tramèr. Evaluations of machine learning privacy defenses misleading. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. CCS 2024, pages 1271-122 ACM, 2024. 4
T. Akiba, Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, pages 2623-2631. ACM, 2019. 6, 30
A. Azize and D. Basu. Some Targets Are Harder to Identify than Others: Quantifying the Target- dependent Membership Leakage. In International Conference on Artificial Intelligence and Statistics, pages 397-405, PMLR, 2025.
E. Bagdasaryan, O. Poursaeed, and V. Shmatiikv.. Differential privacy has disparate impact model accuracy. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. 2
Y. Bai, Pradhan, M. Tobaben, and A. Honkela. Empirical Comparison of Membership Inference Attacks in Deep Transfer Learning. Transactions on Machine Learning Research, 2025. 10
J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems 24; 25th Annual Conference on Neural Information Processing Systems 2011, pages 2546-2554, 2011 30
N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramèr. Membership inference attacks from first principles. In 43rd IEEE Symposium Security and Privacy, SP 2022, pages 1897-1914. IEEE, 2022. 1, 2, 3, 4, 8, 15, 17,37
H. Chang and R. Shokri. On the privacy risks of algorithmic fairness. In 202/ IEEE European Symposium on Security and Privacy (EuroS&P), pages 292-303. IEEE, 2021.
D. Chen, N. Yu, Y. Zhang, and M. Fritz. GAN-leaks: A taxonomy of membership inference attacks against generative models. In CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, pages 343-362, ACM, 2020. 2
G. Cheng, J. Han, and X. Lu. Remote sensing image scene classification; Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865-1888 1883, 2017. 31, 33, 34, 36, 37
C. Clopper and F. S. Pearson. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26(4):404 1934. 3
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021. OpenReview.net, 2021. 6, 30
C. Dwork, F. McSherry, K. Nissim, and A. D. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography, Third Theory of Cryptography Conference, TCC 2006, volume 3876 of Lecture Notes in Computer Science, pages 265-284. Springer, 2006.
V. Feldman and C. Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems 33: Annual Conference Neural Information Processing Systems 2020, NeurIPS 2020, 2020. 2
11

---

F. Guépin, N. Krčo, M. Meeus, and Y.-A. de Montjoye. Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models. ArXiv preprint. abs/2405. 15423, 2024. 2
J. Hayes, I. Shumailov, C. A. Choquette-Choo M. Jagielski, G. Kaissis, K. Lee, M. Nasr, S. Ghalebikesabi, N. Mireshghallah, M. S. M. S. Annamalai, Shilov, M. Meeus, Y. de Montjoye, F. Boenisch, A. Dziedzic, and A. F. Cooper. Exploring the limits of strong membership inference attacks on large language models. In Advances in Neurul Information Processing Systems 39: Annual Conference on Neural Information Processing Systems 2025, NeurlPS 2025, 2025. 10
P. Helber, B. Bischke, A. Dengel, and D. Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217
2019. 31, 33, 34, 36, 37
H. Hu, Z. Salcic, L. Sun, G. Dobbie, P. S. Yu, and X. Zhang. Membership inference attacks on machine learning: A survey, ACM Computing Surveys (CSUR), 54(11s): 2022. 3
Kairouz. S. Oh, and P. Viswanath. The composition theorem for differential privacy. In Proceedings of the 32nd International Conference Machine Learning, ICML 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 1376-1385. JMLR.org. 2015. 9, 45
D. P. Kingma and J. Ba. Adam: method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, 2015. 30
A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung. S. Gelly, and N. Houlsby. Big transfer (BiT): General visual representation learning. In Computer Vision ECCV 2020 - 16th European Conference, volume 12350 of Lecture Notes in Computer Science, pages 491-507. Springer, 2020.
6, 30
A. Krizhevsky. Learning multiple layers of features from tiny images. Master's 's thesis, University of Toronto, 2009. 31, 33, 34, 36, 37
B. Kulynych, M. Yaghini, G. Cherubin, M. Veale, and C. Troncoso. Disparate vulnerability to membership inference attacks. Proceedings on Privacy Enhancing Technologies, 2022:460-480, 2022. 2
T. Leemann, B. Prenkaj, and G. Kasneci. Is My Data Safe? Predicting Instance-Level Membership Inference Success for White-box and Black-box Attacks. In ICMI, 2024 Next Generation of Al Safety Workshop. 2024. 4
M. Meeus, F. Guepin, A.-M. Crequ, and Y.-.. de Montjoye. Achilles' heels: Vulnerable record identification in synthetic data publishing, pages 380-399. Lecture notes in computer science. Springer Nature Switzerland, 2024.
G. D. Németh, M. A. Lozano, N. Quadrianto, and N. Oliver. Privacy and accuracy implications of model complexity and integration in heterogeneous federated learning. IEEE Access, 13: 40258 40274, 2025. 2
Neyman and S. Pearson. IX. On the problem of the most efficient tests of statistical hypotheses. Philosophical transactions of the Royal Society of London, 231:289-333 -338893337111 1933. 2, 3, 5
O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V. Jawahar. Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3498-3505. IEEE Computer Society, 2012. 31, 33, 34, 36, 37
E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. C. Courville. FILM: Visual reasoning with general conditioning layer. Proceedings of the Thirty-Secone AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), pages 3942 3951. AAAI Press, 2018. 8, 30
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and I.. Fei-Fei. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis., 1 15(3):211-255)) 2015. 6, 30
12

---

S. Seabold and J. Perktold. statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference, 2010. 7, 38, 39
R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy, SP 2017, pages 3-18. IEEE Computer Society, 2017. 2,
A. Shysheya, J. Bronskill, M. Patacchiola, S. Nowozin, and R. E. Turner FiT: parameter efficient few-shot transfer learning for personalized and federated image classification. In The Eleventh International Conference on Learning Representations, ICLR 2023. OpenReview.net, 2023. 30
L. Song and P. Mittal, Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2615-2632, 2021. 2
V. M. Suriyakumar, N. Papernot, A. Goldenberg, and M. Ghassemi. Chasing your long tails: Differentially private prediction in health care settings. In Proceedings the 2021 ACM Conference on Fairness, Accountability, and Transparency. ACM, 2021.
M. Tobaben, A. Shysheya, Bronskill, A. Paverd, S. Tople, S. Béguelin, R. E. Turner, and A. Honkela. On the efficacy of differentially private few-shot image classification. Transactions on Machine Learning Research, 2023. 2, 8, 37
S. M. Tonni, F. Farokhi, D. Vatsalan, and D. Kaafar. Data and model dependencies of membership inference attack. ArXiv preprint, abs/2002.06856, 2020. 2
B. Veeling, Linmans. Winkens, T. Cohen, and M. Welling. Rotation equivariant CNNs for digital pathology. In International Conference on Medical image computing and computer-assisted intervention, pages 210-218. Springer, 2018. 31, 33, 34, 36, 37
Yeom, I. Giacomelli, M. Fredrikson, and S. Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pages 268-282. IEEE, 2018.
D. Yu, G. Kamath, J. Kulkarni, T.-Y. Liu, J. Yin, and H. Zhang. Individual privacy accounting for differentially private stochastic gradient descent. Transactions Machine Learning Research, 2023. 2
S. Zarifzadeh, P. Liu, and R. Shokri. Low-cost high-power membership inference attacks. In Forty- first International Conference on Machine Learning, ICML 2024. OpenReview.net, 2024. 3, 21, 22, 41, 42
X. Zhai, J. Puigcerver, A. Kolesnikov, P. Ruyssen, C. Riquelme, M. Lucic, J. Djolonga, A. S. Pinto, M. Neumann, A. Dosovitskiy, et large-scale study of representation learning with the visual task adaptation benchmark. ArXiv preprint, 1994197 2019. 6,
13

---

Supplementary Material
Contents
A Details of Section 3
15
A.I
Formulating LiRA
15
A.2
On the assumption of shared scale
15
A.3
Proof of Lemma
15
A.4
Offline LiRA
17
A.5
Relaxing the assumption of Lemma
18
A.6 Proof of Theorem 2
19
A.7 Proof of Corollary 4
20
Theoretical analysis of RMIA
21
B.1
Formulating RMIA
21
B.2
Per-example RMIA
22
B.3
RMIA power-law
24
C Training details
30
C.1 Parameterization
30
C.2 Hyperparameter tuning
30
C33 Datasets
30
C.4 Compute resources
31
D Additional results
32
D.I Additional results for Section 4
32
D.2
Simpler variant of the prediction model
39
D.3 Empirical results for RMIA
41
D.4 Tabular results for Section 4.4
44
Details on Section 4.5
45
14

---

A Details of Section
A.1 Formulating LiRA
Let M be our target model and ((M(2)), 3) be the loss of the model on a target example (ar, The goal of MIA determine whether (ar, y) E Dearget- This can be formulated as a hypothesis test:
Ho (xx y) Diarget
(A1)
H1 (x, y) Drarget.
(A2)
Following (Carlini et al., 2022), we formulate the Likelihood Ratio Attack (LiRA)). LiRA exploits the difference of losses on the target model under Ho and H₁. To model the IN/OUT loss distributions with few shadow models, LiRA employs a parametric modelling. Particularly, LiRA models to by normal distribution. That is, the hypothesis test formulated above can be rewritten as
Hg In (jontt oou) Hi tw (i In).
(A3)
(A4)
The likelihood ratio is now
(tws Pin o LR(x) N(tai Pout, oout
(A5)
LiRA rejects Hj if and only if
LR(x) 2T,
(A6)
concluding that Hi is true, i.c., identifying the membership of (ax y). Thus, the true positive rate of this hypothesis test given as
TPRIIRA(2)) Dearget DiPI,
Pr
(LR(x)) 2 (zr,)) E Diarget):
(A7)
where M denotes the randomness in the shadow set sampling and shadow model training.
A.2 On the assumption of shared scale
In Section 3 assumed that for LiRA din oout and oin Gout. Using the simplified model formulated in Section 3.4, we show that for large enough number S of examples per class these assumptions are reasonable.
Let Drarget = ((X5,,.),,... (I),S,))fGE Then the IN/OUT LiRA scores are given as
S Fyys + x ))) (x. gys) + (a., 5((0 a xyys))
8
(In) y
(x. S
(A8)
8
(out) (æc
Σ
x
(A9)
y
S
2m1
Since for the simplified model scores follow Gaussian distributions, ain = Gjn and Tout = Tout- It follows that
S (1
i 2 = Var(s
Var( anyx)) - Var((a, ag,i)) =
x² Ex (A10)
g
52
$
1 Var(sfout) = Var( ay.s))
1
out 2
5"
Ex.
(A11)
Thus, the differences din Gout and ôin Gout are negligible for large
A.3 Proof of Lemma
Lemma (Per-example LiRA vulnerability). Suppose thut the true distribution of lo is of location-scall family with locations Hin (ar) ; Pout (x) and scale (x)), and that LiRA models la
by N (x), )xx)2) and N (pout (x), o(x) (2) Assume that attacker has access to the underlying
15

---

distribution D. Then for large enough number of examples per class and infinitely many shadow models, the LiRA vulnerability of a fixed target example is
1 Fz P (PRLIRAA
Min (1) Poot (x)
if piin (2)) Dour(2)
TPRLIA(
(8)
Fz F (FPRLIRA(()) (iu(x) - Pous (a:
if (ax < Mout (o))(
where Fz is the cdf of Z with the standard location and unit scale, assuming that the inverse of Fz exists.
Proof.
We abuse notations by denoting Uín to refer to Min (x) and similarly for other statistics. We
have
(lai Pins o²)
log
2 log
(A12)
Ntaii Plout )2)
t/
Pin
f/ Pout
2 log
(A13)
(2lmiiin
fo + (ont) 2 log
(A14)
282
(Nin Pouu)
Puin Pout) 2 log T
(A15)
262
6² log + frout pim Pout Amm Pout-
tw
Big Mout
(A16)
log + Bio thout
tw Riu Mout
Then if An > Pout, in the limit of infinitely many shadow models
0² log
pinn + Pout
FPRLIRA (x) = Pr
Mout + oz
(A17)
2
jlin
Poout
2
= Pr 2
2 2
8² log T
+ Aiin + Pout
Mout.
(A18)
0 (phnn
Pout)
20
à² log
Am + Aout
Hout
= 1 Fz
+
(A19)
(jiinn
Moutt
20
and if Ain < Pour. similarly,
6² logt
Pin Mout
FPRISRA = Pr Z
Mout + o7 5
(A20)
Pin Poout
2
22 log
Pli + Pout
Hout
Fz
+
(A21)
(pppin
Poout)
20
Thus
2 log r (fpn Aoout
Piin + Aout
Hout
F (1 (PRLIRA(E)) Fz (FPRLIRA
if Piin > Pout if Pin Pout.
+
(A22)
20
It follows that if fim Poout
8² log T
+ Pin + Poout
TPRLIRA(E))
(2)=
=
Pr
Win + oZ >
(A23)
Pii
Pont
2
8² log +
Aii + Pout
= Pr
2
(A24)
2
(jiin
(uout)
20
=1 Fz
( FPRLIA((2))
14n Mout
(A25)
16

---

If iAin < Pout, then
8² gogT
+ ain + Aout
TPRLIRA = Pr
Min + 07 S
(A26)
2
pin
Aout
2
8² log T
Min + Pout
Hin
Pr
Z s
+
(A27)
z
o (itn Pout)
20
Fz
(F (FPRLINA
Fz
Min
pout
(A28)
A.4 Offline LiRA
Carlini al. (2022) proposes offline LiRA that only trains OUT shadow models for computational efficiency. Instead of likelihood ratio, the score for offline LiRA is given as
A(x) ProMout (x) oont (a) s (m))
(A29)
2
where a location-scale distribution with standard location and unit scale. Then TPR of offline
LiRA becomes
TPROOTIRA((E)
Pr Desggne DDDD),
(A(x) 2 T (x), v) E Drarget)
(A30)
where 7 is a tunable parameter. FPR is also given similarly. Below we show a result for offline LiRA similar Lemma 1, but under slightly more strict assumption that attacker accurately estimates the IN/OUT score distributions. In the following, as for online LiRA, we assume Oin Cout-
Lemma A1 (Per-example offline LiRA vulnerability). Suppose that to follows the normal distribution with means Pmm (x), Mour (x) and standard deviation (x)) Assume that an attacker has access the underlying distribution D. Then for large enough number of examples per class and infinitely many shadow models, the offline LiRA vulnerability of a fixed target example
TPRoHLIRA (x) (0001 ( + Min Min(x) Mout (x)
(A31)
o(x)
where de is the standard normal cdf.
Proof. For infinitely many shadow models, we have
FFR Drarzet Dio!
Pr
(A(x) TT( (x. v) $ Dtarget),
(A32)
and the score is now
A(x) Pr(Hont (2:) + [(=)) < tz))
(A33)
where 1) (0,, When (x. v) & Drangets to Hout (x) + oxx) Thus we have
FPRaBLiRA
= Pr (A(xx) 27)
(A34)
2
Pr
Pr( Pr(Mout (2:) [(x)) I (out(x:) x(x)2) 27)
(A35)
2
Pr
Pr(n S 52)2 2)27
(A36)
Pr(((Z) 2n) 2
(A37)
= -17
(A38)
17

---

On the other hand, when (z, E D Pargett to = Min(x) + oz)) Thus we obtain
Pr
Pr Pr((ouu (2:) + a(())) S Min (ar) + (z)) 27)
(A39)
Mu(()) Mout (x)
Pr
Pr
"
11
(2x)
2
2 FPROOLIRA
(A40)
Min
Hout (x)
Pr
Pr
2 S <FPROALIRA
(A41)
"
(
Min (in( - Mout (x)
Pr
(x) - (
(A42)
x-1 (FPROALIRA) + (in( Mout (ar)
(A43)
o(x)
Consequently, the power-law also holds for offline LiRA in the simplified model:
Corollary A2 (Per-example offline LiRA power-law). Fix a target example (xc y)- For the simplified model with arbitrary C and infinitely many shadow models, the per-example offline LiRA vudnerability is given as
log ORR (x) FPROBILRA((T)
(a,, ma)l
log S Φ -etass--aPPPPLLLLLAAaA (FPR + log
log(1 + E(S)) (A44)
Vx Tx Ex
where m. is the true mean of class y and {(S) O(1/VS). For large S we have
(x, x m
Iog (x) (Z)) log S 4 -(P(LLPRA) + log
v V Ex 2 2T
(A45)
A.5 Relaxing the assumption Lemma
In Lemma we assume that an attacker has access to the true underlying distribution. However, in real-world settings the precise underlying distribution may not be available for an attacker. In the following, noting that the Equation (8) mainly relies on the true location parameters Jmm (x), Hout (æ:) and scale parameter a (x:), we relax this assumption of distribution availability so that the attacker only needs an approximated underlying distribution for the optimal LiRA that achieves the performance in Lemma
First, notice that if we completely drop this assumption so that an attacker trains shadow models with an arbitrary underlying distribution, then we may not be able to choose desired FPRLIRA (æ). From Equation (A22) we have
22 log
Íiin + Pout
Mout
F (1 (PRLIRA(Z))
if Pmn Pout if jm Pout
(A46)
a app iPout)
20
Fr (FPRLIRA(EE)
Bin Lin-fuuu flaun
(F P--PIPRRAR (x))
Am fhous
+ Hout
Pin Pout
log T
2
(A47)
Rooss
OF (FPRLIRA (x))
Ahons
Hout
if Pin Pout.
Since it does not make sense to choose rejection region of the likelihood ratio test such that we assume T 2 Then we have
Piu Sin-fhau (oFZ FPRLIRA(E)) Piuntt Nout
2
if Pin Aout
2
(A48)
pin - foout Therefore, sufficient condition about attacker's knowledge on the underlying distribution for Lemma hold
(oFz (FPRLIAA())) Pii t + Mout
2
if Pin Pout-
if Pun Pout F 2 (1 (()))R(())) An + thos + Mout 20
(A49)
2
(PPREIRA (x)) Bin t Mout
if Pin < Pout.
18

---

Rearranging the terms yields
Piu Post
Mout S 22222(1 FIRL)RA
if flin > Pout
(A50)
Hout 2 O (FPRI,RA(
if Pin < Poutt
For example, if the estimated mean Poutt (in) ) is too large compared to the true parameter Mout, the left hand side for the case Jimm Aout becomes very large, thereby foreing to choose sufficiently small FPR (z:) Similarly, if Poot fin) ) is much smaller than Mout then the range possible values of FPRLRRA (a) will be limited. We summarise this discussion in the following:
Lemma A3 (Lemma I with relaxed assumptions), Suppose that the true IN/OUT distributions of ta
are of location-sen scale family with locations Piu (x); Poout (2) and a shared scale (xx) such that the distributions have finite first and second moments. Assume that LiRA models to by N (pm((x) 5 )x)²)
and N (Pout (a).. (xx) ) ), and that in the limit of infinitely many shadow models, estimated parameters
Aiin (z), Pout (z) and (x)) satisfy the following:
Pin font Pout S GoF - (2))
if Pin > Pout
(A51)
Pio tons Hout 2 oFz (FPRLIR(ZE))
Pin Pout-
Then in the limit of infinitely many shadow models, the LiRA vulnerability of a fixed target example (x, y) is
- Fz Fz FPRLIRA(Z))
Pin (ar) Poot (2)) ((m)
if Pin (an(())) > Aoou (z)
TPRLIRA (x)
Fz Fr (FPRIIRA(*))
(in(()) Pout
if (in(x)) < iooue (z);
((x)
(A52)
where Fz is the cdf of with the standard location and unit scale, assuming that the inverse of F2 exists.
A.6 Proof of Theorem 2
Theorem (Per-example LiRA power-law). Fix target example (x, y). For the simplified model with arbitrary C and infinitely many shadow models, the per-example LiRA vulnerability is given as
log(TPRLIRA (x) FPRLIRA (x))
(x, 1e,x-maal log(1 E(S),, (9)
log S - (FFF + log
Vx ExvV2m
where my is the true mean of class y and {(S) O(L/V8). For large S we have
(x, x m log (())) (PR)IRA(E)) ~ log S - (FPR log
(10)
2
Vn Ean 2m
Proof. Let Dearget= {(I,,1,)),.... (2.,.:;)){ Then the LiRA score of the target (æ, y) is
8
(In)
(xr.
1
Σ Fyy. + ))) )
(1,,
Σ
Fyys) (x. / (xx ayyss))
(A53)
5
1=1
s
(out) (x: S
(A54)
respectively, when (x, y) Drarget and when (xx y) & D Diarget. Thus we obtain
Pin pout E(s(in) s(ont)]
1
(2 mzx)
(A55)
1 Var (ax, (ar zgyii)
1 Ex..
d² Var(s
(A56)
5
Noting that the LiRA score follows normal distribution, by Lemma we have
TPR) = + " (x,, x mmx)
(A57)
VSVx" Ex
19

---

where @ is the edf of the standard normal distribution. This completes the first half of the theorem.
Now let (b()) denote the pdf of the standard normal distribution, and let
Kx,, x mz
(A58)
VSVa= Em
Using Taylor expansion of ( POOD~'(FPPLRII (FPRLIAA (x)) + +r) around r = 0, we have
(k)) ( ( (FPRLIRA (x))
TPRLIRA (x)
(A59)
h!
T(k) ($ ((FRRIIA (x)) p.t.
EPRLIRA(E)) +
(A60)
k!
66 ① (FPRLIRA (x)
(FRLIRA +
(A61)
1) 1 1 - (FPRLan((E))) (FFRLLAAA)
(FF
(F)
k!
k=1
(A62)
1) Hey (Φ (0- (FPRLIRA (x)))
= =PRLIRA + ((PPRLIAA (x))
(A (1+1)!
k=
(A63)
where He denotes Hermite polynomials. It follows that log(TPRI,RA (a)) FPRTRRA (a))
(A64)
(--))Ho(QO0 (FFRLIRA ,
=logr- + +log o + ((FPLLIRA (x)) + log
(A65)
(k: + 1)!
1
- - ( + log 1<x, mz)
1
log
(A66)
2
2
V
H (+- ( (FPRLIAA
(1x2, x maa)
+ log
(A.67)
(ke 1)!
VSV D''
k=
② (Z)RLIAAZ)) + +log 11x, x mx))
log
(A66)
2
2
(-1)" (
(aAee((0-FFPPAAAA (FPRLERA (x)))
(xx, x mix) 10
+ log
1
(A69)
(k+11)
VSv " Ex
k=1
( x ma))
log S
2 Φ
(FFRLIRA (x))) + log
+ logg(1 + ((S)),
(A70)
2
Vx T'T'
where we have E(S) ((1/ For large enough S, ignoring the residual term, approximate
log S 3- (FPRIRRAAZ)) +log (xx, x mz
log(TPRIARA()) FPRLIRA (x)) x 2
(A71)
V
A.7 Proof of Corollary 4
Corollary 4 (Average-case LiRA power-law). For the simplified model with arbitrary C, sufficiently large S and infinitely many shadow models, we have
log TPRLIRA (PRLIRA) x log Φ - "(FPRLIRA) + log (E
1
[ x m
Vol Vxe / 2m
(12)
20

---

Proof. By theorem 2 and the law of unconscious statistician, we have for large S
TPRLIRA FPRLIRA
(TPRIRAA) (PRLIRA(::)dd.
(A72)
40
(
e
(EPELIRA)
(x, a mzx
dx
(A73)
2
VSv n En
(a, x mm
1000
(FPRIAAA)
Tx'EE p(x)
de
(A74)
2
VS
1
(FPRLARA) E (2
(0 x mmn
(A75)
VS
V2TV Ex
where p(æ) is the density of DD (x, v), and 9 is the data domain. Note that here we fixed
FPREIRA (x) FPRLIRA for all æ. Then we obtain
1 - (FFRLLAi)))) log (BLaw)D
(0r x mmx
log( TPRLiRA FPRLINA) =- log S x
V2TVV x Ex
(A76)
B Theoretical analysis of RMIA
Similar to LiRA, RMIA is based on shadow model training and computing the attack statistics based on a likelihood ratio. The main difference LiRA that RMIA does compute the likelihood ratio based on aggregated IN/OUT statistics, but instead compares the target data point against random samples (z, yz) from the target data distribution. After computing the likelihood ratios over multiple (z, yx) values, the MIA score is estimated as proportion of the ratios exceeding preset bound. This approach makes RMIA more effective attack when the number of shadow models is low.
B.1 Formulating RMIA
In the following, let us denote a target point by (x, Yn) with y/ being the label of x. Let us restate how RMIA (Zarifzadeh al., 2024) builds the MIA score. RMIA augments the likelihood-ratie with a sample from the target data distribution to calibrate how likely you would obtain the target model if (ar, you is replaced with another sample (z, yz). Denoting the target model parameters with 0, RMIA computes the pairwise likelihood ratio
p(0 x; Vae)
LR(æ, 2)
(A77)
p(0 z. Vz)
and the corresponding MIA score is given
ScoTeRMIA (x)
Pr (LR(x, 2) 7), (#.32)~
(A78)
where denotes the training data distribution. Zarifzadeh et al. (2024) show two approaches to compute LR(x, z): the direct approach and the effecient Bayesian approach. In the following theoretical analysis, we focus on the direct approach that is an approximation of the efficient Bayesian approach, as Zarifzadch et al. (2024) empirically demonstrates that these approaches exhibit similar performances.
Let Pa.b and da.b denote, respectively, the mean and standard deviation of to {(M(b)) 36) estimated from shadow models, where a denotes which of (æ: yn)) and (z. y=) is in the training set. By Equation 11 in (Zarifzadeh et al., 2024), the pairwise likelihood ratio is given
p(0 x, Va) LR(x, 2) p
N( pm,,, 02 W (1z; Pmm 02.2)
= N(ta: f2z,0 22. [...)W(ttt Pz.2.
(A79)
2, v.
where Po.b and ā. are, respectively, the mean and standard deviation of to estimated from shadow models when the training set contains a but not b. Then RMIA exploits the probability of rejecting the pairwise likelihood ratio test over (z) ID::
SCOTeRMIA = Pr (LR(x, x) 2 7).
(A80)
(*.
21

---

where D is the underlying data distribution. Similar to LiRA, the classifier is built by checking ScoTeRMIA (a) > r. In the following, we will use the direct computation of likelihood-ratio as described in Equation 11 of Zarifzadeh et al. (2024) which approximates LR((ar, 2) using normal distributions. Thus, RMIA rejects Ho if and only
Pr
2) x)
(A81)
(*.8=)~D
identifying the membership of æ. Hence the true positive rate of per-example RMIA is given as where @M denotes the randomness in the shadow set sampling and shadow model training.
TPRRMIA (x)
Pr Diarget DIPI, or (x.UE))
Pr (LR(x, 27) 2 T (xx, Vw) Dearget A (x. yoo & Diarget
(A82)
We define the average-case TPR for RMIA by taking the expectation over the data distribution:
TPRRMIA = (m (
(TPRRMIA (z)).
(A83)
B.2 Per-example RMIA
Next we focus the per-example RMIA performance. As in the case of LiRA, assume that tm and lg follow distributions of the location-scale family. We have
Me, + 0w Z
if (ax, Uw) Diarget A (z, Uz) & Dtarget
(A84)
02 + xz
if (xt Ux) Dranget A (z, Ux) Dtarget
t2 Ma, + Z if (xx, Uw) E Dtarget A (z, yz) $ Dtarget
(A85)
Uz, + Oz,xZ if (x, Va) & Dtarget A (z, Ux) E Dtarget.
where Z follows a distribution of location-scall family with the standard location and unit scale. It is important to note that & and Ja denote, respectively, a location and a scale, while previously defined Po.bb and oa,b are, respectively, mean and standard deviation. As for the analysis of LiRA, we assume that the target and shadow sets have a sufficient number of examples per class, and that Ox Oz xz x2 dz.z Oz Ox Ox. Ōx.x and u² σæ.* Ōz,2. where a and o₂ are, respectively, the true scales of ta and fz, and Oअ and Ô² are, respectively, standard deviations of tz and t, estimated from shadow models. Similarly to the case of LiRA, these assumptions justified using the simplified model. We have in the simplified model
b
8. = a. =
VVr(s
(2) Ba
(x)) =
S
S
x Ex.:
(A86)
T Ex
if yu y/2
2
02
Var(s yo
(x))
5)
(A87)
15 Eat
if Yv y/
0 Ø,2
= Var(s (2))
3
8)
2 TEz
if y = yz
(A88)
Ø,
132
if yv * y
1
2 02.2 Var(s yo
(z)
1
Ez.
(A89)
s
s
Therefore, the differences Tn Zz 0n large enough S.
2, Ōn م2,- and م - 02,- are negligible for
Now we derive the per-example RMIA vulnerability in terms of RMIA statistics computed from shadow models.
Lemma A4 (Per-example RMIA vulnerability). Suppose that the true IN/OUT distributions of for and f-2 are of location-scale family with locations pr. Hz ws Ha,z, Hz.z and scales Ga, az Assume that RMIA models tu and tz by normal distributions with parameters computed from shadow models, and that an attacker has access to the underlying distribution D. Then with infinitely many shadow models, the RMIA valnerability of a fixed target example (xe, Vv satisfies
E( >-pl))>>1ll11)))
TPRRMIA (x) S -Fizl [ (1-a) a)
(A90)
E B [[]]]]][[]]]]
22

---

for some constant a 2 (PRRMIA())) where Fizi is the caf of l and
q (n 22 12.0)aa - -Rza)) + (Hm.z 12, (2,,)) P2,2)
(A91)
0
63
Oz (N (z,z),;
A (D 12,))
+
(A92)
22
02
assuming that the inverse of Fiz) exists.
Proof. We abuse notations by denoting probabilities and expectations over sampling Diarge ~ D and (z, 3x) D by subscripts Drarget and We have in the limit of infinitely many shadow models
TPRRMIA (a) = Pr
(LR(xx, z) 2 7) (æz, Vn.) E Diarget A (z, Uz) Dtarget
(A93)
Diurzet
(PR) = Pr
Pr (LR((x 2) 2 7) (x. D A (z, Ux) Dranget)
Dinrect
(A94)
where
V(tzinn.. 0 22, N(txi Pm,
LR(x, 2)
(A95)
N(tz) Pr.æ. 0 W((z: Pr.z,
Note that the probabilities are over target dataset sampling in the limit infinitely many shadow models. We have
A = log(Lo((r,,))) 2))
(A96)
2
2
Panz
V2nae
exp
Gn
V2nu
exp
Bx
= log
(A97)
ls - uzz= Be
1/ -a,
exp
cxp
2
v V2nuz
no
tx
Pa.æ
tw
t2
P
t2
Pz,z
+
+
(A98)
2
x
2
x
2
O2
2
2/
+ 12220) +
M
21z/1z,z + (2.2)) (A99)
202
20
j
P2200
P
iiz,z (2t PA (z,z))
(2t
Pa. P2,x)
(A100)
202
20
When (æ, Vx)) E Drargot and (z, U2) $ Drargoss to = N + oz Z and l1 = Ma, + ogZ. Thus, becomes
Am Pu. uz (2/t,,,rr + 20 7 p (1z,.))
(A101)
282
+ An. iizzz (2 + 20 - Pa. )02,2)
(A102)
202
Similarly, when (I, Vee) $ Drarget and (z, vx) E Drargets to Mzx Jxll and t2 = Mz. + ogZ. Then X becomes
(2 + 20 umm. (2..)
(A (A103)
262
Pux
Pz.2 (2)12,z 2027 P (
(A104)
202
N
((z,)))
(iz,z Pzz,z) Z
(A105)
02
Pa
f2 (2 - ii - Pz +
flxx -xx
(2/2z,z
Aa fuzzz)
(A106)
282
20g
B
=AZ + B.
(A107)
23

---

Notice that A and B are functions of 2 and independent of Z. Also note that taking probability over Z corresponds calculating probability over Dtarget ~ D Thus, since we can set r > using Markov's inequality and the triangle inequality, we have in the limit of infinitely many shadow models
FPRRMIA (I) Pr (Pr(A2 2 log 27)
(A108)
Pr Pr(M 2 1og 2T
(A109)
E= E=|l>=1| 2
$ Pr
(A110)
1og7
(E= [>=ll 2 r log ?)
(A111)
$ Pr (E,
+ Ex 1 [B 2 T log
(A112)
2
Pr [2] - =log Ex [B]]]
(A113)
E=1AA]
Therefore, we can upper-bound FPRRMIA (2)) S by setting
T log E E=
== - F
(A114)
Ex [|A|]
That is,
log 7 Bx
Piz) (1 ox).
(A115)
Bz[ [A]
Now let
(Jmm
Pzz. (Man.o ul2,,)) (Nzz, (z,z) p12,z)
q = Am X2
(A116)
02
02
Note that is also independent of f, therehy E2 [//]] being a constant. By the similar argument using Markov's inequality and the triangle inequality, we have
TPRRMIA (x) PPr
Pr (r 2 log logg) 27
(A117)
$ Pr
Pr([tel 2 logn) 2r
(A118)
2
Ex |alll
<
Pr
2
(A119)
log
Pr (Ez |eell || 2 1og
(A120)
2
$
Pr (E
[E~[|||]
+ E2 B|| + [1gl 2 7 rlog
(A121)
:log E [|]
[=[la]]
Pr |2|
Pr
(A122)
Ez A|]
E2 [
Hence obtain
Ex [1
TPRRMIA (z) $ [لzी
[iz] (1 a)
(A123)
E_ |1 |
Note that, unlike per-example LiRA, we must assume that the attacker has access to the underlying distribution for the optimal RMIA as the Equations (A91) and (A92) depend on the parameters computed from shadow models.
B.3 RMIA power-law
Employing Lemma A4 and the simplified model, we obtain the following upper bound for RMIA performance.
24

---

Theorem 5 (Per-example RMIA power-law). Fix a target example (æ. Vu). For the simplified model with large and infinitely many shadow models, the per-example RMIA vulnerability given as
TPRRMIA 51-Fiz) (Fiz(( -
(12 C)
(A1 (A124)
VS
where a 2 FPRRMIA (z)), Fizi is the cdf of the standard folded normal distribution, and
Ex
(an (=)
||0||
22)
1x|
1/
Yau
(C 1)
(ar T7
||||
{2 m
||
1/2 7 y/
Vr (2:, C²)
Ex
(20.0 x)) =)
|||||
+
|||||
yee
+ + - 1)Es
(0t mm) [
+
(2. (m)
yx F Uee (A125)
In addition, we have
1 log(TPRRMIA (z) (PRRMIA(()) log S
- Fizz 1/12(( (1 (1= log (xx 2
(A126)
vx/2
Proof. To apply Lemma A4, we will calculate Ezlle1l and B=[[A[]. Let Dtarget
Let 1 (a) denote the score of (a, ya when Drarget contains (b, yb) but not the other example. Using similar argument in the proof of Theorem 2, we have
{( ...,, s)),
(2) 3o
(x)
F Un.i + x v.
(A127)
S
(z)
2
(x,
T youi + 2 Fy
Y/ = y/2
(A128)
(z,
if yee 2/2
(2) (2)
(z.
yzn + x .9)
U/2e = 1/2
(A129)
3 i= Tzy..)
if 1/2 + 1/2
(2) (z)
Dza + 2 - Zz S))
(A130)
S
Thus obtain
Pnna = (x,, mye)
+
5 (ar, x mmya)
(A131)
(x, Myja)
3 (x. mmya)
if y/ 2/z
12,x
(A132)
(x, Myj
if Y/2 + y=
zz myn
+ 1
s (2, x my..
if You 1
(A 1333)
my.)
if + 1/2
12,= = (z, mya)
+ 5 (2, -m
(A (A134)
(A 1355
VS
VZTEzz
(A136)
VS
where my is the true class mean of class yz-
Now recall that
9 (1x. 12 (Mx,0 .ix,)) (Ho,, (2xx) (2xa))
(A137)
02
02
A Ow (fx..x px.
(Dx,z (zzx))
(A138)
02
25

---

Since to and x follow normal distributions, the location and scale parameters of the true distributions correspond the mean and standard deviations estimated from infinitely many shadow models, respectively. Thus, we have
2
2
Mzz
Mzz,2
q
(A (A19))
Pz.
Ma.s
(A (4140)
Ox
Oz
Using the law of total expectation, we have
C
E2 |qll Pr(y==
= yv 1E21al v/= = yel +
Σ
Pr(yz E>
Uz f
(A141)
2
j=1.jvue
Bx
1
VSVx." T/ Ex (VSV=TEm))
(2), - 2)
)² +
(z, - z)
Vz V
(A142)
VSV=T TT
C -
1
(xx, 1 Myeel
(2, Myad
+
Ez
+
yz # yr
(A143)
C
VSVX T
VSV x TT
1
(x,
2)²
(z, 2)
CS
E2
x ExE
2T
1/2 = Vae
(A144)
C
1
(0. x my }2
(2, 2 Myya )2
+
E₂
V/ + Vw
(A145)
CS
xx To Σx
TT
and
Eg |||||] = Pr(y2= = yn Ex,E) 32 = yal +
Σ
Pr(y= = j)= 82 = 51
(A146)
j=1,j/b0
(I,2 2)
(z, 2)
E
V/ Vi
(A147)
V3Vv'' Tt' Ex
VSV=T'2
C
(x, maa)
(2, mz)
+
Eg
y/ + ya
(A148)
C
VSV Vx
/5v=TEx
1
(2t, 20 2)
(2, - =)
Ez
1/2 V/
(A 149)
CVS
C
(I.,
mz)
(z,
mzx)
+
Eg
y/ * ya
(A150)
CVS
V Eax
v VzTEz
Hence we obtain
Ez [|o]]
Ez
[]]
1
Rg
2 Eee
(x.x=2)² x)²
V/ = yx
+ (C E)
(=,)=m,)) my,)
*7 Ex
y/ # y
VS
Ez
(00 Var" Van
(s.w=2)
1yz = y
+ (C 1)
(30 me)
(2. -mm) Ve
y/ * Ya
Var 5m
(A151)
Now Lemma A4 yields
1x11
TPRRMIA
S1 (1) 1
[ ak)
(A152)
EZ [[A]
(xr, C*)
= =1-
[1z ox)
(A155) (A15))
Vs
where F/21 is the edf of the folded normal distribution and
Eg
(an.ar
(=,0-=2) 2)?
V/2 V/
+ (C - 1)Eg
(an.ar m.a
(2, m,)
U/ # yx
ar "
(xx C*=
E
(=
y/= Ya
(C 1)E=
(Ir, m7
(z.z- m_))
1/ * Yu (A154)
V Eae
V
v/z" Eee
V
26

---

This completes the first half of the theorem.
Now that from Lemma A4 have
TPRRMIA (2:) = Pr.
Pr(22 + q log 7) 2
(A155)
2
Ez Ez[122] + EE [191]
Pr 2
2
(A156)
log
EZ [=11
= Pr
|2| F F1z| (1 - -a) -a)
(A157)
2
E= E=1|-[]
FPRRMIA (x) = Pr
Pr({> 2 log 2T)
(A158)
2
[x 27
Prr
(A159)
log
Pr (2) Fizl (1 a))
(A160)
We claim that the bound for FPR (a:) (Equation (A (A159)) is as tight as that for TPR (2)
(Equation (A156) for sufficiently large S. Let denote
KTRK (n) Pr(A= + log
(A161)
Kyvu (7) Pr(A> 2 7)
(A162)
Since q O(1/S), for large S we approximate
())p()) (T)n(1) px. (log (logr) Cu
(A163)
for some constant Co- Since A= O(1/VS), the scaled random variable _r VSAg is almost independent of S. Then by the change of variables formula, we have
Px, (log 7) Co = P2g (V'S log 7)
co VS
(A164)
Without loss of generality may set log y 1/VS. Thus, have
KTeR (1/VS) KippR (1/v/) = VS 1 This quantity scales as 0(1/v5). Therefore, TPRRMIA (x) FPRRMIA (x)
Pi.
PP. (1)
(A165)
v
VS
V5"
Pr PI(NiTE PT(Nirx(1/VS3 (1/V8) 2 7) Pr(KiEE (I/VS) 2 T)
(A166)
P (1/v5) (T) (KiPR (1 (1/VS) KTPR (1/V5)
(A167)
P P (1//\) (1) cn
(A168)
VSS
Note that Kyea (1/V5) and, consequently, are independent of S for large enough S since
1 Pr( 1).
Kyex (1/V5) Pr A
Pr
(A169)
VS
VS.
Hence TPRRMIA(#) FPRRMIA (x) = 0(1/ and (A159) we have for sufficiently large S
On the other hand, from Equations (A156)
A
E 1=1
Ezlgg
Pr
Pr
27
(A170)
log
log
log
Pr
(VSEE ||> 2 7) Pr [VSE [| + VS 2T)
(A171)
Pr [B2 22) - Pr
E= +
VS 2T)
(A172)
3/2
~PPel1A-111 (T) VS'
(A173)
27

---

where 01 is some constant. Since for large S, Es [xxl] and T are independent of S, Equation (A173) scales (1/vSS).
Now let
E2 [|x2] || + Ez11o] 27)
TPRLIRA (x) = Pr
UTPR
(A174)
2
log γ
Ez | |
FPRLBA (x) = Pr %
UPPR
(A175)
log 7
for some UYeR, UpeR O which evaluate the tightness of the bounds. Then we have
UTPr
UpER
(A176)
Eg M2 + [xl
Ex 1=
Pr
2 27) -
Pr
27)
((PRRLIAAZ)) -PRLIRA (x))
logy
log 7
(A177)
O(1/V5).
(A178)
Hence we conclude that for sufficiently large S, the bound for FPRLIRA())) is as tight as that for (T).
Therefore, noting that Ez |/ O(L/VS), for large S we obtain
TPRRMIA (x) FPRRMIA(Z)
(A179)
|Z] 2 Fiz) aa) E=11e11
Ex [[A]] )
Pr
Pr FL
(A180)
(Hizl (1 - aa)) Ex 19
P|2|
Ez [|]]]
(A181)
(xx O)
P1 F (1 (1)))
(A182)
VS
Since follows the standard folded normal distribution,
Piz| Fiz) - a))
2
(1-
exp
(A183)
22
follows that
1
1
Fiz (1 -a)² log Vxx
log log(TPRRMIA (x) -PRRMII (()MMIA()))=00 xe log S
(A184)
2
As for the LiRA power-law, bounding || - mm 1 and || -mal will provide a worst-case upper bound for which the power-law holds.
Finally, the per-example RMIA power-law is also extended to the average-case:
Corollary A6 (Average-case RMIA lawer-law). For the simplified model with sufficiently large S and infinitely many shadow models, we have
1 log TPRRMIA FPRRMIA) = - 1ollog log S
V V C3)
1) (A185)
= F (1 (1-a)²
a)² +log
E (0 -B
2
\x/2
where 2 FPRRMIA and Fiz] is the edf of the standard folded normal distribution.
28

---

Proof. By Theorem 5 and the law of unconscious statistician, we have for large S
TPRRMIA
FPRRMIA
(T2)(MP (x)
FPRRMIA (x))
(A186)
1
(1-2)(1-0)* V (x: C)
p(x)
dx
(A187)
V
VS
(1-00)
/ p(x))
(xx C)
dæ.
(A188)
V
VS
1 F
(1
V1
C)
E (M
(A189)
VS
VT/2
where p(x) is the density of (x, Un), and is the data domain. Hence we obtain
(TPRRMAA -PRRMII)
log
Fiz (1 a)2+ +log
E
(A190)
2
(m.
V
29

---

C
Training details
C.1
Parameterization
We utilise pre-trained feature extractors BiT-M-R50xI (R-50) (Kolesnikov et al., 2020) with 23. 33.5M parameters and Vision Transformer ViT-Base-10 16 (ViT-B) (Dosovitskiy et al., 2021) with 85.8M parameters, both pretrained on the ImageNet-21 K dataset (Russakovsky et al., 2015). We download the feature extractor checkpoints from the respective repositories.
Following Tobaben et al. (2023) that show the favorable trade-off of parameter-efficien fine-tuning between computational cost, utility and privacy even for small datasets, we only consider fine-tuning subsets of all feature extractor parameters. We consider the following configurations:
Head: We train a linear layer on top of the pre-trained feature extractor.
FiLM: In addition to the linear layer from Head, fine-tune parameeter-efficient FiLM (Perez et al., 2) adapters scattered throughout the network. While a diverse set of adapters has been proposed, utilise FiLM as it has been shown be competitive in prior work (Shysheya et al., 2023; Tobaben et al., 2023).
C.1.1 Licenses and access
The licenses and means to access the model checkpoints can be found below.
BiT-M-R50x1 (R-50) (Kolesnikov et al., 2020) is licensed with the Apache license and can be obtained through the instructions on transfer.
Vision Transformer ViT-Base-16 (ViT-B) (Dosovitskiy et al., 2021) is licensed with the Apache-2.0 license and can be obtained through the instructions on https://githar com/ noogle-research/viie wwt.iiiiiionttasorrrree.e.o transformer.
C.2 Hyperparameter tuning
Our hyperparameter tuning is heavily inspired by the comprehensive few-shot experiments by Tobaben et al. (2023). We utilise their hyperparameter tuning protocol as it has been proven to yield SOTA results for (DP) few-shot models. Given the input D dataset perform hyperparameter tuning by splitting the into 70% train and 30% validation. We then perform the specified iterations of hyperparameter tuning using the tree-structured Parzen estimator (Bergstra al., 2011) strategy as implemented in Optuna (Akiba et al., 2019) to derive a set of hyperparameters that yield the highest accuracy on the validation split. This set of hyperparameters is subsequently used train all shadow models with the Adam optimizer (Kingma and Ba, 2015). Details on the set of hyperparameters that tuned and their ranges can be found in Table A..
Table A1: Hyperparameter ranges used for the Bayesian optimization with Optuna.
lower bound
upper bound
batch size
10
(D]
clipping norm
0.2
epochs
200
learning rate
le-7
le-2
C.3
Datasets
Table A2 shows the datasets used the paper. We base our experiments on subset of the few-shot benchmark VTAB (Zhai et al., 2019) that achieves a classification accuracy >> 80% and thus would considered be used by practitioner. Additionally, we add CIFAR10 10 which is not part the original VTAB benchmark.
30

---

but Resics45 which required man- Table A2: Used datasets in the paper, their minimum and maximum shots and maximum number of classes C' and their test accuracy when fine-tuning a non-DP VIT-B Head. The test accuracy for EuroSAT and Resics45 computed on the part of training split that not used for training the particular model due to both datasets missing an official test split. Note that LiRA requires 25 for training the shadow models and thus smaller than when only performing fine-tuning.
dataset
(max.)
min. S
test accuracy (min. S)
test accuracy (max. S)
C
$
Patch Camelyon (Veeling et al., 2018)
2
256
65536
82.8%
85.6%
CIFARIO (Krizhevsky, 2009)
2048
92.7%
97.7%
EuroSAT (Helber et al., 2019)
10
8
512
80.2%
96.7%
Pets (Parkhi et al., 2012)
82.3%
90.7%
Resisc45 (Cheng et al., 2017)
45
32
256
83.5%
91.6%
CIFAR 100 (Krizhevsky, 2009)
100
128
82.2%
87.6%
C.3.1 Licenses and access
The licenses and means to access the datasets can be found below. We downloaded all datasets from TensorFlow datasets tttps://www..ensrrflouu temsorflow. org patastttssts ual download.
Patch Camelyon (Veeling et al., 2018) licensed with Creative Commons Zero v1.0 Universal (cc0-1.0) and version 2.0.0 of the dataset specified on datasets/
org/
CIFARIO (Krizhevsky, 2009) is licensed with an unknown license and we use version 3.0.2 dataset as specified on https:/ //tp:://wwww.... www.wenssorfo... org/ catpeett//aaallg//iiiaaa
EuroSAT (Helber et al., 2019) is licensed with MIT and we use version 2.0.0 of the dataset as tensorflow org htt///tttsts/eeeeeeeeeeee catalog/ eurosat.
specified on https:
cataaoo/eurosaa.
Pets (Parkhi et al., 2012) is licensed with CC BY-SA 4.0 Deed and as specified on ht https: //wp://wwww///////////////./// tensorflox wtw.llowwwo.....o.........oo trrg/dtaaeess. dataastsscaaaallgggg..tt catalog/ oowoooddoiidiiooo iii_j_iitppp___
version 3.2.0 of the dataset
Resise45 (Cheng et al., 2017) is licensed with an unknown license and we use version 3.0.0 dataset as specified on https: //ww. http:wtwwwwrfftt////o/////////tt www.rflooooog/ddteeeeo
wrgg/daaasets/caa tatpeetss/aalllooeeseeeeooo
CIFAR 100 (Krizhevsky, 2009) is licensed with an unknown license and we use version 3.0.2 of the
dataset as specified on https: //ww://wwwwwc//o/////. tewsorflos wwwprflwwwwooo//eeeoeoeeee
C.4 Compute resources
All experiments but the R-50 (FiLM) experiments are run on CPU with 8 cores and 16 GB of host memory. The training time depends on model (ViT is cheaper than R-50), number shots S and the number of classes C but ranges for the training of one model from some minutes an hour. This assumes that the images are passed once through the pre-trained backbone and then cached as feature vectors. The provided code implements this optimization.
The R-50 (FiLM) experiments are significantly more expensive and utilise a NVIDIA V100 with 40 GB VRAM, 10 CPU cores and 64 GB of host memory. The training of 257 shadow models then does exceed 24h for the settings that we consider.
We estimate that in total we spend around days of V100 and some dozens of weeks CPU core time but more exact measurements are hard make.
31

---

D
Additional results
In this section, we provide tabular results for our experiments and additional figures that did not fit into the main paper.
D.1
Additional results for Section 4
This Section contains additional results for Section 4.
D.1.1
Vulnerability as a function of shots
This section displays additional results to Figure 1 for FPR E (0.1, 0.001) } for ViT-B and R-50 in in Figure A. and Tables A3 and A4.
(TPR- (TPR-FPR) FPR)
at FPR=0.1
0.1
(TPR FPR)
FPR=0.1
0.1
0.001
0.001
0.0001
0.0001
16
256
1024
4096 16384 65536
16
64
256
1024
4096
16384 65536
Examples per class
Examples per class
Patch Camelyon (C=2)
Pets (C=37)
Patch Camelyon (C=2)
Pets (C=37)
CIFAR 10 (C=10)
- Resics45 (C=45)
CIFARIO (C=10)
Resicu45 (C=45)
EureSAT (C=10)
CIFARI00 (C=100)
EuroSAT (C=10)
CIFARIOO (C=100)
(a) VTT-B Head TPR FPR at FPR 0.1
(b) R-50 Head TPR FPR at FPR 0.1
(TPR- FPR)
at =PR=0.01
0.1
(TPR- FPR) FPR=0.01
0.1
0.01
0.001
0.0001
0.0001
16
256
1024
4096 16384 65536
64
256
1024
4096
16384 65536
Examples per class
Examples per class
Potch Camelyon (C=2)
Poss (C=37)
Patch Camelyon (C=2)
(C=37)
CIFARIO (C=10)
Resics45 (C=45)
CIFARIO (C=10)
Resics45 (C=45)
-- AT (C=10)
CIFAR 100 (C=100)
EuroS (C=10)
CIFAR100 (C=100)
(c) VIT-B Head TPR
FPR at FPR = 0.01
(d) R-50 Head TPR
FPR at FPR 0.01
(TPR-FPR)
at FPR=0,001
0.1
(TPR- FPR)
at FPR=0.001
0.1
0.01
0.01
0.001
0.0001
0.0001
1024
16384
Examples per class
Examples per class
Camelyon (C=2)
(C=37)
Patch Camelyon (C=2)
(C=37)
CIFARIO (C=10)
Resics45 (C=45)
CIFARIO (C=10)
Resics45 (C=45)
EuroSAT (C=10)
CIFAR100 (C=100)
EuroSAT (C=10)
CIFAR100 (C=100)
(c) ViT-B Head TPR FPR at FPR = 0.001
(f) R-50 Head TPR FPR at FPR 0.001
Figure A.1: MIA vulnerability function of shots (examples per class) when attacking pre-trained ViT-B and R-50 Head trained without DP on different downstream datasets. The errorbars display the minimum and maximum Clopper- Pearson Cls over six seeds and the solid line the median.
32

---

Table A3: Median MIA vulnerability over six seeds as function of S (shots) when attacking a Head trained without DP on-top of ViT-B. The ViT-B is pre-trained on ImageNet-21k: dataset
classes (C)
shots (S)
tpr@ frrffpr=0.1 tpr@fpr=0.01
tpr @fpr=000001
Patch Camelyon (Veeling et al., 2018)
256
0,266
0.086
0.032
512
0.223
0.059
0.018
1024
0.191
0.050
0.015
2048
0.037
0,009
4096
0.144
0.028
0.007
8192
0.128
0.021
0.005
16384
0.118
0.017
0.003
32768
0,
0.014
0.002
65536
0.105
0.012
0.002
CIFARTO (Krizhevsky, 2009)
10
0.910
0.660
0.460
16
0.717
0,367
0,201
32
0.619
0.306
0.137
64
0.345
0.132
0.067
128
0.322
0.151
0.082
256
0,227
0.096
0.054
512
0,190
0.068
0.032
1024
0.168
0.056
0.025
2048
0.148
0.039
0.013
EuroSAT (Helber et al., 2019)
10
0.609
0.408
16
0.738
0.420
0.234
32
0.475
0.222
0.113
64
0.400
0.159
0.074
128
0.331
0.155
0.084
256
0.259
0.104
0.049
0.213
0.080
0.037
Pets (Parkhi al., 2012)
0.648
0,343
0.160
16
0.745
0.439
0.259
32
0.599
0.311
0.150
Resics45 (Cheng al., 2017)
45
32
0.672
0.425
0.267
64
0,531
0.295
0.168
128
0,419
0.212
0.115
0.323
0.146
0.072
CIFARTOO (Krizhevsky. 2009)
16
0,508
0.324
32
0.445
0.290
64
0.538
0.302
0.193
128
0.433
0.208
0.114
33

---

Table A4: Median MIA vulnerability over six seeds as a function of S (shots) when attacking a Head trained without DP on-top of R-50. The R-50 is pre-trained ImageNet-2Ik.
dataset
classes (C)
shots (S)
tpr @ fpr=0.1 tpr@fpr fpr=0.01
(pr @ 0.0=0.001
Patch Camelyon (Veeling et al., 2018)
256
0.272
0.076
0.022
0.195
0.045
1024
0.201
0.048
0,011
2048
0.178
0.041
0.010
4096
0.163
0.033
0.008
8192
0.143
0.026
0.006
16384
1.4
0.019
0.004
32768
0.118
0.016
0.003
65536
0.106
0.012
0.002
CIFARIO (Krizhevsky. 2009)
10
0.911
0,574
0,324
0.844
0.526
0.312
32
0.617
0.334
0.183
64
0.444
0.208
0.106
128
0.334
0.159
0.084
256
0.313
0.154
0.086
512
0.251
0.103
0.051
1024
0.214
0.082
0.038
EuroSAT (Helber et al., 2019)
10
0.517
0.275
16
0.699
0.408
0.250
32
0.490
0.236
0.121
64
0.410
0.198
0.105
128
0.332
0.151
0.075
256
0.269
0.111
0.056
0.208
0.077
0.036
Pets (Parkhi al., 2012)
0.631
0,366
16
0.745
0.427
0.227
32
0.588
0.321
0.173
Resics45 (Cheng al., 2017)
45
32
0.671
0.405
0.235
64
0,534
0.289
0.155
128
0.445
0.231
0.121
0.367
0.177
0.088
CIFARTOO (Krizhevsky, 2009)
100
16
0.638
0.429
32
0.763
0.549
0.384
64
0.634
0.414
0.269
34

---

D.1.2 Vulnerability as a function of the number of classes
This section displays additional results to Figure 2 for FPR E {0.1, 0.01, 0.001} for VIT-B and R-50 in in Figure A.2 and Tables A5 and A6.
(TPR- FPR)
atFPR=0.1
(TPR- FPR)
atFPR=0.11
0.1
0.01
0.01
2
16
32
64
2
16
32
64
(classes)
(classes)
--
Patch Camelyon (C=2) - Pets (C=37)
-
Patch Camelyon (C=2)
- Pets (C=37)
CIFARIO (C=10)
Resics45 (C=45)
CIFARIO (C=10)
Resics45 (C=45)
BuroSAT (C=10)
CIFARI00 (C=100)
EureSAT (C=10)
CIFAR100 (C=100)
(a) VTT-B Head TPR
FPR at FPR 0.1
(h) R-50 Head TPR FPR at FPR 0.1
(TPR (TPR-FPRR -FRR)
atFPR=0,00
(TPR FPR) atFPR=0.01
0.1
0.01
2
16
32
64
2
16
32
64
(classes)
(classes)
Patch Camelyon (C=2) - Pess (C=37)
Patch Camelyon (C=2)
Pers (C=37)
CIFARIO (C=10)
- Resies45 (C=45)
CIFARIO (C'10)
- Resics45 (C=45)
ET (C=10)
CIFAR (C=100)
EuroSTT (C=10)
CIFAR (C=100)
(c) VTT-B Head TPR-
FPR FPR = 0.01
(d) R-50 Head TPR FPR at FPR 0.01
(TPR-FE -PR)
atFPR=0.000
(TPR- FPR) -0.PR=0.001
0.1
0.01
0.01
0.001
2
16
32
64
C. (classes)
C (classes)
Patch Camelyon (C=2)
Pets (C=37)
Patch Camelyon (C=2)
Pets (C=37)
CIFAR (C-10)
- Resics45 (C=45)
CI (C=10)
Resics45 (C=45)
AaroSAT (C=10)
CIFAR 100 (C=100)
EuroS (C=10)
CIFAR 100 (C=100)
(c) ViT-B Head TPR FPR at FPR = 0.001
(I) R-50 Head TPR FPR at FPR 0.001
Figure A.2: MIA vulnerability as a function of C (classes) when attacking ViT-B and R-50 Head fine-tuned without DP on different datasets where the classes are randomly sub-sampled and S = 32. The solid line displays the median and the errorbars the min and max clopper-pearson CIs over 12 seeds.
35

---

Table A5; Median MIA vulnerability over 12 seeds as function of C' (classes) when attacking Head trained without DP on-top of a VIT-B. The Vit-B is pre-trained on ImageNet-21K. dataset
shots (S)
classes (C) tpr@ ppr@fpr=0.1
ipr@fpr=0.01
[pr 1pr@fpr00.001
Patch Camelyon (Veeling al., 2018)
0.192
0.080
CIFARIO 2009)
32
0,494
0.167
0.071
0.527
0.217
0.115
0.574
0.262
0.123
EuroSAT (Helber et al., 2019)
32
0.306
0.100
0.039
0.298
0.111
0.047
0,468
0.211
0.103
Pets (Parkhi et al., 2012)
0.232
0.045
0.007
0.324
0.092
0.035
0.296
0.094
0.035
16
0,406
0.158
0.069
32
0.553
0.269
0.136
Resies45 (Cheng et al., 2017)
32
0.272
0.084
0.043
0,322
0.119
0.056
0.496
0.253
0.148
16
0.456
0,204
0.108
32
0.580
0.332
0.195
CIFARTOO (Krizhevsky, 2009)
32
0.334
0,088
0.035
0.445
0.150
0.061
0.491
0.223
0.121
16
0.525
0.256
0.118
32
0.553
0.276
0.153
64
0.612
0.350
0,211
Table A6: Median MIA vulnerability over 12 seeds as function of C (classes) when attacking
Head trained without DP on-top of a R-50. The R-50 is pre-trained on ImageNet-21k.
dataset
shots (S)
classes (C)
aor=fpr=0.1 tpr@fpr=0.01
(pr @ @p0r=00001
Patch Camelyon (Veeling 2018)
0.452
0.151
0.041
CIFARIO (Knzhevkkk,, 2009)
32
0.404
0.146
0.060
0.560
0.266
0.123
0.591
0.318
0.187
EuroSAT (Helber et al., 2019)
32
2
0,309
0.111
0.050
0.356
0.144
0.064
0.480
0.233
0.123
Pets (Parkhi al., 2012)
0.249
0.068
0.029
0.326
0.115
0.056
0.419
0.173
0.075
16
0.493
0.245
0.127
32
0.559
0.294
0.166
Resies15 (Cheng et al., 2017)
32
0,310
0.103
0.059
0,415
0.170
0.083
0.236
0.119
0.311
0.174
32
0.644
0.382
0.218
CIFAR 100 (Krizhevsky, 2009)
32
0.356
0.132
0.054
0.423
0.176
0.087
0.545
0.288
0.163
16
0.580
0.338
0.196
32
0.648
0.402
0.244
64
0.476
0.320
36

---

D.1.3
Data for FiLM and from seratch training
Table A7: MIA vulnerability data used in Figure 5b. Note that the data from Carlini et al. (2022) is only partially tabular, thus we estimated the TPR FPR from the plots in the Appendix of their paper. model
dataset
classes
shots
source
tpr@
tpr@
tpr@
(C)
(S)
fpr=0.1
fpr=0.01
fpr=0.001
R-50 FILM
CIFARIO (Krizhevsky, 2009)
10
50
This work
0.482
0.275
0.165
CIFARTOD
100
10
Tobahen al. (2023)
0.933
0.788
0.525
(Krizhevsky. 2009)
25
Tobaben al. (2023)
0.766
0.576
0.449
50
Tobaben al. (2023)
0.586
0.388
0.227
100
Tobahen al. (2023)
0.448
0.202
0.077
EuroSAT
This work
0.791
0.388
0.144
(Helber al., 2019)
Patch Camelyon
2
256
This work
0.379
0.164
0.076
(Veeling et al., 2018)
Pets
37
This work
0.956
0.665
0.378
(Parkhi et al., 2012)
Resics45
32
This work
0.632
0.379
0.217
(Cheng et al., 2017)
CIFARTO
2500
Carlini (2022)
0.300
0.110
0.084
(Krizhevsky, 2009)
(wide ResNet)
CIFARI00
100
250
Carlini al. (2022)
0.700
0,400
0.276
(Krizhevsky, 2009)
37

---

D.1.4 Predicting dataset vulnerability as function of S and C
This section provides additional results for the model based on Equation (13)
Table A8: Results for fitting Equation (13) with statsmodels Seabold and Perktold (2010) to ViT
Head data at FPR E (0.1, 0.01, 0.001}. We utilize an ordinary least squares. The test R assesses the fit to the data of R-50 Head.
coeff.
FPR
R2
test R²
coeff. value
std. error
p > |=|
coeff. [0.025
cueff. 0.975]
As (for S)
0.1
-0.56
0.011
.4.99%
0.000
0.01
0,946
0.854
-0.555
0.014
-39.788
0.000
-0,582
-0.527
0.001
0,930
0.790
-0.627
0.019
-32.722
0.000
-0.664
-0.589
Do (for C)
0.1
0.952
0,907
0.090
0.021
4.231
0.000
0,048
0.131
0.01
0,946
0.854
0.182
0.026
6,960
0.000
0.131
0.234
0.001
0.930
0.790
0.300
0.036
8.335
0.000
0.229
0.371
Po (intercept)
0.1
0.952
0,907
0,314
0,045
6.953
0,000
0.225
0,402
0.854
0.083
0.056
1.491
-0.027
0.001
0,930
0.790
-0.173
0.077
-.261
0.025
-0,324
-0.022
Figure A.3 shows the performance for all considered FPR.
(TPR-FPR) FPR)
@FPR=0.1
(TPR- FPR)
@ @FPR=0.01 01
0.01
0.01
0.001
16
256
1024
4096
16384 65536
64
256
1024
4096
16384 65536
(shots)
S (shots)
Fitted Line
VIT-B (Train)
Firted Line (C=2)
VIT-B (Train)
Line (C=10)
(Test)
Fitted (C=10)
Fitted Line (C=100)
Filted Line (C=100)
(a) ((TRR FPR) FPR
(TPR FPR) at FPR 0.01
@TPR= 0100
(TPR (TPR-FPR) -FRR)
0.1
0.01
0.001
16
64
256
1024
4096
16384 65536
S (shots)
Fitted Line (C=2)
VIT (Train)
Fitted Line (C=10)
R.50 (Test)
Fitted Line (C=100)
(c) (TPR FPR) FPR 0.001
Figure A.3: Predicted MIA vulnerability as a function of S (shots) using a model based on Equa- tion (13) fitted Table A3 (VTT-B). The triangles show the median TPR FPR for the train set (ViT-B; Table A3) and circle the test set (R-50; Table A4) over six seeds. Note that the triangles and dots for C' 10 are for EuroSAT.
38

---

D.2 Simpler variant of the prediction model
The prediction model in the main text (Equation (13)) avoids predicting TPR FPR in the tail when S is very large. In this section, we analyze a variation of the regression model that is simpler and predicts 1o (TPR) (TPR) instead of 1og (TPR FPR). This variation fits worse to the empirical data and will predict TPR < FPR for high S.
The general form this variant be found in Equation (A191), where Ac and 90 the learnable regression parameters.
10g = 3s (S) + 9C log + 90
(A191)
Table A9 provides tabular results the performance of the variant.
Table A9: Results for fitting Equation (A (A191) with statsmodels Seabold and Perktold (2010) to VIT
Head data at FPR E {0.1, 0.01, 0.001 We utilize an ordinary least squares. The test R assesses the fit to the data of R-50 Head.
coeff.
FPR
test
coeff. value
std. error
p > |=|
coeff. (0.025
coeff. 0.975)
As (for 9)
0.1
0.908
0.764
-0.248
0.008
30.976
0.000
40.264
-0.233
0.01
0.761
-0.416
0.011
76.
0.000
-0.438
-0.393
0.001
0,931
0.782
-0.553
0.017
33.5077
0.000
-0,586
-0.519
De (for C)
0.1
0.908
0.764
0.060
0.015
3.955
0.000
0.030
0.089
0.761
0.169
0.021
7,941
0.127
0.211
0.001
0.931
0.782
0.297
0.032
9.303
0.000
0.234
0.360
Ho (intercept)
0.1
0,908
0.764
0.029
0.032
0.913
0.362
-0.034
0.093
0.940
0.761
-0.118
0.045
-2.133
0.010
-0,208
-0.029
0.001
0.931
0.782
-0,295
0,068
-4.345
0.000
-0.429
-0.161
Figure A.4 plots the performance of the variant similar to Figure 5a in the main text.
39

---

TPR FPR=0.)
TPR@ FPR=0.00 0.011
0.1
0.1
0.001
0.001
16
64
256
1024
4096
16384 65536
16
64
256
1024
4096
16384 65536
S (shots)
S (shots)
Fitted Line (C=2)
VIT-B (Train)
Fitted Line (C=2)
VIT-B (Train)
Fitted Line (C-10)
R-50 (Tes)
Fited Line (C=10)
... R-50 (Test)
Fited Line (C=100)
random MIA classifier
Finted Line (C=100)
random MIA classilier
(a) TPR at FPR 0.1
(b) TPR at FPR = 0.01
TPR @FPR=O..MMM 000
0.1
0.001
256
16384 65536
S (shots)
(C=2)
VTT-B (Train)
Fitted Line (C=10)
(Test)
Fitted Line (C=100)
random MIA classifier
(c) TPR at FPR 0.001
Figure A.4: Predicted MIA vulnerability function of S (shots) using model based Equa- tion (A191) fitted Table A3 (ViT-B). The triangles show the median TPR for the train set (VIT-B; Table A3) and circle the test set (R-50; Table A4) over six seeds. Note that the triangles and dots for 10 are for EUT
40

---

D.3 Empirical results for RMIA
Figures A.5 A.7 report additional results for RMIA Zarifzadeh al. (2024).
(TPR--PP)) (TPR-1PP)@vPR-- 18FPP=0000
(1PR-192)
0.1
0.0001
0.0001
16:02
1e405
16582
16334
(shots)
S (shots)
Camelyon (C=l)
FIm&AT (C=10)
Resice& (C=45)
Canelyon (Cel)
FamSAAT (C=10)
Resize48 (C=43)
STFARIO(C--1O
Pets (C-37)
CIFARIDO (C+100)
CIFARIO (C=10)
Petu (Ch-7)
CIFARI00 (C=+00)
(a) (TPR- FPR) at FPR 0.1
(b) (TPR FPR) EPR 0.01
(198-17%)) F91:=0.00
0.01
0.001
0.0001
1c:05
16
256
1024
4096
16884 65336
(shots)
Pachh Canclpun (C-2)
Fum&AT (G=10)
Raiize5 (C=45)
(C=10)
(C=17)
CIFARIOO (C=100)
(c) (TPR- FPR) at FPR = 0.001
Figure A.5: RMIA (Zarifzadeh al., 2024) vulnerability (TPR FPR at fixed FPR) as function of S (shots) when attacking a VYT-B Head fine-tuned without DP on different datasets. We observe power-law relationship but especially low FPR the relationship is not as clear as with LiRA (compare to Figure A. The solid line displays the median and the error bars the minimum of the lower bounds and maximum
upper bounds for the Clopper-Pearson CIs over six seeds.
41

---

(TPR-FPR) @FPR=0.1
(TPR- FPR)
@ @FF==0001
0.1
0.01
0.01
0.001
16
64
256
1024
4096
16384 65536
S (shots)
S (shots)
Fitted Line (C=2)
RMIA (C=2)
Firted Line (C=2)
RMIA (C=2)
Line (C=10)
(C=10)
Fitted (C=10)
(C=10)
Fitted Line (C=100)
RMIA (C=100)
Fitted Line (C=100)
RMIA (C=100)
(a) (TPR- FPR 0.1
(TPR FPR) 0.01
(TPR-FPR)
@FPR=0.00)
0.01
0.001
16
64
256
1024
4096
16384 65536
S (shots)
Fitted Line (C==2
RMIA (C=2)
Fitted Line (C=10)
RMIA (C=10)
Line (C=100)
(C=100)
(c) (IPR FPR) FPR 0.001
Figure A.6: Predicted MIA vulnerability ((TPR- - FPR) at FPR) based on LiRA vulnerability data as function of (shots) in comparison to observed RMIA (Zarifzadeh al., 2024) vulnerability on the same settings. The triangles show the highest TPR when attacking (ViT-B Head) with RMIA over six seeds (datasets: Patch Camelyon, AT and CIFARI00), Especially FPR 0.1 the relationship behaves very similar for both MIAs, but RMIA shows more noisy behavior at lower FPR.
42

---

(TPR-FPR)
@FRR=
0.1
0.01
0.001
16
256
1024
4096
16384
65536
64
256
1024
4096
16384 65536
(shots)
(shots)
--- Patch Camelyon (C=2)
EuroSAT (C=10) - Resics45 (C=45)
--- LiRA
CIFARIO(C=10))
Pets (C=37)
--- CIFARI00 (C=100)
--- RMIA
(a) (TPR FPR) at FPR 0.1
(TPR-FPR)
FFPR=0001
0.1
0.01
0.001
0.0001
16
256
1024
4096
16384
65536
16
64
256
1024
4096
16384
65536
S (shots)
S (shots)
--
Patch Camelyon (C=2)
EuroAAl (C=10) - Resics45 (C=45)
--- LIRA
CIFARIO
Pets (C=37)
CIFARI00 (C=100)
-x- RMIA
(b) (TPR FPR) at FPR 0.01
(TPR-FPR)
@FPR=0.001
0.1
0.01
0.0001
1c-05
16
256
1024
4096
16384
65536
16
64
256
1024
4096
16384
65536
S (shots)
S (shots)
-- Patch Camelyon (C=2)
EuröAAl (C=10) - Resics45 (C=45)
--- LiRA
(C=10)
Pets (C=37)
CIFAR100 (C=100)
--- RMIA
(c) (TPR FPR) at FPR 0.001
Figure A.7: LiRA and RMIA vulnerability ((TPR FPR)) as function of shots (S) when attacking ViT-B Head fine-tuned without DP on different datasets. For better visibility, we split the datasets into two panels. We observe the power-law for both attacks, but the RMIA more unstable than LiRA. The lines display the median over six seeds.
43

---

D.4 Tabular results for Section 4.4
Table A10 displays the tabular results for Figure 6 in Section 4.4.
Table A10: Tabular results for Figure on when attacking a ViT-B (Head) fine-tuned on PatchCame-
lyon. We display the median over six seeds at FPR 0.1.
S
Max TPR
TPR of 0.999 Quantile
TPR of 0.99 Quantile
TPR of 0.95 Quantile
16384
0.77
0,43
0.22
0.11
23170
0.73
0.37
0.19
0.10
32768
0.65
0.30
0.15
0.08
49152
0.54
0.23
0.13
0.07
65536
0.49
0.19
0.11
0.07
44

---

E Details on Section 4.5
In Section 4.5, compare the results of our empirical models of vulnerability (Sections 4.3 and 4.4) to DP bounds. Below we explain how we make the connection between both.
First, we compute the upper bound on the TPR for given (c, 6D-DP privacy budget at given FPR using Theorem reformulated from Kairouz et al. (2015) below:
Theorem (Kairouz et al. (2015)). mechanism M X + y is (e, 6)-DP if and only if for all adjacent D ~ 'D' TPR S muinee'PPR + 6,1 e e(1
FPR)
(A192)
For given (c, 5) and FPR we then obtain a value for the TPR.
Next, we use the linear model from Section 4.3 solve for the minimum S predicted to be required given = 2 classes in our example. The coefficients can be found in Table A8 for the average case and Section 4.4 for the worst-case. We solve the TPR from the linear model
log1o TPR FPR) 3s (S) + Bc log (C) + 3o
(A193)
TPRR S So 1 + FPR.
(A19) (944))
Now, we find the minimum S that the TPR from Equation (A194) upper bounds the TPR of Equa- tion (A192)
S S8 + FPR min + 6, e 5 =FR))
(A195)
=S mmi 5, - (1 - 5 - FPR) FPR 1/8s
e-
(A196)
C
45

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims that are summarized at the end of Section 1 accuracly reflect the contributions in Sections and
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limiations are discussed in Section 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
46

---

Justification:
The details for the proofs in Section 3 are in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Experimental details are in Appendix C and the documented source code is in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
47

---

Answer: [Yes]
Justification:
The source code including instructions can found in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https:
guwgge/ies///// /wode//ooiiisssteeeeeeee ssionooiicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide details in Appendix C and the remaining details can be found in the documented source code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In Section 4 we provide results over at least six seeds and plot the median as well as errorbars. Many errorbars are based on Clopper-Peante intervals.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
48

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details can be found in Appendix C.4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //te:://eeuiis//e
inne?
Answer: [Yes]
Justification:
The research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss broader impacts in Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
49

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original owners and mention the licenses in Appendices C. and C.3.1.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
50

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The documentation for the code is with the code in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
51

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (bttps://neun) h/tt:///eeu....eeeeee http://uurrs..ccooeeeeeeeeeeeeee cct//oomffeeeeccs/22///1111 for what should or should be described.
52


Reviews:
[{"review_id": "dCWn8zAobh", "reviewer": "Reviewer_c8jK", "summary": "This paper explores membership inference attacks (MIAs) with respect to dataset properties under the non-differential privacy (non-DP) setting for fine-tuned models. Specifically, it establishes a power-law relationship indicating reduced MIA vulnerability as the number of examples per class increases, based on a simplified regression model. The employed MIA methods include LiRA and RMIA. This is a technically solid paper, supported by multiple experiments that demonstrate the effectiveness of the findings.", "questions": "1. Regarding the scope of the research, what is the rationale for studying the power law of MIA on fine-tuned models? Would this relationship be more difficult to observe if the model were trained from scratch?\n2. For LiRA, does the offline version still exhibit the observed power-law behavior?\n3. In Figure 3, could the authors explain why the power-law trend for RMIA appears less stable compared to that of LiRA?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1.  The methodology is technically sound, and the presentation is clear and well-structured.\n\n2. The employed MIA methods, LiRA and RMIA, are among the current state-of-the-art.\n\n3.  The paper studies both average and individual-level vulnerabilities, providing a more complete picture.\n\n\n\n**Weaknesses**\n\n1. The motivation for focusing on fine-tuned models is unclear. Please refer to Question 1.\n\n2. The theoretical power law is derived from a simplified model and does not directly apply to deep learning models, where the relationship between training data and MIA vulnerability has already been extensively studied. This limits the overall contribution.", "was_revised": true, "final_justification": "The authors’ response has addressed my concerns; accordingly, I have raised my score.", "rebuttal": "Thanks for your review.\n\n> Regarding the scope of the research, what is the rationale for studying the power law of MIA on fine-tuned models? Would this relationship be more difficult to observe if the model were trained from scratch?\n\nWe focus on fine-tuned models for the following reasons:\n- By far the most practical way to get practically usable models\n- Lower bound on vulnerability\n- Allows connection to theory (simplified model is last layer fine-tuning)\n- Fine-tuning is computationally much cheaper, enabling large-scale experiments with a reasonable budget.\n\n> For LiRA, does the offline version still exhibit the observed power-law behavior?\n\nThank you for raising an interesting point. We conducted additional theoretical analysis on offline LiRA, and found that the power-law holds in our simplified model. This result will be added to Appendix and mentioned in the first paragraph of Sec 3. In short, if we assume that offline LiRA scores $t_x$ follow Gaussian distributions (unlike the general location-scale family for online LiRA), we obtain a similar closed-form vulnerability like Lemma 1. This in turn implies the same power-law for offline LiRA. \n\nEmpirically for the deep learning models, the LiRA offline attack is less robust and can be much weaker than the online attack. See Fig. 6 of Carlini et al., 2022 and Fig. 3 of Zarifzadeh et al., 2024.\n\n> In Figure 3, could the authors explain why the power-law trend for RMIA appears less stable compared to that of LiRA?\n\nFirst, we would like to say that RMIA is generally stronger than LiRA when only a few shadow models are used, but we use many shadow models where LiRA is strong as can be seen in Fig. 3 of Zarifzadeh et al., 2024. In that regime LiRA seems more stable than RMIA in our experiments.\n\nWe use the implementation of the original authors of RMIA (in their ml_privacy_meter GitHub repository) and believe that we use it correctly as the majority of seeds is performing well (see Figure 3 in our paper). We are unsure why certain seeds degrade heavily with RMIA, but our setting is different from the RMIA paper:\n- Fine-tuning while they train from scratch\n- We evaluate small datasets"}, {"review_id": "nN1CRVmz5X", "reviewer": "Reviewer_7Yjp", "summary": "This paper provides a theoretical analysis of the impact of dataset size and numbers of classes on the performance of MIA attacks. At a high level, it derives relationships between the number of classes, datapoints per class, and the MIA vulnerability of a fine-tuned model when subject to a LIRA or RIMA attacks.", "questions": "See Weaknesses for areas which could be improved.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n+ This is a large area with lots of empirical results - although the general intuitions exist that 1) larger the dataset (i.e. more examples per class) makes MIA attacks harder and 2) the more complex it is in terms of number of classes the MIA performance improves; this paper formalises this with a theoretical analysis to give a more principled grounding and better predictive ability for pre-deployment useage of models.\n\n+ There is a good selection of different models and datasets (albeit, in the image only domain) showing results across a wide cross section.\n\nWeakness\n\n+ The theoretical analysis holds for fine-tuning, although diverges significantly for models trained from scratch in Fig. 5b, with from scratch models being significantly more vulnerable. Tentative explanations are given, though they are not very rigorous. \n\n+ In particular, this can point to the pre-training dataset having a strong influence on the results - which deviate from the theory. If the theory only follows if the last layer, and last layer only, is fine tuned then this further limits the scope of the results.\n\n+ The analysis focuses heavily on two attacks. The proposed theory may not hold in a general sense to future (potentially stronger) attacks that rely on a different attack approach or principle. It is however, worth noting that the paper highlights that a LiRA based approach is the optimal strategy to carry out MIA via Neyman–Pearson lemma, so a strong attack class is considered here.\n\nOverall\n\nOn the whole, even if just relevant to fine-tuning, it is is a very common deployment scenario and, for now, the analysed attacks are some of the strongest MIA attacks to date. Thus, adding theory to practical MIA effectiveness is a useful contribution.", "was_revised": true, "final_justification": "No major discussion with authors - keeping score unchanged", "rebuttal": "Thanks for your review. \n\nWe thank you for the interesting points you raised for future work and we will add them to the discussion section."}, {"review_id": "O9nq1f7N7D", "reviewer": "Reviewer_Hpo1", "summary": "This paper explores how class size affects data points' vulnerability to MIA in a transfer learning setting. Building on an experimentally observed power law relationship between class size and MIA vulnerability, the authors theoretically derive and experimentally verify this relationship, specifically for fine-tuned vision transformers and ResNet. The paper works to connect MIA vulnerability under these conditions to DP guarantees.", "questions": "Why do you think your prediction model ``underestimates the vulnerability of the from-scratch trained target models'' (lines 295-296)? What could be modified in your theory to better account for this, given that from-scratch training remains an important learning paradigm. \n\nHow is the regime explored here \"between DP and typical MIA evaluations\"? This claim is made in intro, but practically it just seems like this work derives theory for and experiments with non-DP fine-tuned classifiers. DP is only mentioned in lines 316-327 and some results shown in Figure 1. Given the strong claims in the intro about this work bridging MIA and DP, I would have expected to see much more substantial discussion of the interplay between these two methods. \n\nMore work is needed to distinguish the contribution of this paper from other works that have previously identified small class size as a risk factor for MIA (e.g. Kulynych et al, Chang and Shokri, etc.) Can the authors clarify where others' work ends and theirs begins?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- Well developed theoretical explanations of how dataset properties relate to MIA vulnerability. This is an important contribution as the community actively works to understand what makes certain points more vulnerable to MIAs. \n- Thorough experimental exploration of power law results in the transfer learning setting with ViTs and ResNet. \n\nWeaknesses: \n- Claims about DP in abstract are confusing/unnecessary given that the majority of the paper proceeds to ignore DP. \n- Unclear how this result generalizes. The power law demonstrated in Figure 1 is derived in a very particular setting (e.g. fine-tuning on a ViT), and it's unclear if this result generalizes to other architectures and/or learning paradigms such as learning from scratch. Theory suggests it will, but it would be nice to see this backed up with experiments and/or argument. \n- The run-on sentence at the end of the abstract was very hard to parse. There are several other run on-sentences in the paper (e.g. last sentence of 1st intro para). Please fix for readability. \n- Figure 1 needs more explanation. What is C? Are all the different colors representing datasets? Why do you use the term \"shots\"? This may all be explained later in the paper, but the first figure should be interpretable on its own. Add content to the legend accordingly.", "was_revised": true, "final_justification": "The authors' rebuttal sufficiently addressed my concerns. Assuming the promised changes are made to the paper, I am now in favor of acceptance.", "rebuttal": "Thanks for your review. \n\nWe will address the points listed in your questions and weaknesses below:\n- From scratch training: We discuss why it is different based on Feldman and Zhang (2020)\n- Link to DP: Will be made clearer in updated manuscript\n- Novelty over other work: See the response below.\n- Claims about DP in abstract: We mention DP to motivate MIA, rather than make claims.\n- Generalization of power-law: We show it for ResNet and other fine-tuning methods, but believe it will not generalize to “from scratch training”\n- Readability of sentences in abstract, intro and caption of Fig. 1: Will be addressed\n\nSee more details below:\n\n# Questions\n> Why do you think your prediction model ``underestimates the vulnerability of the from-scratch trained target models'' (lines 295-296)? What could be modified in your theory to better account for this, given that from-scratch training remains an important learning paradigm.\n\nWe refer in our related work to Feldman and Zhang (2020) who show that pre-training (=from scratch training) with high utility requires memorizing a lot of training data while fine-tuning appears less vulnerable to memorization. It seems likely that pre-training will keep memorising outliers regardless of dataset size, so unlikely to have a power law. One argument for this is the capacity as fine-tuning has limited parameters while pre-training neural networks usually grow with the amount of pre-training data (as recommended by scaling laws).\n\nWe will amend the related work sentence discussing Feldman and Zhang (2020) (in line 59f) as follows to be more clear about this:\n\n*”Feldman and Zhang (2020) showed that neural networks trained from scratch are required to memorize a significant fraction of their training data to obtain high utility, while the memorisation is greatly reduced for fine-tuning.”*\n\nAbout theory, according to Lemma 1 and the proof of Theorem 2, necessary conditions for the power-law is that $(\\mu_\\mathrm{in} - \\mu_\\mathrm{out}) / \\sigma$ converges to zero at rate $O(1/S^\\alpha)$ with $\\alpha > 0$. In our simplified model, this holds with $\\alpha=½$. However, $\\mu_\\mathrm{in} - \\mu_\\mathrm{out} \\rightarrow 0$ would not always be the case for larger neural networks trained from scratch. Therefore, we do not expect similar power-laws in more general training algorithms. We will add this point after Theorem 2 and the from-scratch training part.\n\n> How is the regime explored here \"between DP and typical MIA evaluations\"? This claim is made in intro, but practically it just seems like this work derives theory for and experiments with non-DP fine-tuned classifiers. DP is only mentioned in lines 316-327 and some results shown in Figure 1. Given the strong claims in the intro about this work bridging MIA and DP, I would have expected to see much more substantial discussion of the interplay between these two methods.\n\nWhile DP gives a universal upper bound of privacy loss in a machine learning algorithm regardless of data, MIA, in contrast, provides a statistical lower bound that is often more practically realistic and depends on the training dataset. In our paper, we empirically and theoretically establish the power-law of MIA vulnerability, which bridges this gap by illustrating how the TPR for MIA of a non-DP algorithm decreases to match the TPR guaranteed by strong DP bounds. Obviously the guarantees are not of the same nature and we are not claiming that our approach can provide similar firm guarantees as DP. \n\nWe reformulate that claim to be more precise as follows (line 19-20):\n\n*”In this paper, we seek to explore MIA vulnerability to extrapolate this gap”* \n\n> More work is needed to distinguish the contribution of this paper from other works that have previously identified small class size as a risk factor for MIA (e.g. Kulynych et al, Chang and Shokri, etc.) Can the authors clarify where others' work ends and theirs begins?\n\nOur novelty compared to prior works is the extrapolation of MIA vulnerability to DP guarantee, enabled by the power-law and the worst-case MIA analysis, both evaluated at a low FPR. \n\nAlthough previous studies including Kulynych et al. and Chang and Shokri demonstrated that classes with fewer examples are more vulnerable to MIA, they did not explore the **rate** of change in MIA vulnerability evaluated at a law FPR as the number of shots per class changes. In contrast, we explore this rate, namely the power-law, which helps estimate how much data you would need to obtain a desired MIA vulnerability. In addition, while prior research predominantly focuses on average-case vulnerability, we also show this power-law holds for worst-case vulnerability. The worst-case MIA is more important in that different samples exhibit different degrees of vulnerability.\n\nWe will reformulate the last sentence of the related work part (line 67-68) as follows:\n\n*“Our work significantly expands on these works by a) explicitly identifying a quantitative relationship between dataset properties and MIA vulnerability, i.e., the power-law in Equation (1) and b) focusing on the worst-case vulnerability, both evaluated at a low FPR. This in turn allows us to extrapolate MIA vulnerability to DP guarantee.”*\n\n# Weaknesses\n> Claims about DP in abstract are confusing/unnecessary given that the majority of the paper proceeds to ignore DP.\n\nWe use the sentence “MIAs complement formal guarantees from differential privacy (DP) under a more realistic adversary model.” to motivate the need for MIA evaluations.\n\n> Unclear how this result generalizes. The power law demonstrated in Figure 1 is derived in a very particular setting (e.g. fine-tuning on a ViT), and it's unclear if this result generalizes to other architectures and/or learning paradigms such as learning from scratch. Theory suggests it will, but it would be nice to see this backed up with experiments and/or argument.\n\nIn Fig. 5 we show that our empirical finding generalizes to a ResNet (5a) as well as fine-tuning with adapters (5b, left). We believe that it does not generalize to “learning from scratch” (see your other question above).\n\n> The run-on sentence at the end of the abstract was very hard to parse. There are several other run on-sentences in the paper (e.g. last sentence of 1st intro para). Please fix for readability.\n\nThanks for your suggestions. We will change the last sentence for the abstract as follows:\n\n*”We show that the vulnerability of non-DP models when measured as the attacker advantage at fixed false positive rate reduces according to a simple power law as the number of examples per class increases. A similar power-law applies even for the most vulnerable points, but the dataset size needed for adequate protection of the most vulnerable points is very large.”*\n\nWe will fix the last sentence of 1st intro para:\n\n*”MIAs assume an often more realistic adversary model with access to just the data distribution and the unknown training data becoming latent variables that introduce stochasticity into the attack. However, the practical evaluation is statistical and cannot provide universal guarantees.”*\n\nWe will fix other run on-sentences in the updated version of the paper.\n\n> Figure 1 needs more explanation. What is C? Are all the different colors representing datasets? Why do you use the term \"shots\"? This may all be explained later in the paper, but the first figure should be interpretable on its own. Add content to the legend accordingly.\n\nWith C we denote the number of classes and the different colors are representing different fine-tuning datasets. We use the term shots as it is commonly used to denote examples per class, e.g., in the computer vision domain. We will update the x-axis of Figure to use examples per class. \n\nWe will amend the caption as follows:\n\n*“We observe a power-law relation between MIA vulnerability and examples per class (denoted as $S$ or shots) when attacking a fine-tuned ViT-B Head using LiRA. Each colored line denotes a different fine-tuning dataset where $C$ specifies the number of classes. The solid line is median and the error bars the min/max bounds for the Clopper-Pearson CIs over six seeds.”"}, {"review_id": "doFaNDd1rC", "reviewer": "Reviewer_AdF2", "summary": "This paper both theoretically and empirically establishes a quantitative relationship between black-box membership inference attack (MIA) success and the number of examples per class for fine-tuned image classification models. Leveraging this relationship, the authors propose a method to predict the MIA vulnerability of datasets under transfer learning.", "questions": "What about other data properties, e.g., variability within or between classes?\n\nI suggest adding a short explanation of how RMIA improves performance even with a limited number of shadow models (Line 122). Do you know why RMIA is more unstable than LiRA?", "limitations": "Yes, but I would like to see a more detailed discussion on the assumptions made on the underlying distributions and how these assumptions influence the conclusions.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:  \n- Theoretical analyses: The authors establish a power law relationship between per-example vulnerability and examples per class for a simplified model of membership inference. They then generalize it for average vulnerability. \n- Thorough empirical validation: The empirical findings confirm the theoretical relationship, demonstrating that average MIA vulnerability can be predicted using the number of classes and examples per class. The authors show that this prediction generalizes to different target model and fine-tuning methods, though it tends to underestimate vulnerability when training from scratch. \n- Discussion of how this observation connects to DP guarantees, concluding that the number of examples per class required for protecting the most vulnerable examples is impractically large \n- clearly written, well organized \n\nWeaknesses:  \n- only dataset properties discussed are the number of classes and examples per class (see questions) \n- would benefit from more discussion around the underlying data assumptions (see limitations) \n- As stated, has limited practical relevance (but I still believe the work provides an interesting perspective that emphasizes the necessity of noise addition in transfer learning to mitigate privacy risks)", "was_revised": true, "final_justification": "The work is thorough and clearly written. The authors addressed the questions of all reviewers satisfactorily. I believe the work provides interesting insights but is not groundbreaking.", "rebuttal": "Thanks for your review.\n\n> I would like to see a more detailed discussion on the assumptions made on the underlying distributions and how these assumptions influence the conclusions.\n\nIn the simplified model of MIA, we assume that data is normally distributed around the class centres. As our empirical findings suggest, we believe that this assumption reflects most of typical well-behaved real-world data. We leave it to future work to study other distributions like heavy-tailed distributions. . We will change the limitations as follows:\n\n*”Furthermore, our simplified model assumes well-behaved underlying distributions, meaning that the data is normally distributed around the class centres. We leave the analysis of other data distributions (e.g. heavy-tailed distributions) to future work.”*\n\n> What about other data properties, e.g., variability within or between classes?\n\nThanks for the suggestion, we believe that exploring other data properties is an interesting direction for future work as there is indeed a difference between classes as can be seen by the errorbars when classes are subsampled in Fig. 2.\n\nAdditionally, we focus on settings in Section 4 where the test accuracy is $\\ge80\\%$ and not on datasets that have classes that are not separable, which could be a further dataset property worth exploring.\n\nWe will add both as a suggestions to future work in our discussion by writing:\n\n*”Our experiments show that there is a difference between classes in terms of vulnerability and an interesting direction for future work is to understand the properties of classes that influence this vulnerability, e.g., variability within or between classes or their separability.\"* \n\n> I suggest adding a short explanation of how RMIA improves performance even with a limited number of shadow models (Line 122). \n\nWe are rephrasing the claims of RMIA paper (their Fig. 3) in our background but do not have any data on this explicit claim as we always use many shadow models for our experiments. \n\n> Do you know why RMIA is more unstable than LiRA?\n\nFirst, we would like to say that RMIA is generally stronger than LiRA when only a few shadow models are used, but we use many shadow models where LiRA is strong as can be seen in Fig. 3 of Zarifzadeh et al., 2024. In that regime LiRA seems more stable than RMIA in our experiments.\n\nWe use the implementation of the original authors of RMIA (in their ml_privacy_meter GitHub repository) and believe that we use it correctly as the majority of seeds is performing well (see Figure 3 in our paper). We are unsure why certain seeds degrade heavily with RMIA, but our setting is different from the RMIA paper:\n- We use fine-tuning while they train from scratch\n- We evaluate small datasets while they use large datasets."}]

Author rebuttal:

Meta-review: This paper did an interesting exploration of MIA in the transfer learning and fine-tuning scenarios. Both scenarios are quite important in the current practice of machine learning algorithms and methods. Thus, the studied topic is important and timely. Some presentation issues were pointed out in the initial reviews, and then the authors addressed them well during the rebuttal. In the updated version of this paper, the authors should focus on the main message demonstration and remove some redundant parts, as reviewers pointed out.

After the rebuttal, all reviewers find the merits of this paper and supported the acceptance.
Decision: Accept (poster)