Learnability and Privacy Vulnerability are Entangled in a Few Critical Weights
Authors: Xingli Fang
Jung-Eun Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Privacy
Generalizability
Weights Rewinding
Fine-Tuning
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Prior approaches for membership privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead to unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this paper, we empirically show that only a very small number of weights are liable to membership privacy vulnerability. However, we also identify that those neurons are not only liable to membership privacy breach but also contribute to generalizability. According to these insights, to preserve privacy, instead of discarding those neurons, we rewind only the weights for fine-tuning. We show that through extensive experiments, this mechanism, plugged into other approaches, shows enhanced resilience against Membership Inference Attacks while maintaining utility.

Published as a conference paper at ICLR 2026
LEARNABILITY AND PRIVACY VULNERABILITY ARE ENTANGLED IN A FEW CRITICAL WEIGHTS
Xingli Fang & Jung-Eun Kim'
Department of Computer Science North Carolina State University Raleigh, NC 27695, USA {xfang23, jung-eun. k kim)l encsu. edu
ABSTRACT
Prior approaches for membership privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this work, we observed three insights: privacy vulnerabil- ity exists in a very small fraction of weights; ii) however, most of those weights also critically impact utility performance; iii) the importance of weights stems from their locations rather than their values. According to these insights, to pre- serve privacy. we score critical weights, and instead of discarding those neurons. we rewind only the weights for fine-tuning. We show that, through extensive ex- periments, this mechanism exhibits outperforming resilience most cases against Membership Inference Attacks while maintaining utility.
INTRODUCTION
Membership privacy risks of machine learning models arise from models' behavioral discrepancy between training and non-training data points. Leveraging such discrepancy, attacker can dis- criminate membership information whether a data point was used for training the victim model Shokri et al. (2017). This attack model is called membership inference attacks (MIAs). Existing studies Carlini et al. (2022b); Ye et al. (2024) pointed out that some data points are more privacy- vulnerable than others. Li et al. (2024)
focusing on these data points. However, suggested that better privacy-utility can be achieved by training on the model-ent -edeeeedddd still in a black-box stage. On the other stream of work, early studies Frankle & Carbin (2019); MMlchanov ct al.] (2019)] Lee et al. (2019) have shown that a subnetwork existing in a neural network can achieve competitive performance. identifying that only lesser fraction of weights contributes the model's utility. These prior studies collectively motivate us to raise a reflective question: Do there exist only some weights whose updates lead to privacy leakage of learning models?
To locate them, we first propose a weight-level importance estimation based on Machine Unlearning (MU) to measure fine- -reiiegaidd privacy vulnerability existing in neural networks. With our approach, we find that weights that cause the model to be privacy-vulne vulneeabee are only present in a small frac- tion of the weights. Moreover, we observe that a large portion of these weights overlaps with the learnability-critica weights. It explains why Yuan & Zhang 2022) fails to mitigate privacy risks using general pruning techniques.
One of our very important observations is that the importance of weights-in terms of accu- racy-stems from their locations rather than their values. As long as the most critical weights (the proportion can be even down to 0.1%) remain in the model-i.e.. are not pruned or removed and rewind them in their initial values, the model can recover its accuracy even when these weights left unupdated after retraining fine-tuning. Building on top of these insights, design fine-tuning strategy that curates only privacy-vulnerable weights. To the best of our knowledge, our approach is the first perform nembership-privann privacy-oriented fine-tuning at weight-level granular- ity. Through comprehensive experiments against modern membership inference attacks, LiRA and
"Corresponding author

---

Published as a conference paper at ICLR 2026
Carlini al. (2022a); Zarifzadeh et al. (2024), demonstrate that, in terms of privacy-utille tradeoffs, our strategy outperforms existing defenninn methods that train machine learning models even from scratch. We emphasize the following
insights that we identified through this
paper: (i) Privacy vulnerability exists in a very small fraction of weights. (i) However, most of those weights also critically impact utility performance. (iiè) The importance of weights stems from their locations rather than their values.
2 PRELIMINARIES AND RELATED WORK (MORE CONTINUED IN APPENDIX)
In this section, we introduce fundamental background knowledge regarding Membership Inference Attack, and prior studies regarding Importance estimation of components in neural networks. Due to page limitations, further related work concerning Membership privacy preservation methods and machine unlearning is presented in Appendix
2.1 INTRODUCTION MEMBERSHIP INFERENCE ATTACKS
In our study, we focus on membership privacy on classification tasks. In Membership Inference Attacks (MIAs), the attacker's goal is to determine whether a given sample was part of the training dataset of target (or victim) model. Formally, consider a target model, [[; 0) CCs - Roos, where Cin is the input dimensionality and Cout is the class count of the task. membership infer- ence attack can be formulated as
A: f(= + {0,1),
(I)
where A is binary classifier that outputs 1 if the sample is inferred be member of the training set of (+)00) and () otherwise. The design of the attack function A depends heavily on the attack
strategy. neural network (NN)-based MIAs Shokri et al. (2017): [allm et al. (2019). A itself is machine learning model trained on the predictions of the target model. In contrast, in metric-based approaches (e.g.. threshold based MIAs) Song & Mittal (02T); Del Grosso al. (2022); Carlini et al. ( Leemann et al. (2023); Zarifzadeh et al. (2024), A is defined by manually specified function that computes certain statistics (such as confidence scores or loss values) and compares them against a threshold, typically chosen using auxiliary techniques such as shadow models Shokri et al. 2017); Carlini et al. (002aa).
2.2 IMPORTANCE ESTIMATION OF COMPONENTS IN NEURAL NETWORKS
The importance estimation of components in neural networks has mainly been studied in the context of model pruning. Frankle & Carbin (2019) observed that the potential of weights can be determined, in terms of generalizability. once the model is initialized. Lee al. (2019): Molehanov et al.) (2019) made use of weight gradients in searching for subnetworks with comparable generalizability to the original model. Liebenwein al. (2021) explored possible loss beyond generalizability pruning. Ye et al. Sehwag et al. (2020) explored how to prune neural networks in the adversarial envi- ronment. Tang al. (2020) assessed the reliability importance of neurons by aligning spurious and clean samples through learnable masks. Frankle et al. (2020) observed that weight rewinding helps fine-tuning of extremely sparse models. Renda et al. (2020) found fine-tuning with rewound weights usually outperforms direct (a.k.a, in-place) fine-tuning, Gadhikar & Burkholz (2024) analyzed the factors why learning rate rewinding. along with weight rewinding. recovers utility better. Tran et al.
(022) found that models suffer from faimess deterioration after pruning. [Wang et al. (2023) com-
puted connectivity importance via the influence on the spectrum of the neural tangent kernel (NTK) benefit from magnitude pruning- Sun et al. 2024) applied activation into importance estimation based on the characteristics of large language model. Ye al. (2025) proposed training free importance estimation and pruning on foundation models. Our work is distinct in that we identify privacy-vulnerabiiity of weights.
Jacot et al. (2088) Jia et al. (2023) found machine unlearning
3 MOTIVATION: REMOVING UNIMPORTANT WEIGHTS IS INEFFECTIVE FOR PRIVACY
One of the fundamental weight/neuron importance estimation methods is Taylor First Order (TFO) Molchanov al. (2019). The method estimates the global weight importance via magnitudes of
2

---

Published as a conference paper at ICLR 2026
gradients and weights, which is formulated as follows:
S = (
Σ
Ig Je
de Datr
where S denotes the set of importance scores of weights in the evaluated model, s; denotes the importance score of the weight, Wa, Wi,d denotes the value of the i-th weight of the model before updating with the data point d, 9i.d denotes the i-th weight's gradient computed under data point d, sttr denotes the randomly selected subset of training data Der (i.e., D.rr Dtr), and m denotes the number of weights the model contains. In TFO, the approach usually accumulates the scores in tens of iterations along with the model update each turn of filter removals of the model. Although the TFO groups weight scores into their belonging filters/neurons ultimately for filter/neuron pruning, we use the primitive weight scores for one- shot weight-level pruning.
In detail, to identify the most critical weights, according to the importance estimation method, we prune out the least important weights in one shot instead of iterative and gradual removal as in the original TFO. Figs. Ia and [b exhibit that, even in the very high sparsities, accuracy is maintained, but privacy vulnerability does not improve. Also, at times, the model becomes more vulnerable after pruning, evidenced by the increase of the testing loss of 90% spar- sity from 0%
2.0
Arsuracy
1.5
0.50
1.0
CE
0.25
Train
0.5
Train
Test
Test
80
90
Sparsity (%)
Sparsity (%)
(a) Accuracy w/ fine-tuning
(b) CE loss w/ fine-tuning
(non-pruned) as shown in Fig. [b] and also the observation by Yuan & Zhang 2022) that MIAs on some pruned mod- els become more successful. Overall, these ob- servations lead us to conjecture that,
Figure According to TFO, important weights are pruned over different sparsities. The results are shown on ResNet18 and CIFAR-100
Conjecture:
The performance impact and privacy vulnerability are entangled and exist in very small number of weights.
An intuitive way for verifying this conjecture show a correlation between privacy vulnerability and performance impact. For the goal, we distinguish the traditional estimation of how to maintain utility performance from the estimation of privacy vulnerability. We here refer to the importance estimation for utility performance (i.e., accuracy) in the common pruning techniques as learnability while we refer to how privacy-vulnerabee a weight can become as privacy vulnerability. In the next section, we first propose our approach to estimate privacy vulnerability. Then, the entanglement issue of learnability and privacy vulnerability empirically shown, and we discuss how to solve it.
PROBLEM SETUP AND METHODOLOGY
4.1 PRIVACY VULNERABILITY ESTIMATION
Membership privacy vulnerability is mainly due to the behavioral disparity between member and non-member data. Hence, the intuition of our approach is to determine critical weights of the model that exacerbate the discrepancy between the two prediction distributions preserve privacy. To achieve this goal, we make use of the concept of machine unlearning Bourtoule et al. (202T) to design mechanism let the model learn member data while unlearning non-member data, respectively.
Our privacy vulnerability estimation approach (Fig. 2bb consists of unprotected model, Map: vanilla model, M member set, Drr; and non-member set, Dre. The Drr is the set on which the Map trained. The non-member set, Dre. is held-out set of data points that the Mup has never seen during training. and it is also disjoined from the testing data in the evaluation phase. The two models, Mup and Men. are in the same structure, f(:;6), but with different parameters, 0up and 0.n. respectively. Oup are pretrained on training data Drr while 6 are the values at initialization before being trained on D
3

---

Published as a conference paper at ICLR 2026
For member data, Detr, we force the model to minimize the loss as much as possible. In contrast, for non-members, D Derc. encourage the predictions close to the vanilla model, Mon, rather than ground truths. This process can be formulated follows:
grad
Unprotected Model
Train
leam
(a) Existing learnability estimation
arg min (a.n) Loo Mup)] Bup
grad
Unprotected Model
learn
E Br [21(22 Map Man)]
(3)
grad
un leam
where Lee denotes the cross-entropy loss function, and Cst denotes Kullback-Leibler (KL) divergence Csiszár (1755);
Vanilla Model
Ref
Inference
Hinton et al. (2015). Through this process (Eqq.) 3). the model tries to learn information that only effective for recogniz- ing member data points that it maintain low loss the train set when unlearning the non non-member set, which does not contribute the privacy vulnerability of the model since the data points are all non-member. In details, we fine-tune the unprotected model, Map. using the following objective function:
(h) Our proposed privacy vulnerability timation
Our approach takes into account privacy vulnerability for im- portance estimation, while TFO only measures learnability for accuracy.
Figure 2:
Live = (1- (f Oup), ytr) + (f f(rei 0on))
(4)
where (fer and Tre are mini-batch samples randomly sampled from Dtr and Dre. respectively; À is hyper-parameter to balance the learning and unlearning losses so that the fine-tuned model can maintain accuracy on D,r while losing accuracy on Dre as much as possible. The final privacy vulnerability estimation function is the same Eq. 2 but with these aforementioned processes and constraints. It accumulates the weight-level importance with respect to privacy vulnerability, via gradients and magnitudes each step, along with the update of Oup-
4.2 LEARNABILITY AND PRIVACY VULNERABILITY ARE ENTANGLED
To verify our conjecture in Sec. 3 we visualize the weight-level privacy vulnerability scores and learnability scores Fig. 3 and quantify their correlations Tab. [ with two architectures: ResNet18 8 He et al. (2016) and VIT Dosovitskiy et al.] (202I). Shown by the charts for all train- able weights (the leftmost column) in Fig. B most of the weights are neither privacy-vulnerable nor cimmabblity-critican which aligns with the experimental results in Fig. T
It tells again that pruning
Weights
Layers Weights
Layers Weights
Layers Weights
1.53
1.22
1.29
0.50
0.22
0.35
0.25
0.30
0003
0.00
60000
01032
(1001 100 5000)
0.000
0.060 0.004
5000
0000
5/002
5101 1155 0L008 Abtllty
0,003 0.000 3.00 E000 (1004
(a) ResNet18
All Trstnoble Weights
AI1+MLP Weights
Nermakzaiton Layers Weights
10
0.25
PaNcyy (Mternkiy
0.29
015
0.8
0.39
00
0.08
Teaming
Taarring dailty
Laaning Abitb
Lawitnng
(b) ViT
Figure 3: The visualization of weight-level learnability scores and privacy vulnerability scores. Privacy vulnerability and accuracy are significantly correlated and this correlation varies in different components. Due to the significant scale discrepancy, the ranges of axes of the four charts in ViT are not consistent. (The same data points as Tab
4

---

Published as a conference paper at ICLR 2026
ceamability-noncritiaaa (not critical for accuracy) weights does not remove the privacy risks (predic- tion discrepancy).
The other weights, much fewer than these non-critical weights, can be categorized into three types: learnability-critiee and both. Tab. shows the Pearson correlation coefficient between privacy vulnerability and learning ability. We find that the results of the two architectures are consistent that the correlation in normalization layers (batch normalization loffe & Szegedy (2015) in R8 and layer normalization [Ba al. (2016) in ViT) are the lowest while the cor- relation in main components of the models (convolution layers in ResNet18 and Attention & MLP layers in ViT) are the highest. Weights belonging normalization layers occupy only a tiny propor- tion of weights-less than 1%. However, some of them are the highly privacy-vulneraby weights of the models as shown in the charts of normalization layers weights (the 3rd column) in Fig. 3 Since these weights are also critical for learnability (many weights in normalization layers exhibit high learnability scores), pruning them by common pruning techniques will impair the performance.
Moreover, the majority of the weights be- long to convolution/ateentio//LLL layers, and they show strong correlations-over 0.9 in Pearson correlation privacy-vulnerabiiity and Icarnability (see Tab. D The correlations are significantly higher than normalization layers. This re- sult indicates that many privacy-vulnerable weights are also crucial for learnability. In addition, compared to CNNs, transform- ers exhibit higher privacy vulnerability (see charts of convolution layers weights and Att+MLP layers weights (2nd column in Fig. 3D), which is also supported in part by the observation of Zhang et al. (2024) that at- tention layers lead to worse privacy risks.
Table The correlation between privacy vulner- ability and learnability in two architectures. PCC denotes Pearson Correlation Coefficient. Att+MIL denotes the weights of the attention layers and MLP layers in transformer blocks. (The same data points as in Fig. 3b
betuuiient-etteer
Model
Weight Type
PCC
Proportion
All
0,8329
100.00%
Conv
0.9410
99.50%
ResNet18
Linear
0,8096
0.45%
Norm
0,6776
0.05%
All 0.7667
100.00%
Att+MLP
0.9068
99.39%
VIT
Linear
0.8642
0.54%
Norm
0.7336
0.07%
Finally, the linear layers in Tab. denote the last few linear layers. We find that most
weights in them are not privacy- vaavacc--uunerable, while some of them could be leamabilit-crrticaal
In summary, most privacy-vulnerable weights impact learnability (utility performance). This is the fundamental reason why the existing standard pruning techniques fail to effectively reduce pri- vacy risks. To address this issue, we propose Critical Weights Rewinding and Finetuning (CWRF) in the next section to promote the model to achieve better privacy-accuran trade-offs.
4.3 CRITICAL WEIGHTS REWINDING AND FINETUNING (CWRF)
Our approach (CWRF) consists of three steps: (i) estimating privacy vuinerability, (ii) rewinding & freezing privacy-vulneable weights, and (iii) fine-tuning the other weights with privacy-presee preservation training approaches. Since privacy vulnerability estimation has been elaborated in Sec 4.1 we start
our discussion from the second step.
Weights Rewinding. Weights rewinding Renda et al. (2020); Frankle et al. (2020) strategy that rolls back weights earlier values in training. In approach, the weights rewound the initial status, at which point the weights are privacy-safe because no data has been exposed to the model. Once calculating the privacy vulnerability estimation scores Syne in the way described in 4.c
two masks for weights rewinding and fine-tuning can be produced as follows:
B, [1[$ 2 ([Spue:))]B.sSs,
=
(S)
where B, denotes weight rewinding mask, B, denotes weight freezing mask, II(-) denotes indicator function, Q(--)) denotes the combination of sort function descending order and quantile function, and denotes the predefined rewinding rate we opt to. After producing the masks, a portion of the weights of the trained model is rewound from Oup to 8yn (defined in Sec 4.1 as follows:
Orw B3 ② Oup + B, O.n
5

---

Published as a conference paper at ICLR 2026
Algorithm 1: Pseudocode of CWRF
Input: Unprotected model Mup with parameters Bup. vanilla model Men with parameters Own member (train) set Drr, and non-member (reference) set Dre. batch size B, privacy-preserving training approach P, the number iterations for score estimation T, the number of fine-tuning epoches E, the learning rate for estimation Te. the learning rate for fine-tuning ft--
Result: Privacy--inee fine-tuned Map with parameters Oup
Initialize {0, = 03 which are corresponded weights of Oup
Copy unprotected model, denoted as M with parameters dup
for i 1...7' do
Get sample batches {(x}", ,"""",,""" -,("))== C Der and {(af", )) Dre
Forward and compute loss Spee (M (xt") vt", M Maplat (xt")) Men (xn()
(For Lyees refer to Eq. 4 Approximate gradient I
Vol fpee
Compute scores @ + Φ + IIA upl (refer to Eq. 2
Update unprotected model 8 6up neI
10 end
11 Get the two masks B, = [T[s; 2
Eppp B3=1- -1-B. B,
(refer to Eq.
12 Rewind the unprotected model up t By 3 up + Br @ Pyn
(refer to Eq. 6
13 for epoch
do for i = 1 K do
14
15
(K denotes the number of mini-batches)
16
Get sample batches {dp- = (x v ") C
Drr Dtr
(Some preserving approaches may additionally require reference data)
Train the unprotected model with privacy approach P (MMup d(")
19
Approximate gradient < Ve.p? P
20
Update the model Mup with masks Oup + Oup - IIIBB
(refer to Eq.
21
end
end
where @ denotes Hadamard product and Orw is the updated weights with partially rewound weights after the two masks are overlaid. After rewinding, the most privacy-risky weights can return to being privacy-safe. However, due to entanglement between privacy- vulnerbbiinr and learnability, the rewinding also leads to the utility deterioration of the model. More precisely, usually leads to lenoo--ueess-leell utility. Hence, the model needs be fine-tuned recover its utility.
Weights Freezing & Privacy Fine-Tuning The final step is fine-tuning the model to achieve better privacy-utility trade-offs. consists of two parts: Weights freezing & privacy fine-tuning. For training 0rw preserve privacy, can plug in any privacy-preserving approaches and train the model. Note that the approaches need to train the model from scratch, but by being plugged into method, they only require partial weights be rewound and frozen, and then the rest of the weights are fine-tuned. From the perspective of implementing weight freezing, masking the gradients is a sensible option stop the update of the non-rewound weights. Given the gradients, Gp. obtained by the preseevvig training approach with the rewound weights, frww at each fine-tuning iteration, we can filter out the gradients of the frozen weights so that only the rewound weights can be updated:
Gp + B, B o
During the fine-tuning process, we do not train model at a fixed learning rate because neither too small or too large fixed learning rate is good at recovering the model from random guess status. Instead, the learning rate is also rewound to the earliest learning rate at which the model started.
The way is similar to learning rate rewinding (LRR) Frankle et al. (2020); Gadhikar & Burkholz 2024), although we rewind the learning rate to the very initial one. The self-contained procedure of CWRF is described in Alg. [ The CWRF contains three stages: (i) scoring privacy vulnerability, (ii) rewinding and freezing privacy-vulnerable weights according scores, and (iii) fine-tuning the

---

Published as a conference paper at ICLR 2026
rest of the trainable weights with privacy- preeeeriigg approach. CWRF can adapt arbitrary privacy training approaches by plugging them into the third stage of CWRF for srraacypoot-training We note that it might be somewhat counterintuitive fine-tune the privacy-invulnerable weights rather than the privacy-vulnerable. There are two reasons why the model fine-tuned that way: (i) the privacy risks of the privacy-vulnerabe weights have been fully removed thanks rewinding. Fine- tuning the rest of less- or in-vulnerable weights help the model with further mitigation of privacy risks. (ii) based on our hypothesis and empirical investigation elaborated and explained in Sec. [.44 fine-tuning invulleallleee weights help the model recover its utility better than doing that on privacy-vulnerable weights. We explain this in detail in the next section.
4.4 THE PRIVACY- VULNERABLE WEIGHTS ARE UNNECESSARY TO BE TRAINED
Finally, we explain why we fine-tune the privacy- invulneable weights rather than the vulnerable. The lottery hypothesis Frankle & Carbin (2019) proposed and validated that the learnability of weights in neural network is determined at the initialization phase. Moti- vated by the insight, we propose and validate a hypothesis in this section:
Test Accuracy
0.4
0.2
Hypothesis: The learnability of a weight in a neu- ral network is determined by its position rather than its value (magnitude & sign.)
0.0
M1
M2
M3
Model
Figure 4: The performance of M1. M2, & M3 on ResNet18 & CIFAR-100.
This be observed and understood through model pruning. For the verification, we devised three models:
MI: unpruned model trained from scratch.
M2: 85% pruned model from MI and then rewound to the initial values and retrained.
M3: 85% pruned model from MI with no mee-tuning/retarinnngg
M2 and M3 pruned with the same masks based MI.
Their comparisons are shown in Fig. 4 Let us focus on the saamability-unimortaa
Test Test Accuracy
weights that are pruned away in M2 and M3.) By looking at the almost same final accuracy MI and M2, we can infer that MI the
present in M1 (which
0.6
0.4
(Ours)
learnability- ceamaabllty-uuiiportaan weights shared knowledge and role with weights. This is also checke
0.2
the learnability by the accuracy drop of M3 (from M1) where the learnability- unimportant are discarded. hints the potential of the pruned weights (which were regarded as not important for learnability, though) toward learnability to some extent. Overall, encour- aged not update cammaailiit--mmortan weights by the Hypoth- weights by Fig.
0.0
0.1
1.0
3.0
5.0
Portion (%)
Figure 5: The performance of A1, A2, & A3 along with moving/rewidding ratios. The dotted line represents baseline performance of a model trained from scratch with the same privacy-preservin approach
esis, but to finetune On top of that, by considering that vuunealee weights weights, we only rewind
are entangled with learnability- the privacy-vulnerable weights as not hurt the accuracy, but fine-tune only privacy-invulerrbbe weights not to expose the privacy-vulnerable weights the data again reduce privacy risk.
Based on the insights, to verify the hypothesis and validate our approach, CWRF, we compare the following three approaches:
Al: Remove privacy-vulnerable weights & fine-tune privacy-invulneran weights;
* A2: Rewind privacy-vulnerable weights & fine-tune privacy-vulnerable weights;
A3 (CWRF): Rewind privacy-vulnerable weights & fine-tune privacy-invulnerabl weights.
As for srivacy-preserving training, here we apply RelaxLoss Chen et al. 2022 to fine-tune the three approaches. Shown in Fig. 5 it is very clear that discarding privacy-vulnerablee
weights (A1) leads
to unrecoverable accuracy crash for the model, unlike the cases of A2 & A3. The performance discrepancy stems from "removing" (A1) vs. "rewinding" weights (A2 & A3). That is because
7

---

Published as a conference paper at ICLR 2026
removing alters the locations of the weights, but rewinding does not. This comparison successfully validates our hypothesis that the locations of weights are of paramount importance for learnability. As long the crucial locations in the model are retained, the model pre- serves the capability recover its curacy. Another point to pay atten- tion to is the performance gap between A2 and A3. By retaining the locations of privacy-vulnerable weights (A3), the model can recover its accuracy when very small portion of privacy-vulnerable weights are rewound, and it even outper- forms the baseline model that is trained from scratch using RelaxLoss with the same training configurations except for epochs. As for privacy-related information, Tab. displays the model's prediction loss distributions on train and test set at various configurations. It exhibits that CWRF (A3) shows significantly bet- ter loss gap compared to A2 and the model trained from scratch, especially at portions of 3.0% & 5.0% while they are at the same testing accuracy at these ratios. Overall, it tells us that fine-tuning on privacy-invulnee invulnerable weights (A3) has less negative impact on the testing distribution compared with A2 (fine-tuning on privacy-vulnerable weights.)
Table 2: The Cross-entropy loss after fine-tuning with a privacy-preserving approach, according to the portion of rewound weights.
Approach
0.1 %
1.0%
3.0%
5.0%
A2 train
1.2268
0.8570
0.4326
0.4619
A2 test
1.3797
1.2728
0.9288
0,9610
A3 train
0.1502
0.3376
0,4473
0,4815
A3 test
0.7720
0.7433
0.8044
0.8330
From scratch train
0.8087
From scratch test
1.5398
5 EMPIRICAL STUDY
5.1 EXPERIMENTAL SETUPS
Datasets. We evaluate defense approaches on three datasets: CIFAR-IO & -100 Krizheezyyyyky et al. 2009) and MANIC-10Darloon et al. (2018). CINIC-I0 contains 270, 000 images, evenly distributed into training. validation, and testing subsets. The size of the images in the CINIC-10 is resized to 32 x 32, which the same as the CIFAR datasets. some data points from the training data, which are disjoined from the data points used for train- ing the specific single model. More details regarding sampling are described in MIAs' setting in Appendix B
all three datasets, we randomly sampled
Models. To adequately evaluate our approach against compared approaches, two commonly used architectures, ResNer18 He al. (2016) and Vision Transformer (ViT) Dosovitskiy al. (202T), are used in the experiments. When evaluating with ResNet18, we adapt the model configurations designed for the CIFAR datasets the original paper. As for VIT, the inputs of images are divided into patches of 4 x 4, which is smaller than the VIT designed for the ImageNet dataset Deng et al. (2009) in the original paper.
methods against membership inference attacks, two recent MIAs techniques, Likelihood Ratio Attack (Li RA)CCrlii at. and Robust Membership Inference Attack (RMIA) Zarifzadeh et al. (2024), are adopted in our defense evaluation. In addition, the strategy of adaptive attacks Song & Mittal (202I) is applied to all MIAs to rigorously evaluate the defense approaches. We evaluate the model's 's reliance ability against attacks along two metrics: (i) AUC and (ii) TPR at low FPR. Specificly, the TPRs at 10-3 and 10-5 FPRs are reported in our paper. More details of attacks are elaborated in Appendix B
Attacks. To show the superiority of our approach in boosting
Defenses. To verify the universality of our approach, we provide extensive comparisons with four privacy-preserving training approaches: Differentially private stochastic gradient descent (DP-SGD) Abadi et al. (2016). relaxed loss (Re axxosss) Chen et al. (022). High accuracy and membership loss (CCL) Liu al.) (2024), and privacy-aware sparsity tuning (PAST) Hu et al. (2024) are deployed to train the models against MIAs. We adopt the implementation DP-SGD provided by the Opacus library Yousefpour et al. 2022) while adopt the official implementation of other defense approaches. Due compatibility issues between DP-SGD, Batch Normalization, and Dropout techniques, DP-SGD is only applied to VIT. In addition, since we compare the model's 's internal privacy-defense ability, the training part of HAMP is deployed when we use it.
privacy (HAMP) Chen Pattabiraman (2024). convex-concatl
8

---

Published as a conference paper at ICLR 2026
General Configurations. Adam optimizer Kingma & Ba 20155 is applied to train all models. We
set the apper-paramtters 31 = 0.9, 32 = 0.999 and the weight decay to x 10 10-4 For the learning rate, we train the model by setting the initial learning rate
10 and changing the learning rate along steps with the cosine annealing scheduler Loshchilov & Hutter (2017) The batch size and epochs of all tasks training from scratch are set to 256 and T00, respectively. As for defenses, follow the original paper's hyperparameter settings for each approach that we compare with. As for attacks, eight shadow models, including four 'IN' models and four 'OUT" models that are required by LiRA and RMIA, are deployed for both attacks. We report all results in three independent runs. As for the experimental environment, some important information of the computation device is listed as follows:
CPU
GPU
RAM
OS
CUDA
Python
PyTorch
AMD Ryzen 7 7700X
NVIDIA GeForee RIX S090
6-1 GB
Ubuntu 24.04 LTS
12.9
3.12.3
2.80
Customized Configurations In our approach, on the privacy vulnerability estimation stage, 30 iterations and 256 mini-batch size are applied. The À is set 0.7 for CIFAR-I0 and CINIC-10 while it is 0.9 for CIFAR-100, As for fine-tuning epochs, we set it to 40 with the same initial learning rate using in training from scratch. The same learning rate scheduler is also applied. We perform grid search to select the rewinding rate E [1%, 10%] in all experiments shown in the main context.
5.2 CWRF (OURS) WITH VARIOUS PRIVACY-PRESERVIA APPROACHESS
In CIFAR-10, we report results with both ResNet18 8 and VIT in Tab. In the evaluation of ResNet18 three approaches, RelaxLoss, HAMP and CCL are all effective in exhibit that our approach successfully improves the models' resilience against SOTA MIAs by plug- ging other privccytrrining approaches. Especially, approaches with CWRF all achieve significant mitigation of privacy risks under LiRA. However, under RMIA, the combo of RelaxLoss and CWRF suffers from some slight increase in privacy risks. This is to some extent due to the instability of solely deploying RelaxLoss the significantly higher variance of test accuracy. With such instabil- ity, the shadow models of RMIA become harder to model the target model's behavior. As for ViT, the performance of CWRF becomes even better: combining with all four spproaches-DP-SGD. RelaxLoss, HAMP, and CCL, CWRF shows most effective improvements in reliance against the attacks while, in some instances, the testing accuracy becomes even better (DP-SGD + CWRF).
The results
Table 3: The performance of four seracy-preservation approaches with and without CWRF (Ours) on CIFAR-10. Higher is better in test accuracy (1) while lower is better in Privacy (L).
LIRA (1)
RMIA(L)
Model
Defense
Test Acc. (%, t)
TPR % PEF
AUC
AUC(%)
0.1%
0.1%6o
0.1%
0.1%oe
No Definse
79.44
a5.00
2.18(0.38)
1.78(0.3))
74.76(1.59)
5.88 (0)
3.90
RelaxLoss
(2.51 21.)
(.38
(o.2()
66.60(1.07)
0.52 34)
(.12
CWRF (Ours)
(0.
68.31
(0.00)
0.03 (o.3)..05)
68.18(1.68)
1.22(0.97)
(.27(0.19)
ResNet18
HAMP
77.79
79.71.0.20;
3.33 (o.73)
(.80
80.07,0.08)
(0.08)
7.28(1 64)
1,93 28)
CWRF (Ours)
(1.43
77.96(0.13;
(.53(0.58)
0.07 (007
80.26(0.413
4.30(1 as)
1.66(o.6))
CCL
79.56(0.58)
83.95 (o.30)
(.50
(.80(..01)
76.04;o.50)
4.23ç0.84)
(.22(1.85)
CWRF (Ours)
(7.77
(4.82
(.22
0.10
(o.23
2.80(0-45)
(.93
Defense
(6.45(o.se)
82.88
(100().1))
(.92
84.44
1.52 81)
0.45 (2.)))
DP-SGD
(7.65
(4.97
(..
60.86;o.18) (.17
0.23 16)
0.18 08)
CWRF (Ours)
(0.45
55 (0.68
0.13
(.00
60.46;1.00)
0.13
(.03(0.05)
RelaxLoss
(0.75 (7.
(.38
0.37 (o.1))
72.87
(.85
0.23 23)
CWRF (Ours)
55.88 50.88 (6.82
(2
0.03 (.03
63. (3..3)).77))
0.38 31)
0. (o.1).
VIT
HAMP
50.53/o.41; 01.
0.07
(0.00.
54.42(o.58) (.00
0.27
0.05 (0.04)
CWRF (Ours)
32.50
50.15
0.05 (.05
(.00
51.50
0.13ço
0.02
CCL
54 54.25(o.11)
52. 18
0.02
0.00
(6.33
0.12(0.08)
0.00
CWRF (Ours)
63.45(0.65)
(1.68
(.00
0.00
(1.32
0.07 06)
0.00
PAST
54.84 54.66/0.57)
54.30
(.
(.08
62. 62.99 62.99(1.49)
0.97 25)
(.25
CWRF (Qurs)
53.86
(.
(.08
62.10(o.o8)
0.68
0.22

---

Published as a conference paper at ICLR 2026
Web CWRF
wi OWRF 1Ours)
Geseraliralily
LIAA
LIA
LIRA
RMIA
RMBA
RMAA
Accurway (80
2
AUG (35)
AUCCOO
80
60
RS1.
HAMP
CCL
HANP CC1
ROL.
HAMP GCL
Rud.
HAMP
GoL
RXL.
HAND
Rad.
HAMP
RaL
HAND
Approuch
Approuch
Apgerach
Approach
Appeoach
Approach
Appronus
(a) CINIC-10
Accunacy (8)
Genernlizabiity
1.00
LIRA
TPRM8000 NPPPR
LIRA
TFRDS/B0 0015TPR.
EJRA
100-
RM14
TPRrss vag...o...... NNPP
RM1A
30
RMIA
AUC 1900
Lo-
A220 150
40
to
20
10
20
t0
Rat.
BAMP
Rdl.
HAMP
GOL
Rdl.
HAMP
Rd.
BAMP CCL
Red.
8AMP
GoL
Mal.
HAMP Gc2
Rdi
HAMP
COL
Approach
Approach
Appreach
Approach
Appmach
Appeoarh
Approsch
(b) CIFAR- 100
Figure 6: The performance of ResNet18 trained with three privacy- senaacy-pessevvatoon approaches with and without CWRF (Ours). The dotted line represents baseline performance of model trained from scratch with regular training approach, Cross-Entropy.
CINIC-10 has more data points, thus showing more stable trends (see Fig. 66 Considering the utility-priaacy tradeoffs, the best combo is HAMP with CWRF: it shows not only significant ad- vance in test accuracy-even substantially more than the undefended model-but also best privacy resilience against both attacks. However, the CCL is not fully effective under RMIA, the perfor- mance becomes worse in terms of AUC and TPR when FPR is fixed at 0.1%. After the addition of CWRF, it becomes further worse in RMIA,, while the privacy risks mitigated under LiRA. In RelaxLoss, training with CWRF helps the model stably improve its generalizability and privacy.
In CIFAR-100, the results Fig. 6b -vary a lot due the more difficult task, but limited training samples. We note that the model solely trained with HAMP fails to converge. In contrast, the model can achieve better utility when it is trained with both HAMP and CWRF As for CCL, the trend is consistent with that in CINIC-10. These results explain that our approach can definitely boost the privacy-preserving approaches only when the approaches can be effective against MIAs. As for RelaxLoss with CWRF. it shows stable improvements in both generalizability and privacy. In addition, in the evaluation of LiRA with 128 shadow models (discussed Sec. C. in the appendix), CWRF shows the consistent advantages by combining each of the three approaches.
In summary, when the applied privacy- presevvii approach is effective in the specific situations, our approach, CWRF, can always boost it to achieve better privacy-utility tradeoffs. We also empha- size that our approach can assist the stability of privacy-pres siracc-preseeving training by stabilizing testing accuracy variance through multiple independent runs and avoiding model collapse.
6 CONCLUSION
We design a method to estimate weight-level privacy vulnerability. By exploring the correlation between privacy vulnerability and learning ability, we explained and showed why general neural network pruning is not effective in eliminating model privacy vulnerabilities in previous studies. Throughout this paper, we found that privacy vulnerability exists in very small fraction of weights entangled with learnability. We also recognized the importance of weights stems more from their locations rather than their values. Based on those insights, we propose strategy to mitigate member- ship privacy risks of the model that rewinds partial privacy-vulnerable weights and freezes the others, and then does that our strategy achieves more effective balance between accuracy and privacy than directly ap- plying existing mrivcc-ppresevvng methods that train from scratch.
fine-tuning. Through comprehensive experiments, we demonstrate
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016. URL
https://arxiv. http://axxxx..o.....////// /0wp./10..... 06450
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP), pp. 141-159. IEEE, 2021.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463-480. IEEE, 2015.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Mem- bership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897-1914. IEEE, 2022a.
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 13263-13276. Curran Associates, Inc., 2022b. URL https:::////t pttptceee/ppoooeeedeeee ngs. www.rew..sccmmm wttp:pprspppppppppppppppeeeeeeee waapiperrrffeeeeeeeeeee wwtps//app/se/a//2///// 984
fi F51e/566b558228bbee 23t.aaa4446600224443344433333333 Canfeeeeeeee----eaeeeeneeeeeeee pddf
Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee. Learn- ing to unlearn: Instance- wise unlearning for pre-trained classifiers. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 1 11186-11194 2024.
Dingfan Chen, Ning Yu, and Mario Fritz. Relaxloss: Defending membership inference attacks without losing utility. In International Conference on Learning Representations, 2022. URL
https://openreviev. net wtt.:/orrmmmddd-eeee
Zitao Chen and Karthik Pattabiraman. Overconfidence is a dangerous thing: Mitigating member- ship inference attacks by enforeing less confident prediction. Network and Distributed System Security (NDSS) Symposium, 2024.
Imre Csiszár. I-divergence geometry of probability distributions and minimization problems. The annals of probability, pp. 158, 1975.
Luke N. Darlow, Elliot J. Crowley, Antreas Antoniou, and Amos J. Storkey. Cinic- 10 is not imagenet or cifar-10, 10, 2018.
1810. 03505
Ganesh Del Grosso, Hamid Jalalzai, Georg Pichler, Catuscia Palamidessi, and Pablo Piantanida.
Leveraging adversarial examples to quantify membership information leakage. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10399-10409, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009. doi: 10. 109////R..0000 5206844.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference on Learning Representations, 2021. URL https: net htw.:arwwww...ttttaammmaaaaaaaaa
Xingli Fang and Jung-Eur Kim. Center-based relaxed learning against membership inference at- tacks. In The 40th Conference on Uncertainty in Artificial Intelligence, 2024a. URL https: //openreview. net. net/ffoumm? d=un WrunF

---

Published as a conference paper at ICLR 2026
Xingli Fang and Jung-Eun Kim. Representation magnitude has liability to privacy vulnerability. In Proceedings of the AAAI/ACM Conference on Al, Ethics, and Society, volume 7, pp. 411-420, 2024b.
Xingli Fang and Jung-Eun Kim. Decoupling genarraiityygg and membership privacy risks neural networks, 2026. URL http::: URL.atppss//xxx...rrr///////////
02296
Xingli Fang, Jianwei Li, Varun Mulchandani, andg Kim. Trustworthy ai: Safety, bias, and privacy a survey, 2025. U.L./hcpee./aarrrrr.orraaa////..
Jonathan Frankle and Michael Carbin. The lottery ticket ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019. URL https://
openreview wwtpeeewwwwweeeeeeeeeeeeeeeeeeee nett/forrmmm00rr1133333333
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode con- nectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pp. 2259-3269. PMLR, 2020.
Advait Harshal Gadhikar and Rebekka Burkholz. Masks, signs, and learning rate rewinding. In The Twelfth International Conference on Learning Representations, 2024. URL https:
//openzeview. www.a/eeviiw.nnt/orrrnee.
Kristian Georgiev, Roy Rinberg, Sung Min Park, Shivam Garg, Andrew Hyas, Aleksander Madry, and Seth Neel. Machine unlearning via simulated oracle matching. In The Thirteenth Interna- tional Conference on Learning Representations, 2025. URL forum?2dd=3xp2200222
Laura Graves, Vineel Nagisetty, and Vijay Ganesh, Amnesiac machine learning, 2020. URL 10981
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition JOVPR),June 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network, 2015. URL
02531
Qiang Hu, Hengxiang Zhang, and Hongxin Wei. Defending membership inference attacks via 06814
privacy-aware sparsity tuning, 2024. URL
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference chine Learning Research, pp. 448 456, Lille, France, 07-09 Jul 2015. PMLR. URL https: //proceedings ml press http://///oooff..occcoco
Machine Learning, volume 37 of Proceedings of Ma-
Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and gen- cralization in neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018 URL ht htttpp://ppoceeeiinnoe .errrineuuuur www.eurrrrrccco wwtppppcccppppppppppcpppppppp pdf
Eiless/aper:/2018/ (tt.l///flee//aaeb.ee F233ee2bbbaaeee66312222222222222
Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao. Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. Advances in Neural Informa- fion Processing Systems, volume 36, pp. 51584-51605. 2023.
Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. Memguard: Defending against black-box membership inference attacks via adversarial examples. In Proceed- ings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 19. pp. 259-274, New York, NY, USA, 2019. Association for Computing Machinery. ISBN doi: 10.14573319535.333220
9781450367479.
URL bitps::/dol.. htt.//....org/1..1111111
3 9535 36662011
12

---

Published as a conference paper at ICLR 2026
Yigitean Kaya, Sanghyun Hong, and Tudor Dumitras. On the effeettivnness of regularization against 05336
http://.ggggbbbb/baaaba httpp:///araav.ooo/aa/////000000 membership inference attacks, 2020. URL BRRLattps://axxivi
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http:
7/ wwwpiv..rg/abb///ee
6980
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr. SNIP: Single-shot network pruning based on connection sensitivity. In International Conference on Learning Representations, 2019. URL
htttps://openreviees n http://ormmmiiiem wtt..//nee//ooormmm/d///////3/1.
Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Gaussian membership inference privacy.
In Thirty-seventhh Conference on Neural Information Processing Systems, 2023. URL https:
//openreview.rr net wttp://wwwweeeeetttoe/eeeddddddd 2NU TTMMS
Jiacheng Li, Ninghui Li, and Bruno Ribeiro. MIST:
Defending against membership inference ceabbeshhpp-nnvaiiann subspace training. In 33rd USENIX Security Sympo- sium (USENIX Security 24), pp. 2387-2404, Philadelphia, PA, August 2024. USENIX As- sociation. ISBN 978- 1-91133-44111 URL https: /www. [userr/keccurrr22///rreeeeeooooo
attacks through
jiacheng
Lucas Liebenwein, Cenk Baykal, Brandon Carter, David Gifford, and Daniela Rus. Lost in pruning: The effects of pruning neural networks beyond test accuracy. In A. Smola, A. Di- makis, and I. Stoica (eds.), Proceedings of Machine Learning and Systems, volume 3, pp. 93- 138, 2021. URL https: File/ [BRL[tttpps:/prrceeednng http://aww....llllsoo/ooe/eeeeee org.raper..pppoooooooo.eoeeee..e Files/paper/2021
Zhenlong Liu, Lei Feng, Huiping Zhuang, Xiaofeng Cao, and Hongxin Wei. Mitigating privacy risk in membership inference by convex-concave loss. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Pro- ceedings of Machine Learning Research, pp. -0998-31114.. PMLR, 21-27 Jul 2024. URL mlr https://proceedingg http://nww.......s/eeeeeesee www.pprsssv3333 wwt.///i223//1122..... html
Iyy Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. / net / stw.ummiidsskgg9900
Pavlo Molchanov, Arun Mallya, Stephen Tyree, luri Frosio, and Jan Kautz. Importance estimation for neural network pruning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 11264-1 1122, 2019.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC Conference Computer and Communications Security, CCS '18, pp. 634-646, New York. NY, USA, 2018. Association for Computing Machinery. ISBN 97845563669 doi: 10.1145/32433344.24888
httpst::
//doi. 1141.001111333323331 3243855
Alex Renda, Jonathan Frankle, and Michael Carbin. Comparing rewinding and fine-tuning in neural network pruning. In International Conference on Learning Representations, 2020. URL https:
www.:ne/ffoommiiii Striiddd1llh S1g. 1elgg JONKBB
Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. 01 2019, doi: 10. (47222/dss.2019.222 1991.
Sebastian Schelter. amnesia-towarde machine learning models that can forget user data very fast. In 1st International Workshop on Applied Al for Database Systems and Applications (AIDBI9), 2019.
13

---

Published as a conference paper at ICLR 2026
Sebastian Schelter, Stefan Grafberger, and Ted Dunning. Hedgecut: Maintaining randomised trees for low-latency machine unlearning. In Proceedings of the 2021 International Conference on Management of Data, pp.
2021.
Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversari- ally robust neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Procexsing Systems, volume 33, pp. 19655- 19666, 2020. URL https: http:ss:///poceeeeeeeeeee wwwripsunni.. cttpeiipssccpappeeooee cc/paper files/paper/ G000FELLL44222221111057777772222
Jing Shang, Jian Wang, Kailun Wang, Jiqiang Liu, Nan Jiang, Md Armanuzzaman, and Ziming Zhao. Defending against membership inference attacks on iteratively pruned deep neural net- works. In NDSS, 2025.
Virat Shejwalkar and Amir Houmansadr. Membership privacy for machine learning models through knowledge transfer. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp.
2021.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE symposium security and privacy (SP), pp. 3-18. IEEE, 2017.
Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX security symposium (USENIX security 21). pp. 2615-2632, 2021.
Mingjie Sun, Zhuang Liu, Anna Bair, and Zico Kolter. simple and effective pruning approach for large language models. In The Twelfth International Conference on Learning Representations, 2024. URL h
ew net/ formmoormm?? o
Xinyu Tang, Saeed Mahloujifar, Liwei Song, Virat Shejwalkar, Milad Nasr, Amir Houmansadr, and Prateek Mittal. Mitigating membership inference attacks by (Self-Distillatie) through a novel ensemble architecture. In 31st USENIX Security Symposium (USENIX Security 22), pp. 1433- 1450, 2022.
Yehui Tang, Yunhe Wang, Yixing Xu, Dacheng Tao, Chunjing XU, Chao Xu. and Chang Xu. Scop: Scientific control for reliable neural network pruning.
H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural In- formation Processing Systems, volume 33, pp. 10936-10947. Curran Associates, Inc.,
2020. URL
newrips wwtrriissccppaaeeee cc/paper files/paper/2020. Papee.appppa.a
Cuong Tran, Ferdinando Fioretto, Jung-Eun Kim, and Rakshit Naidu. Pruning has a disparate impact on model accuracy. In S. Koyejo, Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 17652 17664, 2022. URL https::/ neurips http:ppssspppppppppeeeeeeeee files/paperrrrrr 2022/fi 25222//h077000000022222110222222 Papee Confereece pdf
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL htttts::/proceeddiiie ngg.. nwurieuurs.... cc stwerrffillespppe/2/1777 File/3f 4354 7dee9 fbd053cl 844 .pof
Yite Wang, Dawei Li, and Ruoyu Sun. NTK-SAP: Improving neural network pruning by aligning training dynamics. In The Eleventh International Conference on Learning Representations, 2023.
URL httl:htts///penreeee
net/ forum?id==-SEtht 4qWP
Bo Yang. Hongwei Yang, Renhao Lu, Hui He, Weizhe Zhang, Haoyu He, and Rahul Yadav. Loss- control: Defending membership inference attacks by controlling the loss. In ICASSP 2025-2025 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
14

---

Published as a conference paper at ICLR 2026
Jaayuan Ye, Anastasia Borovykh, Soufiane Hayou, and Reza Shokri. Leave-one-ee distinguishabil- ity in machine learning. In The Twelfth International Conference on Learning Representations, 2024. URL http:s/://ppeeeeeee/e/ net/ wtt.eeeerom...ae-REOOOOOOOO
Shaokai Ye, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin. Adversarial robustness vs. model compression, or both? Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
Weihao Ye, Qiong Wu, Wenhao Lin, and Yiyi Zhou. Fit and prune: Fast and training- free visual token pruning for multi-modal large language models. 39:22128-22136, Apr. 2025. doi: htt9:/aaii.33i21..3333333 view/34366
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj. Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendl differential privacy library in pytorch, 2022. URL https:
//axxiv.org/aas//222
12298
Xiaoyong Yuan and Lan Zhang. Membership inference attacks and defenses in neural network pruning. In 31 USENIX Security Symposium (USENIX Security 22), pp. 4561-4578, 2022.
Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-power membership inference attacks. In Forty-first International Conference on Machine Learning, 2024. URL https:
//openzeview. net. dow.mmma..sTTTTSSSSC
Guangsheng Zhang, Bo Liu, Huan Tian, Tianqing Zhu, Ming Ding, and Wanlei Zhou. How does deep learning model architecture impact its privacy? a comprehensive study of privacy attacks on {CNNs} and transformers. 33rd USENIX Security Symposium (USENIX Security 24), pp- 6795-6813 12, 2024.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https://pproceeddggs. http://ss...uurrs......oooaaaaea file
files/paper/2015//
Yunpeng Zhao and Jie Zhang. Does training with synthetic data truly protect privacy? In The Thirteenth International Conference on Learning Representations, 2025. URL https:
//openzeview. wttp://ww.......ooooooooooceeeee
15

---

Published as a conference paper at ICLR 2026
A FURTHER RELATED WORK
A.1 MEMBERSHIP PRIVACY PRESERVATION METHODS
Prior membership privacy preservation research Fang al. (025) mainly focused data-end and training components. Abaaii et al. (2016) attempted to prevent data points from being over-learned via gradient clipping and noise confusion. Nasr et al. (2018) tried to align member and non-member predictions via adversarial learning. Jia ct al. (2019) attempted to mitigate privacy breaches by ob- fuscating prediction probabilities. Kaya et al. (020 found that the sense of privacy provided by the regularization mechanisms is false. [Chen et al. (2022) designed a iitiiiiioondissibuttioo aaligning loss function via reducing the generalization gap and increasing the variance of the training loss dis- tribution. Fang & Kim (2024a bb) attempted to mitigate privacy breach by explicitly facilitating rep- resentation aligmment in latent space. [iu et al. (2024) achieved privacy preservation by embedding concave term into convex losses, which help the model predictions with high variance in training losses. Fang & Kim (2026) tried to identify and mitigate the layer-wise start of privacy risks. Zhang et al. (2024) determined that components such as attention modules lead ViTs' privacy vulnerability to be significant than CNNs. Carlini et al. (2022b) observed that simply removing the data identifi- able by MIAs from the training dataset induces new privacy leakages in the model. Ye et al. (2024) quantified sample-level privacy vulnerabilities via a leave-one-out approach. Li et al. (2024) tried to separately handle privacy-ries data points that are leaked from model. Yuan & Zhang (2022) ob- served that common accuracy-ore -rrintee pruning & fine-tuning techniques cannot e eliminate privacy risks neural networks. [Shang et al. (2025) identified privacy-risky samples to mitigate the privacy risks of the model by rotating the phases of destroying memorization and relearning selective sam- ples during the accuracy-oriented iterative pruning. [Shejwalkar & Houmansadr (202T): Tang et al.) 2022); Yang et al. (2025) facilitated the mitigation of privacy leakage during training by producing driacc-friendlly soft labels. Chen & Pattabiraman (2024) attempted to avoid overconfidence in both training and inference stages. IZhao & Zhang (2025) claimed prior data synthesis approaches cannot prevent privacy leakage. However, past studies did not identify where the privacy risks are inside neural networks. In our paper, we locate and analyze weight-level privacy vulnerabilities.
A.2 MACHINE UNLEARNING
A general goal of machine unlearning (MU) is to get rid of the impacts of some data points. Current MU approaches can be categorized into two types: (i) data reorganization and (ii) model manipula- tion. The data reorganization approaches usually modify data labels to achieve unlearning, such
as label obfuscation Graves et al. 2020), data pruning [Bourtoule et al. (2021). or data replacement Cao & YYang (013)) As for model manipulation, it mainly consists of two directions: updating the
model weights Schelter (2019); Cha et al. (2024): Georgiev et al. (2025). and replacing components
Schelter et al. (2021). In our paper, we mainly study the way of updating model weights to explore the weight-level privacy vulnerability in neural networks.
B EXPERIMENTAL SETUPS
Attacks. To show the superiority of our approach in boost- ing privacy-press prrecy-pres methods against membership inference attacks, two recent MIAs techniques, Likelihood Ratio Attack (LiRA) Carlini et al. (2022a) and Robust Membership Inference Attack (RMIA) ZZaiizadeh et al. (0024), are adopted in our de- fense evaluation. To simulate the scenario where the shadow model technique (hokrr et al.) (2017): Carlini et al. (2022a) is applied, only a small portion of the data is sampled as training data and reference data for each model. In our study, we follow LiRA's sampling strategy, while the
Table 4: The number of data points sampled from the entire non-testing set.
Dataset
Training
Reference
CIFAR-1O
000
2, 0.000
CIFAR-100
18, 000
4, 0.000
CINIC-10
25, 25,000
3,000
precise quantities are different. The specific quantities for each dataset are provided in Tab. 4 In addition, the strategy of adaptive attacks SSng & Mittal (202 1> is applied to all MIAS to rigorously evaluate the defense approaches. We evaluate the model's reliance ability against attacks along two metries: (i) AUC: by integrating the ROC curve across all thresholds, the AUC reflects the degree to which the attacker can distinguish the membership of the data points for the target model that is attacked by attacker, (ii) TPR at low FPR: we also use true true-positive rate (TPR) at low false-positive
rates (FPR) metric to show the model's privacy vulnerability since Carlini al. 2022al state
16

---

Published as a conference paper at ICLR 2026
W/o CVIRF
W/ CWRF (Ours)
Generalleakiiyy
LIRA
LIRA
LJRA
1.00
TIP/% 00 INFES
4.0
20-
20
10-
HAMP
HAMP
HAMP
HAMP
Approuch
Apprtruc
Approucd
Appituucs
Figure 7: The performance against LiRA when 128 shadow models (64 'IN' and 64 'OUT' models) are deployed for ResNet18 trained with three privacy-preee preservation approaches (RelaxLoss, HAMP, and CCL) with and without CWRF (Ours) in CIAFR-100. The dotted line represents a baseline performance of a model trained from scratch with regular training approach, Cross-Entropy.
wro CWRF
w/ CWRF (Ours)
That Accuracy (%0
37.5
TER/NiGO N00 NEPA
0.075
AUI elg
60
1.0
1P90 D01 NEPP
0.4
60
AUC
1900%
2
TPRCKIBB
0.5
60
$2.5
0.025
HAMP
56.0
HAMP
RbL
HAMP
Ror.
HAMP
Pbol
HAMP
HAMP
Rob
HAMP
Approach
Approuc
Approual
Approsch
Appronch
Appesaci
Figure 8: The performance of transformer trained with three privacy-preservation approaches with and without CWRF (Ours) in DBpedia-14. 14. The dotted line represents a baseline performance of model trained from scratch with regular training approach, Cross- Cooss-Entropy.
that neither attack accuracy nor AUC scores adequately reflect an attack's ability to confidently determine membership while TPR at low FPR identifies it better. A perfect defense mechanism corresponds to AUC 0.5 in the first metric while TPR = 0 in the second metric. Specifically, the TPRs 10 and 10 FPRs reported in paper.
C FURTHER EXPERIMENTAL RESULTS AND DISCUSSION
C.I MORE SHADOW MODELS
To reinforce the empirical evidence of our experiments, we further explore how our approach and others perform when evaluate ResNet18 under LiRA with more shadow models in the CIFAR-100 classification task. As shown in Fig. 7 when 128 shadow models, stronger attacks, are deployed, all approaches show more significant privacy flaws, compared with Fig. (bb] Among these approaches, RelaxLoss and CCL show better resisting ability while the utility performance is even slightly better when they are plugged into CWRE, our approach. As for the HAMP, the trends remain the same as Fig. 6b Through the results, regardless of the number of shadow models, our approach shows consistent advantages when combining with other privacy-training approaches.
C.2 EVALUATION ON NLP DOMAIN DATASET
To reinforce the empirical evidence of our experiments, we further explore our approach for an NLP dataset - DBpedia- 14 Zhang et al.] (2015). The DBpedia-14 is an NLP classification dataset that contains 560, 000 training samples and 70, 000 testing samples for fourteen classes from DBpedia. As shown in Fig. g we evaluate the approaches with transformer Vaswani et al. (2017). At a similar utility level, combining with CWRF shows improvement privacy.
C.3 PRIVACY-UTILITY CURVE
To reinforce the empirical evidence of experiments, further explore how approach and others perform with privacy-utility trade-offss via ResNet18 trained with the CIFAR-106 100 classifica- tion task. As shown in Fig. 9 show the privacy-utility curve, including the configuration points
17

---

Published as a conference paper at ICLR 2026
w/o CWRF
CWRF
LIRA
TPRNN/ 1NPPPR
LIRA
TPRe%ogo 001 16796
LIRA
RMIA
TPRUNDOO 1,F2R
RIMIAA
RMIA
-5
90
ACC (86)
10
8-
85
TPR0%/g
52
42 Tist Act. (%6
(8))
Add. (%)
(56)
Acc.
Text. Anc. (%6
(a) RelaxLoss
70
LRA
TPRIN90BO. 198PPR
LiRA
TRRCN 001 0.101 NTER
0.10
GRA
RMIA
- N l
RMIA
TPRNNO 0.000101NN00
RMJ
(%)
0.4
70-
60
022
AtiC
50
0.0
0.00
50
Test Ace. (%)
Test Acc. (%)
TestAAc.. Aec. (%)
Test Acc. (%6
Test Acc. (%)
Test Acc. (%)
(b) HAMP
LIRA
LIRA
TTPP% 00 IAPPR
LIRA
RMIA
RMJA
RMTA
330
1G
20
40
85
90.00
10
92.5
10
30
40
30
40
30
40
30
40
30
40
30
40
Test Acc. (%)
Test Acc. (%)
Test Acc.
Test Ace.
Test Acc. (%)
Test. Ace. (%)
(c) CCL
Figure 9: Privacy-utility curve of ResNet18 in CIFAR- 100. The bottom right corner (low MIAs yet high test accuracy) the best performance terms of privacy-utility.
in 6bg Compared with the case with each of the three approaches solely, plugging CWRF shows consistent advantages by combining privacy-training approach.
18


Reviews:
[{"review_id": "0tQxe21gEZ", "reviewer": "Reviewer_wcwv", "summary": "This paper investigates the relationship between model learnability and privacy, finding that privacy vulnerability is \"entangled\" with utility performance within a very small fraction of critical weights. The authors propose a key insight that the importance of these weights stems from their location rather than their trained values. Based on this, the paper introduces Critical Weights Rewinding and Finetuning (CWRF) , a strategy that identifies this small set of privacy-vulnerable weights. Instead of pruning, CWRF rewinds only these high-risk weights to their initial, privacy-safe values and then freezes them. Finally, the model recovers its utility by fine-tuning only the remaining \"privacy-invulnerable\" weights, leveraging the fact that the critical weights' locations are preserved. Experiments demonstrate that CWRF successfully boosts the effectiveness of existing privacy-preserving training methods, achieving a superior privacy-utility tradeoff.", "questions": "1. How are the hyperparameters chosen for the baseline methods reported in this paper? For example, the loss threshold for RelaxLoss and the $\\gamma$ for RMIA.\n2. How do the authors split the dataset for training the target model and reference models?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The work offers a significant contribution by clearly explaining why standard model pruning fails to mitigate privacy risks, linking it directly to this entanglement.\n2. The paper proposes CWRF that cleverly combines machine unlearning for vulnerability estimation with weight rewinding, which boosts existing privacy-preserving methods to achieve.", "weaknesses": "1. The paper provides no sensitivity analysis for the hyperparameter rewinding rate $r$ and $\\lambda$, making it unclear how to set it efficiently.\n2. The empirical validation is limited to small-scale models (ResNet18, small ViT) and datasets (CIFAR, CINIC). It is not demonstrated whether the vulnerability estimation step is computationally feasible or if the core insight scales to LLMs. \n3. It is recommended to include full privacy–utility curves (similar to those reported for RelaxLoss and CCL) rather than only isolated metric points, as these curves may provide a more comprehensive representation of the privacy-utility trade-off.  \n4. Previous work [1] also discusses which parameters substantially impact privacy risk. Please add a discussion and comparison with this work. \n\n[1] \"Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning\" (2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7MvnREHaPU", "reviewer": "Reviewer_fnse", "summary": "This paper introduces a method, Critical Weights Rewinding and Finetuning (CWRF), for optimizing model utility and membership privacy simultaneously. It attempts to resolve the issue of \"entanglement\", where learnability (utility) and privacy vulnerability are concentrated in the same small set of critical weights. The new approach is applied as a \"booster\" to existing privacy-preserving training methods (DP-SGD, RelaxLoss, HAMP, and CCL) and is evaluated on both model utility and its resilience to Membership Inference Attacks (MIAs). The algorithmic contribution is demonstrated with a suite of experiments in classic benchmark domains (CIFAR-10, CIFAR-100, and CINIC-10) and on modern architectures (ResNet18 and ViT).", "questions": "Please refer to the Weaknesses section.\n\nSome minor points:\n\n- The abstract states CWRF \"exhibits outperforming resilience\", but the actual results show mixed performance on RMIA for ResNet18 e.g RelaxLoss is better without CWRF\n\n- In Figure 3, the paper notes that the axes for the ViT plots are not consistent. While this is acknowledged, it makes visual comparison of the correlations across different layer types (e.g., Att+MLP vs. Norm) very difficult.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1. The paper is well written and easy to follow\n\nS2. The insights are interesting and helpful", "weaknesses": "I found the claims in the paper are interesting but unconvincing due to several points:\n\nW1. Lacks of theoretical motivation. The paper's central hypothesis that a weight's importance stems from its location, not its value is a strong one, but it is presented without theoretical proof and is supported only by a specific set of ablation studies. The paper argues that because A3 (CWRF) successfully recovers accuracy while A1 (Remove) fails, the hypothesis is validated. While the result is compelling, it is not a proof. Can the authors provide any formal argument for this? How does this \"location-over-value\" hypothesis relate to existing work like the Lottery Ticket Hypothesis, which is mentioned but not deeply connected?\n\nW2. Missing datasets. Previous baselines such as RelaxLoss or HAMP include tabular dataset such as Purchase or Texas besides image dataset.\n\nW3. The justification for fine-tuning invulnerable weights (A3) over vulnerable weights (A2) is the paper's most critical experimental result. However, this experiment appears to have been run only using RelaxLoss as the fine-tuning defense. Does this crucial finding hold for DP-SGD, HAMP, and CCL as well?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hDRqk08elt", "reviewer": "Reviewer_XTZA", "summary": "The paper provides evidence that privacy vulnerability (as quantified by susceptibility to membership inference attacks, or MIAs) is concentrated in certain locations/weights of a model. Additionally, these weights are also those that correspond to learnability/generalizability. With these insights, the authors propose a method that rewinds and freezes these privacy-vulnerable weights and fine-tunes the other unfrozen weights with (potentially private) information with privacy-preserving approaches such as DP-SGD and more recent approaches. They further corroborate their claims with an extensive array of results across different percentages of rewound weights, different methods, ablating which weights are rewound/fine-tuned, etc. along with an analysis of privacy-utility tradeoff for their method and the reduction in attack success observed for different state-of-the-art MIAs (LiRA and RMIA).", "questions": "**[Q1]** Can you please address W1 and add a self-contained description of CWRF in the final version of the paper for better presentation?\n\n**[Q2]** Could you also ideally address W2 as well?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "**[S1]** Provides valuable insights on which weights correspond to MIA vulnerability and how they are also largely the same as the ones that are most important to generalizability. The methods used to identify such weights are sound and convincing.\n\n**[S2]** Insights provided in Fig. 5 to motivate the design of CWRF are interesting; it clearly demonstrates why it is key to both rewind privacy vulnerable weights and fine-tune privacy-invulnerable (so to speak) weights. It is also interesting how fine-tuning privacy-invulnerable weights, even if they might not be most correlated with learnability, helps significantly with achieving a good privacy-utility tradeoff/test accuracy. In other terms, the observation that the location of these weights, and not their values, is what matters is fascinating, albeit not very intuitive.\n\n**[S3]** Demonstrates the effectiveness of the proposed method (CWRF) when paired with multiple prominent differentially private training algorithms.\n\n**[S4]** Uses the strongest possible attacks to test their proposed defense against. In addition, they report attack success in low FPR regions, which is absolutely essential to effectively communicate privacy attack success rates (as argued by [1]).\n\n**[S5]** In-depth descriptions of the hyperparameters, libraries/software, and hardware used are provided, providing confidence in the reproducibility of the work.\n\nAll in all, this paper appears to provide a valuable contribution to the community: a method that can reliably defend against powerful MIAs while maintaining a good level of test accuracy/utility. I find the contributions of this paper valuable and appealing.\n\n---\n\n## References:\n\n**[1]** Carlini, Nicholas et al. “Membership Inference Attacks From First Principles.” 2022 IEEE Symposium on Security and Privacy (SP) (2021): 1897-1914.", "weaknesses": "**[W1]** The description of the method could be done more clearly. While the motivation of the methodology by discussing prior approaches to rewinding, weights freezing and fine-tuning, and ablation studies is a great choice, it will be highly beneficial to have a concise self-contained discussion about the steps involved in CWRF along with pseudocode to avoid ambiguity and for easy reference.\n\n**[W2]** The text in tables 1 and 2 is very small and not very appropriate for a potential camera-ready version.\n\n---\n\nThis paper is pretty solid otherwise.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wL0m2v53gk", "reviewer": "Reviewer_wW66", "summary": "This paper proposes a new algorithm, Critical Weights Rewinding and Finetuning (CWRF), for privacy preservation [without significantly affecting the utility of the model] based on the observation that there is a relatively small fraction of weights that cause privacy leakage in models. They also demonstrate that a large fraction of these privacy-vulnerable weights are critical to the model's learnability. CRWF works by keeping the learnability/ privacy-critical weights intact while fine-tuning the other privacy-invulnerable weights. This work incorporates the concept of importance estimation of weights in a neural network, which is largely used in the context of model pruning (Frankle & Carbin [1]).", "questions": "**Questions**: I would urge the author(s) to address the weaknesses detailed above.\n\n**Suggestions**:\n\nAuthor(s) can improve the presentation of the paper by considering the following suggested edits:\n\n- Minor suggestion #1: It might be good to rearrange the Fig 3 and Tab 1's results by decreasing order of PCC for an easy read.\n- Minor suggestion #2: The PCC reported for linear layer weights for ResNet18 in Tab 1 does not appear aligned with the results as depicted in 1st row, 4th column of Fig 3. If possible, use a different y-axis to demonstrate the high correlation between privacy vulnerability and learnability for linear layers (~0.8).\n- Minor Suggestion #3: I would urge the authors to use the full form of the abbreviated terms, such as HAMP and CCL, in Line 377, when referencing them for the first time. Thereafter, they can use the abbreviation. It would be erroneous to presume a reader knows the abbreviations beforehand. \n- Minor Correction #1: Line 811, Specifically <-> \"Specificly\".", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Unlike previous works, instead of data points, the author(s) in this paper are concerned with addressing model-level privacy vulnerability in neural networks. They propose a new metric for privacy-vulnerability estimation for individual weights in a neural network based on Bourtoule et al.'s [2] work on machine unlearning.\n- They provide empirical evidence to support their conjectures: (a) Privacy-critical weights are concentrated in the layers responsible for learnability (Section 4.1), (b) there is a correlation between privacy-critical and learning-critical weights (Section 4.2), and (c) privacy-vulnerable weights are also least updated during training (Section 4.4).\n- CWRF is modular as it allows plugging in any privacy-preserving approach during the fine-tuning step, as demonstrated in Table 3 by integration of CWRF with other privacy-preserving methods such as HAMP, DP-SGD and others. Furthermore, in some cases, CWRF not only improves resistance to MIAs but also boosts the utility (measured in terms of the test accuracy) of the models. \n- This work also contributes to the field of Machine Unlearning, where the model trainer aims to minimise/ remove the influence of select data points on a trained machine learning model. With CRWF, the author(s) aim to train a model to mitigate weight-level privacy vulnerability without significantly affecting its utility.", "weaknesses": "- In CRWF, per my interpretation of the approach, the model is fine-tuned using the same training dataset as the one used to train $M_{up}$ from scratch in the privacy vulnerability estimation step. However, the author(s) do not clarify this in Section 4.3.\nThe author(s) do not clarify what privacy parameters, for example, $(\\epsilon, \\delta)$, were used in the experiments with DP-SGD. \n- In lines 388-389, the author(s) state that they use a total of 8 shadow models for both LiRA and RMIA. While RMIA is said to work well with a low number of shadow models, comparing it against a powerful version of LiRA would necessitate training more than 8 shadow models. Carlini et al. [3] use 256 shadow models. If the author(s) intent is to convey the strength of their proposed privacy-preservation approach (CWRF), they ought to evaluate it using strong attacks and not its weaker versions.\n- Lines 404-405 that \"It is verified that our approach successfully boosts the performance of all of them.\" ought to be framed in the context of the performance of CWRF + other privacy-preserving methods against SOTA MIAs.\n- Carlini et al. [4] demonstrated the onion-effect wherein removing privacy-vulnerable data points from training exposes earlier invulnerable data points to replace them. Do the author(s) probe whether something similar happens when using CWRF, which involves rewinding privacy-vulnerable weights to their initial values? Could this cause previously privacy-invulnerable weights to become privacy-critical?\n- The author(s) do not share the code for the paper.\n\n[1] Frankle, J., and Carbin, M. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. ICLR 2019.\n\n[2] Bourtoule, L. et al. Machine Unlearning.” IEEE S&P 2021.\n\n[3] Carlini, N., et al. Membership Inference Attacks From First Principles. IEEE S&P 2022.\n\n[4] Carlini, N. et al. The Privacy Onion Effect: Memorization is Relative. NeurIPS 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hIwmfqlfwU", "reviewer": "Reviewer_Jji4", "summary": "This paper finds that performance impact and privacy vulnerability are entangled and exist in a very small fraction of weights. The authors propose a fine-tuning method that curates only privacy-vulnerable weights to defend against membership inference attacks. Experiments show the performance of this method on three datasets.", "questions": "1. In Figure 1(b), does the training loss approach zero, suggesting that the model is severely overfitting? Does CE loss represent privacy vulnerability? \n\n2. How to formally define privacy-vulnerable weight. As described in Section 4, the author uses the weight importance estimation method Eq.2 to estimate privacy vulnerability. Can importance estimation serve as a proxy for privacy vulnerability?\n3. Existing paper[1] also propose a membership-privacy-oriented fine-tuning defence method.\n4. How to compute the learnability score? Is this the accuracy of models?\n5. What does the PCC in Table 1 represent? For example, does a high PCC indicate both high accuracy and a high MIA AUC?\n6. Can CWRF be used independently as a defence method?  Why does Table 3 only show the results when CWRF is plugged into other defence methods?\n7. The proposed method is similar to pruning techniques. Why are pruning-based defence methods not included in the comparison?\n\n[1] Defending membership inference attacks via privacy-aware sparsity tuning[J]. arXiv preprint arXiv:2410.06814, 2024.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The proposed question is interesting. The authors argue that performance impact and privacy vulnerability are entangled and exist in a very small fraction of weights. This is a valuable research problem in the membership inference attack field. \n\n2. The author makes an attempt to confirm the proposed insights by empirical evidence. Based on these observations, this paper proposes a method to mitigate the privacy vulnerability introduced by MIAs.", "weaknesses": "1. The presentation of this paper could be improved. A well-organised structure makes it easier for readers to understand the paper and follow the arguments. This paper lacks readability, making it difficult for readers to grasp the viewpoints the author wants to convey. For example, the authors should introduce the importance estimation in neural networks in the preliminaries (not as a related work) if they use the tools as an analytical method.\n\n2. The paper does not establish a clear logical connection between the three proposed insights, the motivation, and the method. The authors claim the importance of weights stems from their locations rather than their values. However, the TFO method estimates weight importance via magnitudes of gradients and weights. Moreover, why does the proposed method not consider fine-tune weights at particular locations, even though the importance of weights stems from their locations? In addition, the authors claim that most privacy-vulnerable weights impact utility performance. Why does rewinding privacy-vulnerable weights, while freezing these weights, have little impact on performance?\n\n3. The experimental results provided don’t sufficiently substantiate the conclusions drawn in this paper. For an empirical paper, authors should conduct extensive experiments to evaluate the effectiveness of the proposed method, including the various attack and defence methods, different types of datasets, and more models. In addition, the authors should conduct experiments to analyse the performance of the method under different configurations, such as hyperparameters, ablation studies on key components. Moreover, the authors should plot privacy-utility curves to show the privacy-utility trade-offs, rather than reporting only a single point result as in Table 3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)