Membership Privacy Risks of Sharpness Aware Minimization
Authors: Young In Kim
Andrea Agiollo
Pratiksha Agrawal
Johannes O. Royset
Rajiv Khanna
Venue: ICLR 2026 Poster (2026)
Keywords: membership inference attack
sam
sharpness aware minimization
memorization
benign overfitting
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Sharpness Aware Minimization is more vulnerable to membership privacy attacks than SGD although it generalizes better due to the property of memorizing atypical subclass features.

Abstract: Optimization algorithms that seek flatter minima, such as Sharpness-Aware Minimization (SAM), are credited with improved generalization and robustness to noise. We ask whether such gains impact membership privacy. Surprisingly, we find that SAM is more prone to Membership Inference Attacks (MIA) than classical SGD across multiple datasets and attack methods, despite achieving lower test error. This suggests that the geometric mechanism of SAM that improves generalization simultaneously exacerbates membership leakage. We investigate this phenomenon through extensive analysis of memorization and influence scores. Our results reveal that SAM is more capable of capturing atypical subpatterns, leading to higher memorization scores of samples. Conversely, SGD depends more heavily on majority features, exhibiting worse generalization on atypical subgroups and lower memorization. Crucially, this characteristic of SAM can be linked to lower variance in the prediction confidence of unseen samples, thereby amplifying membership signals. Finally, we model SAM under a perfectly interpolating linear regime and theoretically show that sharpness regularization inherently reduces variance, guaranteeing a higher MIA advantage for confidence and likelihood ratio attacks.

Published as a conference paper at ICLR 2026
MEMBERSHIP PRIVACY RISKS OF SHARPNESS AWARE MINIMIZATION
Young In Kim', Andrea Agiollo', Pratiksha Agrawal', Johannes O. Royset Rajiv Khannal
1 Department of Computer Science, Purdue University
2 Faculty of Electrical Engineering, Mathematics and Computer Science, TU Delft
3 Department of Industrial and Systems Engineering, University of Southern California
ABSTRACT
Optimization algorithms that seek flatter minima, such Sharpness-Aware Min- imization (SAM), are credited with improved generalization and robustness to noise. We ask whether such gains impact membership privacy. Surprisingly, we find that SAM is more prone to Membership Inference Attacks (MIA) than clas- sical SGD across multiple datasets and attack methods, despite achieving lower test error. This suggests that the geometric mechanism of SAM that improves generalization simultaneously exacerbates membership leakage. We investigate this phenomenon through extensive analysis of memorization and influence scores. Our results reveal that SAM is more capable of capturing atypical subpatterns, leading to higher memorization scores of samples. Conversely, SGD depends more heavily on majority features, exhibiting worse generalization on atypical subgroups and lower memorization. Crucially, this characteristic of SAM can be linked lower variance in the prediction confidence of unseen samples, thereby amplifying membership signals. Finally, model SAM under a perfectly interpolating linear regime and theoretically show that sharpness regularization inherently reduces variance, guarantceing a higher MIA advantage for confidence and likelihood ratio attacks.
INTRODUCTION
Sharpness-Awan Minimization (SAM) has emerged as prominent optimization technique for improving generalization in deep learning by encouraging flatter minima-ia. i.e., similar loss values for weight perturbations of certain degree around the optima- in the loss landscape (Norton & Royset
2021 Foret et al. 2020 Wu et al. 2020 Kim et al. 2022 Du et al. 2022 Kwon 1022 Flatter
optima have been linked to robustness to noise and improved test performance (Chen et al. 2023
Foret al. 2020 Back al., 2024)) while tension exists between whether SAM's implicit bias
is geared more towards diversity (Springer et al. 2024) or simplicity (Andriushchenko et al. 2023
Chang & Khannal 2025) of features.
Models that generalize well are thought to rely less on memorizing specific training examples, which should also improve privacy. Consider membership inference attacks (MIAs) in which an attacker exploits the model behavior gap between training and unseen data to infer if a data point was part of the training data or not (Shokri et al. 2017). Intuitively, when model strongly overfits (training error
< test error), MIA would become easier. Yeom et al. (2018) formally showed that, under certain assumptions, the advantage of a threshold- based MIA is upper bounded by the model's 's generalization error. In light of this, one would naturally expect that a technique like SAM - which demonstrably improves generalization should also decrease model's susceptibility to MIAs.
Contrary to this expectation, we find that models trained with SAM are actually more vulnerable to MIAS than SGD consistently across diverse datasets and attack methods, even as they achieve better generalization (see Tables minima-good from a privacy standpoint, calling for a deeper investigation into the relationship between generalization, memorization, and privacy to unearth this phenomenon both empirically and theoretically. Our work is the first to systematically demonstrate higher membership privacy leakage for sharpness-based algorithm known to generalize better, connecting loss sharpness MIA
[ and 2 Furthermore, this finding challenges the notion that flatter

---

Published as a conference paper at ICLR 2026
Dataset
Optimizer
NN
Confidence
Entropy
M-entropy
Test Acc
SGD
76.62%
77.19%
76.61%
77.30%
80.30%
CIFAR-100
SAM
77.99%
79.10%
78.66%
79.25%
81.60%
SGD
50%
59.37%
59.09%
59.51%
96.00%
CIFAR-10
SAM
50.08%
61.64%
61.64%
61.70%
96.72%
SGD 66.00%
66.76%
64.78%
67.13%
85.50%
Purchase- 100
SAM
66.62%
67.30%
65.35%
67.54%
85.54%
SGD 59.81%
65.20%
55.74%
65.13%
50.83%
Texas-100
SAM
59.56%
66.59%
57.14%
65.42%
51.34%
SGD
73.62%
73.40%
68.50%
73.40%
73.67%
EyePacs
SAM
77.73%
77.07%
73.37%
77.36%
75.41%
Table 1: Attack accuracy of direct threshold MIA on SGD and SAM showing tradeoffss in test accuracy and MIA attacks. In green we highlight the best performing model on the test set and in orange the model against which MIA is more successful. SAM is more prone to direct threshold attacks.
vulnerability. We note that there have been previous works that exhibit utility-privacy tradcoff's (Long
ct al., 2018 Carlini et al. 2021 Chen et al. 2022 Liu et al. (024) demonstrate that higher genera lizatin does not necessarlyy decrease privacy leakage
Kaya & Dumitras, 2021 Del Grosso
et al. 2023).
The difference model behavior on data points that were part of the training set versus those that were not be more precisely quantified using memorization scores (Feldman) 2020 Feldman & Zhang 2020), defined via Leave-One-Oun (LOO) error. Memorization scores measure the change in model performance when specific training sample is removed, and thus serve proxy for how much the model has memorized that sample. Motivated by this connection, we analyze the memorization scores of samples trained with SAM and find that SAM exhibits more memorization than SGD, indicating stronger reliance on individual training samples. While this increased memorization provides plausible explanation for SAM's s heightened vulnerability MIA, it raises key question: "How can model that memorizes more generalize better?"
We hypothesize that the answer lies what being memorized. Under overparameterization (Allen-Zhu et al., 2019), models can learn not only noise in the data, but also atypical patterns in under-represented subpopulations through memorization-ia. few white tiger images with numerous yellow tiger images. This distinction is important as real world datasets are known to have long tail of such rare subclasses (Feldman 2020). We conjecture that SAM is capable of doing more structured memorization, selectively focusing atypical subclass patterns, which contributes positively to generalization. Corroborating this hypothesis, we observe that SAM's memorization score distribution concentrated in the mid range, rather than the high end-whie is typically associated with noise memorization (refer to Section 1. This suggests that SAM emphasizes samples that are neither trivially learned nor purely noisy, but instead represent rare, but generalizable sub-patterns.
To further validate this finding, we analyze influence woores-wiich measure the impact of individual training samples on test predictions (see Section 4.2) Our results show that, for SAM, samples corresponding to moderate memorization exert higher influence on test predictions compared to SGD, confirming that SAM's generalization gains derive from its ability better capture rare sub-pattems. Conversely, for SGD, the lower influence of such points implies that the majority pattern is learned dominantly: since this feature is redundant across many samples, the marginal influence any single point is diluted. These results seem to suggest more that SAM's implicit bias is towards diversity as opposed to simplicity.
We support our intuition further by introducing novel metric that quantifies the degree of memo- rization involved in predicting a test sample in Section 4.3 Using this metric, we dissect SAM's performance gains and identify that SAM's improvements mostly stem from its performance atypical test samples that depend heavily on a handful of memorized training points. Meanwhile, SGD performs slightly better typical samples that rely
broadly learned features. This is in
2

---

Published as a conference paper at ICLR 2026
SGD
SAM
Dataset
Attack
Test Acc
AUC
Attack Acc
TPR@.
Test Ace
AUC
Attack Acc
TPR @
RMIA
90.4%
80.8%
21.0%
91.6%
82.2%
23.4%
CIFAR-100
67.7%
69.1%
LiRA
92.6%
82.9%
27.0%
93.7%
84.1%
31.0%
RMIA
71.4%
63.5%
4.8%
74.9%
65.9%
6.7%
CIFAR-10
92.3%
93.1%
LiRA
73.0%
64.2%
8.8%
76.4%
66.7%
12.5%
RMIA LiRA
68.8%
62,7% 02.7%
1.5%
70.2%
63.7%
1.7%
Purchase100
76.5%
77.4%
68.9%
62.6%
1.4%
70.2%
63.4%
1.6%
RMIA
79.8%
70.8%
2.9%
80.6%
71.5%
2.8%
100
46.9%
49.2%
LIRA
80.8%
71.3%
6.9%
81.6%
72.0%
8.2%
Table 2: Comparison of online shadow-model MIA on SGD and SAM. In green we highlight the best performing model on the test set, and in orange the model with higher privacy leakage (higher AUC, Attack Accuracy, and @PR@0.1 %FPR). SAM is more prone shadow-modes attacks.
line with recent works that leverage model behavior differential between SAM and SGD for dataset pruning (Agiollo et al. 2024) or active learning (Kim et al., 2025). Stronger influence of minority samples can lead to greater membership privacy risk due the increased retention of information an attacker can exploit.
However, what intrinsic mechanism drives this structured memorization? Results for confidence threshold attack indicates that SGD produces more predictions with extreme confidence compared to SAM. These samples reside beyond the threshold and are source of attacker's error. This suggests distinct geometric effect: SAM induces shrinkage in the variance of the model's output predictions. This property translates to structured memorization and suppression of majority sub-feature. Relying heavily a single/few majority subclass feature classify diverse inputs requires the model to assign large weights to that feature. Geometrically, this creates a steep decision boundary and, consequently, high output variance under perturbation. By penalizing this sharpness, SAM prohibits the amplification of the majority feature, effectively forcing the model to distribute its reliance across diverse, subclass-specific features.
Completing this conjecture, we provide a theoretical foundation for this mechanism Section 5 We prove that the interpolating solution favored by sharpness-aue geometry inherently reduces the variance of the output logits. We then demonstrate how this variance reduction amplifies the attacker's advantage for both confidence-based and likelihood ratio attacks. Our proofs highlight strong result: SAM is more vuinerable at any fixed threshold. We empirically corroborate this with ROC curves in Figure 7 Lastly, theoretically analyze dataset with majority and minority subclasses and show how capturing minority feature better leads to enhanced generalization (see Appendix B).
Contributions In summary, our contributions are the following: (i) we are the first empirically demonstrate that SAM-trained models exhibit higher membership privacy risk than SGD-trained models, serving as a cautionary tale against flatter minima=good notion from privacy standpoint; (ūi) we offer detailed and conceptually grounded analysis of the root causes of SAM's generaliza- nonn-memorization relationship, suggesting SAM's implicit bias towards diversity; (iii) we introduce novel methodology to dissect generalization gains, proving that SAM's generalization gains stem from its performance on unseen atypical samples; (iv) we theoretically show variance shrinkage effect of interpolating sharpness-awe -aware aware solutions and how it increases MIA risk for both confidence and likelihood ratio attacks; and (v) we theoretically formulate data distribution composed of subclasses where stronger alignment with minority subclass features enhances generalization.
2 BACKGROUND & PRELIMINARIES
Memorization & Influence scores For training algorithm A that is used train the model
f(+) using dataset D (x),9)).... (X,., )), the amount of label memorization by A on a sample (x,, y:) D is defined by Equation (T). The probability is taken over randomness of the algorithm
3

---

Published as a conference paper at ICLR 2026
such weight initialization.
mem(A, == Pr [f(x:) Vil
Pr |eA(D\( (xc.
[fss)) y/
(I)
f=A(D)
Influence score of a training example (xi, yi) on test example (x), vf) is defined by:
Pr in.fl((,D,,.)))) = (=A(D) ( [f(x)) = U/yl [-A(D((x....) [f(x)) = 8/y
(2)
Sharpness Aware Minimization (SAM) Consider a model f X - Y parameterized by a weight vector and a per-sample loss function 1: W x X x Y R4. Given a dataset S [(X1, Wi..... (Xn, yo)] sampled i.idd. from data distribution, the training loss is defined as Ls(w) = Cy: f(xi, w)) /n. Sharpness Aware Minimization combines traditional loss with sharpness term to minimize the difference between maximum loss in the vicinity (a Ball of radius Bp)) of defined as the following:
the current minima. Formally,
LSAM(W)= = min Ls (w) + max Ls(w + E) Ls(w)] = min max Ls(w + "EB(())
(3)
w
or <EB(p)
2.1 MEMBERSHIP INFERENCE ATTACKS
Consider a victim model fa trained on dataset D ~ 'D and attack model fa- In a black-box setting, attacker infers whether sample (x,y) belongs D (IN) not (OUT). In this paper, consider two types of attacks: Direct threshold attacks, which directly learn threshold from obtained member/non-mentt member data, such as confidence and entropy attacks; and Shadow model attacks, which train proxy models to calibrate membership scores, such as Likelihood Ratio Attack (LiRA) and Robust Membership Inference Attack (RMIA) (Carlini et al. 2022, Zarifzadeh et al. [0224).
We quantify privacy risk using the empirical attack accuracy, defined as the average of the true positive (TPR) and true negative rates (I-FPR):
1
Panm
ACCMIA
[ff((x,)))
[fa((x)))) 01) =].
2m
2nom
where Tom, Am are the counts of IN and OUT samples. Additionally, metrics such as Area Under ROC Curve (AUC) and TPR low FPR are employed to characterize vulnerability. Further details provided in Appendix D.2
PRIVACY RISKS OF SAM
Inspired by the link between SAM and generalization and how MIAs should exploit poor generaliza- tion, we here scrutinize the membership privacy risk of SAM by comparing the membership attack accuracy (see Equation (4) and Table different MIAS against SAM- and SGD-trained models across five different benchmark datasets for direct threshold attacks and four different benchmark datasets for shadow model attacks.
We utilize datasets and target models that widely employed in studies MIAs and defenses (Yeom et al. 2018 Fang & Kim 2024 Chen et al. 2022 Jia et al., 2011)). Furthermore, we assume that the attacker has access some portion of the training data and non-training data that it uses to train the attack models-a common assumption in the MIA literature.
Datasets We use CIFAR-10, CIFAR-100 and Purchase-100 100 along with two medical datasets Texas- 100 and EyePacs. For direct threshold attacks, we follow Tang et al. (2022) determine the partition between training and test data and to determine the subset that constitutes the attacker's prior knowledge For shadow model attacks, we use a different dataset split to account for shadow model training More details about the datasets and the experimental setup can be found in Appendix G
We adopt and extend the code in ht http:///ttthu..cm//////rrrrrrora htt.a/reeeerrroo///adedeeeeeeees
We adopt and extend the code in http://tww///gggttt.oo.ooooooona

---

Published as a conference paper at ICLR 2026
Methods For direct threshold attacks, we train set of models and choose the one achieving highest validation accuracy. We then employ different MIA methods namely NN-based, entropy-basee -asedd and modified entropy-based attacks (see Appendix D.2 for a detailed formulation of each MIA) to evaluate the attack accuracy on the target model. For shadow model attacks, we generate 256 random half-splits of the training dataset into member set and non-member set and train a model for each split. One model is chosen as the target model and all the other models are used shadow models for reference. More details about the experimental settings can be found in Appendix H
refidenne--aased
Results Direct Threshold Attacks We report the attack accuracy and test accuracy for each model in T We observe that while SAM achieves higher generalization performance, it also incurs highest attack accuracy for almost all settings. To further investigate the connection between flatness of minima and membership privacy, we report the results for other sharpness-ane -erppeess-awrre optimizers and custom designed optimizer that explicitly aims to find sharper minima in Section D The results support relationship between loss landscape geometry and membership privacy, with other optimizers exhibiting similar behavior. For confidence threshold attack, we observe that SGD model incur higher number of extremely confident predictions compared to SAM for both correctly classified and wrongly classified non-members. Because the threshold is typically set near a high value (i.e. 0.92), non-member samples with high confidence are missed by the attacker. From this observation, conjecture that variance of the model output is an important factor in MIA risk. To confirm that these results are not mooeel-dependent we report an ablation study in Appendix | and verify that similar findings can be observed for different model architectures over the same datasets.
Shadow Model Attacks Table 2 summarizes the results for online Robust MIA (RMIA) and Likelihood Ratio Attack (LiRA), reporting Attack Accuracy, AUC, and TPR at 0.1% FPR averaged over 10 attack splits (Zarifzadeh et al. 2024] Carlimi al. 2022). Consistent with the previous results, SAM achieves higher generalization while incurring non-trivially higher privacy leakage across nearly all settings. We note that the reported test accuracies differ from Table to difference data splits for shadow model training.
The results for offline attacks are presented in Table 4 Analogous to the online setting, SAM exhibits superior generalization paired with heightened MIA vulnerability. Since shadow model attacks represent the state-of-the-arr in membership inference, these results suggest that SAM's vulnerability is not merely artifact of global threshold shifts. Instead, it points to an intrinsic geometric property of SAM that persists even under rigorous, sample-specific SOTA attacks.
4 SAM LEARNS ATYPICAL SUBCLASS FEATURES MORE
To investigate the source SAM's 's increased membership privacy risk, we analyze its optimization behavior through the lens of sample memorization and influence. We follow the procedure of Feldman Zhang (2020) to compute the memorization and influence scores for SAM-trained models on -UFAR-100. These scores are computationally burdensome to compute, hence we make them publicly available We then compare these scores against the publicly available scores for SGD- trained models on the same datase enabling direct comparison of sample-level behavior between the two optimizers.
4.1 SAM MEMORIZES ATYPICAL SUB-PATTERNS MORE
We first focus comparing the memorization behavior of SAM and SGD. Figure ((a) shows kernel density estimates of memorization scores for both models. Although the overall shapes of the distributions are similar- reflecting the long-tailed nature of the dataset -, SAM exhibits lower density at the lowest end of the spectrum, with the mass redistributed more evenly across the rest of the range. This indicates that SAM assigns higher memorization scores more broadly, suggesting a structured memorization of more diverse patterns compared to SGD.
To further investigate this behavior, we plot the memorization scores of individual CIFAR-100 samples under both SAM and SGD in Figure ((b) Each sample is represented as blue dot, with its
httrass/attthbb.cm///ffooon////s men ine___eeo_soses
http:://phoskkro.gtttooooooo/
5

---

Published as a conference paper at ICLR 2026
SGD
1.0
1.0
Over lpp
Denskor
0.6
0.6
0.2
0.2
Ridge inteppaation
Ridge Interpoation Equal Memortzation
0.0
Equal Memoniation
0.0
mom
(a)
(b)
(c)
Figure 1: (a): Memorization score density plot for SAM SGD. SAM has less density in the lowest range, but more density spread evenly across the remaining range. (b): Memorization scores of CIFAR-100 100 training samples under SAM and SGD. The regression curve (in green) shows consistent deviation from the identity line (in red), indicating that SAM memorizes a larger subset of samples in the lower score range which likely to be atypical subclass samples. (c): Visualization of samples more memorized by SAM for the tiger class following the same setting of (b).
and y-coordinates corresponding its memorization score under SGD and SAM, respectively. The red diagonal line denotes equal memorization across both optimizers. Samples above this line and the left (top-left quadrant) are more memorized by SAM, while those below and the right (bottom-right quadrant) are more memorized by SGD.
regression analysis over all samples shown via the green curve reveals a consistent deviation from the identity red line, skewed towards the top-left quadrant. This indicates a systematic increase in memorization for large subset of samples under SAM. Crucially, this deviation is not concentrated at the high end of the memorization spectrum. This finding together with the kernel density plot - supports our hypothesis that SAM does not simply memorize pure noise samples, but rather focuses on non-dominant, atypical subclass samples that are underrepresented in the training distribution. Indeed, SAM were picking up sample-specific noise, we would expect a sharp concentration of kernel density at the highest end of the spectrum and deviation of the regression curve the top-right quadrant.
Figure [ce illustrates this phenomenon within the tiger class. Samples with higher SAM memoriza- tion relative to SGD (top-left region) tend to depict clean samples containing atypical sub- uub-pattems- e.g., close-ups of tiger heads, tigers in water, multiple tigers in single image. These visually distinct yet semantically consistent with the class label. In contrast, samples with high memorization under both SAM and SGD (top-right region) often contain sample-specific noise, - e.g., tiger with shiny paws on a pitch-black background which are less likely to generalize.
4.2 SAM INCREASES INFLUENCE OF HIGH MEMORIZATION SAMPLES
We here analyze how memorization affects the influence training samples test predictions. using the influence metric defined in Equation (2). Following the setup of Feldman & Zhang (2020), we first filter training-test sample pairs with influence scores above 0.2 to exclude non-influential cases. We then group training samples by memorization intervals defined as / mem(.A, D, < with I and ranging from 0 to in steps of 0.05 and, for each interval, we select the 20 training samples achieving the highest influence score test data. This yields distribution of influence scores of the most influential training samples conditioned on their memorization levels.
Figures ( and 2(b) show the resulting distributions for SGD and SAM, respectively. As in Figure (b) we regression curve (green line) to highlight the trends. For SGD, influence scores incur in steep transition from lowly influent samples to highly influent points at the upper end of the memorization spectrum. This indicates SGD's reliance on a very narrow set of highly memorized samples. In contrast, SAM exhibits smoother transition curve, with larger set of high (and mid-to-high) memorization samples contributing more consistently to test predictions. This supports earlier finding that SAM emphasizes of atypical, non-dominant subclass patterns.

---

Published as a conference paper at ICLR 2026
1.0
Training Samples
Ridge Regression
1.-
Ridge Regression
Samples
0.50
Training Samples
0.00
0..50
-1.00
0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
10
-0.6-0.4-0.2 0.2 0.4
menhoo
memtue
(a)
(b)
Figure 2: (a) and (b): Distribution of the influence scores of the 20 most influential training samples over each memorization interval for SGD (a) and SAM (b). The regression analysis (green lines) shows that SAM maintains smoother influence distribution, relying 0.85) memorization samples (subclass features) than SGD. (c) Difference in influence scores between SAM and SGD function of memorization low-memorized samples and selectively amplifies the influence of mid-to-high memorization samples.
mid-to-high (0.6
differences. SAM downweighs influence for
(a) Bicycle
(b) Tiger
Figure 3:
Test images (boxed) from buckets I and 5 and their respective top-10 10 influential training images. For each object the top row is image from bucket and the bottom row is an image from bucket 5. For bucket images (higher memorization.top row), notice that the images are atypical for their classes, and there is a near duplicate in the training data that was important for generalizing this test image. For bucket 5 images, on the other hand, the top influential images are reminiscent of the test image conceptual level.
To further validate this, we examine the difference in influence scores between SAM and SGD as a function of their memorization score differences (Figure 2(c)). Training samples which are more memorized by SGD tend to have lower influence under SAM. suggesting that SAM down-weighs the influence of its low- meworized samples. Conversely, samples with similar memorization under both optimizers but higher influence under SAM tend to lie in the mid-to-hee memorization range. These are precisely the samples containing atypical subpatterns that SAM selectively amplifies, confirming our intuitions.
7

---

Published as a conference paper at ICLR 2026
4.3 SAM's GENERALIZATION GAIN COMES FROM HIGHER MEMORIZATION OF SUB- BUB-PATTERNS
In this section, dissect the generalization gains of SAM finer granularity by constructing metric that divides the test data points into groups based on the amount of memorization used for predicting them. We then compare the performance on each group between SGD and SAM.
We measure the typicality of a test data point as the entropy of its corresponding training samples' influence scores. We rely on this measure since, in practice, the prediction of a typical unseen sample would be evenly influenced by numerous training data points within the same class, while atypical counterparts would be heavily influenced by handful of training samples that are themselves also atypical. To measure the even spread, we normalize the influence scores and leverage entropy. Even influence spread would follow a more uniform distribution resulting in high entropy, while uneven spread incurs low entropy. Formally, for each test data point i, let S, be the set of influence scores of all the training points in the same class. Let S be the influence score of jun training point and m be the number of training points in the same class. Then, our entropy metrie Tent is defined as:
S, E,S,.
Tent [i]
Σ 1=1
Pi.j log where, Pi.j
(5)
We group test data points into 5 buckets in the order of lowest Tent to highest Zent. We present some test images and their top-10 10 influential training images in Figure 3 from bucket and bucket 5. The figure illustrates that images from bucket tend to be atypical images e.g., bicycle alongside people, white tiger, etc, for their respective labels while images from bucket tend to be more typical images- e.g., typical bicycle and yellow tiger. For quantitative verification, of memorization scores of the highest influencing training points from each bucket. We observe that lower numbered buckets are associated with high memorization and vice versa (see Figures 4bb) and 4(c)). The results for other buckets interpolate between those of bucket 1 and 5, and are skipped for brevity.
plot the distribution
We compare the generalization gains of SAM against SGD on each of these buckets and show the results in Figure 4(a For test data points bucket 5, SGD achieves negligible performance gain, while for bucket SAM achieves a significant gain w.r.t. SGD. Thus, the performance gains of SAM can be attributed to it correctly predicting more atypical data points which need more memorization of atypical sub-patterns to be classified correctly. For further validation, we generate a synthetic dataset and illustrate SAM's capability of learning atypical subclasses better than SGD in Appendix We theoretically analyze how stronger alignment with minority subclass features can lead to better generalization in Appendix B
Summary of experimental findings These results together with those in Sections 4.1 and provide strong empirical evidence that SAM's increased performance derives from better capturing atypical but informative sub-patterns, allowing predictions to be less dominated by the majority feature. For SGD, we posit that this results in a higher output sensitivity, yielding more confident predictions on unseen data. Paradoxically, this high variance acts as cloak for membership privacy, as it mimics the high confidence usually reserved for members. In the next section, we theoretically analyze how sharpness-regularizztioo can inherently suppress prediction variance on unseen data.
5 THEORETICAL ANALYSIS
In this section, we provide a theoretical foundation for the variance-shrinkage effect of geometry- aware solutions and its implication for membership inference risk. To obtain a clean and analyzable characterization of SAM's geometric bias, we study perfectly interpolating regime of overparame- terized linear models. Closest our analysis in spirit is that of Tan et al. (2022). which explains how parameter size and ridge regression affects membership risk via a variance gap. In contrast, we introduce curvature-aligned geometry modeling SAM and show that it provably increases membership risk.
'These results do not consider image transformations (e.g. random crop, rotations), however we have also replicated the experiment with transfommatoos, obtaining similar trend.
8

---

Published as a conference paper at ICLR 2026
0.95
0.90
930
0.85
0.50
Count
0.75
0.70
0.65
2
Iae bucket
man of top-influancinn train points
mem of top-iinituenchng train points
(a)
(b)
(c)
Figure 4: (a): Test accuracy on Tent groups as evaluated by equation 5 SAM's performance gains comes from it correctly predicting more atypical data points that need memorization of atypical sub-patterns to be classified correctly. (b) and (c): Distribution of top-1 most influential training point's memorization scores for Tent buckets and 5. Testing samples falling in the lower (higher) numbered buckets influenced by training points with higher (lower) memorization.
Our proofs are written the regression form XO y. We note that the connection to classification is direct. For separable binary classification with losses such as logistic exponential, gradient descent known to implicitly maximize the E margin, and the resulting predictor geometrically equivalent to the minimum- -noom interpolator of corresponding regression problem (Soudry et al.) 2024 Gunasekar et al. 2017 Muthukumar al. 202T). Motivated by this equivalence, we can consider classification regression to gigh-magnitude targets ys E {-M, +M) for large constant M >> 1 under MSE loss. In this regime, training points are interpolated to +M and therefore have fixed high confidence, whereas test points yield outputs that fluctuate around the decision boundary (0).
Model and notation. We work in finite dimension >> n. Let the population feature covariance be Baxd with E O. Draw iid. samples
24 2),
V: 0 x; Eis
& NO oB), independent of Fi, i I,..., n.
Let X E Rad have rows x. (y,,..., ........,)) Bn) and assume rank(X) = n. Define the orthogonal projector onto the data span and the covariance matrix
- X,
P X
PPP-PPT
Σ
2: ].
The model is defined fa(#) == or.. In the squared-loss linear model, the population Hessian H equals Σ and the empirical Hessian equals E
Now introduce the geometries employ to model SGD and SAM.
Min-G interpolation. G-norm interpolant:
For any symmetric positive definite matrix G > O, consider the minimum-
t == arg min T s.t. X0 = y. dege 2
We compare the standard Euclidean case,
Go = la
with the sharpness-aveee aware geometry,
G = 7
17 0.
Classical implicit-bias results identify standard SGD with the Euclidean case under step size and weight initialization assumptions (Zhang al. 2017). Sharpness-aware geometry, the other hand, reflects SAM's penalty on local sharpness, which can be shown by expanding the minimization objective. We prove that, relative to the Fuclidean interpolation, the geometry G. strictly reduces

---

Published as a conference paper at ICLR 2026
the variance of non-member outputs, thereby enlarging the separation between members and non-
members and increasing the advantage of membership inference attacks.
We first establish that SAM strictly reduces the variance of the model's output on unseen data. By
Lemma for a non-menne non-member sample Xout. and the output follows fa(Xout) NiO, o²)-
Theorem I (Variance strictly decreases for SAM geometry). Under Assumption there exists 7jo > 0
such that for all v E (0, 1o),
a < GG. with probability o(1).
Remark Theorem formalizes a simple geometric picture. The minimum-G interpolant balances fitting the training span against penalizing components in directions where G' is large. Moving from Go to G, increases the penalty precisely along curh-curvature directions of the Hessian. Lemma 2 shows that, under the overlap condition in Assumption I
this reweighting strictly suppresses the - P)E0G. component of the interpolant. Since non-member predictions depend on 0g only through the quadratic form 0g200. this suppression translates directly into a strict decrease of the output variance unseen data.
Next, we consider confidence-thresholl attack and likelihood ratio attack.
Confidence-thresholld attack. The attacker uses the confidence score Confg(2:) := [x)) and predicts "member" iff Confa(x) 2 Let (Xins Yin) be random training pair and Xout (0, E) an independent non-member. Define attack advantage
Adv := sup (TPRo(T) - (PRc(T)).
+20
Theorem 2 (SAM strictly increases confidence- based MIA advantage). Under Assumption 7 for all sufficiently small 1j 0,
Adv Adv with probability 1 o(1)
Remark 2. For the confidence-thresholdd attack, interpolation implies that member confidences geometry-invarian: the training logits are fixed (up to label noise) for any choice of G. Thus, changing the geometry from Go G., leaves (F) unchanged for every threshold т. while Theorem D strictly reduces the non-member variance and hence lowers FPRc(r) at every т > 0. In other words, SAM pushes non-mene member scores closer to the decision boundary, sharpening the separation between the two confidence distributions.
Likelihood Ratio (LR) attack.
The oracle LRA predicts "member" iff Ac(fc(z)) 2 Define
(s)
(s log
Adv LR
sup (Pr(Ac(fc(Xm)) - 2 t)). tER
Pout (s; G)
Theorem 3 (SAM strictly increases LR-attack advantage). Under Assumption 7 for all sufficiently small 1) > O,
AdvLR
Adv
with probability o(1),
where G, I+=
Remark The argument is similar to Theorem Theorem analyzes an oracle LR attacker that uses single global IN/OUT distribution. Corollary strengthens this to sample-adaptive IN/OUT distributions, matching the per-query calibration used by LiRA/RMIA. We note that while Theorem and Theorem state an improvement in the supremum advantage, the proof in fact shows stronger statement: for all sufficiently small r) > the entire ROC curve of the confidence-bsse attack under G, strictly dominates that under Go. This is empirically verified in Figure 7 where SAM's 's ROC curve above that of SGD's for nearly the entire range across most settings.
CONCLUSION AND FUTURE WORK
This work investigates the mechanism behind SAM's dual effect: superior generalization coupled with heightened privacy leakage. As algorithms seeking flatter minima are widely employed, our work serves cautionary tale users should be aware of. Furthermore, extending to devise an effective defense against Membership Inference Attacks would be beneficial to the community. Finally, future research is encouraged to further scrutinize the implicit bias of SAM.
findings
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the European Commission under the Horizon Europe Programme as part of the projects MLSysOps (Grant Agreement #101092912) and REWIRE (Grant Agreement #101070627).
REFERENCES
Andrea Agiollo, Young In Kim, and Rajiv Khanna. Approximating memorization using loss surface geometry for dataset pruning and summarization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, pp. 17-28, New York. NY. USA. 2024. Association for Computing Machinery. ISBN 9798400704901. doi: 10.1145/3637528. 3671985. URL https::::///o org/10 114..1145550 1145/3637528 3671985
Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overparameterized neural networks, going beyond two layers. Advances neural information processing systems, 32, 2019.
Maksym Andriushchenko, Dara Bahri, Hossein Mobahi, and Nicolas Flammarion. Sharpness- aware minimization leads to low-rank features. arXiv preprint arXiv:2305. /62, 2023. URL NeurIPS 2023 poster version.
httpss:/aaraiv.orr//aaass
6292
Christina Baek, Zico Kolter, and Aditi Raghunathan. Why is sam robust to label noise?, 2024. URL
httpss//acxivvoor///bb/ htt.a/////20.3366
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voor Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In USENIX Security Symposium, volume 6, 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles, 2022. URL httlttts////xiv.oort
2112.03570
Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34:22405-22411 2021.
Wei-Kai Chang and Rajiv Khanna. A unified stability analysis sam vs sgd: Role data coherence and emergence of simplicity bias, 2025. URLlhttts://axxii// wwtp:/xwwwwraaa///e/axaabaaa
Dingfan Chen, Ning Yu, and Mario Fritz. Relaxloss: Defending membership inference attacks without losing utility, 2022. URL hul_leppss//axxi..crr//////////
Zixiang Chen, Junkai Zhang, Yiwen Kou, Xiangning Chen, Cho-Jui Hsich, and Quanquan Gu. Why does sharpness-aware minimization generalize better than sgd?, 2023, URL https: //arxii
231..b///20102 /xc//bb/2331.00725
orq bog/abs//20
Ganesh Del Grosso, Georg Pichler, Catuscia Palamidessi, and Pablo Piantanida. Bounding informa- tion leakage in machine learning. Neurocomputing, 534:1-17, May 2023. ISSN 0925-2312, doi: 10.1016/j.neccom..o www.com.222222222 2023,m002202222 02..058....
URL/http:/ dxx doi org. 101. nww.om. 2023
02. 058
Jiawei Du, Zhou Daquan, Jiashi Feng, Vincent Tan, and Joey Tianyi Zhou. Sharpness-awan training for free. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Xingli Fang and Jung-Fun Kim. Center-based relaxed learning against membership inference attacks, 2024.
Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium Theory of Computing, pp. 954-959, 2020.

---

Published as a conference paper at ICLR 2026
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881-2891,, 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020.
Suriya Gunasekar, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, and Nathan Srebro. Implicit regularization in matrix factorization, 2017. URL httpt::///ati...t//////a
09280
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Av- eraging weights leads to wider optima and better generalization. arXiv preprint (XXiv:800300500 2018.
Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang. and Neil Zhenqiang Gong, Memguard: De- fending against black-box membership inference attacks via adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp. 259-274, 2019.
Yigitcan Kaya and Tudor Dumitras. When does data augmentation help with membership inference attacks? In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 5345-5355. URL https: http::///occeeedeeee mlr S.l..ppres/vv33
PMLR, 18-24 Jul 2021.
html
Minyoung Kim, Da Li, Shell X Hu, and Timothy Hospedales. Fisher SAM: Information geometry and sharpness aware minimisation. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 11148- 11161. PMLR, 17-23 Jul 2022.
Young In Kim, Andrea Agiollo, and Rajiv Khanna. Samosa: Sharpness aware minimization for open set active learning, 2025. htttttss://axxi...rrrrtt 26777
Yiwen Kou, Zixiang Chen, Yuanzhou Chen, and Quanquan Gu. Benign overfitting for two-layer relu convolutional neural networks, 2023. htlthttss///xxiii oww.ass/.rg/a///////oooo... 220.p///3333.4433
Jungmin Kwon, Jeongseop Kim, Hyunseo Park, and In Kwon Choi. Asam: Adaptive tharpness-aware minimization for scale-invariant learning of deep neural networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 5905-5914 PMLR, 18 18-2412 Jul 2021.
Zhenlong Liu, Lei Feng, Huiping Zhuang, Xiaofeng Cao, and Hongxin Wei. Mitigating privacy risk in membership inference by convex-concave loss, 2024. URL httpt:t////c....... https::/axxi..or///////. 24C2. 05453
Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen. Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04880 2018
Vidya Muthukumar, Adhyyan Narang, Vignesh Subramanian, Mikhail Belkin, Daniel Hsu, and Anant Sahai. Classification vs regression in overparameterized regimes: Does the loss function matter?, 2021. htRLhhttts///axiv....t
abs 2005. 20050080544000
Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regularization. In Proceedings the 2018 ACM SIGSAC conference on computer and communications security, pp. 634-646, 2018
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-bol box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pp. 739-753. IEEE, 2019.
12

---

Published as a conference paper at ICLR 2026
Matthew D Norton and Johannes O Royset. Diametrical risk minimization: Theory and computations. Machine Learning, pp. 1-19, 2021.
Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprini 1806.018800022444 2018.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017.
Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In USENIX Security Symposium, volume 1, pp. 4, 2021.
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson. Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent on separable data, 2024. URL wttph:ttttpssttteeee http//awxx..aggaaaaaaaa/aa 1710
10345
Jacob Mitchell Springer, Vaishnavh Nagarajan, and Aditi Raghunathan. Sharpeessawwrre tion enhances feature quality via balanced learning, 2024. URL
24C5 20439
Jasper Tan, Blake Mason, Hamid Javadi, and Richard G. Baraniuk. Parameters privacy: A provable tradeoff between overparameterrizatoo and membership inference, 2022. URL https:
Xinyu Tang, Saeed Mahloujifar, Liwei Song. Virat Shejwalkar, Milad Nasr, Amir Houmansadr, and Prateek Mittal. Mitigating membership inference attacks by "Self-Distillation) through novel ensemble architecture. In s1 USENIX Security Symposium (USENIX Security 22), pp. 1433-1450, 2022.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 33:2958-2990 2020.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy Risk in Machine Learning: Analyzing the Connection Overfitting In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp. 268 282, Los Alamitos, CA, USA, July 2018. IEEE Computer Society. doi: 10.1109/CSF.001 wtt.:/SS...1..02222222 org/100 1109/C5F 2018 00027
URL httpp://codiiieecccoprreeeeeee
Samuel Yeom, Irene Giacomelli, Alan Menaged, Matt Fredrikson, and Somesh Jha. Overfitting, robustness, and malicious algorithms: A study of potential causes of privacy risk in machine learning. Journal of Computer Security, 28(1):35-70. 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.0714 2016.
Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-pone power membership inference attacks, 2024. URL
03262.
Chiyuan Zhang, Qianli Liao, Alexander Rakhlin, Brando Miranda, Noah Golowich, and Tomaso Poggio. Musings deep learning: Properties of sgd. Technical Report CBMM Memo No. 067, Center for Brains, Minds and Machines (CBMM), MIT, 2017. URL https: //tpm/.c.mmmmmm mit hebb///ttes//fffall//lleees.s
06 000
Juntang Zhuang. Boging Gong, Liangzhe Yuan, Yin Cui, Hartwig Adam, Nicha Dvornek, Sekhar Tatikonda, James Duncan, and Ting Liu. Surrogate gap minimization improves sharpness-aware training, 2022. URL https: //arxiv... org/ abt.///..22222/. 08065
13

---

Published as a conference paper at ICLR 2026
Clern
Coso halial
Oan hole
2
-sa -62 -1s
4g (c) SAM vs SGD (Large sample size)
(a) Class contours
(b) SAM vs SGD (Few atypical ex- amples)
Figure 5: A synthetic construction illustrating the generalization ability of SAM over SGD for atypical examples. Fig (a) shows class density contours of a two-class, 2-dimensional classification problem, along with the bayes optimal solution. The red class has two 'clusters', one representing typical examples and one representing atypical examples. Fig (b) shows an instance of data sampled from densities shown in (a); the larger cluster of red dots represent typical examples in the red class, and the red '+' points represent lot fewer atypical examples. SAM generalizes better than SGD in this case. Fig (c) shows that if there are enough samples generated from both typical and atypical clusters, SAM and SGD coincide with the Bayes Optimal classifier.
A SYNTHETIC DATASET
In this section, we provide a simple synthetic construction that illustrates how SAM can achieve better generalization performance w.r.t. vanilla SGD. The example illustrated in Figure The data is generated from two-dimensional densities illustrated in Figure 5(a] The densities are supported in two dimensions labelled as Tı and x2- There are classes the red class and the blue class. Figure ((a) also shows the Bayes Optimal classifier. The red class has two 'clusters', one representing the typical examples (e.g. yellow tigers), and the other representing the atypical examples (e.g. white tigers). The data is sampled in such a way that we have several samples from the typical cluster, while there are only a few samples from the atypical cluster in the red class (as shown in Figure 5b)). Figure ((b) further shows that seeking flatter minima using the SAM optimizer learns a classifier that is closer to the bayes optimal solution than the one learnt using vanilla SGD. Therefore, SAM generalizes better. This difference in performance vanishes in Figure 5(c) when have large sample size for the atypical examples. This synthetic construction shows that one possible reason that SAM can perform better is its tendency to memorize atypical samples more than vanilla SGD. In other words, the gain in generalization could potentially come from those atypical data subgroups. In the next subsection, empirically verify this conjecture for the CIFAR-100 dataset.
B BETTER MINORITY SUBCLASS ALIGNMENT LEADS TO HIGHER GENERALIZATION
In this section, we theoretically analyze setting involving majority subclass samples, minority subclass samples, and pure noise samples to illustrate how an overfitting model generalizes better. Driven by the motivations in the previous sections, we discern model by the amount of minority subclass feature it captures. Theorem 4 shows that this property leads to higher generalization. The proofs are in Section C33
Definition 1 (Data Model). Training dataset D = +xxx,,)))= is sampled i.i.d. from the data distribution D. We define x = [x1, xa), y E [+1], X1 E IRd: X2 E Rd2 x3 E Rds, We consider overparameterized regime where d1, d2, d3 > We consider linear predictor for classification, (x) = sign (w, x) with w = W1, W2, wa]. We assume perfect interpolation satisfying finite margin Vi 0 S mg S m, M co, where margin m, w. xi). Let M, S,N denote the majority, minority, and noise subsets, respectively. D = MUS N. We define
a(=) = 1/(1 + exp=x)
14

---

Published as a conference paper at ICLR 2026
D is mixture distribution
D = PMMMDM DM + Ps ppsDa Ds + PN DNs
with mixture weights Pm + ps + PN 1, and PM > ps. PN. Here, DM generates majority samples with x, = y X2 ~N,00, Id,), X3 ~ (0, Iag) where μn is a fixed vector; Dss generates minority samples with X1 V, X2 yuu. X3 NN, Ida) where μ2 is a fixed vector and is random vector anti-aligned with W1 ((W1,V) < 0); and Dxx generates pure noise samples with x (O, dd:-44dd))) Let (n.M, ng, ny) denote the counts of majorit, minority, and noise samples in S.
Our model easily translates to a logistic regression model we set Jw (x) = ( x)) and change the label to {0, 13. The setup in which part of the input contains the true signal has been commonly used in previous works (Chen et al. 2023 Kou et al. 2023) but we generalize their setup by including and analyzing subclasses in the design. Concretely, one can think of a minority sample as belonging to a long-tail subgroup of the class that requires a different feature to be recognized. We formalize anti-alignan anti-alignment condition: the minority features arranged such that any model that heavily prioritizes the majority feature u will perform poorly on the minority. Intuitively, fitting the minority subgroup requires the model to memorize an alternative pattern that independent of (or even interfering with) the main decision boundary for the majority. We assume the minority subgroup very small, so that by default standard empirical risk minimizer might deem it negligible. This resonates with the long-tail phenomena observed in practice-i.c. a handful of unusual examples exist that a model could easily ignore without significant impact on overall training loss. However, those examples are crucial for tail generalization as they represent rare yet valid concepts that an ideal model should learn. Our assumptions reflect prior findings that real datasets contain such long- tailld subpopulations which must be memorized to achieve the best possible accuracy [eldman (2020). Formally,
Condition 1. For each minority point i S, define
Bi == - -/s (s( v3) > O,
A (W2,22)
Let B be a random variable with CDF (A(() Pr(B < A) such that B 2 B: (i.e. Fn is the law/distribution of the B 's when is drawn uniformly from S). We assume
A <
B Bmax
Buax == sup b e IR Fb(b) < <1}.
This condition assures that the majority feature still dominates globally. Furthermore, we formulate v as a random variable to effectively capture multiple atypical subclasses. For example, there can be wide range of tigers that purely white yellowish-white By modeling v as random variable, it provides variation on the strength of anti-alignment with the majority feature.
Driven by the empirical motivations in Section 4 define ordering of the models as how much minority subclass alignment (MSA) they achieved. Formally,
Definition 2 (Minority Subelass Alignment Order). Given two interpolating solutions w(A), w (B) trained on the same S, define
(A))
(A) .y22),
(B) (B) = (w2
.1122).
We write
(()) MSA
(
>
and say that (A) has higher minority subclass alignment than w if
(A)
2 A and
(B) Better Generalization). Let w(A), (B) be two interpolating MSA
A < Bmax-
Theorem 4 (Higher MSA
solutions trained on the same D. Under Definition and regulatory conditions, if ((A) then
(B),
Pr (y(w x) > 0| (x. 1) ~ D) 2 Pr (v(w( ()),)) > 2 ~ D), with strict inequality (F) > 0.
Theorem 4 shows that the interpolating model that aligns with the minority subclass feature more generalizes better.
15

---

Published as a conference paper at ICLR 2026
C
PROOFS
Assumption 1.
We make the following assumptions:
(i) (Kernel overlap) Let P == X (Xx")=- be the orthogonal projector onto span(X )).
The population-sharpeese direction has nontrivial mass outside the data span:
II P) 0G. cc0
with probability 1 ((1) for some constant c.
(ii) (Bounded interpolator) The Euclidean interpolator satisfies ||0c.11 = Or(1).
Remark 4 (Justification of assumptions). Assumption Ti) generic in the overparameterized regime (d > n²): the vector Edg, a population signal direction, while span(X]) is a random n- dimensional subspace. Under mild spectral regularity of Σ, their alignment is not perfect with high probability, yielding a non-negligible (T - P) component.
Assumption I (ii) is standard for benign overfitting and ensures finite test variance. If ||0G, ll diverged, the Fuclidean interpolator would have exploding prediction variance and thus be unstable on unseen data.
Lemma 1, The unique solution of equation 6 is
8 =- (XG-XXT) -1
and X0G y-
V.
Proof. Form the Lagrangian T.(0,)) 30 G0 The KKT conditions are G0+XT X O and X8 = y. Eliminate - to get -X = y, hence À = >
Substituting back yields the stated 0g. Since G > 0, the objective is strictly convex and the solution unique.
Lemma 2. Ler 0(m) = GG,, with Gy = I + 1/ Then
(0) = - (())
d
= -- Ed
d1
===
Proof. By Lemmal I
0(1) = x
(x'X((XG,XX]
Differentiate a) 0.
i) 4 1 - -G so dn
d G m== =.
iii Let A(r) == XG x T Then A'(0) = T and
A
7=0 (x0
Combining (i)((i),
(0) = - X (xX - (xxx) (XXX)((XX))) -1 P)P AG.
Lemma (Distribution of model output on non-member data). Let Xout ~ NN Σ) independently of the training set. Conditioned on dg. the prediction fa(Xout) = Xour satisfies
fo(Xout) o3),
ae 0g.
Proof. Condition on 0g. Since Xout is Gaussian and fa is linear in Xouts the claim follows with variance 0% E.g.
16

---

Published as a conference paper at ICLR 2026
Lemma 4 (First derivative of non-member variance at 17 0). With G, + mp., ler (2) ((n) _0()))))n)))))) Then
o (0) = 20)) - -
Proof. Differentiate: 0²0 = 2 2. 2 (0) since 2 is symmetric. Insert Lemma
000 = -2 20 Σ(I P) = -2 - P)IdG, I|
using symmetry and idempotence of (I P).
Proof of Theorem
Proof. By Lemma
x²'(0) = - 112
Assumption
1 yields I - P 2 c. hence o²'(0) - -2c² with probability 1 o(1).
Therefore, on the same high- probability event, there exists 170 > 0 such that for all (0, nol.
Since (()) smooth 1j for G, > 0² (n) differentiable
o (n) 0² 0² (00) + o(7) (0).
This implies o² < < for all sufficiently small > 0 with probability 1 o(1).
C.I CONFIDENCE-TTIRSSIIIN ATTACK
Let Im be the index set of member training points, and let {(zs, Wi)ee, denote the training samples. For geometry G O, let 0G IRd be the (interpolating) solution of equation 6 Define the signed score and confidence by
fa(z) = 1z
Confo(x) = fa(x)) Ozz
We model black-box confidence- thrreshold attacker as follows. Let / be a random index drawn from Ii (e.g., uniformly), and let the random member pair be
(Xin- Yin) == (x1,
Let Xout ~ NN Σ) be an independent non-member (test) input, independent of the training set and algorithmic randomness. For threshold 0, define
TPRc(T) = Pr(Confa (XLu) 2 T),
(7)
FPRo(T) := Pr(Confe (Xout) 2 T),
(8)
Adv = sup (PRR(T)) FFRR(T))
(9)
720
Lemma 5. For any geometry G-0, O,
Confo(Xin)
Yinl almost surely.
Proof. Because Og interpolates the training data, we have Of = for every i E Im. Therefore
for the random member index T,
Confa(XX.)) 06 lyrl |M/ a.s.
Lemma 6. Conditioned on iG.
Confa(Xont) 121 -~ ~~1000
Proof. By Lemma 3 and absolute value.
17

---

Published as a conference paper at ICLR 2026
Lemma 7. Let 0 on d2 and let Z₃ (( (1.2).
Then for every
Pr 2 7) < Pr( 2 7).
Proof. Let (O, 1). Then 2 d 0x UkU, so
Pr(1Ze| 2 T) = Pr U0 -
ok
The function t + Pr([U] 2 t) is strictly decreasing on (0, oo). Since T/o1 > T/02, the claim follows.
Lemma 8. Assume Pr(
0)
(true whenever labels have any continuous noise). Then
Adv comf sup
- (2)(()))FFP((T))
+20
Proof. A = 0, (P)a(0) = (0) = 1, so the gap equals 0. Because Pr([Vin > 0) > 0, we have (T) > 0 for some J > Also, by Lemma FPRc(7) - 0 as - 0o. Hence there exists T > such that (P) - (P) > 0. Therefore the supremum cannot be attained at 0, and we may restrict to r >
Proof of Theorem 2
Proof. By Lemma member confidences are geometry-invariant, so TPRg, (T) TPRG, for all 2 0.
By Theorem for all sufficiently small vı >>
G < Z with probability 1 - o(1).
Condition this high-probability event. Lemma 6 implies Confa, (Xout) |zyl with Zn (0 o and similarly for Go. Then Lemma 7 yields that for every T > 0,
FPRG, (r) Pr([Z,) 2 T) Pr([Zol r) (P)
Therefore, for every T >>0 0,
TPRa, (T) FPRG, (T) TPRG. (T) - FPRG.
Taking the supremum over and using Lemma
Adv conf = sup TPRo, (7) FPRG, (T) G
(T) > sup (T) -FRR..(T) = Ady conf +>0
Go
>>0
This holds with probability 1 1 o(1.
C.2 LIKELIHOOD -AATTOOOO--RATOO RATIO ATTACK
We consider oracle likelihood-ratio attack that the model output fa(x) 03a The attacker knows the true member and non-member score distributions and performs the Neyman Pearsson likelihood-ratio test.
Score distributions. Recall the data model (0, Σ) and 8 x + with ~ NN0 of) independent of æ. Let (Xin. Yin) be random member training pair obtained by sampling a random index from the training set. Unconditionally over the training data draw, we have the same marginal law as a fresh sample:
X ~ A00 E), Lemma (Member score is geometry-invariane Gaussian). For any geometry G. the member score satisfies
Y,, T + E.
fa(Xmn) = Ym a.s.
and hence
fo(Xin) ~ NO vad;
where Unn - 500
oz.
18

---

Published as a conference paper at ICLR 2026
Proof. Interpolation gives X0g y, for every training point 00 yv- Thus for random member index I,
fa(Xin) = 0/ = yr Yin a.s.
Unconditionally, Yn = 1 Xin + & with Xin ~ N( 2) and ~ N(0, oy) independent, so it is
mean-zero Gaussian with variance Vin-
Lemma 10 Non--member score is geometry-dependee Gaussian), Let Xout ~ (N,00, 2 indepen- dent of the training set. Conditioned on fg.
fx = Xout N(O, Vout (G)),
Vut (G) : D0;
Proof. Same argument as Lemma
Oracle tikelihood-ratio test. Let and Pout be the densities of (O, Unn) and N(0, Vout Vout (G)), respectively. The oracle LR score
(s log Pin (s)
Pout
Lemma 11. For zero-mean Gaussians with variances Vin and Vout (G) > O, the LR test (s(s) 2 is equivalent to |s] 2 for some 2 0. Moreover, the optimal test for any fixed FPR is of this form.
Proof. For 8 E IR
1 Ac(s)= ( log ei + log Vout (G)
5² 20in
x2
82
+
= CG +
2
2vout (G)
2 Vout (G)
Vin
where CG does not depend on Thus As(s) 2 t is equivalent s² > 72 for some 2 i.e. |s| 2 т. Neyman-Pearson gives optimality of the LR test. Note that as we are in a setting where members are highly confident, Vin Vout-
Define the LR-attack advantage as the best achievable TPR-FPR gap over all two-sided thresholds:
Adv = sup (Pr(Ifo(Xmm) 2 Pr(Ifa(Xout)) 2)).
r20
Proof of Theorem 3
Proof. By Lemma
fa, (Xin) NNO, o.)) for all 17 2 0, so the member tail Pr([sa, (Xin)) 2 T) geometry-invariant for every т 2 0.
By Lemma 10
(Xaut) ~ (O, Nout (G,,),, Theorem gives that, for all sufficiently small 0,
Vout (Gn) oz.
Cout (Gr) < Yout (Go) with probability 1 1-off o(1).
Condition this probabbtbbbblitty event. Then Lemma 7 (applied to (Sc yields that for every
7 T>0 O,
Pr[IJa, 7) Px(nt,,(Xontt
Therefore, for every 7 0,
Pr(Ifa, (Xim)) 2 7) Pr(Ifa, (Xout) 2 Pr(]foo(Xin) 7) -Pr( 2 7).
Taking the supremum over 0 on both sides gives Adv LRt
Adv LR. (As in Lemma 8 0
yields zero gap, so the supremum attained for some > 0.) This holds with probability 1 (1) n
19

---

Published as a conference paper at ICLR 2026
Corollary 5 (Sample-adapp adaptive LR monotonicity). Fix query point
fins NO0 Um,,z): four, (G) NO, Vout, (G)).
Let Gas G5 be two geometries such that = (G,) < Vout, ( Then for every false positive rate (0, 1), the optimal likelihood-ratio test at level a attains strictly larger true positive rate under Ga than under Gb- Equivalently, the per-sample ROC curve under Ga strictly dominates that under Gb, so any MIA advantage is strictly larger under Ga.
Proof. Fix z and k E {a, The LR between N(0, Vin,s) and (O, Vout,z (Gk)) is
Yout, (GR)
s²
As(s;G)) - log
+
Vin,z
2
(ont,=(GRk)
Vin.z
If Vin,s * Vout, (Gx) then (s G&) is strictly monotone function of |s], so by the Neyman- Pearson lemma the optimal level-a test is equivalent to two-sided magnitude test |s] 2 Tx(a) for some unique threshold Tx (x) >
Write (Gutt,((L)) & Uout, (Gx) U with U (0, 1). The constraint Pr([fout, 2 rT(a)) a is then
a= Pr (P 2 V //oun.((Gr)
Since Pr([U] u) is strictly decreasing (0, oc) and Vout, < (oott; this forces
Ta Ta(a)
To(a)
Ta(a) < (a))..
Vout, (Ga)
V /out,:
The member distribution fin, is the same under G2 and Gb, so
Pr fl 2 Ta(a)) > Pr(Ifm.sl 2 rl(a)).
Thus at every FPR level a the optimal LR/LiRA test has strictly larger TPR under Ga. which implies strict ROC dominance and the claimed advantage comparison.
C.3 HIGHER SUBCLASS ALIGNMENT LEADS TO HIGHER GENERALIZATION
Lemma 12 (High-dimensionall near-orthogonality). Lel X XN E Rd have i...d. N 1) entries. Then, as d + D0,
(x,, xx)
d and
| ||,|| = o(1)
for each fixed i + j. with probability tending to Moreover, the two conclusions hold uniformly over all i ( j with probability tending to provided log N = o(d).
Proof. For norms, ||x;||2 ~ (d). Laurent inequality implies that for all / O,
Pr( [x, d 2 2 dt + 2t) Se-
Pr(d- - |x,| 2 2v dt) S _t
Taking & t= edd gives |x 2 = d(1 + O())) with probability at least 2e =e d, hence ||x;|| d(1 + o(1)) w.h.p. and ||x:|| V O1,
For inner products, write xx) = -E== Z& with Ze = K,KK which are i.i.d., mean and sub-exponential. Bernstein's 's inequality yields
x)(K(,,,,)) 2 < 2 exp (- -emin mmmmmtt///// f)
for a universal e 0. Taking = Cvd shows (xx,,,))| = (v Combining with ||x||| [|x]| =
d(1 + og()),
xxx, = Op (1).
|x, ||X|||
union bound over the (2) pairs then gives the uniform statement whenever log o(d), since both tails are exp(-O(d))
20

---

Published as a conference paper at ICLR 2026
Noise weights By the representer theorem, write
W
Σ ==1
By X3.j-
Then, for any training point 6,
yi (w3, xa,i) = 3. |x3,,11 + Si,
;=
33
Under Lemma 12
|x,,i| 2 (()) ds and the cross inner products are x3, |xx,,]]] o(ds)
Condition 1. For each minority point E S, define
B, = - ys (w1, v.) > O.
A = (W2, p2).
Let B be random variable with CDF (B(A) Pr(B < A) such that B - B, (i.c., FB is the law/distribution of the By 's when is drawn uniformly from S). We assume
A < Bmax:
Bmax sup E R (b <1);
Remark 5. For each minority sample i, the anti-alige alti-alignment magnitude B, - (w.. v,) > summarizes how strongly the majority anchor opposes the minority anchor for that sample. We assume (es i.i.d. draws from common distribution Fn supported on (0, Bmax]. Unseen minority test point has margin m = - -B + A + S', with B FB, so Prs(m' > 0) = Pr(B < A) = FB(A) (up the o(1) fluctuation ç' from Lemma 12). Intuitively, (B(A) is the fraction of minority subclasses whose majority anti alti-alignment is not too strong relative to the learned shared minority signal A.
Definition 3 (Generalization gap). The generalization gap Dx E Ds, M,, Dx is defined as
gap for a training point i from distribution
R (w) = Pr(u:(w,x,) > 0}
>). (x,3') Dx.
(10)
Assumption 2 (Majority alignment). We consider models whose first block weights align with the majority subclass feature:
W1 = a
This captures the implicit bias of common ERM procedures (e.g., logistic regression trained by gradient descent, or minimum-fy, -a- interpolation) to align with the dominant signal in the data. Assumption 3 (Majority dominance). We assume the majority signal dominates the stochastic parts:
(Wi 11)
00.
Assumption 4 (Perfect interpolation and fanite-margin). trained model w has finite margins. That is, there exist 0 < mo M Do such that
mg
yo (w) <
M
Vi.
Lemma 13 (Generalization gap of majority subclass samples). Let E M. Then with probability 1 1- ol),),
R.((w) = o())
Proof. By Assumption 4 y: (w, > O. hence Pr (w, x.) > 0}
Draw (x',1)) DM. 0 M1 2/y (O, Ia 2/3 (0, (0, Ida). independent of y'. Then y (w,)'') (W1, 11) (W2, y (Wa, 2n)) (W1, m11) + Z,
where Z is mean-zero sub-Gaussian random variable with variance proxy VarProxy (Z) | ||wa||? ?, since y (2,, ~ (0, ?) and y/ 3W3,, xg) ~ (00 // are independent and centered.
21

---

Published as a conference paper at ICLR 2026
For any >> 0, standard sub-uuusses Goossaussian tail bound yields
Pr(Z - -a} 1 exp
2( (ws
Taking a = (W1, 141)
Pr (u 50} = PP([2= S -(wi, S exp(
<WI po1)
2([[wel + w. ww)l22y/
Hence
Pr(y'(,, 200 0} 2 1--08 -cxp(
(WI. |11
2[[1w2/|2 //w33
By Assumption
(w) Pr/y/(w, > s exp( w(/]wa 2 + || (wal2))
(W1, p41)
=(1)
Lemma 14 (Generalization gap minority subclass samples), Let E S. R (x) = - FB(A) (up to o(1) terms).
Proof. By Assumption 4 3/ (w. x,) 0, hence (r(y x,) > 0} =.
Draw (x',8') Ds. X3 is independent of {xs,}} } and mean- zero; hence y(W3, xS) E,By00 (Xa.j. x3) is a mean-zero fluctuation with variance vanishing relative to x2112 set this fluctuation to Ç² = (1) using Lemma 12 Then, 1/ (w, x') = - A
Pr(y' (w, x²) > 0) = Pr(B < A- E (FB(A- - E), FB (B(A+s))
Letting e J. 0 gives the stated identities up to o(1).
Lemma 15 (Generalization gap for pure noise samples). Let i E N (pure noise).
R (w)
Proof. By Assumption 4 31 (w(,,:;) 0, hence Prfsx(w, x.) >0} 1.
For an unseen noise (x'.8") ~ Dx. each term (',, w (k = 1,2, 3) is a centered continuous symmetric random variable (linear form of mean-zero isotropic vector, independent of s/). The sum remains centered and symmetric; hence Pr(y/ (w, x') > 0) 1/2.
Proof of Theorem
Proof. By Lemma 14 minority test accuracy equals (A). This is monotone increasing in A. The
majority subclass and noise samples yield the same result for both models by Lemmas 13 and
D ADDITIONAL RELATED WORKS
D.I CONNECTION OF FLATTER MINIMA WITH GENERALIZATION GAP
There have been numerous studies Foret et al., 2020 Izmailov et al. 2018 Cha et al. 2021 Norton &
Roysel 2021 Wu et al.) (2200) which account for the worst-case empirical risks within neighborhoods in parameter space. Diametrical Risk Minimization (DRM) was first proposed by (Norton & Royset 2021) and they asserted that the practical and theoretical performance of Empirical Risk Minimization (ERM) tends to suffer when dealing with loss functions that exhibit poor behavior characterized by large Lipschitz moduli and spurious sharp minimizers. They tackled this concern by employing DRM, which offers generalization bounds that are unaffected by Lipschitz moduli, applicable to
22

---

Published as a conference paper at ICLR 2026
both convex and non-convex problems. Another algorithm that improves generalization is Sharpness Aware Minimization (SAM) (Foret et al. [2000] which performs gradient descent while regularizing for the highest loss in the neighborhood of radius of the parameter space. (Izmailov et al.) 2018 proposed Stochastic Weight Averaging (SWA) that performs averaging of weights with a cyclical constant learning rate which leads to better generalization than conventional training. They also prove that the optima chosen by the single model is in fact a flatter minima than the SGD solution. Further, (Cha et al., 202T) argues that simply performing the Empirical Risk Minimization (ERM) is not enough to achieve at good generalization, in particular, domain generalization. Hence, they introduce SWAD which seeks for flatter optima and hence, will generalize well across domain shifts.
D.2 DIFFERENT MEMBERSHIP INFERENCE ATTACKS
There are many variants of Direct Single-query attacks (DSQ) based on the approach of the attack and below we describe the ones used in our experiments:
NN-based attack (Shokri et al. 2017; Tang et al., 2022; Nasr et al. 2018) This is the first MIA proposed by Shokri et al. (2017) where they use binary classifier to distinguish between the training members and the non-members using the victim model's behavior on these data points. The adversary can utilize the prediction vectors from the target model and incorporate them along with the one-hot encoded ground truth labels as inputs. Then, they can construct a neural network (INN) called attack model.
Confidence- attack (Yeom et al., 2020; Salem et al., 2018; Song & Mittal 2021) the highest prediction confidence of an input record exceeds a predetermined threshold, the adversary considers a member; otherwise, it is inferred as a non-member. This approach based on the understanding that the target model is trained to minimize prediction loss using its training data, implying that the maximum confidence score of a prediction vector for a training member should be near 1. The attack / is defined as follows:
/eon/((y/xx) = 1 (max p(y 2)
(11)
Here, 11() is an indicator function which returns 1 if the predicate inside it holds True else the function evaluates to O.
Entropy-based attack (Nasr et al. 2019: Song & Mittal, 2021: Tang et al. 2022) When the prediction entropy of an input record falls below a predetermined threshold, the adversary considers it member. Conversely, if the prediction entropy exceeds the threshold, the adversary infers that the record is a non-member. This inference is based on the observation that there are notable disparities in the prediction entropy distributions between training and test data. Typically, the target model exhibits higher prediction entropy on its test data compared to its training data. The entropy of prediction vector p(g (z) defined as follows:
H(p(ūlx)= = --Inoooogpi
(12)
where p: the confidence score in p(ilx). Then, the attack Lentr is given as:
Tentr (ener(((yx)) (f(u))),)) (H()(()))) <T
(13)
Modified entropy-based attack (Song & Mittal 2021 Song et al.[15] introduced an enhanced prediction entropy metric that integrates both the entropy metric and the ground truth labels. The modified entropy metric tends yield lower values for training samples compared to testing samples. To infer membership. either a class-dependent threshold Ty class-independent threshold Tattack applied.
T Mentr (f(vlx), y) = 1 (Mentr(p(ū)))) Ty)
(14)
where M Mentr( (entr(p((xx)) (p(ī)x) for (x.y) data sample is given by combination of entropy information and ground truth label as:
Mentr(p(ūlx)) =((11 ((lix)))Nog(((())))
[(((i)x))og(( p(())),)))
(15)
it
23

---

Published as a conference paper at ICLR 2026
Likelihood Ratio Attack (LiRA) (Carlini et al., 2022) LiRA is chadow-mode based sin- gle-query attack that explicitly models the distributions a scalar score for members and non- members and then performs a likelihood ratio test. For sample (x. y), the attacker first defines one-dimensional score s(x, y) from the target model, typically the negative cross-entropy loss or the (log-) >ooffidnceec (x, y). the attacker estimates two score distributions: one for members (IN) and one for non-members (OUT). In practice, LiRA fits parametric Gaussians
the true label y. Using multiple shadow models trained with and without
s(x, 8) IN ~ (Min Ii)), and computes the log-likelihood ratio
s(x, OUT ~ (Nout, oott)
o(s(x y): Pin, o
(x, 8) log ((s(x. y); Mouts )ut)
where ((5 denotes the Gaussian density. The LiRA decision rule is then (PR)((()))); y) (ALIRA (x, 2 TLRR),, for some threshold TLIRA chosen to trade off between TPR and FPR. In the "online" variant, the attacker fits both IN and OUT distributions from shadow models; in the "offline" variant, only the OUT distribution is estimated and low likelihood under the OUT model treated as evidence of membership.
(16)
Robust MIA (RMIA) (Zarifzadeh et al. 2024) RMIA reframes membership inference as a calibrated hypothesis test based on pairwise likelihood ratio between query (x, y) and many population samples (z, 32). For each pair (x, z), RMIA compares how the (approximate) probability of and change when conditioning on the event that RMIA defines
used to train the target model. Concretely,
Pr(x ()
Pr(x)
LR(x, z) = Pr(z
0)
Pr(z)'
where Pr(- ()) denotes the target model's likelihood and Pr(.) is population prior. Intuitively, if
including x in the target training set fits x disproportionately better than many other population points the ratio LR(x, z) becomes large. RMIA samples many z from the population and defines a robust membership score
R(x) = Σ (LR((X 2) >>)) 7),
where is a fixed pairwise LR threshold and n. is number of population (non-member) samples. The attack then declares membership if R(() exceeds global threshold r; by sweeping one obtains calibrated ROC curve, and for chosen FPR one can directly pick the corresponding т. In the offline mode, all reference models are OUT models trained once on population data; the online mode, the attacker additionally trains IN reference models that explicitly include in their training set, which yields a more accurate approximation of the conditional likelihoods but more computationally expensive.
E OTHER SHARPNESS-AWAN OPTIMIZERS
In this section, we discuss other variants of SAM, namely Adaptive SAM (ASAM) (Kwon et al.. 2021), Guided SAM (GSAM) (Zhuang et al. 2022), and custom designed optimizer, namely Sharp. Sharp objective is designed to explicitly find a sharper minima. The objective function of Sharp is,
Csharp (w)rrp()==r ) = L(w) - max L(w + e). eEB(p)
(17)
This objective be as minimizing the loss at current while maximizing the loss in the vicinity. We empirically verify that this objective does lead to a sharper minima measuring its hessian trace. Results and discussion about Sharp available in Appendix H.4
The results on CIFAR1O, CIFAR100, Purchase 100, and Texas 100 are reported in Table B Other sharpness- aware optimizers shown to achieve similar generalization gain, albeit the cost of higher membership attack accuracy. On the other hand, optimizer that explicitly looks for a sharp minima does worse in terms of generalization, but has better membership privacy.
24

---

Published as a conference paper at ICLR 2026
Dataset
Optimizer
NN
Confidence
Entropy
M-entropy
Test Acc
SGD
76.62%
77.19%
76.61% %
77.30%
80.30%
Sharp
57.62%
59.69%
57.88
59.69%
76.14%
CIFAR-100
ASAM
78.92%
79.22%
78.86%
79.31%
81.80%
GSAM
78.63%
79.23%
79.00%
79.23%
82.16%
SGD
50.00%
59.37%
59.09%
59.51%
96.00%
Sharp
50.22%
52.86%
52.47%
52.78%
92.86%
CIFAR-10
ASAM
50.48%
61.39%
61.20%
61.32%
96.66%
GSAM
50.00%
61.46%
61.38%
61.54%
96.64%
SGD
66.00%
66.76%
64.78%
67.13%
85.50%
Sharp
59.58%
60.96%
58.04%
61.16%
84.31%
Purchase 100
ASAM
66.85%
66.84%
65.39%
67.03%
85.54%
GSAM
67.45%
67.72%
66.51%
67.87%
85.82%
SGD
59.81%
65.20%
55.74%
65.13%
50.83%
Sharp 51.11%
59.89%
53.46%
59.36%
49.97%
Texas
ASAM 60.92%
67.50%
58.80%
67.10%
53.17%
GSAM 54.89%
67.07%
57.93%
67.13%
52.04%
Table Attack accuracy of direct threshold MIA on SGD, Sharp, ASAM, and GSAM. green we highlight the best performing model on the test set and in orange the model against which MIA is more successful. Analogous to SAM, optimization methods that improve generalization (ASAM, GSAM) through finding flatter minima tend to be optimization that looks for sharp minima instead is more robust to MIA attack while being worse off in generalization.
prone to direct threshold attacks, while
SGD
SAM
Datasct
Attack
Test Acc
AUC
Attack Acc
TPR@.1
Test Acc
AUC
Attack Acc
TPR @
RMTA
86.8%
77.3%
17.3%
87.7%
78.1%
18.9%
CIFAR-100
67.7%
69.1%
LiRA
76.2%
71.9%
16.9%
77.8%
73.2%
19.4%
RMTA
69.4%
62.3%
4.3%
72.7%
64.6%
5.7%
CIFAR-10
92.3%
93.1%
LÍRA
14.
55.9%
4.1%
58.7%
58.3%
7.0%
RMIA LIRA
68.7%
62.6%
1.7%
70.2%
63.6%
1.9%
Purchase100
76.5%
77.4%
52.9%
53.4%
0.1%
53.7%
54.2%
0.1%
RMIA
74.8%
67.4%
3.6%
78.8%
69.8%
5.6%
Texas100
46.9%
49.2%
LIRA
56.9%
58.3%
0.8%
61.7%
61.5%
2.6%
Table 4: Comparison of offline shadow model MIA on SGD and SAM. In green we highlight the best performing model on test set, and in orange the model with higher privacy leakage (higher AUC, Attack Accuracy, and TPR TPR@0.1%FPR)
F OFFLINE SHADOW MODEL ATTACKS
We report the results for offline shadow model attacks in Table The results in line and support the finding that SAM tends to incur higher membership privacy Teakage. Excluding cases for tabular datasets where TPR at 0.1 FPR is near zero for both models. SAM has higher values for all other attack metrics. Consistent with the literature, RMIA is more effective for offline setting compared to LiRA (Zarifzadeh et al. 2024). For online setting, we use a different experimental setup (WideResNet targets and shadows, our own training pipeline, and different choice of auxiliary z-points). In this setting, LiRA is slightly stronger than RMIA across most metrics (Table 3). but the gaps are modest and much smaller than those reported for the offline comparison in Zarifzadeh et al. (2024). We therefore view our results as broadly consistent with prior work: RMIA and LiRA are competitive state-of-the-an shadow-model attacks. and their exact ranking training choices. Our main conclusions - in particular, that SAM consistently increases vulnerability to both LiRA and RMIA compared to SGD unaffected by these small differences.
depend architectural and
25

---

Published as a conference paper at ICLR 2026
G DATASETS
Here we introduce the four benchmark datasets used in the experiments and they have been widely used in prior works on MI attacks:
This is a benchmark dataset for image classification task. The dataset consists of 60,000 color images of 32x32 size. There are 6,000 images from 10 classes where 5,000 images per class belong to the training dataset and 1,000 images per class belong to the test dataset.
CIFAR-10
CIFAR-100 The dataset designed to more challenging than CIFAR-10 10 as it contains greater number of classes and more fine- eeneggrined distinctions between objects. There are total of 60,000 images from 100 classes. Each subclass consists of 600 images, and within each subclass, there are 500 training images and 100 testing images. This distribution ensures balanced representation of each class in both the training and testing sets.
Purchase-100 This a 100 class classification task with 197,324 data samples and consists 600 binary feature; each dimension corresponds to a product and its value states if corresponding customer purchased the product; the corresponding label represents the shopping habit of the customer. We use the pre-processed and simplified version provided by Shokri al. (2017) and used by Tang et al. (2022).
Texas- 100 This dataset is based on the Hospital Discharge Data public files with information about inpatients stays in several health facilities released by the Texas Department of State Health Services from 2006 to 2009. We used prepossessed and simplified version of this dataset provided by (Shokri et al. 2017) and used by (Tang et af.) 2022) which is composed of 67 67,330 data samples with 6,170 binary features. Each feature represents a patient's medical attribute like the external causes of injury, the diagnosis and other generic information. The classification task is to classify patients into 100 output classes which represent the main procedure that was performed the patient.
EyePacs 10 The pre-processed version of this dataset is obtained from Kaggle and it was originally used for Diabetic Retinopathy Detection challenge. The dataset consists of 782 colour fundus images, including 35,126 samples for training and 53,576 samples for testing. The images were captured under various conditions by various devices at multiple primary care sites throughout California and elsewhere. For each subject, two images of the left and right eyes were collected, with the same resolution. A clinician 0-4 according to the Early Treatment Diabetic Retinopathy Study (ETDRS) scale. Note that for this dataset only training set (35k images) used since the labels for testing set is not publicly available. The images in the dataset vary in their image resolution and we resized all the images to 128x 288 pixels for our experiments.
asked to rate each image for the presence of DR with a scale of
H EXPERIMENTAL SETUP
H.
MODELS
For CIFAR- 100 and CIFAR-10, we use WideResNet (WRN) Zagoruyko & Komodakis 2016) with 16 layer depth and as width factor. For Purchase- 100 and Texas- we follow the setting in Tang et al. (202)) and use a l-yer fully connected neural network with layer sizes [1024, 512, 256, 100]. For ByePacs, we use R8.Net-18.
"https://www.cs http://wwwwwww.wt.ooooo//o coronto.edu/ kriz/eifar.html
7 httpp:://wwwwcs. toronto.edu/ kriz/cifar.htmll
'https://www.l www.le
hhtpss//www.dssh. dshs. texas. www.ddsss.....oooot.tt.t... www.TTTCCCCCHHoiiiiioeoooooooo.o
10 hhtps://www.. www.wwww....c.oo wtans//wwww..ggg....maaaa/ssssi www.a///eeeaeetttttaacccemmomeoe
26

---

Bucket Published as a conference paper at ICLR 2026
H.2 Tent EXPERIMENT
We discuss how test data points were grouped into 5 buckets according to different Zent levels. posed of test points where all 500 training points have 0 influence score. This means that the prediction output for that test point does not change had the model been trained without any one particular training data point. Be- cause influence scores for all training points are equal, these test points have highest Tent 11 Figure dis- plays distribution of Zent for remaining test data points. We group those above 6.1 into bucket 4. For the rest of the points, we calculate the mean and stan- dard deviation and use them for grouping. We group points below -0.4o from the mean into bucket I, points between -0.4o and 0.4o into bucket 2, and points above 0.4o into bucket 3. Final number of test points in each buckets [Bucket 1: 1924, Bucket 2: 2996, Bucket 3: 2392, Bucket 4: 535, Bucket 5: 2153]. For SAM's buckets, final number of test points are [Bucket I: 1913, Bucket 2: 3181, Bucket
contains highest Tent level, and com-
55
56
33
2.5
3.0
Figure 6: Ient distribution excluding bucket for SGD and SAM
3: 2548. Bucket 4: 502, Bucket 1856]. Number of overlapping indices were [Bucket I: 1116, Bucket 2: 1625, Bucket 3: 1199, Bucket 4: 133, Bucket 5: 1678].
H.3 ATTACK SETUP & SIZE DATA SPLITS
We adopt the attack setting from TTang al. 2022 Nasr al. 2018) to determine the partition between training data and test data and to determine the subset of the training and test data that constitutes attacker's prior knowledge for CIFAR- 100, Purchase- 100 and Texas- 100 datasets. We use similar strategy to determine the data split for CIFAR-10. Specifically, the attacker's 's knowledge corresponds to half of the training and test data, and the MIA success is evaluated over the remaining half. For shadow model attacks, the total sample pool used is 50000 for CIFAR10 and CIFAR 100, 40000 for Purchase 100, and 20000 for Texas 100. For RMIA attack, we used 7 = 1 and selected all of the z samples within the training pool that were not part of the target model's 's training set. On CIFAR10, for example, number samples was 25000.
H.4 HYPERPARAMETER TUNING AND EMPIRICAL VALIDATION OF FLATNESS FOR SHARP
For the Sharp objective (see Equation 17 we fine-tuned and p which result model that exhibits sufficient difference in test accuracy and sharpness of the minima compared to SAM and SGD. The final hyperparameters of the model reported were 0.01, 8 0.6818 for CIFAR-100 and CIFAR-10,
Optimizer
Trace of the Hessian
0.01, = 0.83 for Purchase- 100, p = 0.001, B = 0.513 for 0.001, 8 0.18 for EyePacs. To verify that Sharp actually finds a sharper minima, we computed the trace of the hessian matrix using Hutchinson's 's method for SGD, SAM, and Sharp models on CIFAR-100, The results are in Table Higher trace indicates sharper minima and vice versa. The trace is the largest for Sharp and smallest for SAM.
Texas- 100. and
Sharp SGD SAM
1556.54
307.87
84.18
Table 5: Comparison of Hessian trace values across methods.
H.4.1 BALL OF RADIUS p
For SAM loss, sharp minima loss, and our proposed loss, we approximate the maximum loss in the ball of radius p around the minima. Norton & Royset (2021)
W'een actually calculating Tent with Equation 5 this evaluates to due to probability normalization, but represents highest value.
27

---

Published as a conference paper at ICLR 2026
Dataset
Model
Optimizer
Test Acc
Best Attack Acc
SGD
78.42%
74.31%
1
SAM
78.74%
77.45%
CIFAR- 100
SGD
77.44%
77.22%
Inception V4
SAM
79.60%
80.82%
SGD
95.18%
57.90%
Resnet 18
SAM
96.16%
60.05%
CIFAR-10
SGD
94.26%
61.60%
Inception V4
SAM
95.76%
64.41%
Table 6: Privacy vs generalization tradeoff for SAM and SGD using the Inception V4 and ResNet- 18 models. green we highlight the best performing model on the test set and in orange the model against which MIA is more successful. The results confirm the architecture-indeeeddenn nature of our findings.
found that the type of norm that is used for defining the ball has large impact along with actual value. For all our experiments, we use L2 norm for our ball of radius p.
H.4.2 HYPERPARAMETER TUNING FOR CIFAR-10 & CIFAR-100
We trained each model for 200 epochs and chose the model with highest validation accuracy held-out validation set. We used initial learning rate of 0.1 with learning rate decay of 0.2 at 60th, 120th, and 160th epoch with batch size of 128. We trained the models with weight decay 0.0005 and Nesterov momentum of 0.9. For SWA on CIFAR-100 we trained first 150 epoch with vanilla SGD and used weight averaging for the rest of the epochs.
H.4.3 HYPERPARAMETER TUNING FOR TEXAS- 100 & PURCHASE-100
We chose the best model as discussed before for CIFAR-10/100. We trained models with a learning rate of 0.1 with weight decay 0.0005 and Nesterov momentum of 0.9. We trained the models on Purchase- 100 for total of 100 epochs and on Texas-100 for total on 75 epochs. During training, we employed batch size of 512 for the Purchase- 100 dataset and a batch size of 128 for the Tex dataset.
H.4.4 HYPERPARAMETER TUNING FOR EYEPACS
We trained ResNet-18 with SGD, SAM and our proposed loss using EyePacs dataset for 100 epochs. Since, the dataset highly imbalanced with about 25k data points of 35k training data points belonging to one of the five classes, we used the balanced batch sampling strategy and lower learning rate of 0.01 with learning rate decay of 0.2 60th epoch. As before, we also used weight decay 0.0005 and Nesterov momentum of 0.9. For our experiments, we utilized batch size of 100, consisting of samples from each of the classes.
I ABLATION STUDY: COMPARISON OF DIFFERENT ARCHITECTURES
To validate consistency across different model architectures, we report direct threshold attack results in Table using Inception V4 [2] and ResNet-18 13 for CIFAR- and CIFAR-IO. We kept the same across all model architectures with value 0.1. The results are consistent with our findings that SAM tends to have higher test accuracy while having higher membership attack accuracy the same time. Overall best attack accuracy higher for SAM for all the cases although we find mixed findings for query attack accuracy specifically.
12
13 http:///ttttccoooomiioomom/////s
28

---

Published as a conference paper at ICLR 2026
RMIA (Online)
RMIA (Offline)
LiRA (Online)
LiRA (Offline)
CIFAR10 True Positive Rate
10-3
10-2
10
True Positive Rate
10
CIFAR100
10-3
True Positive Rate True Positive Rate
Purchase
10-2
Texas
10-3
10-5
10-1
10-1
10-1
10-1
10-1
10-1
10-2
10-1
False Positive Rate
False Positive Rate
False Positive Rate
False Positive Rate
SGD
SAM
Figure 7: ROC curves comparing SAM (Orange) vs. SGD (Blue) across all datasets and attack modes on log-log scale. Rows represent datasets (CIFAR-10, CIFAR-100, Purchase100 Texas 100). Columns represent the attack configuration. The ROC curve for SAM (orange) is above the ROC curve for SGD (blue) for nearly the entire range for most settings.
29


Reviews:
[{"review_id": "T3Uc6XWdIx", "reviewer": "Reviewer_4axt", "summary": "This paper challenges the conventional belief that improved model generalization implies better membership privacy by investigating the SAM algorithm. The authors surprisingly find that SAM-trained models, despite achieving better generalization than standard SGD, may be more vulnerable to MIAs. The authors hypothesize this occurs because SAM selectively learn atypical but generalizable sub-patterns more effectively than SGD. This enhanced memorization of rare features improves performance on atypical test samples, but consequently increases the privacy risk by making training samples more distinguishable. The paper empirically validates this mechanism by analyzing memorization and influence scores and provides a theoretical framework showing how a model that better captures minority features can simultaneously achieve high generalization and high MIA vulnerability.", "questions": "I do not understand why Assumption 4 is reasonable. Could you explain it?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper first states that SAM increases vulnerability to MIAs.\n2. The paper is exceptionally clear, presenting its counter-intuitive finding and complex hypothesis in a logical, well-structured, and easy-to-follow manner.", "weaknesses": "1. The authors claim to challenge the conventional assumption that improved generalization implies stronger privacy. However, this insight has already been discussed in previous works [1,2,3,4].  \n2. The paper's claims are based solely on the original SAM algorithm. It is unclear whether these findings on increased privacy risk generalize to other sharpness-aware optimizers (e.g., ASAM, GSAM). An investigation into these variants is recommended.  \n3. As a privacy metric, accuracy has been criticized in many works [4,5]. It is recommended to provide TPR at a low FPR and evaluate the privacy risk using recent MIA methods, such as LiRa and RMIA [4,6].\n4. The \"anti-alignment\" assumption means a model that gets better at classifying the majority subclass inherently gets worse at classifying the minority subclass, forcing a direct trade-off. This is a very strong and specific setup.\n\n\n\n[1] \"Understanding membership inferences on well-generalized learning models\" (arXiv, 2018)\n\n[2] \"When does data augmentation help with membership inference attacks\" (ICML, 2021)\n\n[3] \"Bounding Information Leakage in Machine Learning\" (Neurocomputing, 2023)\n\n[4] \"Membership Inference Attacks From First Principles\" (SP 2022)\n\n[5] \"Evaluations of Machine Learning Privacy Defenses are Misleading\" (CCS, 2024)\n\n[6] \"Low-Cost High-Power Membership Inference Attacks\" (ICML, 2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fYgVO1b0RU", "reviewer": "Reviewer_U9Jc", "summary": "This paper presents a counterintuitive yet rigorously studied phenomenon: Sharpness-Aware Minimization (SAM), an optimization method known for improving generalization, unexpectedly increases vulnerability to membership inference attacks (MIAs). Through extensive experiments on multiple datasets and theoretical analysis, the authors demonstrate that SAM’s generalization gains stem from structured memorization of atypical subclass patterns(e.g., rare features in long-tailed distributions), which simultaneously enhances test performance and privacy risks. The work challenges the conventional belief that better generalization implies lower privacy risk, offering novel insights into the trade-offs between optimization, generalization, and privacy.", "questions": "See the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper offers novel insights into the trade-offs between generalization and privacy, which challenges the conventional belief that better generalization implies lower privacy risk.\n2. The authors combine empirical evidence with theoretical guarantees to support their claims. The consistent results across datasets and models strengthen their conclusion.\n3. The paper is well-written and easy to follow.", "weaknesses": "1. While the paper highlights SAM’s privacy vulnerability, it would be good to propose or evaluate defensive strategies to mitigate this risk to improve practical applicability.\n2. The theoretical analysis relies on a simplified linear model and strong assumptions (e.g., perfect interpolation). It lacks both theoretical and empirical validations on more advanced non-linear architectures like Transformer-based or diffusion models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ax38bKV1wU", "reviewer": "Reviewer_2e1z", "summary": "This paper reveals a critical trade-off in Sharpness-Aware Minimization (SAM): while it improves model generalization, it also increases privacy risk by making models more vulnerable to Membership Inference Attacks (MI). The authors demonstrate that this occurs because SAM's performance gains stem from its enhanced ability to memorize rare sub-patterns within the training data.", "questions": "See the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written and easy to follow.\n\n- The finding of the relationship between the SAM and memorization is interesting and may shed light to privacy defense.\n\n- The theoretical analysis is interesting and insightful.", "weaknesses": "+ (Major) Experimental Evidence. The central claim regarding SAM's heightened privacy risk is not yet fully convincing. The results in Table 1 show that SAM's privacy risk (measured by ASR) is not consistently or significantly higher than that of SGD (in Purchase-100 and Texas-100). Besides, Table 3 reports only the best attack accuracy, unlike Table 1 which shows results for all MIA methods.\n\n+ (Major) Uncertain connection between ASR and memorization score. The link between the memorization score and the Membership Inference Attack (MIA) success rate is presented as a given. However, this relationship appears to be an assumption rather than an empirically demonstrated fact. This connection should be either validated or more cautiously framed as a hypothesis.\n\n+ (Major) What can the observation bring to the future work on MIA or SAM? A more precise presentation of the study's potential implications for relevant fields would substantially strengthen the significance of this research.\n\n+ (Minor) The solidness of the experiments would be strengthened by evaluating a wider range of modern MIA methods to ensure the findings are not specific to the selected attacks. \n\n+ (Minor) The experiments are primarily conducted on small datasets. It is unclear if the observed privacy trade-offs persist on larger, more complex benchmarks such as ImageNet.\n\n## Score Justification\n\nThe paper is well-organized and introduces valuable theoretical analysis. However, the experimental evidence currently feels incomplete, which undermines the strength of its conclusions. Besides, the most significant shortcoming is the lack of a clear discussion on the broader impact and implications of these findings for the community. While briefly mentioned in the conclusion, a detailed discussion on how this new understanding of SAM's properties (especially concerning long-tailed data) can influence future research and practice is crucial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JDRKvmufEu", "reviewer": "Reviewer_g2BK", "summary": "This paper studies the vulnerability of neural networks trained using algorithms that improve generalisation, such as the Sharpness-Aware Minimisation (SAM), to membership inference attacks (MIAs). It provides empirical evidence that suggests SAM-optimised networks are more vulnerable to MIAs when compared to SGD-optimised networks. The author(s) also provide theoretical justification for the high MIA vulnerability of SAM-optimised networks despite improved generalisation, since that goes against the conventional expectation where high MIA vulnerability is associated with low generalisation (and vice-versa).", "questions": "**Questions**: I would urge the author(s) to address the weaknesses detailed above. I am amenable to updating my initial assessment thereafter.\n\n**Suggestions**: I suggest the following edits to improve the presentation of the paper:\n- Minor Suggestion #1: Memorisation scores are measured w.r.t. samples, so this statement, \"Motivated by this connection, we analyse the memorisation scores of SAM-trained models...\", is somewhat misleading. It would be better to amend it to frame it w.r.t. individual samples.\n- Minor Suggestion #2: Can you report the correlation coefficient between mem_SAM and mem_SGD for Figure 1(b) and 1(c)?\n- Minor Suggestions #3: Lines 350-352 are written in a complicated and difficult-to-read manner. It would be best to rewrite it focusing on one relationship (for example, lower number bucket is associated with higher memorisation and vice versa).", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper presents evidence that suggests that optimising models using SAM encourages memorisation of atypical samples (mid-range memorisation scores), which contribute to their improved generalisation.\n- The author(s)' proposed metric of influence entropy $\\mathcal{I}_{ent}$ in Eq(6) helps verify the claim that SAM enhances generalisation by encouraging the network to memorization more atypical samples compared to SGD-based optimisation (as seen in Figure 4(a)). \n- The empirical results show that the claims in the paper are not model-dependent but extend to different experimental settings.\n- The theoretical justification for the observed phenomenon of improved generalisation leading to high memorisation when optimising a model with SAM is well described.", "weaknesses": "- Computing memorisation scores using Feldman and Zhang's [1] method is computationally expensive as it often requires (re)training hundreds of models. Why did the author(s) not use Ye et al.'s [2] more efficient version to compute these scores?\n- Use of balanced accuracy as the attack metric and not TPR at fixed FPR [3], which informs more about an attack's ability to correctly identify membership signal at [preferably] low FPR (or low chances of predicting a member as a non-member).  \n- It does not use SOTA MIAs such as LiRA [3] / Quantile-MIA [4] / RMIA [5] to measure the sensitivity of SAM-optimised networks, which are known to provide better estimates of a model's MIA vulnerability compared to attacks that rely on predetermined thresholds, such as the Entropy- or Confidence-based attacks. There is also the case that the theoretical results for Theorem 2 depend on a single [data-dependent] threshold, whereas a factor contributing to the success of SOTA MIAs is that they incorporate sample-level thresholds. \n\n[1] Feldman, V., and Zhang, C. “What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation.” NeurIPS 2020.\n\n[2] Ye, J. et al. \"Leave-One-Out-Distinguishability in Machine Learning.\" ICLR 2024.\n\n[3] Carlini, N. et al. \"Membership Inference Attacks From First Principles.\" SP 2022.\n\n[4] Zarifzadeh, S. et al. “Low-Cost High-Power Membership Inference Attacks.” ICML 2024.\n\n[5] Bertrán, M. et al. \"Scalable Membership Inference Attacks via Quantile Regression.\" NeurIPS 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)