Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks
Authors: Lukas Struppek
Dominik Hintersdorf
Kristian Kersting
Venue: ICLR 2024 poster (2024)
Keywords: model inversion attacks
privacy
label smoothing
defense
Primary area: societal considerations including fairness, safety, privacy
TL;DR: Label smoothing regularization can reinforce a model's privacy leakage but can also act as a practical and powerful defense mechanism.

Abstract: Label smoothing – using softened labels instead of hard ones – is a widely adopted regularization method for deep learning, showing diverse benefits such as enhanced generalization and calibration. Its implications for preserving model privacy, however, have remained unexplored. To fill this gap, we investigate the impact of label smoothing on model inversion attacks (MIAs), which aim to generate class-representative samples by exploiting the knowledge encoded in a classifier, thereby inferring sensitive information about its training data. Through extensive analyses, we uncover that traditional label smoothing fosters MIAs, thereby increasing a model's privacy leakage. Even more, we reveal that smoothing with negative factors counters this trend, impeding the extraction of class-related information and leading to privacy preservation, beating state-of-the-art defenses. This establishes a practical and powerful novel way for enhancing model resilience against MIAs.

Published as a conference paper at ICLR 2024
BE CAREFUL WHAT YOU SMOOTH FOR: LABEL SMOOTHING CAN BE A PRIVACY SHIELD BUT ALSO A CATALYST FOR MODEL INVERSION ATTACKS
Lukas Struppek
Dominik Hintersdorf
Technical University of Darmstadt German Center for Artificial Intelligence struppeke cs. tu-da rmst adt... de
Technical University of Darmstadt
German Center for Artificial Intelligence hintersdor f cs. tu-da rmstadt..dd de
Kristian Kersting
Technical University of Darmstadt
Centre for Cognitive Science of TU Darmstadt Hessian Center for AI (hessian.Al)
German Center for Artificial Intelligence
Label smoothing using softened labels instead of hard ones - is widely adopted regularization method for deep learning, showing diverse benefits such as enhanced generalization and calibration. Its implications for preserving model privacy, however, have remained unexplored. To fill this gap, investigate the impact label smoothing on model inversion attacks (MIAs), which aim to generate classreepresentative samples by exploiting the knowledge encoded in classifier, thereby inferring sensitive information about its training data. Through extensive analyses, we uncover that traditional label smoothing fosters MIAs, thereby increasing a model's privacy leakage. Even more, we reveal that smoothing with negative factors counters this trend, impeding the extraction of class-related information and leading to privacy preservation, beating stac-offthe-aa defenses. This establishes a practical and powerful novel way for enhancing model resilience against MIAs. Source code: Inttps://githii com httpa//..o...tttttttteeeeeeeeeee
and Play-Attase
INTRODUCTION
Deep learning classifiers continue to achieve remarkable performance across a wide spectrum of domains (Radford et al., 2021] [Ramesh et al. 2022, OpenAl 2023). due in part to powerful regularization techniques. The common Label Smoothing (LS) regularization (Szegedy et al. 2016) replaces labels with smoothed version by mixing the hard labels with uniform distribution to improve generalization and model calibration (Pereyra et al., 2017 Müller et al. 2019). However, the very capabilities that make these models astonishing also render them susceptible to privacy attacks, potentially resulting in the leakage of sensitive information about their training data.
One category of privacy breaches arises from model inversion attacks (MIAs) (Fredrikson al. 2015). a class of attacks designed to extract characteristic visual features from a trained classifier about individual classes from its training data. In the commonly investigated setting of face recognition, the target model is trained on facial images to predict a person's 's identity. Without any further information about the individual identities. MIAs exploit the target model's learned knowledge create synthetic images that reveal the visual characteristics of specific classes. As practical example, let us take high-security facility that uses face recognition model for access control. MIAS could enable unauthorized adversaries to reconstruct facial features by accessing the face recognition model without any further information required and with the goal of inferring the identity of authorized staff. In this case, successful attack can lead to access control breaches and potential security and privacy threats to individuals.
While recent MIA literature (Zhang et al. 2020 Struppek et al. 2022a) keeps pushing the attacks' performance, the influence of the target model's 's training procedure on the attack success has not been studied so far. We the first to start investigations in this direction with focus on LS regularization, whose effects on a model's privacy have not yet been considered. We connect LS to a model's 's privacy leakage in the light of MIAS and reveal that training with positive, i.e., standard label smoothing

---

Published as a conference paper at ICLR 2024
increases model's vulnerability, particularly in low data regimes. Moreover, reveal that by smoothing labels with a negative smoothing factor and, therefore, penalizing a model's confidence in any class different from the true label, one make models robust to MIAs and decrease their privacy leakage significantly without large performance drops on its classification tasks, offering a better utility-privacy trade-off than existing defense approaches. Our extensive evaluation studies the various effects of both positive and negative label smoothing on MIAs and explores possible reasons for these phenomena. These insights not only contribute to our understanding of model privacy but also bear practical implications for deep learning applications. By strategically deploying LS, model providers can balance model performance with security and mitigate the risks associated with privacy breaches in various real-world scenarios.
In summary, we make the following contributions:
We are the first to demonstrate that positive label smoothing increases a model's privacy leakage in light of model inversion attacks, particularly in low data regimes.
We reveal that negative label smoothing counteracts this effect and offers a practical defense, beating state-of-the-an approaches with a better utility-privacy trade-off.
We introduce a novel attack metric and provide rigorous investigations on the effects of label smoothing on the target models and the individual stages of model inversion attacks.
BACKGROUND AND RELATED WORK
We start by introducing model inversion attacks (Sec. 2.1 and label smoothing (Sec. 2.2).
2.1 MODEL INVERSION ATTACKS
In the image classification setting, let M Margett X + [0, 1Jc be classifier that takes images E X and computes for each class e E {1,.... C} probability Pe E [0, 1). Model inversion attacks (MIAs) aim to construct synthetic images that reflect and reveal characteristic features of specific class learned by Maarget. In the standard MIA setting, the adversary has only access to MMtrge and knows the general data domain but has detailed information about the individual classes. In the face recognition domain, a common setting for MIAs, each class corresponds to an individual identity, but the names and appearances of these identities are unknown to the adversary. A successful MIA allows for inferring the visual appearance and identity of the different classes from the training data without direct data access, leading notable security breach.
The first MIAs were limited to linear regression (Fredrikson et al. 2014) and shallow neural net- works (Fredrikson et al., 2015) by using gradient-based sample optimization reveal features. Direct sample optimization is prone to producing adversarial examples (Szegedy et al. (0044). i.e., inputs that look nothing like the target class but are still assigned high prediction scores by the target model. To overcome this problem, Zhang et al. (2020) added a generative adversarial network (GAN) (Good) fellow al. 2014) as prior to enable attacks against deeper networks and improve the image quality of the generated images. GANs consist of two components, generator network G: Z + X prrior. which is trained to generate images æ E Xprsor from latent vectors z E Z, and a discriminator network D used to distinguish between generated images and real images. Generative MIAs then try to solve the following optimization goal by optimizing a latent vector 2:
min L(Mtarget G,D,å,c).
Here, L denotes a suitable loss function to maximize the target model's confidence in the target class c, e.g., a cross-entropy loss computed on the outputs of Maarget for images generated by G. Broadly speaking, the adversary tries to find spot on the generative model's manifold that represents the visual features of the target class and, ideally, allows inferring the person's 's identity. To find such a spot, the target model's knowledge is exploited to provide guidance through the latent space. This is not a trivial task since MIAs face various challenges, including misleading feature reconstruction. distributional shifts, and complex optimization landscapes (Struppek et al. 2022a). To tackle these challenges, generative MIAs have been improved, e.g., by training sarget-specfice GANs (Chen et al.) 2021 Yuan et al. 2023) or changing the attack's 's objective function (Wang et al. 202 Nguyen al. 2023). Whereas most MIAs require white-box model access, some gradient- frree approaches
2

---

Published as a conference paper at ICLR 2024
were also proposed. Han et al. (2023) introduced reinforcement learning- based black- box attack, and Kahla et al. (022) and Zhu et al. (2023) proposed label-only MIAs.
However, all mentioned attacks focus low-resolution tasks and have yet to prove successful in high-resolution data regimes. Recently, StruplpySt et al. (2022a) introduced Plug & Play Attacks (PPA), which decoupled the attack and target model from the underlying GAN, allowing the use of any suitable pre-trained GANs from the target domain. The attack has demonstrated greater robustness to distributional shifts, increased flexibility in the choice of generative prior and target model, and high effectiveness in the high-resolution setting. Therefore, we base most of our experiments on PPA since allows us to investigate a more realistic attack scenario with high high-resolution data. We provide a more detailed overview of PPA's specifics in Appx. B.4
2.2 LABEL SMOOTHING REGULARIZATION
Image classifiers are typically trained using a cross-entropy loss Lor(y, - 1/ log Pk. where the ground-truth class label vector y assigns a value of 1 to the correct class and sets all other values to zero. Here, p E [0, 1] represents the model's output probability vector for the current sample. In the standard hard label setting, each training sample is strictly assigned to single class c, simplifying the loss to Lc = log During training, the model learns to produce high values for the predicted class with Pe > Pk. and c + k, which often results in overconfident models. To mitigate this effect, (Szegedy et al. (2016) introduced label smoothing (LS) regularization. LS replaces the hard label with mixture of the hard-coded label and uniformly distributed vector. Formally, the target vector y' for positive LS with a smoothing factor a E (0, 1] and C' classes is defined by
(1-a)
a
C
For example, y = (1, 0) one-hot encoded target vector. Smoothing with a = 0.3 replaces the vector by yoss (0.8, 0.1, 0.1)". This target vector represents uncertainty about the true label and encodes the correlation of the sample to classes different from the assigned hard label. In combination with LCE, LS effectively replaces LCE loss with weighted combination of losses:
L (y, = (1- -e) p)
(cE(L.)) C
k=1
Here, denotes a vector of length with all entries set to 1. Previous research has shown that LS improves generalization (Szegedy al. [2016] Pereyra al. 2017), model calibration (Müller et al.) 2019), language modeling (Chorowski & Jaitly] 2017]. p. and learning in low label noise regimes (Lukasik et al., 2020). Wei et al. (2022) generalized the LS formulation by allowing ( (-00, and demonstrated that smoothing with a negative factor can improve model performance in high label noise regimes. Smoothing with a negative factor creates target vectors that are no longer valid probability distributions. Repeating the previous example with negative smoothing factor = -0.3 results in the target vector yheg = (1.2. 00.1, -0.1"" Training with negative LS not only encourages the model to learn the target class but also penalizes confidence in other classes. This can formally be seen in Eq. (3). where the second term becomes negative. more formal analysis of LS is provided in Appx. A While LS demonstrated performance improvements in various domains (Szegedy et al.) 2016 Zoph et al., [0188] HHe et al. 2019), it important to highlight that LS also increases model's 's privacy leakage, aspect that has not been investigated yet.
a E
3 ILLUSTRATING THE DUAL USE OF LABEL SMOOTHING FOR MIAS
To motivate our investigation into the effects of LS on model privacy, we begin with a simple toy example based on a two-dimensional dataset with three classes: blue squares, green circles, and orange pentagons. We first illustrate the impact of positive LS. For this, we trained three-layer neural network with both hard labels (a = 0) and soft labels (a = 0.05). In Fig. we visualize the decision boundaries and model confidences over the input space. The model trained on hard labels (Fig. la assigns tigh-coofidence predictions to most inputs, even if they are far away from the decision boundary. In contrast, the model trained with positive LS (Fig. Tb) assigns high confidence only to samples close to the training data. We further simulated a simple MIA by taking a random starting point, here sample from the class (green circles), and optimizing this sample maximize
3

---

Published as a conference paper at ICLR 2024
Steps 763
Steps Taken:
Steps 53
Closest Sample Distance: 3.23
Closest Sample Distance: 0.65
Clasess Sample Distance: 6.75
1.00
0.75
(a) Hard Labels (a 0.0)
(b) Positive LS (a 0.05)
(c) Negative LS (a 0.05)
Figure 1: Simple MIA on 2D toy dataset with three classes. The Background color indicates the models' prediction confidence, and the yellow lines show the intermediate optimization steps of the attack. The optimization starts from random position, here sample from the green circle class, and tries to reconstruct a sample from the orange pentagons class. The attack against the positive LS model (Tb) constructs sample very close to the targeted training data. In contrast, attacking the negative LS model (Tc) saturates close to the decision boundary and far away from the training data.
the model's confidence for another class (orange pentagons) over 95%. The goal is to reveal the features of the target class, which are, this simplified setting, just the coordinates of training samples. While the attack is much faster the model trained with hard labels 763 optimization steps compared to 3113 steps the resulting data point is significantly further away from the training data 3.23 compared to 0.65 f2 distance to the closest training sample. By clustering the training samples in a high-confidence area, positive LS training reveals their position more precisely to the inversion attack and promotes attack results closer to the target class distribution.
If positive LS improves the attack success, one would expect that negative smoothing counteracts this effect and renders the inversion attack more challenging execute. Retraining the model with a negative smoothing factor (a -0.05) indeed shows reversed effects (Fig. [c]. The model's confidence very high everywhere except for the decision boundaries. This leads an inversion attack that achieves its goal already after 53 steps but ends far away from the training data. Thus, it appears that there exists trade-off between regularizing a model with L.S and the success of MIAs.
EXPERIMENTAL EVALUATION
With the previous illustration in mind, we now turn to a real-world scenario: face recognition of high- resolution images. Plug & Play Attacks (PPA) (Struppek et al.) (022a) are the current state-of-the-an MIA on which we base our experimental evaluations. Furthermore, we explored the effects LS on other MIAs in a low-resolution setting in Appx. C.5 to validate our findings. First, introduce experimental protocol (Sec. 4.1 before demonstrating the general impact of LS on the attack success (Sec. 4.2) model's l's embedding space (Sec. and the individual stages of MIAs (Sec. (Sec.F..4). For additional experimental details and results, we refer to Appx. B and Appx. C
respectively.
4.1
EXPERIMENTAL PROTOCOL
To ensure consistent conditions and avoid confounding factors, we maintain identical training and attack hyperparameters and seeds across different runs, with adjustments solely made to the smoothing factor during target model training. Specifically, the training samples, including their augmentations, remain consistent across For reproducibility, recreate the results. Additionally, we state all training and attack hyperparameters Appx.
source code includes configuration files to
Datasets: In line with previous MIA literature, we focus our investigation the FaceScrub Ng & Winkler 2014) and CelebA (Liu et al., 2013) datasets for facial recognition. FaceScrub comprises images of 530 different identities with equal gender split. While CelebA contains samples of 10.177 identities, we adhere to the standard MIA evaluation protocol (Zhang et al. 2020) and take only the 1,000 identities with the most samples. All images are resized to 224 x 224 for model training.
4

---

Published as a conference paper at ICLR 2024
Table 1: PPA attack results against ResNet-152 models trained on FaceScrub and CclebA. Results are compared to state-of-the-art defenses MID and BiDO. While positive LS (a = 0.1) amplifies the attacks, negative LS (a = -0.05) beats existing defenses terms of mulity-privacy trade-off. Green indicates attack improvements, whereas red shows better defense compared to the baseline.
FaceScrub
CelebA
Model
Test Acc 1 Acc@1
4 6Jac
t Emaii
t Test Ace 1 Acc@1
$ 8 ffcx
1 Strain
Standurd
94.9%
94.3%
0.71
61.2%
87.1%
81.8%
0.74
59.8%
Pos. LS
97.4%
95.2% (+0.9)
0.63 (-0.08)
71.0% (+9.8)
95.1%
92.9% (+11.1)
0.61 (-0.13)
66.1% (+6.3)
Neg. LS
91.5%
14.3% (-80.0)
1.23 (+0.52)
16.5% (-41.7)
83.6%
26.4% (-55.3)
1.04 (+0.3)
7.1% (-52.7)
MID
91.1%
92.0% (-2.3)
0.72 (40.01)
73.0% (+11.8)
80.4%
78. (-3.8)
0.74 (40.0)
70.9% (+11.1)
BiDO
87.1%
45.4% (-18.9)
0.91 (+0.2)
59.3%
(-1.9)
79.9%
63.7% (-18.1)
0.81 (+0.07)
60.6% (+0.8)
Models: We trained ResNet-152 (He al. 2016), DenseNet-121 (Huang al. 2017), and ResNeXt- 50 (Xie et al., (0017) as target models. All results from the main paper are based on ResNet- 152 models; results for other architectures are stated in Appx. C For negative LS, we trained the first epochs without any smoothing and then gradually increased the negative smoothing to stabilize the training and prevent models from getting trapped in poor minima during the initial epochs.
Attack Parameters: We used the official implementations of the different attacks to perform MIAs, employing default parameters. This choice ensures that differences attack performance do not arise from specific parameter selections. Due to the remarkably high time requirements for MIAs, we performed a single attack against each target model. To reduce random influences, we generated a total of 50 samples per target class, which is significantly more than most related research evaluated.
Metrics: We employ various metrics consistent with prior research (Zhang et al., 2020 Struppek et al. 2022a) evaluate the impact of LS. The target models' utility from the user's perspective is quantified by their accuracy on a holdout test set (Test Acc). The following metrics quantify the attack success from the adversary's perspective. Additional metrics provided in Appx. C
Attack Accuracy: To imitate a human evaluator that judges if reconstructed images depict the target class, a separate Inception-v3 (Szegedy et al., 2016) evaluation model is trained the target model's training data. The attack is then evaluated by computing the proportions of predictions on the synthetic images that match the target class, i.e., the top-1 (Aco@) /) and top-5 (Acc@5) 5) accuracy.
This metric measures the average l distance jface between the reconstructed images and the nearest samples from the target model's training data in the embedding space of pre-trained FaceNet (Schroff et al., 2015) model, which predicts visual similarity between faces. We also computed the distance ò deval in the evaluation model's 's penultimate feature space. Lower distances indicate that the reconstructed samples more closely resemble the training data.
Feature Distance:
For measuring the extracted discriminative information about distinct classes, we introduce novel metric. Specifically, we train a surrogate ResNet-50 (He et al. 2016) classifier on the synthetic attack results and measure its top-1 classification accuracy Etrai on the target model's 's original training data. The intuition behind this metric is that the more successful the inversion attack, the better the surrogate model's ability to distinguish between the classes
Knowledge Extraction Score:
4.2 THE IMPACT OF LABEL SMOOTHING ON A MODEL'S PRIVACY LEAKAGE
We begin by showcasing the general effects of positive and negative LS on MIAs. Our attack results in Tab. | for models trained on the complete FaceScrub and CelebA datasets, respectively, demonstrate that positive LS (second row) indeed amplifies a model's privacy leakage and enables the attacks to extract characteristic class features more closely related to the training data, as indicated by both jaece and Etrosn Moreover, smoothing with a negative factor (third row) substantially diminishes the attacks' success with only small reduction in model's test accuracy. Comparing the defensive effect of negative LS to tfate-oftthe-et defenses, MID (Wang et al. 2021c) and BiDO (Peng et al. 2022), even suggests more favorable utility- -efeene trade-off, all without requiring architecture adjustments or complex loss functions to be optimized. In the following, we focus our analyses on the FaceScrub ResNet- 152 models.
5

---

Published as a conference paper at ICLR 2024
75
Metric Accig
Smoething e- 0.1
Metric
Acci1
Advantage [%6
50
FaceNet Distance
Knowladga Extraction
0.05
FaceNet Distance
Knowledge Extraction
-25
-50
40
50
-0.05
-0.01
-0.
Samples per Class
Smoothing Factor
(a) Number training samples per class
(b) Training with different smoothing strength
Figure 2: Attack results for FaceSerub models trained with varying numbers of training samples per class (2a) and different smoothing factors (b). Results are stated as the relative improvement, denoted advantage, compared to the model trained with hard labels. While positive LS has larger impact on low-data regimes, negative LS acts as a stronger defense when trained on more samples.
Real
==0.0
-0005 0.1
Figure 3: Attack samples from the FaceScrub models trained with 30 samples per class. Samples are not cherry-picked but show the most robust attack results based on PPA's selection procedure. The model trained with positive LS (a 0.1) clearly reveals more visual characteristics the target identities, whereas attacks on the negative LS model (a = -..05) generate misleading results.
To further investigate the impact the available number of training samples on the effects of LS, conducted experiments by training the targets with fixed number of samples per class assess whether the effect of L.S depends on the training set size. In Fig. [2 we present the computed metrics the relative advantage compared training without LS make the differences more apparent. Notably, all models trained with positive LS exhibit increased privacy leakage, with the effect being more pronounced low-data regimes. Conversely, the defensive effects of negative LS relatively improve as the number of training samples increases. The impact of LS is also reflected in the resulting attack samples depicted Fig. A sensitivity analysis Fig. 2b further demonstrates that smoothing factors above 0.1 only marginally contribute to increased privacy leakage, whereas negative factors exhibit an increasingly beneficial defensive effect.
4.3
LABEL SMOOTHING'S s SHAPING EFFECTS ON EMBEDDING SPACES
The effectiveness of MIAs relies upon model's 's ability to discern class-specific features and dis- tinguish them from those of other classes. To assess the influence of LS, we turn our focus to the embedding spaces within the penultimate layer of our ResNet-152 models. These spaces offer lower- dimensional representations of input samples, where inputs considered similar by model are placed closer together, while dissimilar inputs are placed farther apart. In Fig. 4 we employ 1-SNE (van der Maaten & Hinton 2008) to visualize the embeddings of the derived from training samples across 100 different classes. Without LS (Fig. 4a). the model tends to form clusters among samples from the same class, yet the distinction from other clusters remains rather subtle, with some clusters overlapping. Positive LS (Fig. (b)), however, noticeably enhances the separation between samples from different classes and tightens sample clusters from the same class, which has also been observed

---

Published as a conference paper at ICLR 2024
(a) Hard Labels (a 0.0)
(b) Positive LS (a 0.1)
(c) Negative (a 0.05)
Figure 4: Penultimate activations of training samples from 100 FaceNet classes (colors are reused). Compared to training with hard labels 44a. training with positive LS (4b) clusters samples from the same class together. Smoothing the labels with negative factor (4c] reverses this effect and instead places samples from different classes closer together to build a less clearly separated space.
3.0
Inire Was
Intredtass
Intradies
Interilexs
Interisss
Insert lass
Intret was Nearest Neightor
Inteec wes
Intraciass
Intenc
Intrac
Interc asa Nearast Neigh boor
Class Average
Nearest Neighbor
Class Average
Class Average
(a) Hard Labels (a 0.0)
(b) Positive L.S (a 0.1)
(c) Negative LS (a 0.05)
Figure 5: Distribution of mcale le feature distances between penultimate layer activations. The left-hand side of each plot depicts the average distance between each training sample and all other samples from the same class (intraclass) and other classes (interclass). The right-hand side of each plot shows the distances to the closest sample. Positive LS reduces the relative intraclass distances while increasing the distance to other samples, whereas negative LS partly reverts this effect and moves some samples from other classes closer samples of particular class.
by previous research Müülerr et al. 2019: Chandrasegaran et al. 2022 This observation suggests that the model has effectively captured discriminative features crucial for identity recognition. On the other hand, training with negative LS (Fig. 4c) partially counteracts this effect, as it promotes increased overlap among different clusters, thereby undermining the clarity separation.
To establish guantitative foundation for observations, computed the lo distances between the embeddings of all training samples. The violin plots presented Fig. deppic the average distances between samples within the class (intraclass) and all samples from other classes (interclass). Additionally, we calculated the average distance from each sample to its nearest neighbor. These results confirm the trends observed in our embedding space visualizations. Specifically, comparison to training with hard labels (Fig. 5ab. the introduction of positive LS (Fig. [b)) effectively diminishes the relative distance between samples belonging the same class, while simultaneously increasing the separation from samples of other classes. Training with negative LS (Fig, 5c also increases the relative distances between samples different classes, but the nearest neighbor interclass distances are comparable to the average interclass distance. This observation suggests that samples within single cluster exhibit a higher degree of label inconsistency.
In a broader perspective, generative MIAs can be described as the process of exploring the data manifold of the generative model, guided by the target model, to find meaningful representations of specific classes. In this regard, positive LS is expected to enhance the exploration by offering better guidance since the target model is able to better distinguish between the features different classes. The guidance signal of models trained with negative LS, in turn, contains less clear information, as class embeddings overlap, obfuscating the characteristic features individual classes. We delve deeper into the impact of L.S the various stages of MIAs in the following section.
7

---

Published as a conference paper at ICLR 2024
30
Poditive 15
(2-0.1)
20-
Negative LS (n--0.051
11.95%
Grodient Similartty
0.5-
10
4.64%
4,00%
Advantage
-10
10.26%
-20-
-0.5
Hand Labels
Positive LS
(a-0.0)
(c-0.1)
-10-
-2527%
28.57%
-1.0
Negative (ew-0 (.000.05)
Facehet
Eveluation
Evaluation
20
40
66
80
Optimization Step
(a) Quality of initially selected vectors
(b) Similarity between consecutive image gradients
Figure 6: Fig. shows the advantage of the distance and confidence metrics computed on the initial latent vectors selected by different models. Values are relative to the results of the model trained without LS. Positive smoothing improves the sampling, whereas the samples selected by the negative smoothing model perform markedly worse. Fig. 66) visualizes the mean cosine similarity and their standard deviation between consecutive image gradients for the individual optimization steps. While the gradient directions are stable for models trained with hard labels and positive smoothing, the optimization path on the negative smoothing model is characterized by many changes of direction.
4.4 ABLATION STUDY: WHICH STAGES OF MODEL INVERSION ARE AFFECTED BY LS?
The inference process of MIAs can be grouped into three stages: latent vector sampling, optimization, and result selection. We will now individually analyze the impact of LS each stage while isolating influences from the other stages. The analyses are based on the FaceScrub models.
Stage 1 Sampling (Affected): The initial phase involves the selection of latent vectors to be optimized by the attack. PPA first samples a larger pool of latent vectors and subsequently chooses fixed set of vectors for each target class whose corresponding images achieve the highest classification probability on the target model under random transformations. For our analysis, we created fixed pool of 10,000 random latent vectors and then let each model select 50 samples for each class. To assess the quality of the initial sampling, we computed feature distances between the corresponding generated images and the training data, analogous to our evaluation metrics. Furthermore, we considered the confidences assigned by the evaluation model to determine which model selected samples that visually resembled the target classes most closely. The results, which are presented in Fig. 6al again state the relative advantage compared to the model trained with hard labels. All three metrics indicate that the samples selected by the positive LS model indeed more closely resemble the target classes compared degradation across all three metrics. To further validate the sampling' 's influence, we performed PPA's optimization process on the model trained without L.S using the three different sets of initial latent vectors. As expected, the results confirm that samples selected with the positive LS model outperform those from the standard model, while the samples selected by the negative LS model underperform. Overall, LS seems to have a notable impact on the sampling stage of MIAs.
the standard model. Conversely, the negative LS model exhibits
Stage 2 Optimization (Heavily Affected): This stage comprises the attack process, wherein the latent vectors are optimized to reconstruct characcharacteiiittc features. During this iterative procedure, the generated images x are fed into the target model compute a loss based on its prediction Margee (a:) for the target class The latent vectors are then updated to reduce the loss. To gain insights into the optimization stability, we sampled a set of 1,000 initial vectors and targets. At each step their similarity to the gradients from the previous step t - 1 using the cosine similarity Sc defined by
we computed the loss gradients ( w.r.t. the current images æ and measured
( ((-1) |V [
Sc (VC(), V
with v Vacs L (Murrgel (x (z(()).
to examine the dynamics of gradient direction changes across consecutive optimization steps. A stable optimization process implies that successive gradients should remain consistent in their direction. The visual representation of the mean similarity between consecutive gradients depicted in Fig. 6b Remarkably, the models trained with hard labels and positive LS exhibit high degree of gradient similarity, indicating stable optimization. In contrast, the negative LS model shows substantial variations in gradient directions. This connotes that the optimization frequently changes direction,
8

---

Published as a conference paper at ICLR 2024
with later optimization steps pointing in orthogonal even opposing directions. This observation is further reflected in the attack metrics, which report poor results for the optimization performed on the negative LS model and again improved results for the model trained with positive LS. Consequently, LS also substantially influences the optimization stage of MIAs with negative LS hurting its stability. The unstable gradient directions explain the attack failures on models trained with negative smoothing.
Stage 3 - Selection (Barely Affected); After the optimization, PPA selects a subset for each target class by filtering out those results for which the target model shows the least robust confidence. The selection is done by feeding various transformed versions of each corresponding image into the target model, computing its prediction score for the target class, and averaging across all transformations. Then, the latent vectors with the highest mean confidences are selected as attack results. To measure the effects of the various models, we took the optimization results of 200 samples per target class from the model trained without LS and trained with negatíve LS and then repeated the filtering approach for both sample sets on the three models to see which model selects the most promising samples. The previously observed pattern the positive LS model improves results and the negative LS model degrades them- - is still apparent, but differences are rather small. Consequently, LS seems to have only small effect on the selection stage and all models perform rather similarly.
5 IMPACT, FUTURE WORK AND LIMITATIONS
Deep learning promises impressive potential in virtually all areas of our life. However, its applications have to be secure and protect user and data privacy, which can be compromised by MIAs. Our findings show that LS regularization techniques can amplify model inversion attacks a previously unexplored dimension of privacy leakage. We show how to turn an attack leverage into an attack blocker and that LS also offers straightforward mitigation strategy by smoothing with a negative smoothing factor, which trades model calibration and small amounts of utility for a strong defense against MIAs, particularly gradient- based attacks. Importantly, this process requires no complex adjustments the training procedure or model architecture. An interesting direction for further research involves the investigation of other regularization methods. Another critical question is whether existing attacks can be adjusted to improve their results on negative LS models, e.g., by also taking the distance to decision boundaries during the optimization into account. Furthermore, we envision that the information reduction effects of negative LS in model's 's confidence scores can also be valuable in mitigating other privacy (Shokri et al. 2017 Hintersdorf et al. 2022 Struppek et al.] 2023a) or model stealing attacks (Tramer et al.) 2016), that exploit model confidences in black-box fashion.
Nevertheless, the investigation of MIAs also faces various challenges, and our work is no exception in this regard. First, performing MIAs is very time- and sesorcc--onssmingg This limitation adds constraints on the extent to which we can conduct detailed hyperparameter analyses. We anticipate that more exhaustive grid search of smoothing parameters and schedules could potentially yield an even more favorable balance between model utility and defense against MIAs, suggesting that our results may actually underestimate the true impact of LS. However, we found a -0.05 generally serves as promising initial value and our findings present compelling evidence for the dual effects LS within the realm of privacy. Moreover, the field of MIAs still lacks a comprehensive theoretical framework. In contrast to other attack classes, e.g., adversarial examples, MIAs are considerably more intricate and require deeper understanding of the features learned and encoded in a model's weights. Our research is a first step in advancing the understanding of MIAs.
6 CONCLUSION
In contrast to previous literature on MIAs, we conducted the first study on the impact of model regularization on the attacks' success. Specifically, we investigated the impact of label smoothing regularization on the vulnerability of image classifiers to MIAs. Our findings reveal a remarkable phenomenon: training a model with a positive smoothing factor increases its privacy leakage, particularly in settings with limited training data. In contrast, training a model with negative label smoothing counteracts this trend and emerges as a practical and viable defense mechanism. No architectural modifications or complex training procedures model utility. Our work underlines the importance of delving more into factors that influence a model's privacy leakage, moving the research paradigm from improving the attacks themselves.
required, and it only slightly reduces
9

---

Published as a conference paper at ICLR 2024
ETHICS STATEMENT
This work investigates model inversion attacks, a common privacy attack against deep learning classifiers. Successful attacks bear the risk of privacy leakage and cause serious harm to individuals in privacy-sensitive settings. Whereas we did not propose any novel attack algorithms, our insights into label smoothing regularization can indeed have practical implications for the privacy of models. However, we believe that informing the community about the dual role of label smoothing important to raise awareness of these risks. Moreover, we also present practical defense strategy by training classifiers with negative label smoothing to mitigate model inversion attacks and the corresponding privacy risks. Overall, we are convinced that the henefits of the present work outweigh any potential risk.
ACKNOWLEDGMENTS
This work was supported by the German Ministry of Education and Research (BMBF) within the framework program "Research for Civil Security" of the German Federal Govermment, project KISTRA (reference no. 13N15343). We gratefully acknowledge support from the German Center for Artificial Intelligence (DFKI) project "SAINT".
REFERENCES
Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. VGGFace2: A Dataset for Recognising Faces across Pose and Age. In International Conference on Automatic Face & Gesture Recognition, pp. 67-74, 2018.
Keshigeyan Chandrasegaran, MroonTTrngggn Tran, Yunqing Zhao, and Ngai-Man Cheung. Revisiting label smoothing and knowledge distillation compatibility: What was missing? In International Conference on Machine Learning (ICML), pp. 2890-2916, 2022.
Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi. Knowledge- Eerriegg-Enrrccedd Distributional Model Inversion Attacks. In International Conference on Computer Vision (ICCV), pp. 16178-16187, 2021.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint, 171.0527.,. 2017.
Jan Chorowski and Navdeep Jaitly. Towards Better Decoding and Language Model Integration in Sequence to Sequence Models. In Conference of the International Speech Communication Association (Interspeech), pp. 523-527, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Conference Computer Vision and Pattern Recognition (CVPR), pp. 248-255, 2009.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model Inversion Attacks that Exploit Confi- dence Information and Basic Countermeasures. In Conference on Computer and Commurications Security (CCS), pp. 1322-1333, 2015.
Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon M. Lin, David Page, and Thomas Ristenpart. Privacy Pharnacogenetics: An End-to-EE Case Study of Personalized Warfarin Dosing. In USENIX Security Symposium, pp. 17-32, 2014.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. Conference on Neural Information Processing Systems (NeurIPS), 2014.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations (ICLR), 2015.
Tianyu Gu, Brendan Dolan-Gavine and Siddharth Garg. Badnets: Identifying vulnerabilities in the
machine learning model supply chain. arXiv preprint, 17XXvv1708..661 2017.
10

---

Published as a conference paper at ICLR 2024
Gyojin Han, Jachyun Choi, Hacil Lee, and Junmo Kim. Reinforcement Learning Based Black-Box Model Inversion Attacks. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20504-20513, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Conference on Computer Vision and Puttern Recognition (CVPR), pp. 770-778, 2016.
Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. Bag of Tricks for Image Classification with Convolutional Neural Networks. In Conference on Computer Vision and Puttern Recognition (CVPR), pp. 558-567, 2019.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by Two Time-Scale Update Rule Converge to Local Nash Equilibrium. In Conference on Neural Information Processing Systems (NeurIPS), pp. 6626-6637, 2017.
Dominik Hintersdorf, Lukas Struppek, Manuel Brack, Felix Friedrich, Patrick Schramowski, and Kristian Kersting. Does clip know my face? arXiv preprint, arXiv:220900734,, 2022.
Jeremy Howard. Imagenette, 2019. URL httt::://gtthub..cm//sttaaaa/// Accessed: 2023-11-14.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely Connected Convolutional Networks. In Conference on Computer Vision and Puttern Recognition (CVPR), pp. 2261-2269, 2017.
Mostafa Kahla, Si Chen, Hoang Anh Just, and Ruoxi Jia. Label-Only Model Inversion Attacks via Boundary Repulsion. In Conference on Computer Vision and Patlern Recognition (CVPR), pp. 15025-15033, 2022.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and Improving the Image Quality of StyleGAN. In Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Diederik P. Kingma and Jimmy Ba. Adam: Method for Stochastic Optimization. In International Conference Learning Representations (ICLR), 2015.
Alexey Kurakin, lan J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference on Learning Representations (ICLR), Workshop Track, 2017.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild. In International Conference on Computer Vision (ICCV), 2015.
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, and Sanjiv Kumar. Does Label Smoothing Mitigate Label Noise? In International Conference on Machine Learning (ICML), volume 119, pp. 4458-6458. PMLR, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations (ICLR), 2018.
Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When Does Label Smoothing Help? In Conference on Neural Information Processing Systems (NeurIPS), pp. 4696-4705, 2019.
Mahdi Pakdaman Nacini. Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In AAAI Conference on Artificial Intelligence (AAAI), pp. 2901-2907, 2015.
Hongwei Ng and Stefan Winkler. A Data-Driven Approach to Cleaning Large Face Datasets. In IEEE International Conference Image Processing (ICIP), pp. 343-347, 2014.
Ngoc-Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Cheung. Re-thinking model inversion attacks against deep neural networks. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16393, 2023.
11

---

Published as a conference paper at ICLR 2024
OpenAl. GPT-4 Technical Report. arXiv preprint, atXiv:3303.08774, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zachary Vitoto, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style. High-Performance Deep Learning Library. In Conference on Neural Information Processing Systems (NewrIPS), pp. 8024-8035. 2019.
Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han. Bilateral Dependency Optimization: Defending Against Model-Inversion Attacks. In Conference on Knowledge Discovery and Data Mining (KDD)), pp. 1358-1361 1367, 2022.
Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. Regularizing Neural Networks by Penalizing Confident Output Distributions. In International Conference on Learning Representations (ICLR) Workshop, 2017
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ily Sutskever. Learning Transferable Visual Models From Natural Language Supervision. International Conference on Machine Learning (ICML), pp. 8748-8763, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical Text- Conditional Image Generation with CLIP Latents. arXiv preprint, arXiv:2204.66125 2022.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering. In Conference Computer Vision and Pattern Recognition (CVPR), pp. 815-823, 2015.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference Attacks Against Machine Learning Models. In Symposium on Security and Privacy (S&P), pp. 3-18, 2017.
Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correia, Antonia Adler, and Kristian Kersting. Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks. In International Conference Machine Learning (ICML), pp. 20522 205455, 2022a.
Lukas Struppek, Dominik Hintersdorf, Daniel Neider, and Kristian Kersting. Learning to break deep perceptual hashing: The use neuralhash. In ACM Conference Fairness, Accountabiliry, and Transparency (FAccT), pp. 58-69, 2022b.
Lukas Struppek, Dominik Hintersdorf, Felix Friedrich, Manuel Brack, Patrick Schramowski, and Kristian Kersting. Class attribute inference attacks: Inferring sensitive class information by diffusion-based attribute manipulations. arXiv preprint, arXiv:2303.09239 2023a.
Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. Rickrolling the artist: Injecting backdoors into text encoders for text-to-image synthesis. In International Conference Computer Vision (ICCV), 2023b.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. Transactions on Evolutionary Computation, pp. 828-841, 2019.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing Properties of Neural Networks. In International Conference on Learning Representations (ICLR), 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the Inception Architecture for Computer Vision. In Conference Computer Vision and Pattern Recognition (CVPR), pp. 2818-2826, 2016.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A. Alemi Inception-v4, Enception-ResNee ResNet and the Impact of Residual Connections on Learning. In AAAI Conference on Artificial Intelligence (AAAI), pp. 4278-4284, 2017.
12

---

Published as a conference paper at ICLR 2024
Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. Stealing Machine Learning Models via Prediction APIs. In USENIX Security Symposium, pp. 601-618, 2016.
Laurens van der Maaten and Geoffrey E. Hinton. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9:2579-2605, 2008.
Kuan-Chieh Wang, Yan Fu, Ke Liand Ashish Khisti, Richard Zemel, and Alireza Makhzani. Vari- ational Model Inversion Attacks. In Conference on Neural Information Processing Systems (NeurIPS), 2021a.
Qingzhong Wang, Pengfei Zhang, Haoyi Xiong, and Jian Zhao. Face.evoLVe: High-Performance Face Recognition Library. arXiv preprint, utXiv:2107.08621, 2021b.
Tianhao Wang, Yuheng Zhang, and Ruoxi Jia. Improving Robustness to Model Inversion Attacks via Mutual Information Regularization. In AAAI Conference on Artificial Intelligence (AAAI), pp. 2021c.
Jiaheng Wei, Hangyu Liu, Tongliang Liu, Gang Niu, Masashi Sugiyama, and Yang Liu, To Smooth or Not? When Label Smoothing Meets Noisy I Labels. In International Conference Machine Learning (ICML), volume 162, pp. 23589 23614. PMLR, 2022.
Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated Residual Transformations for Deep Neural Networks. In Conference Computer Vision and Pattern Recognition (CVPR), pp. 5987-5995, 2017.
Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang. Pseudo LaiellGuided Model Inversion Attack via Conditional Generative Adversarial Network. In AAAI Conference on Artificial Intelligence (AAAI). 2023.
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song, The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 250-258, 2020.
Tianqing Zhu, Dayong Ye, Shuai Zhou, Bo Liu, and Wanlei Zhou. Label-Only Model Inversion Attacks: Attack With the Least Information. IEEE Transactions on Information Forensics and Security, 18:991-1005, 2023.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoe Le. Learning Transferable Architectures for Scalable Image Recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8697-8710, 2018
13

---

Published as a conference paper at ICLR 2024
A FORMAL ANALYSIS OF LABEL SMOOTHING REGULARIZATION
We provide a more formal analysis of Label Smoothing regularization to demonstrate its impact on the model training. Be E {0, _j with ly| = - = 1 the one-hot encoded ground-true label of training sample with C possible classes. Here, yk denotes the k-th entry in y. The label vector contains C' entries, which are all set to 0 except the correct label set to 1.
Let [0, 1JSh with Ip- Ex k= Pk- further denote the softmax probability vector computed by classifier for input æ. Again, pa denotes the k-th entry of p. The sofumax function is computed on the output logits z and corresponds to the model'ss confidence for the j-th class. It is computed as
a (z)j
(5)
k== 8
Neural network classifiers are usually trained by minimizing a cross-entropy loss, defined by
Co p) - ye Px
(6)
Given the standard empirical risk minimization framework, a classifier M is trained by searching the hypothesis space H. for model M that minimizes the loss on the training set S with N samples:
N
1
M = arg min MEH N
Σ 2=1
Cy ())).
Here, y(i) denotes the label vector for the i-th training sample, and (()) is model's predicted confidence vector for this sample. Practically, this is done by adjusting model's parameters with gradient descent. Training with standard cross-entropy loss encourages the model to increase the logit values for the predicted class. This can be seen by computing the gradients of Lce with respect to the j-th logit value Zz.
First, we compute the derivative of the softmax score pt with respect to the logit value 2y. Let us start with the case j. i.e., the index of the softmax score and the logit vector are identical:
Opj 0z,
e Ee E E
Pj 22 = pp - (p-p))- (8)
(EEe) ezh)
D c
1==1
For the case i # j, the derivative is computed as follows:
3p:
- e
02,
C km
e*
-P.P.-
(9)
With the softmax derivatives from Eq. and Eq. respect the output logits:
9
we can now compute the derivatives with
aLce (y.p)
Σ
Uk
a llg(pss
U/ Opk
yi p5
Ps)
Σ
Uk
PkP;
Öp;
0zj
Pk 0zj
Pj
Pj
==1
1=
(10)
C
= - p.) + P
Σ yk -
+ Σ yk =
knj
Given the resulting gradient function for a model's logits, it becomes clear that gradient descent updates the model weights to increase the logit values for the target index while decreasing the values for all other classes.
Next, let us analyze the gradients for training with label smoothing regularization, which replaces the one-hot encoded label vector y with its smoothed variant y defined by
y = (1 - a)
a
(11)
c
14

---

For negative label smoothing, on the other hand, this saturation effect never occurs, since 1-a+ C > 1 always holds true for C 1 and < O. Therefore, the weights for computing the logits of the target class are always increased, even if the predicted confidence approaches 1.0. This explains why Published as a conference paper at ICLR 2024
Here, = (-o0, 1] denotes the smoothing factor. Standard label smoothing uses > whereas negative label smoothing applies a < 0. The case of a = corresponds to the hard label setting, Recall that label smoothing still maintains the condition lyl 1, independent of the selected smoothing factor. Plugged into the logit gradient formula from Eq. (TO). it allows us to analyze the impact of label smoothing during training.
First, we take a look at the gradients for the j-th logit vector entry corresponding to the ground-true class. In this case, label smoothing replaces yjj = 1 by o// 1 a+ &. which leads the following derivative:
DCce(y's .p) Pj 1/ pj -(1 -1- - 2) dp;
(12)
Analogously, the gradients for the i-th entries in the logit vector that do not correspond to the ground-true class are computed by:
dc P)) = P:
LS
a P. C
(13)
0p:
Given the nature of gradient descent, which subtracts the gradients (weighted by the learning rate) from each parameter, label smoothing offers interesting effects on the updates of the weights used to compute the logit vector. Generally, the weights associated with the j-th logit, which corresponds the ground-true class j. are increased, as long as PS
n < 0 holds:
Pj < 1, Pp 1 C.
i
- LS
<
(14)
otherwise.
We can see that for training without label smoothing the weights for the target class are almost always increased but the gradient update saturates when Pj approaches 1.0. For positive label smoothing. this saturation effect occurs earlier, when P/ approaches 1 + S. Let us take an example with = 0.1 and C 10 classes. In this case, the weights are increased until p5 0.91. If the predicted confidence for sample exceeds this value, the gradients change directions and the resulting weights are reduced. This explains the calibration effects label smoothing, which prevents the model from being overconfident in its predictions.
models trained with negative label smoothing are overconfident in their predictions and usually only barely calibrated.
Revert effects can be shown for the weights of the remaining logits. The weights associated with those outputs are decreased as long as the following holds true:
p: 0, =
P: y
(15)
Pi C otherwise.
Again, training with hard labels always reduces the weights but the gradients saturate for pt approach- ing 0.0. For positive label smoothing, the gradient directions change as soon as < 8. supporting the mitigation of overconfidence. For negative label smoothing pt - > 0 always holds true since
a is always smaller than 0.0 because > 0 and a < 0.
15

---

Published as a conference paper at ICLR 2024
B EXPERIMENTAL DETAILS
Here, we state the technical details of our experiments to improve reproducibility and eliminate ambiguities.
B.1 HARD- AND SOFTWARE DETAILS
We performed all our experiments on NVIDIA DGX machines running NVIDIA DGX Server Version 5.2.0 and Ubuntu 20.04.4 LTS. The machines have 1TB of RAM and contain NVIDIA L100-XXM4400GB GPUs and AMD EPYC 7742 64-core CPUs. We further relied on CUDA 11.4, Python 3.8. 10, and PyTorch 2.0.0 with Torchvision 15.1 Paszke et al. (2019) for our experiments. If not stated otherwise, we used the model architecture implementations and pre-trained ImageNet weights provided by Torchvision. We further provide Dockerfile together with our code to make the reproduction of our results easier. In addition, all training and attack configuration files are available to reproduce the results stated in this paper. Our main experiments are built around the Plug & Play Áttacks (Struppek et al. 2022a) repository available at https:: / /qithub...... com/ Lukaasttruppe/ andg--and--Paaae
Note that we updated the Py Torch version, which may lead to small differences when repeating the experiments with older version.
B.2 EVALUATION MODELS
For experiments based on Plug & Play Attacks (PPA), we used the pre-trained Inception-v3 evalua- tion models provided with the code repository at putttps:: //tt:::://ithubb com/ LukasSt P Plug- and Play At tacks For training details, refer to SSruppek et al. (2022a). The models
achieve a test accuracy of 96. 26.20% (FaceScrub) and 93.28% (CelebA), respectivel..
We also used the pre- trainrr FaceNet (Schoff et al. (2015) from https:/gghhub.com/mmmeee/////a/e pytorch to measure the distance between training samples and attack results on the facial recognition tasks. The FaceNet model is based the Inception-ResNet-.. SZzgedy al. (2017) architecture and has been trained on VGGFace2 Cao et al. (2018).
Schroff
For experiments based CelebA classifiers with smaller image resolutions, we used the evalu- ation model provided at thttps://// /ttt::::/ggtttii com/ SCccc21 Mence1/Knnoan //noo/owwiil -eweeeeegeeeeeeeeeeeeeeeeeee EEEnichhed -MI for download. The model is built around the Face.evoLVe (Wang et al. 2021b) framework with modified ResNet50 backbone and achieves stated test accuracy of 95. 85.8878 %. For training details, we refer Zhang et al. (2020).
B.3 TARGET MODELS
For training target models with PPA, we relied the training scripts and hyperparameters provided in the corresponding code repository and described in Struppek et al.] (2022a) The only training parameter we changed was the smoothing factor of the label smoothing loss. All models were trained for 100 epochs with the Adam optimizer (Kingma & Ba 2015) and an initial learning rate of 0.001 and B = (0.9, 0.999). We multiplied the learning rate after 75 and 90 epochs by factor of 0.1. The batch size was set to 128 during training. All data samples were normalized with resized to 224 224. The training samples were then augmented by random cropping with scale of [0.85, 1.0] and a fixed ratio of 1.0. Crops were then resized back to 224 x 224. We also applied random color jitter with brightness and contrast factors of 0.2 and saturation and hue factors of 0.1. Finally, samples were horizontally flipped in 50% of the cases.
= 0.5 and
The target models trained on 64 x 64 CelebA images were trained with the training script provided at https: thub. com/ SCccc21 Knowl wwweeegee-aa Enri ched DMI To use more recent and advanced architecture, we trained Re models initialized with pre-trained ImageNet weights. The models were trained for 100 epochs with the SGD optimizer with an initial learning rate of 0.01 and a momentum term of 0.9. The batch size was set to 64. We reduced the learning rate after 50 and 75 epochs by factor of 0.1. Weight decay has not been used for training.
16

---

Published as a conference paper at ICLR 2024
1.0
Gradient Similarity
0.5
0.0
-0.5
Hard Labels (a=0.0)
Positive LS
(r==011)
Negative (a=-0.05)
-1.0
20
40
60
80
Optimization Step
Figure 7: Similarity between consecutive image gradients based cross-entropy loss instead of the Poincaré loss used in Fig.
B.4 PLUG PLAY ATTACKS
PPA consists of three stages: latent vector sampling, optimization, and result selection. We provide brief overview of each stage and refer for more comprehensive introduction to Struppek et al. (2022a).
The attack parameters follow those of the paper and used a pre-trained FFHQ StyleGAN 2 KKrras
et al. 2020). available at httpps:: //g t.ub. com/ NV1 aallab///a sty egan2 aca pytorch
Stage 1: Latent Vector Sampling: PPA first samples a large number of latent vectors as candidates for the attack. Out of this set of latent vectors, the attack selects for each target class a fixed number of vectors as starting point for the optimization. The selection is done by generating the corresponding images for each latent vector and then measuring the target model's prediction confidence on the augmented version of these images. The top l samples with the highest mean confidence assigned then selected as starting points.
During the sampling stage, we sampled 200 candidates for each target class out total search space of 2,000 (FaceSerub) and 5,000 (CelebA). Since the StyleGAN model generates images of size 1024 x 1024, samples were first center cropped with size 800 x 800 and then resized to 224 x 224.
Stage 2: Optimization: Instead of standard cross-entropy loss, PPA uses Poincaré loss function to mitigate the problem of vanishing gradients:
Posncare d(u, v)
= arcosh (
2 uu
(16)
(1 |u [2](2)((1)))))))))))))))
10
Here, u = Jallz Ialla are the normalized output logits and is the one-hot encoded target vector with the target label set to 0.9999 instead of 1.0. The attack further applies random augmentations on the images generated by the GAN before feeding them into the target model increase the attack's robustness and avoid the generation of adversarial examples.
In our experiments, samples were optimized for 50 (FaceSerub) and 70 (CelebA) steps, respectively.
Before feeding the cropped and resized samples into the target model, random resized crop with a scale between [0.9, 1.0] was applied and the cropped images resized back to 224 x 224. For optimizing the latent vectors, the Adam optimizer with a learning rate of 0.005 and 3 0.11 was used.
We repeated the experiment with a standard cross-entropy loss instead of PPA's Poincaré loss to demonstrate that the gradient instability does not arise from the model's optimization goal. We kept all other parameters identical. The results in Fig. demonstrate that also with another loss function used the gradients on the model trained with negative LS are still unstable, and the similarity between consecutive gradients is still low.
Stage 3: Result Selection: Out of the set of optimized latent vectors for each target class, PPA selects subset for which the target model shows the highest robustness under random augmentations.
17

---

Published as a conference paper at ICLR 2024
Steps Taken: 4999
Steps Taken; 4999
Steps Taken: 4999
Closest Sample Distance: 0.82
Closest Sample Distance: 0
Closest Sample Distance: 5.64
1.00
0.75
(a) Hard Labels (a 0.0)
(b) Positive LS (a 0.05)
(c) Negative LS (a 0.05)
Figure 8: Variation of our motivational example without using an attack-stopping criterion. Instead, the attack in all three settings is run for 5,000 steps.
More specifically, each image corresponding to an optimized latent vector undergoes strong random transformations to create variations of it. All augmented variations are then fed into the target model to compute the mean prediction confidence on the target class, Out of all candidates, the top samples with the highest robust confidence are selected as final attack results.
During the final selection stage, 50 samples out of the 200 optimized samples were selected for each target class. As random transformations horizontal flipping with p 0.5 and random cropping with scale between [0.5, 0.9] and a ratio of [0.8, 1.2] are performed 100 times. The resulting samples were resized back to 224 224 before being fed into the target models.
B.5 COMPARISON TO EXISTING DEFENSE MECHANISMS
For comparison to previous defense approaches, we trained ResNet-152 models with BiDO (Peng et al. 2022) and MID (Wang al. 2021c). The model architecture is based the official Py Torch implementation. For MID, we added the information bottleneck based on the variational method between the average pooling and the final linear layer and the bottleneck size BiDO, we used the outputs of the four major ResNet blocks as inputs for the regularization loss. We further relied on the Hilbert-Schmice independence criterion (HSIC) as dependency measure since the corresponding paper reported better results compared to the constrained covariance (COCO) dependency. The remaining training hyperparameters and data augmentations are identical those stated in Appx. B.3
1024. For
B.6 TOY EXAMPLE FROM SEC.
The network architecture used for the motivational example in Sec. 3 is simple 3-layer fully- connected network. The hidden layer consists of 20 neurons. A batch norm layer is placed after the first and second layers, each followed by ReLU activation. All models were trained with a standard cross-entropy loss (see Eq. 3 with different label smoothing factors a {0, 0.05, -0.05). The models were then optimized for 5000 iterations using standard SGD with a learning rate of 0.001 and a momentum of 0.9.
The attack starts from (fixed) random point from the green circles class and updates it to maximize the model's ' prediction score for the orange pentagons class by minimizing an identity loss (cross- entropy loss) proposed by /Zhang et al. (2020) (GMI). Since no image prior is applied, there are no additional loss terms. For optimization, SGD with a learning rate of 0.1 and no momentum is used. For the experiment, we stopped the optimization as soon the model's confidence for the class orange pentagons exceeds 95%.
In addition, repeated the attack process without any stopping criterion and optimized the samples for 5000 steps. The results depicted in Fig. g draw a similar picture. For training without LS, the attack approaches the training data but the results compared to training with positive LS markedly further away from the data. For the negative I.S model, the attack again fails to approach the training set and stays close to the decision boundary.
18

---

Published as a conference paper at ICLR 2024
C ADDITIONAL EXPERIMENTAL RESULTS
In this section, we state additional attack results that did not into the main part of the paper. Besides the metries from the main paper, we also computed the common Fréchet inception distance (FID) (Heusel et al. 2017) between synthetic samples and the training data. Moreover, the knowledge extraction score Ştest is also computed the surrogate model's prediction aceuracy on the target model's 's test data. To measure a model's ' calibration, we computed the expected calibration error (ECE) (Naeini et al. 2015) with 10 bins and the 22 norm on the individual test splits.
C. PLUG PLAY ATTACKS RESULTS FOR VARIOUS ARCHITECTURES
Tab. states the results for PPA performed against various architectures trained FaceScrub CelebA, respectively. We further varied the LS smoothing factor to showcase the influence of difference values. All training and attack hyperparameters are identical between the different runs. The results extend Tab. and correspond to Fig. 2b from the main paper.
Table 2: Additional PPA attack results for attacks against models trained with label smoothing.
Architecture
f Test Ace
ECE
f Accc
Acc
+ 6 face
$ &ceat
LFID
Eirain
f Erest
0.3
96.96%
0.3248
95.80%
+
98.17%
0.6108
107.76
11.22
72.83%
70.01%
0.2
97.23%
0,2176
94.00%
+ 2.9
98.77%
0.656T
109.46
14.10
71.08%
69.67%
ResNet-152
0.1
177.99%
0.1933
95.20%
1. 2.9
98.47%
0.6343
107.36
43.33
70.95%
68.77%
FaceSerub
0.0
94.93%
0,0619
94.32%
+ 1.9
99.37%
0.7060
124.30
10.58
61%1
58.69%
-0,005
94.01%
0.1389
80.66%
2.0
95.82%
0.8186
138.64
45.12
58.429%
55.52%
-0.01
83.77%
0.1262
50.87%
4.4.9
86.44%
0.9225
158.07
49.84
53.0095
52.03%
0.05
91.45%
0.1474
14.34%
1. 7.6
30.94%
1.2320
239.02
59.38
16.45%
15. 73%%
97.54%
95.95%
29%.55%
0.6346
107.38
733.88%
71.30%
ResNeXt-50
0.0
95.25%
0.0771
91.97% + 1.6
99.38%
0.6997
119.51
11.61
67.78%
64. 86%
22.77%
1.2790
15.31%
0.1
97. 15%
0.0452
94.86% + 2.9
97.70%
0.6416
110.31
43.18
67. 07.42
64.8.15
DenseNet-121
0.0
95.72 72%
0.1865
96.00%
+ 1,2
99 63%
0.6795
116.40
13.11
73. 0396
71.30%
0.05
92.13%
0. 1525
40.60%
+ 7.1
69.60%
0.9733
179.53
49.91
32.7995
36.76%
0.1
95.11%
0.4545
92.85% + 3.4
96.40%
0.6065
275.30
38.77 73
66. 1396
59.65%
ResNet-112
0.0
57.05%
0.0899
81.76% + 1.3
95.03%
0.7406
318.09
36.12
59.77%6
50.67%
0.05
83.59%
0.2179
26.41% 4 3.4
49.96%
1.0420
441.66
61.30
7.0895
5.80%
CelebA
0.1
95 27%
0,4002
93.37%
96.81%
0.6010
275. 48
59
65, 75%%
58, 95%
ResNeXt-50
0.0
87.85%
0.1174
85, 139% 1: 1.1
96.07%%
0.7310
307.00
33..9
633. 14%
55. 55%7
32.833%
56.61%
1.0140
430.08
33.54%
28.86%
0.1
92. 92.88%
0.4113
90. 90%9 + 3.3
96.35%
0.64884
376. 46
89.54 54.554
70.841%
62. 48%
DeaseNet-121
0.0
86.05%
0.0063
76. 4994 1.0
92.23%
0.7410
383.77 74
87. 73
60.42%
51.00%
-0.05
86.48%
0,2062
72.70% + 2.2
90.80%
0.7866
467.67
94.78
50.22%
44.00%%
19

---

Published as a conference paper at ICLR 2024
C.2 ADDITIONAL RESULTS FOR DEFENSE MECHANISMS
We compared the defensive effects of negative LS to state-to-the-art defenses MID (Wang et al. 2021c) and BiDO (Pegg al. 2022). The implementation of the defense mechanisms is based https: / //tpt//ttbbbbbb/// com/ A Pen.nngg997.. //eee fend_Mi We adjusted the implementation to support the ResNet-112 architecture. Training and attack hyperparameters Compared to the original evaluations, we tested the defense mechanisms on high-resolution data for which both, MID and BiDO, only provide a partial defense to MIAs. Smoothing the labels with small negative factor beats both approaches by keeping more of the model's 's utility while significantly decreasing the attacks' success.
identical for all models.
Table PPA results against state-the-art-defennss and negative LS as defense mechanism.
Defense Parameter
t
JECE
A.ce@5
4 fuce
LIID
Crowin
Sceat
No Defense
a 0.0
94.90%
0.0619
94.12%
1.9
90.17%
0.7060
124.30
40.89
61.19%
58.699%
Label Smoothing
0.05
91.45%
0.1474
14.34% + 7.6
30.94%
1.2020
230.02
59.38
16.45%
15.73%
FaceSerub
8 0.001
83.14%
0.0599
94.57%
1.6
99.48%
0.6746
111.15
43.74
70.58%
70.51%
0.7163
115.07
72.95%
8 0.01
90,6956
0.0722
88.46%
+ 3.0
98.01%
0.7212
119.01
44.05
69.0663
67.24%
10,05, (.5)
03.7256
0.1042
80.28% + 2.0
98.25%
0.7536
124.20
44.16
70.58%
67.40%%
BIDO-HII
10,05, 1.0)
80.1156
0,1004
85.00%
2.6 45.422%
97.32%
0.7796
130.86
45.09
67.43%
65.47%
(0J06, 2.5]
87-1456
0.1606
73.60%
0.9083
154.49
49.21
50.300
55.91%
No Defense
0.D
87.0556
0.0890
81.76%
+: 1.3
95.03%
0.7406
318.09
36:12
58.773
50.57%
Label Smoothing
D.005
83.505
0.2179
26.411%
3.4
40.96%
1,0420
441.67
61.
7.08%
3.89%
Celeba
83.95%
75.36%
01.%
8 0.005
80.4356
0.1076
78.022%6-1
00..38%
0.7353
353.55
94.30
70.9006
64.01%
8 = 0.01
77.10%
0.0907
73.941%
1:
91.52%
0.7952
351.79
97.44
70.769
61.18%
HISO-HIIC
00.05, 0.5)
79.8955
0.2034
63.68 03..8%
+ 1.5
69.10%
83.92%
0.8135
382.40
90.71
60.647%
56.4350
52.26%
(0,05, 2.5]
42256
0.2911
8.13%
0:9
21.83%
1.1340
423.00
.309%
30.93%
Real
a= 0.0
a
--.00
MID
BIDO
Figure 9: Attack samples from ResNet-152 models trained on FaceScrub. Samples are not cherry- picked but show the most robust attack results based on PPA's selection procedure. Results for MIL) and BiDO are taken from the models trained with À = 0.005 and À = (0.05, 0.5), respectively.
20

---

Published as a conference paper at ICLR 2024
Real
a=0.0
d=0.1
MID
BIDO
Figure 10: Attack samples from ResNet-152 models trained on CelebA. Samples are not cherry- picked but show the most robust attack results based PPA's selection procedure. Results for MID and BiDO are taken from the models trained with À = 0.005 and À = (0.05, 0.5), respectively.
21

---

Published as a conference paper at ICLR 2024
C.3 VARYING NUMBER OF TRAINING SAMPLES
We compared the impact of training target models with LS for a varying number of training samples available. More specifically, we sampled fixed number of samples for each class of the training data and trained the models on these smaller datasets. Whereas the impact of positive LS on model's s privacy leakage is larger for settings with fewer training samples available, negative LS improves the defense in settings with more data. Tab. states the results for attacks against the various models. The results correspond to Fig. 2a from the main paper.
Table 4: PPA results against models trained on varying number of samples per class.
+Samples
1 Test
ECE
Acce1
t A
face
LFID
Geoacn
Exeat
0.1
81.52%
(+15.44)
0.5367
60.385%
(+17.81)
83.81%
0.8532 (40.8)
145.06
41.90
45.78%
42.37%
10
0.0
06 0856
0.2108
42.5756
71.34%
0.9337
158.8
30.88
37.64%
34. 9335
(-1.78)
0.1265
32.91% (-9.06)
60.58%
0.9590 (+0.05)
106.17
13.30
37.89%
36.22%
0.1 16% (+9.08)
0.1392
57. 26%
(+14, 13)
95.69%
0.7155 (-0.11)
126.03
12.50
39.90 39.92%
56.89%
20
0.0
84.15%
0.1079
82.83%
92.24%
0.8201
142.74
42.85
55.55%
52 0300
FaceSerub
0.05
82.47%
(-1 -1.66)
0.2483
(-31.771)
0.9818
170.33
50.59
48.000%
45.78%
0.1
96.915
(+7.13)
0.3773
6906 (48.5)
97.41%
6636
(-0:11)
116.99
42.76
65, 60%6
70%
30
0.0
88, 78%
0.1022
81.100
96.87%
0.7708
135
13.55
58, 929
56. 02.50%
-0.05
87.67%
(-1.11)
0.1850
24.53% (-59.57)
51.15%
1,0800 (+0.31)
158.27
55.59
$3.84%
32.00 32.025
(+4.78)
0.3247
94.40%
(+4.27)
97.67%
0.60144
(10.11)
111.84
45.22
67.72%
42%
40
0.0
92. 2256
0.0750
9106
98.64%
0.7429
131.69
41.79
50.96676
56.635
-0.05
89.025
(33.23)
0.1622
12.87% (-77,26)
31.35%
1950 (+0. 45)
217.91
59.89
22, 35%
96%
7056 (+4.25)
0.3207
94.54%
(+5.31)
(20.12)
114.65
63.2356
50
0.0
92.45%
0.0850
99.2355
98.1
0.7383
131.68
41.75
30.1976
56.36
0.05
88 2856 (-4.17)
0,1808
18.945 (-70,29)
42.81%
1.1340 (+0.37)
190.76
58.99
32.25%
31. 0255
98%
(+20.07)
0.5768
40%
(+21,06)
0.8335
(-0.10)
313.67
47.65%
88%
10
0.0
58.830 80%
0.2346
38.345%
63.42%
0.9348
377.917
33.705
39.9076
32. 19%
-0.05
57.0656
(-1.80
0.4630
33.1 (-3.21)
38.75%%
0.91440 (40.004)
396.79
38.115
38. 1256
31. 3254
CelebA
103 2253
(+111.09)
0.5141
(+177.17)
56.06%
65092 (-0-13)
259.63
62/21%
66. 1956
20
0.0
22%
0.1126
71. 62662556
90.29%
0.7912
333.11
31.31
56.06%
17 57%
-.05
80.36% (-1.86)
0,2600
35.57%
(-36.05)
61.11%
0. 0.9868
(+0. 20)
416.94
54.02
34.16%
30.899%
0.1
92.215% 1.32)
0.5180
91. 91.611%
(1.1%. (4))
96.68%
0.6 0.6166 (-0.16) 16)
269.37
37.46
60.52%
54. 54%39%
30
80.8995
0.1353
69156
9X2.47%6
0.7767
326.20
34.01
57. 37.29556
8016
-0.05
78. 235223506 (-2.66)
0.2158
17. 69.69
(-59.00 (0)
36.76%
1,1200 (+1.331)
472.90
61.22
22.835
20.71%
22

---

Published as a conference paper at ICLR 2024
C.4 ANALYSIS ATTACK STAGES
Here, we provide additional numerical results for our attack stage analysis in Sec. 4.4 More specifically, we investigated three ResNet-152 models trained FaceScrub with hard labels, positive or negative LS. Tab. 5 states the results for PPA performed on the FaceScrub ResNet- 152 model trained without LS but with initial latent vectors selected by different models. Tab. further states attack results for which a fixed set of random latent vectors has been optimized with the different target models. Finally, Tab. 7 contains results for which the three models selected a subset of the attack results computed on the model trained with hard labels or negative L.S.
Table 5: Evaluation metrics for PPA performed with the same 15s which has been trained on FaceScrub without label smoothing. The only difference between runs the initial latent vectors used during the attack optimization. For each target class, 50 latent vectors have been selected from total 10,000 candidates. The samples were selected by three different ResNet-152 once trained without label smoothing, once with positive smoothing (a = 0.1), and once with negative smoothing (a = -0.05). No final selection of the results was performed.
Smoothing of Sampling Model
Acc@1
Acc@5
Seeal
JFID
Etrain
t Exeut
0.1
86. 86.09% (+2.75)
90.58%
(-7.63--0003)
0.7764
127.92
42.54
79.98%
75.79%
0.0
83.34%
95.12%
0.8057
127.51
41.99
78.55%
73.92%
0.05
70.20% (-13.14)
85.48%
0.8808 (+0.08)
142.23
44.47
77.70%
72.89%
Table 6: Evaluation metrics for PPA performed with three different ResNet-152, trained on FaceScrub with and without label smoothing. For each target class, 50 latent vectors have been selected from total of 10,000 candidates by the ResNet- 152 trained with hard labels. The runs differ only in the target model used to optimize the latent vectors. No final selection of the results was performed.
Smoothing a Optimization Model
f Acc@1
Acc
6race
4 6evot
(FII)
t Ecrain
1 Eteat
85.37% (+2.03)
93.
0.7476 (-0.66)
83.48%
79.38%
0.0
83.31%
95.42%
0,8057
127.51
11.99
78.55%
73.92%
-0.05
17.14% (-06.20)
38. 76%
1.1640 (+0.34)
205.77
48.82
36.63%
35.45%
Table 7: Evaluation metrics for PPA performed on ResNet-152 models trained FaceScrub with hard labels or negative LS, respectively. Each attack produces 200 optimized vectors for each target class. The attack's final result selection stage was then performed on models trained with different smoothing factors. All metrics were then computed on those selected subsets, each consisting of 50 samples per class.
Optimization Model
Smoothing of Selection Model
Acc88
Joee
JFID
Ereoin
0.1
06.50%% (+1.985
99. 73%
0.6638 (-0.01)
118.03
40.10
08 585
61%
0.0
0.0
94.57%
99.39%
0.7051
121.50
40.77
67.74%
64.
0.05
94.92% (+0.33)
99.45%
0.6880 (-0.02)
120.81
40.79
67.74%
64.60%
17.98% (v0.233
41.47%
(-0.01)
54.33
13:24%
0.05
0.0
17. 76%
40. 42%
1.1290
210.11
53.19
21 92.9256
20.93%
-0.05
14.36% (-3.39)
30.95%
1.2520 (-0.10)
299.36
59.38
15.58%
1
23

---

Published as a conference paper at ICLR 2024
C.5 RESULTS ON LOw- RESOLUTION MODEL INVERSION ATTACKS
In addition to conducting experiments on high-resolution data, we also explored the impact of LS on low-resolution MIAs. More specifically, we conducted Generative MIA (GMI) ZZang et al. 2020), Knowledge Enriched MIA (KED) (Chen et al. 2021), Logit MAximization & Model Augmentation (LOMMA) (Nguyen et al. 2023). Pseudo Label-
MIA (PLG-MI)(Yua)) et al.) 2023), Reinforcement Learning Based MIA (RLB-MI) (Han et al. 2023), and Boundary Repulsion MIA (BREP-MI) (Kahla et al., 2022). Our experiments followed the standard evaluation protocol of the papers using 64 x 64 CelebA images from the 1,000 identities with the highest sample count and training VGG-16 target models. Both the training and attack phases were done using the official attack implementations. We only adjusted the target model training by adding the smoothing factors and setting the number of training epochs to 100. Additionally, for training with negative LS, we applied the same smoothing scheduler as employed in the high-resolution experiments. All attack hyperparameters remained at their default settings. For GMI, KED, LOMMA, and BREP-MI, we carried out attacks all 1, 1,000 identities. For PI.G-MI, identities. However, for RIB-MII, due to the extensive time requirements of the attack, we adopted the evaluation procedure outlined in the original paper and targeted only 100 randomly selected identities.
followed the paper and attacked 300
We compare the effect of LS to training with the MID (Wang al., 2021c) and BiDO (HSIC) (Peng et al. 2022) defenses. Following the original papers, we trained two models for each defense. For MID, we used 3 0.003 and / 0.01 defense parameters, and for BiDO (0.05, 0.5) and A (0.05, 1.0), respectively.
Tab. [ states the evaluation results of the different attacks. Here, ōKNN denotes the k- k-nearest neighbor distance, which states the shortest distance from the attack results to the training data from the target class. The distance is measured as the & distance in the evaluation model's feature space. As evaluation model acts pre-trained FaceNet model. We refer to Zhang et al. (2220) for more details on this metric. We emphasize that RLB-MI and BREP-MI only compute the attack accuracy, which is why the FID score and 8KNN are not stated for these attacks. Both attacks also provide no standard deviations.
Our findings consistently revealed that for most of the attacks, the positive LS amplifies privacy leakage, while negative LS mitigates this effect. In the case of the label-only BREP-MI, LS appeared to have negligible impact on the attack results. We hypothesize that this phenomenon is attributable to the optimization strategy employed in BREP-MI, which solely relies on distance estimations to decision boundaries. LS, in contrast, influences the information content within the target model's logits and prediction scores. This explains why MIAs that rely on these components for optimization are more noticeably influenced by the choice of smoothing procedure. The introduction of boundary- based guidance into the optimization strategies of gradient- based and black-box -box attacks represents an intriguing avenue for further research, holding the potential to enhance the effectiveness of existing attacks. Also, negative LS has no noticeable impact on the success of PLG-MI, where the attack results are comparable to the model trained with hard labels. However, when qualitatively analyzing the attack samples in Fig. II it becomes clear that the attack against the negative LS model mainly produces low low-quality samples without much variance between the samples, i.e., the attack, in some sense, collapsed into adversarial sample.
Comparing the defensive effects of negative LS with MID and BiDO, we can conclude that negative L.S provides better defense against most attack algorithms while keeping similar model utility. For stronger MID and BiDO defense parameters, substantially more model utility is lost to enable a better defense. Overall, and BiDO with the tendency to outperform it in the low-resolution setting.
conclude that negative LS at least offers the same utility-privacy as MID
24

---

Published as a conference paper at ICLR 2024
Table 8: Results of MIAs against VGG-16 models trained on CelebA with 64 x 64 resolution.
Attack
Type
Training
Parameter
Test Acc
Accc@1
A
FID
Baseline
85.74%
16 00.00%
1: 3.75
36 60% 1.37
1043.
52,90
(+2.36)
25.40 25.40%
5.06
1067.55
GMI Zhing ei X
White-Box
LS
MID
# 0.003
-0.05
77.56% (-8
80.02% (-5.72)
14.60%
5.9255
1: 2,80 (-1.40)
2.31 (-10.08)
30.00%
19.80%
2: 4.15
3.91
1079.17
1078.57
56.52
70.87
MID
0.01
07.45%
(-18.29)
12.58%
3.79 (:3.44)
31.00%
2 3.69
1085.05
50.41
79.06%
7.00%
(-8.00)
1142.11
HSIC
A
(0.05, 1.0)
70.18% (15.56)
2.76% 1: 1.36 (-13:210)
8.60%
2:21
1227.64
81.60
Baseline
0.0
85.74%
43.64% + 3.67
71.00%
=: 3.41
897.54
42.59
LS
0.1
10% (+2.36)
68 8898
3.25 (+25.24)
86, 86.20%
2.43
791.11
24.10
(-5.72)
3,06
54
Coon et 2021
White-Boo
MID
8
0
77.56%
18)
60 40%
+ 2.40 (66676)
87. 807%
2.32
797.99
27.58
MID
0.01
07.45%
(-18.22 29)
44%
4: 2.21 (+6.50)
79. 20%
2: 1.80
821.01
27.02
HSIC
À
(0.05, 0.5)
79.06% (-6.68)
12.72%
4.25 (-0.92)
71 60%
805.94
29.96
(0.05, L.)
N.13 (-14.64)
58,
902.78
Baseline
a
0.0
85.74%
53.645%
4.64
79.607 3.55
878.36
42.28
LS
0.1
88. (+2.36)
50 96%
1: 3.52 (-2.68)
71 71.00% 2: 4.68
958.44
47.44
LOMMA (GMD)
18
(-5.72)
(-14.48)
68 68.0076
30.24
Nguyen all 2023
MID
0.01
67.45%
(-18.22 29)
17.18%
4. 3.71 (:38. 45)
9.50
36.807 6.47
1004.63
50.07
HSIC
(0.05. 0.5)
79.0656
(-0.68)
47 8456
+ 4.02 (-5,80)
74.40% 4.51
882.00
41.76
HSIC
(0.05. L.)
70.18%
(-15.58)
30 84.8
(-22.80)
54.20%
963.49
44.36
22.9685 1,20
90.00%
18
0.1
88 10% (42.36)
76 5256
+ 1:331 (-3.56)
92. 40%
0.05
780.76 76
$3.01
Nauyen
LOMMA (KKD)
White Box
1S
MID
0.003
0.05
77.567 (.8.18)
80.0236 (-5.72)
67.45%
(-18 200
(3 60%
5875
6956
+ 1.37 (-9.30)
1.11 (-9.40)
1.40 (-22 28)
76. 60P6 5.55%
90.00%
86.00% 0.70
831.31
792.74
784.43
20.00
40.97
36.87
HSIC
(0.05. 0.5)
79.06%
(-6.68)
65. 68%
+ 1.41 (-7.28)
86.007 0.67
$10.61
35.95
HSIC
(0.05, 1.0)
70.1856
(-16.56) 543
47 64%
1,228 (-25.30)
75.80% 0.99
860.08
38.77
Bascine
0.0
745
1358.
88 (+2.335)
4.44 (-0.00)
92
13299.0
1.
0.05
80.02% (-6.72)
72 000%
+ 2.50 1.000
88.007
2.00
1544.82
78.98
PLG-MI Yunn 2023
White-Box
MID
0.003
TT. 77.5656 (-8
72 00%
1 6.08 (-1.00)
80 00% 5.00
1378.36
20.74
MII
2.45 (-12.00)
HSIC
(0.05, 0.5)
19.06% 68)
70 0000
1.0 (-1.00)
85 00% 3.10
25.37
HSIC
(0.05, 1.0)
70.18% (-16 566
42 + 4,80 (-20.00)
62.00% 6 1.
1586. 335
30.52
Baseling
0.O
85.745 7456
$2 00%
75.00%
10% (42.36)
84.00%
RLB-MI Han erall 20SS
Black-Boo Box
LS
MID
0.003
-0.05
77.560 5656 (-8 189
80.02% (-5.72)
27 0006 (-25 (1))
19 000 (-33.00)
41 100%
48 000
MID
0.01
67.45% (-16
0005 (-32
42 0005
HSIC
(0.05, 1.0)
70.185 34)
25 00%
48
Biseline
0.0
85. 74%
19 001%
730 67%
LS
0.1
88 10% (+2.36)
56 33%
77. 0055
70%
BRRP-MI
Label Only
MID
B 0
77.56% 188
63.6 (+4.44)
78.0555
MID
# 0.01
07. 45% (-18.22 (-1
43.17% (-5 85)
68. 25
HSIC
A (0.05, 0.5)
79.0656 (-6.68)
60% (-6, (00)
60. 60.9096
1.0)
(-15.56)
(-21.9)
50.30%
25

---

Published as a conference paper at ICLR 2024
Real
Baseliie
- 0.05
8- 0.003
MID
BIDO = (0.05.0.5) 0.51
Figure 11: Visualization of attack results for PLG-MI performed against models trained with hard labels (Baseline), negative LS, MID, and BiDO as defenses. Whereas the attack accuracy for the models trained with hard labels and negative LS indicate comparable attack success, the attack results for the negative LS models have substantially reduced image quality and, consequently, reveal fewer private features.
26

---

Published as a conference paper at ICLR 2024
C.6 INVESTIGATING THE IMPACT OF PREDICTION ACCURACY
Classifiers trained with positive LS often improve their prediction accuracy on unseen test data compared models trained with standard cross-entropy loss. Analogously, negative LS training can reduce a model's prediction accuracy. To investigate if the defensive effect of negative LS on MIAs arises from a reduced model performance, we re-trained ResNet-101 models targeting the same test accuracy as the negative LS models achieved. More specifically, we trained models without LS (a = 0.0) and with positive LS (a 0.1) and stopped the training as soon as the models reached the same test accuracy as the model trained with negative LS (a -0.05). Since we measured the accuracy after each training epoch, the resulting test accuracy is slightly higher for the models trained with 2 0.0 but still in the same range the negative LS models. Importantly, the models trained with positive LS already achieved the target test accuracy after only 4 (FaceScrub) and 7 (CelebA) training epochs, respectively. In particular, the FaceScrub model has seen substantially fewer samples than the other models. It explains the attack results being slightly below those the model trained without any L.S applied, which was trained for 19 epochs.
Still, the results clearly indicate that training with negative LS leads to markedly lower attack metrics in all cases, even if the models' test accuracy is comparable. We, therefore, conclude that model's slightly reduced prediction accuracy is not the main reason for the defensive effect of negative LS. Still, is also clear that models with poor generalization and understanding of class characteristics can only leak less information about individual classes. Consequently, when attacking the models trained with early stopping, the attack metrics degrade compared the models trained for all epochs. However, the gap between those models and the negative L.S models is still substantially large, indicating the strong defense effect of negative L.S.
Currently, results are reported for 50 gender-balanced balanced classes in each setting. The results for targeting all classes will be added as soon as they become available. Since the Knowledge Extraction Score is computed over all classes in a dataset, we will add those values later to avoid misleading results.
Table 9: PPA results against models trained with early stopping to achieve similar prediction accuracy
results on the test data. Even for matching test accuracy, the attacks achieve much higher success on models trained with standard coos-yy loss or with positive LS.
Defense
f Test Acc
Acc@1
f Acce5
J.vol
JFIDD
1 Etroin
Etest
No Label Smoothing
0.D
91.74%
89.68%
99.00%
0.7382
133.12
50.43
24. 60%
23.68%
Positive Label Smoothing
0.1
13%
87.16%
97.20%
0.7766
135.73
49.56
27.19%
Negative Label Smoothing
-0.05
91.45%
14.34%
30.94%
1.2320
239.02
59.38
16.45%
15.73%
No Label Smoothing
0.0
02.02%
69.36%
90.60%
0.7899
344.82
49.60
22. 9..00%
20.89%
Positive I Label Smoothing
0.1
88.78%
76.64%
92.84%
0.785
332.53
50.05
20.43%
18.35%
Negative Label Smoothing
-0.0
83.59%
26.41%
49.96%
1.0420
441.67
61.30
7.08%
5.89%
27

---

Published as a conference paper at ICLR 2024
C.7 CHALLENGING THE DEFENSIVE EFFECT OF NEGATIVE LABEL SMOOTHING
We further investigated the defensive effect of negative LS on MIAs, particularly PPA in the high-resolution setting, is due the design of the attack algorithm and if attack adjustments able to break the defense. For this, we investigated the following settings:
We increased the number of initially sampled latent vectors to 10, 000.
We repeated the attack with a lower learning rate of 0.0005.
We repeated the attack with a higher learning of 0.05.
We replaced the Poincaré loss with standard cross-entropy loss, following (Zhang et al. 2020).
We added prior loss based on the GAN's discriminator weighted by 0.01, following Zhang et al. (2020).
We replaced the Poincaré loss with a logit identity loss including a regularization term
Preg (pppnn following the implementation of Nguyen al. (2023).
In all settings, only changed single aspect of the attack and kept all other parameters fixed. All experiments were conducted on the same ResNet-101 101 models trained with negative LS (a = -0.05).
Table 10: Attack results for various modification of PPA's optimization method.
Variant
f Acc@1 1
f Acc@5
face
4 deval
Etrain
t Steat
Default
14.34%
30.94%
1.2320
239.02
59.38
16.45%
15.73%
Increased Latent Vector Sampling
19.24%
39.64%
1.178
225.39
63.87
16.92%
16.67%
FaceScrub
Lower Learning Rate
2.72%
9.40%
1.381
222.13
75.94
10.17%
8.77%
Higher Learning Rate
19.44%
36.60%
1.258
1031.79
86.44
17.11%
15.79%
Cross-Entropy Loss (Zhang et 2020
23.56%
55.92%
1.01
156.79
62.39
25.72%
25.15%
Prior Loss (Zhung 2020
12.04%
29.60%
1.224
218.58
64,83
15.339%
12.57%
Identity Loss Ngguyen el al. 2023
22.84%
40.56%
1.197
186.52
84,48
13.89%
12.87%
28

---

Published as a conference paper at ICLR 2024
C.8 IMPACT OF LABEL SMOOTHING TRAINING ON ADVERSARIAL ROBUSTNESS
In this section, we investigate if training with LS has an impact on a model's robustness to adversarial examples. Adversarial examples slightly perturbed inputs that seem benign to the human eye but can change a model's 's prediction substantially. In recent years, various attack algorithms and settings have been proposed (Szegedy al., 2014: Goodfellow et al., 2015: Struppek et al. 2022b). For evaluation, we apply the following attack algorithms for crafting adversarial examples:
Fast Gradient Sign Method (FGSM) (Goodfellow et al.
2015 Single-step. white-box
attack. Hyperparameters: 8/255.
Projected Gradient Descent (PGD) (Madry et al. 2018 Mtlti-step, white-box attack.
Hyperparameters: e = 8/255, step size 2/255, steps = 10, random start = True.
Basic Iterative Method (BIM) (Kurakin al. 2017): Multi-step, white-box attack. Hyper-
parameters: 8/255, step size 2/255, steps = 10.
One-Pixel-Attace (Su et al. 2019): Multi-step, black-box attack. Hyperparameters:
pixels I, steps 10. population size 10.
All attacks were performed on the unseen test data. We repeated each attack in a targeted and untargeted setting. In the untargeted case, an attack is successful if the prediction of the true label is prevented under adversarial perturbations. For the more challenging targeted case, randomly selected a target label for each input. An attack is successful if the added perturbations are able to force the model to predict the target label. For both settings, we compute the attacks' success rate, iee, the lower the success rate, the more robust a model is to adversarial perturbations.
The results in Tab. 11 demonstratee that training a model with positive L. can make model more robust to adversarial examples. For instance, the success rate of FGSM decreases on almost all models compared training without any LS. However, training with negative LS can have an even higher impact on a model's robustness to adversarial perturbations. The attack success rates for untargeted attacks are substantially lower in almost every case compared models trained without LS or positive LS. Training model with negative LS, therefore, not only makes MIAs harder to perform but also makes models more robust to adversarial examples.
Table 11: Adversarial robustness of models against various targeted and untargeted adversarial attacks. We computed the attack success rates in all cases. The lower the success rate, the more robust model is to the individual attack algorithm.
Architecture
Clean Test Ace
Untargeted
Attacks 1 BBM
Targeted
FGSM
1 LPGM
One-Pisel
1. FFGSM
JPGGM
BBM
OOne-Pixel
0.1
97.39%
70.05%
100.00%
100.00%
3.93%
7.26%
92.42%%
93.51%
0.00%
Res Net-111
94.03%
96.00%
100-00%
100.00%
7.63%
56-6256
99.96%
99.97%
D.03%%
FaceScrub
0.05
91.45%
13.38%
13.94%
13.44%
10.43%
22.005
98.52%
98.80%
0.08%
0.1
97.54%
50.21%
100.00%
100.00%
3.54%
6.84%
83.58%
85.56%
B.00%%
ResNeXt.50
0.0
95.25%
97.30%
100.00%
100.00%%
7.70%
52.46%
100.00%
90.977
8.05%
92.40%
11.388
11.11%
24.00%
0.08%6
0.1
13%
755
100.00%
000%
8.83%
9.16%
93.11%
94.30%%
B000%
DenseNet-121
0.0
95.72%
97. 07.0
100.00%
100.00%
5.44%
30.7356
100.00%
100.00%6
9.005
0.05
13%
31.84%
32, 22.74%
32.15%
9.61%
18.72%
86.17%
88.5276
0.08%
0.1
95.11%
44%%
100.00%
100.007%
6.49%
17.64%
99.53%
99.901%
0.00%
ResNott112
8.0
87.03%
08. 90%
100.00%
000%
20.67%
31. 10%
100,00%
99.97%
BO00%
0.05
$3.50%
1396
30.16%
28. 16%
20.0776
8.165
92.08%
93.815
BU0055
Celeb.A.
0.1
95.27%
94.81%
100.00%
100.00%
6.29%
11 11.68%
98.84%
99.177
BJ0056
ResNeXt-50
0.0
87.8.%%
96.84%%
100.00%
100.000
19,88%
28.36%
100.00%
99:9017
0.03%
B.06
84.70%
23.201%
24.40%
23.34%
20.07%
9.02%
95.27%
96.74%
B
0.1
82.8%%
98 677%
100.00%
100.00%
144022%
98.04%
99. 10%
0.00%
DenseNet-121
0.0
96.0559 0.5%
98.57%
100.00%
100.00%
15.78%
20.0156
99. 99.900%
99.00376
D.03%%
0.05
96.48%
67.48%
68.544
68.18%
15.057%
8.965
67.54%
09.415%
0.00%%
29

---

Published as a conference paper at ICLR 2024
C.9 IMPACT OF LABEL SMOOTHING TRAINING ON BACKDOOR ATTACKS
In addition to adversarial robustness, we also investigate if training with LS has an impact on backdoor attacks. Backdoor attacks aim to integrate a secret behavior into model that only activated for inputs containing a pre-defined trigger (Gu et al. 2017 [Struppek et al., 2023b). Due to the relatively small number of samples per class in FaceSerub and CelebA, we were not able stably train the models independently of the smoothing factor used. Therefore, we trained ResNet-152 models on poisoned ImageNette (Howard, (019) datasets, which a subset of ImageNet (Deng al, 2009) classes. Specifically, we investigate the following common attack methods:
BadNets (Gu et al. 2017): We added a 9 x 9 checkerboard pattern to the lower right corner of each image. In total, 10% of all images were poisoned and labeled as class
Blended (Chen et al. 2017): We interpolated each poisoned image with fixed Gaussian noise pattern. The blend ratio was set to 0.1. In total, 10% all images were poisoned and labeled class O.
For evaluation, we computed the model's clean prediction accuracy the test splits. The attack success is then measured by adding the triggers to test images (excluding samples from the target class) and computed the share of poisoned samples that were classified as the target class O. The lower the attack success ratc, the more robust the model is to an attack.
Table 12: Robustness of models against various backdoor attacks. We computed the attack success rates in all cases. The lower the success rate, the more robust a model is to the individual backdoor algorithm.
Trigger
t Clean Accuracy
Attack Success Rate
Clean
81.71%
0.1
BadNets
98.64% 80.41%
80.59%
ImageNette
Blended
98.25%
Clean 82.37%
0.0
BadNets
81.17%
100.0%
Blended
80.31%
95.48%
Clean 78.17%
-0.05
BadNets
14.62%
100.0%
Blended
72.76%
48.05%
30

---

Published as a conference paper at ICLR 2024
Examples of Target Person
Option 1
Option 2
Option 3
Figure 12:
Labeling example from the user study, The annotators have choose between the three options (MID, BiDO and negative LS) that look the least similar to the examples of the target person.
Table 13: PPA attack results against ResNet-152 models trained FaceScrub and CelebA. The User Preference states the relative share of attack samples that annotators find to look the least similar to the target class.
FaceScrub
CelebA
Model 1 Test Ace Acce1
Suein
User Preference
Test Acc
7'Axe@11
4.6
1
User Prefereace
Neg 18
91.59
14.3% (-50.0)
1.23 (20.5))
16.5% (-46.7) 47.76%
83.6%
26 4% (-55.3)
1.04 (+0.3)
7.1% (-52.7
7L555%
MID
91.1%
82.0% (-2.3)
(.7201
73.0% (+11.8) 17.167
80.4%
78 07% (-3.8)
0.74 (40.0)
70.95 (+11.1)
10,2255
BiDO
87.19
45.4% (-18.9
0.91 (+0.)
39.3% (-1.) 35.07%
70.0%
63.7% (-1.1)
(+0.07)
(00.8
18.25%
C.10 USER-STUDY
In addition to the quantitative evaluation metrics used to evaluate the attacks' success, conducted a qualitative user study to compare the effects of different defense methods on the visual similarity between attack results and the true identity. We compared the attack results the ResNet-152 FaceScrub and CelebA models trained with negative LS (a = -0.05) to the attack results against models trained with MID and BiDO, respectively. The models the study was conducted using the manual labeling service of thehive.ai More specifically, we randomly sampled 1550 target classes from the FaceScrub and CelebA datasets, respectively. We then took for each target model and target class the attack results that achieved the highest robustness on the target model using PPA's final selection algorithm. For each class, the annotators were presented with randomly selected images of the true identity and the 3 attack results from the attack against negative L.S, MID, and BiDO models. The annotators were then asked to "select the person (options 1-3) that looks the least similar to the person depicted in the example images". Noticeably, the annotators were not provided any information on the different defense mechanisms or even the attack setting at all to avoid undesired side effects during the labeling process. Fig. 12 shows example from the user study. Each sample was labeled by three different annotators and the final decision was made on a majority vote. For 16 FaceScrub samples and 13 CelebA samples, the annotators could not decide on a single favorite. We excluded those samples from the final preference computation. In total, 37 annotators took part in the labeling process. To filter out unreliable annotators, some honeypot tasks were added to ensure the annotation quality.
as stated in Tab. The
Tab. 13 states the user study results the relative preference for the individual defense mechanisms. For the FaceScrub model, the annotators found about 48% of the attack results from attacks against the negative LS model look the least similar the target class. For the CelebA experiment, even 71.5% of the results were preferred. Overall, this means that the defensive effect of negative LS was qualitatively higher valued than MID and BiDO as defense mechanisms, particularly on the CelebA models. At the same time, negative LS also maintained the model's 's utility in terms of test accuracy better than the other defense mechanisms.
31


Reviews:
[{"review_id": "soowZUKynq", "reviewer": "Reviewer_22v2", "summary": "The paper shows that model trained with label smoothing can be more vulnerable to model inversion attacks while negative label smoothing can be a defense for the attacks.", "questions": "In Fig 6b, why is the gradient similarity for label smoothing lower & with higher variance than that of hard label? I must admit that I don’t have a good intuition here but I was kind of expecting the lines of smoothing and negative smoothing to stay on two sides of the hard label’s line.\n\n(And those mentioned above.)", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The paper looks at an interesting and important problem of how model training can affect model’s vulnerability to MIAs.  \nThe experimental results in general justifies the statements well.\nThe presentation is clear.", "weaknesses": "I’m still wondering how much we can conclude the relation between vulnerability of MIAs and training methods (normal / smoothing / negative smoothing) from the empirical results shown here. For example,\n\n- different training methods lead to different model test accuracy (Table 1) especially for CelebA. I think it’s fairer to compare the attack accuracy under the same model accuracy (e.g. early stop the label-smoothed model at a lower accuracy iteration), as it might be normal for a higher accuracy model leak more. It might also be interesting to look at training accuracy of the models to understand how well they generalize.\n\n- we’re looking at one particular attack algorithm here. I think it’s natural to ask whether another algorithm, or maybe an adjusted version this this algorithm can achieve a different result. For example, if the attacker knows that a model is trained with negative label smoothing (and thus has a different calibration than a normally-trained model), can they possibly sample more initial latent embeddings in the first stage, or adjust their objective function to incorporate with the calibration of this model in the second stage?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ge0NsER78l", "reviewer": "Reviewer_dNRe", "summary": "This work considers the problem of defending against Model Inversion Attacks in white-box settings. The major contributions of this work are:\n\n1) In the context of model inversion attacks, this paper observes that positive label smoothing increases a model’s privacy leakage and negative label smoothing counteracts such effects.\n\n2) Consequently, negative label smoothing is proposed as a proactive measure to defend against Model Inversion Attacks.", "questions": "Please see Weaknesses section above for a list of all questions.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) This paper is written well and it is easy to follow.\n\n2) To my knowledge, this is the first work to explore label smoothing in the context of defense against model inversion attacks.", "weaknesses": "At a high level, Model Inversion attack procedures can be decomposed into 2 stages: 1) Classifier training, and 2) Model Inversion attack on the classifier. My review below addresses weaknesses of this work corresponding to each stage.\n\n**Classifier training:**\n\n1) Section 3. Analysis is only remotely related to contemporary Model Inversion setups. In particular, it is unclear as to how observation from Section 3 lays groundwork to the remainder of the paper.\n\n- What is the optimization objective for both classifier training and model inversion attack for Figure 1? A clearly formulated problem definition with relevant equations is critical to understand this part.\\\n\n- How would Figure 1 change if the number of iterations = 5K for all setups. \n\n2) Section 4.3. A large number of observations from Sec 4.3 for Standard and Positive Label Smoothing has already been thoroughly investigated in prior works. \n\n- Muller et al. [A] and Chandrasegaran et al. [B] have already shown that positive label smoothing erases some relative information in the logits resulting in better class-wise separation of penultimate layer representations under positive LS compared standard training.\n\n- Figure 4, column 3 is unclear. What are the Training and Test accuracies of the classifiers used in Figure 4?  Recall that if the classifier trained with negative label smoothing is good, penultimate layer representations should be linearly separable. Therefore, does column 3 correspond to a poor classifier trained with negative label smoothing?\n\n3) Can the authors explain why standard training is required in the first few iterations before “gradually” increasing the negative label smoothing?\n\n\n\n**Model Inversion Attacks:**\n\n1) Limited empirical study. This work only studies one attack method for evaluating defense (No comparison against MID and BiDO defense setups even in Table 8 in Supp.). I agree that PPA works well in high-resolution setups, but SOTA attacks in well established test beds are required to understand the efficacy of the proposed defense.\n\n- It is important to include GMI, KEDMI, VMI [C], LOMMA [4] and PLG-MI [5] attacks to study the efficacy of the proposed defense (against other SOTA defense methods). Currently it is not possible to compare the proposed method with results reported in the MID and BiDO defense papers.\n\n2) There is no evidence (both qualitative and quantitative) to establish that unstable gradient directions during Model Inversion attack is due to negative label smoothing. Is it possible that such shortcomings could be due to the PPA attack optimization objectives? Addressing 1) above can answer this question to some extent. \n\n3) User studies are necessary to show the defense/ leakage of privacy shown by the inversion results. Since this work focuses on private data reconstruction, it is important to conduct user study to understand the improvements (See [F]).\n\n4) Significant compromise in model utility when using negative label smoothing questioning the findings/ applicability of this approach. Table 1 and Table 8 results suggest that Neg. LS reduces the Model Accuracy (model utility) by huge amounts, i.e.: A 3.5% reduction in Test Accuracy for CelebA (Table 1) compared to Standard training could be serious. Recall that lower model accuracy leads to lower MI attack results. I agree that generally some compromise in model utility might be required for defense, but large reduction in model utility makes this approach questionable, i.e., In practice, no one would deploy/ attack a weaker model.\n\n5) Error bars/ Standard deviation for experiments are missing.\n\n6) Missing related works [C, D, E].\n\n\nOverall I enjoyed reading this paper. But in my opinion, the weaknesses of this paper outweigh the strengths. But I’m willing to change my opinion based on the rebuttal. \n\n===\n\n[A] Müller, Rafael, Simon Kornblith, and Geoffrey E. Hinton. \"When does label smoothing help?.\" Advances in neural information processing systems 32 (2019).\n\n[B] Chandrasegaran, Keshigeyan, et al. \"Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing?.\" International Conference on Machine Learning. PMLR, 2022.\n\n[C] Wang, Kuan-Chieh, et al. \"Variational model inversion attacks.\" Advances in Neural Information Processing Systems 34 (2021): 9706-9719.\n\n[D] Nguyen, Ngoc-Bao, et al. \"Re-thinking Model Inversion Attacks Against Deep Neural Networks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[E] Yuan, Xiaojian, et al. \"Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network.\" AAAI 2023 (2023).\n\n[F] [MIRROR] An, Shengwei et al. MIRROR: Model Inversion for Deep Learning Network with High Fidelity. Proceedings of the 29th Network and Distributed System Security Symposium.\n\n\n=======================================================\n\nPost-rebuttal\n\nThank you authors for the extensive rebuttal. I've adjusted my rating to 6 to acknowledge your efforts, though I still have reservations about the analytical support for your claims. I'd like to offer some final thoughts for consideration:\n\n- **Clarifying the paper's focus primarily on white-box attacks could be beneficial.**  It may enhance the paper to discuss how the findings, particularly regarding negative label smoothing's impact on optimization, apply to black-box and label-only model inversion attacks that employ gradient-free approaches (Table 8). \n\n- Additionally, it would be great to mention a short description reg. any anomalous results in the captions of relevant tables. \n\nI appreciate your extensive efforts and involvement during the rebuttal phase.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J62rMx5tHM", "reviewer": "Reviewer_wJ7G", "summary": "This paper studies the impact of label smoothing, a commonly used regularization techniques in deep learning, on the privacy vulnerability of models to model inversion attacks (MIAs), which aim to reconstruct the characteristic features of each class. It is shown that traditional label smoothing with positive factors may inadvertently aid MIAs, increasing the privacy leakage of a model. The paper also finds that smoothing with negative factors can counteract this trend, impeding the extraction of class-related information.", "questions": "How does label smoothing impact the model's vulnerability to other attacks (e.g., adversarial attacks, backdoor attacks, etc)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- MIAs represent a major threat to the privacy of machine learning models. This seems to be the first paper studying the relationship between label smoothing and MIAs. \n- The work provides both empirical and analytical justification for the connections between label smoothing and MIAs. \n- The ablation study is interesting to show that label smoothing impacts MIAs mainly in the optimization stage.", "weaknesses": "- The findings are not very surprising. Given that label smoothing helps the model generalize, leading to better representation of each class (e.g., more smooth decision boundaries). It is intuitive that more smooth decision boundaries allow gradient-based method to better optimize the representation of each class. \n- There is trade-off between the model's accuracy and its vulnerability to MIAs. Label smoothing with negative factors reduces the privacy risks at the cost of model accuracy (see Table 1). Intuitively, a poorly trained model is more robust to MIAs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7jO4Ojr9sd", "reviewer": "Reviewer_uLpV", "summary": "This paper studies how label smoothing (during model training) can affect the performance of the model inversion attacks.\nSpecifically, the authors find that a positive label smoothing factor would facilitate the inversion attacks, while a negative factor would suppress the attacks.\nThis phenomenon underlines the importance of delving more into factors that influence machine learning privacy leakage.", "questions": "None.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper reveals a novel phenomenon that the positivity/negativity of label smoothing factors can affect privacy leakage in model inversion attacks. Actually, the mechanism behind this can also be explained as the robustness-privacy trade-off: [r1] first find that label smoothing with positive factor can improve adversarial robustness, and [r2] first find that there is a trade-off between adversarial robustness and membership inference attacks. Nevertheless, although the relationship between label smoothing and model inversion attacks is forecastable based on the aforementioned works, this paper is the first that empirically demonstrates the relationship. As a result, I think the results of this paper are fundamental and important.\n\n2. The paper is easy to follow and provides sufficient experiments details for reproducing its results.\n\n\n**References:**\n\n[r1] Shafahi et al. \"Label smoothing and logit squeezing: A replacement for adversarial training?\" arXiv 2019.\n\n[r2] Song et al. \"Privacy risks of securing machine learning models against adversarial examples\". CCS 2019.", "weaknesses": "1. In Section 3 and Figure 1, the authors provide an intuitive explanation of why label smoothing can affect model inversion attacks based on a simple experiment on 2D data. The explanation is based on the hypothesis that a more clear decision boundary would make training data less likely to be leaked through model inversion attacks. This hypothesis is odd (at least for me) and I think the authors may need to put more effort into explaining why the hypothesis makes sense.\n\n\n2. Negative-factor label smoothing would result in the smoothed label no longer a probability simplex. I think the authors may need to justify why this type of label smoothing is appropriate.\n\n\n3. Suggestion: As explained in Section \"Strengths\", the found phenomenon can be seen as a robustness-privacy trade-off. Since this paper finds that negative-factor label smoothing can mitigate privacy leakage by model inversion attacks, I suspect it could also harm the adversarial robustness of the model. Therefore, I suggest the authors include a discussion on the potential harm to models' robustness when protecting training data privacy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HiO1OYVHot", "reviewer": "Reviewer_EAeB", "summary": "The manuscript analyzes the potential risk of increased privacy leakage associated with traditional positive Label Smoothing in the context of Model Inversion Attacks. Additionally, it provides an analysis on how utilizing negative values can counter this risk.", "questions": "1. Can an analysis be provided comparing the performance of using negative label smoothing to the performance under positive label smoothing?\n\n2. Are there any known vulnerabilities introduced by using negative label smoothing?\n\n3. How generalizable are the findings of this paper? Are the observed effects of label smoothing on privacy specific to the datasets and models used in the experiments, or can they be applied to other domains and architectures as well?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The authors have for the first time considered the potential privacy leakage risks associated with the use of positive label smoothing under model inversion attacks, and have provided insights into the privacy benefits that may arise from using negative label smoothing. Additionally, the authors have presented a framework for analyzing the exposure of privacy in model inversion attacks due to label smoothing, and have examined how label smoothing may affect the sampling and optimization phases of model inversion attacks. Furthermore, the authors have provided a geometric intuition in the EMBEDDING SPACES to explain this phenomenon, all of which are relatively novel contributions. The quality of writing in the article is excellent, the presentation is clear, the experiments are comprehensive, and the work holds significant importance.", "weaknesses": "1. The paper lacks a theoretical analysis framework. The authors claim that negative label smoothing performs better than state-of-the-art defenses, but in fact, it only performs better under model inversion attacks within the analysis framework of this paper. However, privacy protection mechanisms are often capable of resisting multiple types of attacks (or even arbitrary attacks), and it is still unknown whether the approach of negative label smoothing is effective in protecting privacy against other types of attacks.\n\n2. It is generally considered that achieving both privacy and efficiency in Euclidean space through privacy protection mechanisms is an NP-hard problem. The paper does not discuss whether the performance improvement of using negative label smoothing is significantly less than that of positive label smoothing. It is noted that when negative label smoothing was previously proposed, the range of parameter choices was from negative infinity to 1. In this case, good performance does not mean that always choosing negative label smoothing will result in good performance. The authors do not discuss in the article how the performance of using negative label smoothing compares to that of positive LS or even not using this regularization measure at all, as well as whether negative LS is not applicable in many tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)