Data-Centric Defense: Shaping Loss Landscape with Augmentations to Counter Model Inversion
Authors: Si Chen
Feiyang Kang
Nikhil Abhyankar
Ming Jin
Ruoxi Jia
Venue: Accepted by TMLR ()

Abstract: Machine Learning models have shown susceptibility to various privacy attacks, with model inversion (MI) attacks posing a significant threat. Current defense techniques are mostly \emph{model-centric}, involving modifying model training or inference. However, these approaches require model trainers' cooperation, are computationally expensive, and often result in a significant privacy-utility tradeoff. To address these limitations, we propose a novel \emph{data-centric} approach to mitigate MI attacks. Compared to traditional model-centric techniques, our approach offers the unique advantage of enabling each individual user to control their data's privacy risk, aligning with findings from a Cisco survey that only a minority actively seek privacy protection. Specifically, we introduce several privacy-focused data augmentations that modify the private data uploaded to the model trainer. These augmentations shape the resulting model's loss landscape, making it challenging for attackers to generate private target samples. Additionally, we provide theoretical analysis to explain why such augmentations can reduce the risk of model inversion. We evaluate our approach against state-of-the-art MI attacks and demonstrate its effectiveness and robustness across various model architectures and datasets. Specifically, in standard face recognition benchmarks, we reduce face reconstruction success rates to $\leq5\%$, while maintaining high utility with only a 2\% classification accuracy drop, significantly surpassing state-of-the-art model-centric defenses. This is the first study to propose a data-centric approach for mitigating model inversion attacks, showing promising potential for decentralized privacy protection.

Published in Transactions on Machine Learning Research (11/2024)
Data-Centrie Defense: Shaping Loss Landscape with Aug- mentations to Counter Model Inversion
Si Chen Virginia Tech
chensi@nt edu
Feiyang Kang
fyketout. edu
Virginia Tech
Nikhil Abhyankar
nikhilsa edu
Virginia Tech
Ming Jin
jinminggutt edu
Virginia Tech
Ruoxi Jia Vinginia Tech
ruomijio Grtt edu
Reviewed on OpenReview: https: openreview. net/ forum? id= r8wx
Abstract
Machine Learning models have shown susceptibility to various privacy attacks, with model inversion (MT) attacks posing significant. threat. Current defense techniques are mostly model-centric, involving modifying model training or inference. However, these approaches require model trainers' cooperation, are computationally expensive, and often result in a significant privacy-utility tradeoff. To address these limitations, we propose a novel data- centric approach to mitigate MI attacks. Compared to traditional model-centric techniques, our approach offers the unique advantage of enabling each individual user control their data's privacy risk, aligning with findings from Cisco survey that only minority actively seek privacy protection. Specifically, we introduce several privacy-focused data augmenta- tions that modify the private data uploaded to the model trainer. These augmentations shape the resulting model's loss landscape, making it challenging for allackers to generate private target samples. Additionally, we provide theoretical analysis to explain why such augmentations can reduce the risk of model inversion. We evaluate our approach against state-of-theeart MI attacks and demonstrate its effectiveness and robustness across various model architectures and datasets. Specifically, in standard face recognition benchmarks, reduce face reconstruction success rates to < 5%, while maintaining high utility with only a 2% classification accuracy drop, significantly surpassing state-of-the-an model-centric defenses. This the first study to propose a data-centric approach for mitigating model inversion attacks, showing promising potential for docentralized privacy protection. Our code is available at https://giii wtweehuub.co/Sccc/el/11ee git..
1 Introduction
Thanks to advances in computation and the availability of large-scale datasets collected globally, Machine Learning (ML) has experienced significant growth in recent years, showing great potential in various domains, such as computer vision, natural language processing, and healtheare, among others. However, the use of ML models trained on sensitive data can leak private information (Fredrikson et al., 2014: Shokri et al., 2017). While some data contributors may have a neutral stance lack concern about their data privacy,

---

Published in Transactions on Machine Learning Research (11/2024)
Data-Centric Defense
Model-Centric Defense
Privacy Actives
Privacy Actives
Model Training
Model Training
Others
Adversary
Others
Adversary
Server
Server
Figure 1: Pata-Centric Defense vs. Model-Centric Defense.
survey conducted by Cisco (Cisco, 2022) in 2019 identified group known as "privacy actives' who value and actively take steps protect their privacy, including switching companies or providers. Moreover, existing legislations (e.g., the General Data Protection Regulation (GDPR) in the European Union (Magdziarezyk, 2019) and the California Consumer Privacy Act in the United States (Pardau, 2018)) stipulates the right of individual users to exercise control over their own data.
Existing defenses primarily focus on protecting privacy in a model-centric manner (Abadi et al., 2016; Jia et al., 2019; Wang et al., 2021; Yang et al., 2020), which involves altering model training (Abadi et al., 2016) or inference procodures (Jia et al., 2019). Common tochniques include differentially private stochastic gradicnt descent. (DP-SGD) (Abadi et al., 2016), which involves clipping and noising the gradients during training. These approaches often result in performance degradation and increased computation time. Moreover, they require users to trust the model trainer (e.g., the data-harvesting companies) to ensure privacy, limiting user control over their privacy risks. More critically, they often present a binary stance on privacy protection, offering protection to all users or none, overlooking the nuanced needs of individual users.
Real-world surveys Cisco (2022); Review (2020); Bongiovanni et al. (2020) reveal that only a small portion of users (i.e., 32%) are privacy actives, but the binary nature of existing solutions implies significant compromise in utility for the sake of protecting the privacy of minority. This motivates our exploration into dato-centric defenses: strategies that individuals can to mitigate privacy attacks by modifying their dala before uplouding it to the central model Iruiner. This empowers individuals to control their privacy risks in decentralized manner. The randomized response (Warner, 1965), long-standing strategy in social sciences, serves as an example, although encounters challenges with high-dimensional data common in modern ML tasks.
In this paper, we focus on model inversion (MI) attacks to investigate the feasibility of effective data-centric defense. MI attacks, which reconstruct training data from a trained model, are well-roscarched and have been successful in both white-box and black-box scenarios (Fredrikson al., 2014; Zhang et al., 2020b; Chen et al., 2021; Kahla et al., 2022; Struppek et al., 2022; An et al., 2022). Compared to other common privacy attacks such as membership inference attacks (Shokri et al., 2017; Nasr et al., 2019) (which infers whether certain data is used for training) and property inference attacks (Ganju et al., 2018; Melis et al., 2019; Song & Raghunathan, 2020) (which infers whether a dataset has certain global properties), MT attacks recover much more fino-grained information such as training images, posing a signíficant threat user privacy. This work develops the first dala-centric defense for MI attacks, making the following contributions:
1 MI Defense via Privacy-Focused Augmentations. We propose privacy-focused data augmentations that can be injected by individual data contributors to mitigate their MI risks. Unlike traditional augmen- tations like cropping, rotation, and flipping that aim to improve model generalization, our augmentations are specifically tailored to thwart MI attacks. We present several ideas for designing such augmentations, with a central theme of shaping the loss landscape in ways that mislcad MI attacks to recover irrelevant samples. This central theme distinguishes our ideas from the early simple randomized response, wherein the design of the noise injected into the data does not consider its impact on model behaviors. Also, in contrast
2

---

Published in Transactions on Machine Learning Research (11/2024)
to existing MI defenses, our proposed approach, named DCD, requires no access to the victim model training data from other contributors.
2 A Novel Privacy Protection Mechanism Setup. Contrasting with the model-centric defense mech- anisms that uniformly apply privacy protections to all users, we introduce a novel setup, inspired by insights from real-world surveys Cisco (2022); Review (2020); Bongiovanni et al. (2020). The novel setup is designed to cater specifically to the varying privacy needs of different users. It uniquely enables a selective privacy framework, where only specific group of users those who prioritize privacy according to their prefer- ences and needs engage in enhanced privacy measures. This innovative setup acknowledges and addresses the different privacy concerns in the user community, marking significant shift from conventional binary protection setups that either extend privacy safeguards to all users or none.
3 Theoretical Analysis for Privacy-Focused Augmentations. We provide theoretical justification for DCD, demonstrating that: 1) the proposed augmentations reshape the loss landscape near the target and inject irrelevant samples; 2) these treatments cause existing MI attacks relying on gradient-based optimization to converge to the irrelevant samples rather than the target samples.
4 Evaluation. We evaluate DCD against varions statc-of-the-art MI attacks and demonstrate the ro- bustness of DCD across different datasets, model architectures, and attack strategies. DCD outperforms the baselines by achieving a significantly improved privacy-utility tradeoff.
2 Background and Related Work
Model Inversion Attacks. In an MI attack, an adversary aims to reconstruct representative training samples for any target class of victim model given access to the model. For example, in the context of face recognition, the adversary seeks to reconstruct face images of specific target identity. To recover training data from given model fo for any target class y, the key idea of MI is to find an input that minimizes the prediction loss of y: Tayn E arg min,
However, solving this optimization over the high-dimensional space without any constraints generates noise- like features that lack semantic information and give unsatisfactory model inversion performance. Recently, GMI (Zhang et al., 2020b) proposed to optimize over the latent space of a pre-trained GAN instead: Zayn G(2"),2* arg min, L(fo (G(=)), v) ((G())) where G and D represent the generator and the discriminator of the GAN, respectively. Chen et al. (2021); An et al. (2022); Struppek et al. (2022) follow the idea of using GAN and further improve the quality of reconstructed images with different techniques, e.g., knowledge distillation from the target model; latent space disentanglement via a StyleGAN (Karras et al., 2019; 2020a), etc. These works show that the samples synthesized by the GAN-based MI technique above can maintain high visual similarity to the original training data of so. The backbone of existing MI attacks involves solving an optimization objective, containing the prediction loss of the target class, i.e., L(fe ((=))) y), and other quality-enhanciie loss terms, via gradient descent. To recover multiple images, one could run gradient descent multiple times, each of which uses a randomly selected initialization value.
Defense Techniques. Existing defenses against MI involve altering the training process or model architec- tures. Differential privacy (DP) was deployed to defend MI in Fredrikson et al. (2014); Zhang et al. (2020b), which empirically show that DP can mitigate MI attacks only when the injected noise is large enough and as a side effect, the model suffers significant, performance degradation. Wang et al. (2021) studied the the- oretical basis of the inefficacy of DP in defending MI and introduced information bottleneck-based learning objectives to decrease the correlation between model outputs and training data. While improved over DP, it still suffers a significant privacy-utility tradeoff. Peng et al. (2022) proposed to minimize the dependency between the latent space and input while maximizing the dependency between the latent space and model outputs, enhancing utility. This, however, also requires modifications to model architectures. It's worth noting that all these defenses lack user control, relying on model trainers and imposing unnecessary utility sacrifices for privacy, especially when only a minority prioritize data privacy. In comparison, our approach involves only modification to data, which can be achieved by individual users who want to protect their privacy. Also, as we will show later, our defense effectively preserves the model's utility.
3

---

Published in Transactions on Machine Learning Research (11/2024)
Connection between Data Augmentation and Privacy. The impact of augmentations on privacy risks has been studied recently in the context of membership inference attacks (Kaya & Dumitras, 2021; Tramer et al., 2022; Chen et al., 2022). These attacks aim to determine if specific data samples were part of a model's training data. Kaya & Dumitras (2021) studied common data augmentations used for improving model generalization (e.g,, random cropping and Gaussian augmentation) and empirically identified which ones mitigate or amplify membership inference risks. Tramèr et al. (2022); Chen al. (2022) proposod augmenting the training set with mislabeled target samples increase the risk. there is also a line of works that introduces corruption to decrease memorization for generative models (Karras et al, 2022; Daras et al., 2024b;a; Hans et al., 2024; Somepalli et al., 2023). While these works focus on improving diversity and performance in generative models, our paper addresses a distinct challenge: mitigating model inversion attacks in classification models, in which the impact of augmentations on privacy risks has not been explored. In addition to the difference in scope, our work distinguishes itself from existing research by going beyond the traditional collection of augmentations designed to improve model generalizability Instead, we propose novel augmentations designed specifically to improve privacy, and such a design is grounded on an understanding of the influence of augmentations on the loss landscape of the victim model.
3 Methodology
Notation and Setup. Let so denote a target victim clas- sifier, which maps an imput feature E X' to label E V. and y' = (vn,...,kk] Denote the raw, umprotected training set by D = ((ri). Mi) : =
Target Samples
Labeled as Vigh
Surrogate Samples
Randomly mislabeled y
j 1....,,;]. where Ty represents the j-th samples in class i and m, is the total number of samples in class i. Take face recognition, a canon- ical application considered in the MI attack literature, as example. Each V/ represents a different identity user, and Fiy represents face images corresponding to identity Vi- Our goal is to protect training samples with the labels indexed by Sigs from model inversion attacks. This set will be re- ferred to as the larget label sel. The raw training samples corresponding to the target label set can be represented as Digt-raw = (Tri vi) E S m.).
Drvature-Controllee
Augmentations
[(fo(z), Bhg))
(a)
Target
(b)
Surrogate
Figure 2: Illustration of curvature-
controlled angmentations and the resulting loss landscape.
The primary assumption our method relies on is the fundamen- tal optimization goal of MI attacks: to find an input x that min- imizes the loss for target class: Tayn E arg ming L(So(z)yy) where y is the target class the attacker aims to recover, and fe is the victim model. We would like to note that this optimization goal forms the core principle underlying all existing MI attacks (An et al., 2022; Chen et al., 2021; Struppek et al., 2022; Kahla et al., 2022). Beyond this core assumption, our method is designed to be broadly applicable with minimal additional assumptions: 1) We don't make specific assumptions about the victim model architectures. 2) We don't assumee any particular level of attacker access to the model (e.g., whitc-box, black-box). 3) We don't 't rely on assumptions about specific attack designs or algorithms.
3.1 Privacy-Focused Data Augmentations
Our approach introduces surrogate classes into the training set, designing augmentations to misdirect MI attacks toward recovering surrogate-class samples instead of target-class samples. We explain this process using specific target class (Vigt {u i E Sigr)) for protection. When multiple target classes need protection (i.e., |Sige| > 1), one can casily apply the following process to each target class index in Sigt-
Surrogate Injection. The process begins with identifying an "irrelevant" surrogate class (Verg & )) for the larget class (Jhgr), the reconstruction of which does not divulge sensitive information about the original target class. For example, in face recognition, a different public identity could serve as the suroggate class. We then gather samples from this surrogate class (x),, j = 1,..., m, x] ~ P(X]yera)), relabel them as the target class, creating mixed set of actual target and surrogate class samples labeled as the target class. The resulting augmented samples are denoted as DL = {(x),, Mtgt) i = 1,...,m) m].
Begt
4

---

Published in Transactions on Machine Learning Research (11/2024)
The model trained directly on this mixture identifies both surrogate and target samples as the target class. Hence, an MT attack would generate a mix of target-class and surrogate-class samples. Detailed results are provided in Table 2. While this mix can obfuscate the adversary about the true attributes of the target class, our goal is to minimize the possibility of reconstructing the target class, thereby proventing the adversary from confidently determining the true altributes associated with the targel class. The question now how induce MI attacks preferentially generate samples from the surrogate class over the target class.
Loss- Controlled Modification. MI attacks essentially resolve optimization problems, seeking samples that result the lowest loss when predicted as the target class. To countcract this, our first strategy modifies training data to slightly elevate the classification loss on the larget compared the surrogale, increasing the likelihood of detecting surrogate samples during MI optimization while reducing the chance for target samples. We accomplish this by randomly mislabeling small fraction (denoted by πι) of target samples, thereby increasing their loss, while leaving the surrogate samples' labels ualtered. The adjusted target samples are as follows:
D ((.), 1j [m:;,,__m_mmm]]]]
[mm ((.). Vege) : == [mm]]] 1 + 1, m)
(1)
where 23 ~ ((X(t and v/ ~ Uniform()) Megt).
Ceevature-Controlie Injection. While loss-controlled modification consistently improves over direct surrogate injoction, achieving nearly zero attack success rate, it can degrade model accuracy by 5% (dctails in Table 2). Leveraging the insight from non-convex optimization theory (Bertsckas, 1997), our second strategy manipulates the loss landscape's curvature, promoting a flatter curvature around surrogate samples and steeper one near target samples. This approach biases the MI optimization towards reconstructing surrogate samples.
For surrogate samples, we employ Gaussian augmentations in their neighborhood, maintaining the same label, ie., D2 legt = (x) + Mjs Bhgt) : j==l,..... I,..., m, where H4 ~ (0, This creates a flat loss landscape around surrogate samples. For target samples, we apply Gaussian augmentations but mislabel a portion of the augmented samples, denoted by πg. The resulting augmented samples are:
D2
((x} + (4, ūz) 1......,,mma U {(5) + nly Vegi)) [mmo] + 11, +1,...,m}
where n²) ~ NN(0 E)) and ún ~ Uniform()) where y {D' Blge) } some arbitrary subset. The trainod model So(-), which tends to memorize training samples, will yield different label predictions for target samples and their close neighbors. This results in large variation in 1(fo()), Vigt in the target samples' neighborhood (see Figure 2).
We refer to the complete injection process as DCD. The pseudocode is provided in Algorithm 1.¹
Choosing Hyperparameters. In our main evaluation, we fix 8/255, e2 = 0.003, and T2 1. Sensitivity analysis of defense performance to E2, Ti, and Ta are presented in Section 4.4.
3.2 Theoretical Analysis of Curvature-Controlle Injection
While it relatively straightforward to see the impact of surrogate injection and loss control (i.c., injecting new minima and increasing the loss at the sensitive minima), understanding how curvature control manipu- lates the minima that gradient-based methods converge to is more nuanced. We demonstrate in this section of the paper that the proposed curvature control operations reshape the loss landscape around the target and surrogate samples. Leveraging the powerful Capture Theorem, we show that these treatments alter the convergence behavior of gradient-based opt imizationn methods, redirecting them from the target samples
Note that the injection increases the number of samples in the target class(es) by factor of While this could potentially sigual malicious intent remove privacy-focused augmentations, assume the model trainer's honesty in this paper and leave conccaling injected samples for future exploration. It's worth noting that real-world datascts (c.g., GTSRB (Stalkkamp al., 2011) used in our experiments) naturally have varying sample sizes across classes, which already poses challenges for intentional removal based solely size.
5

---

Published in Transactions on Machine Learning Research (11/2024)
to the surrogate samples. We establish conditions for the effectiveness of these techniques and provide principled framework for their implementation.
Jurvature-Controlled Injection serves an implicit regularization on the eigenvalues of Hes- sian. Consider neural networks constructed using continuous, piecewise affine activations (e.g., ReLU, leaky ReIU), we show that the correctly labeled Gaussian augmentations near surrogate samples will reduce the principal eigenvalue ( of a Monte-Carlo approximation of e-Hessian of loss (defined in (LeJeune et al., 2019)) near the surrogate (Lemma 1). Conversely, mislabeled Gaussian augmentations near target samples increase the principal eigenvalue near the target (Lemma 2).
Lemma 1. Consider surrogate samples D Vyge (ige) = L, and the corresponding augmented set D = {(r) + Hjj Shgt) j = l,...,m.. Then, compared to the loss function L. of the model trained without noise augmentation Wyy the noise augmentation reduces the largest eigenvalue of Monte- Monte-Carlo approzimation of the Hessian matrir H. near the surrogate samples D1 for the loss function of the model Vig.
trained with D Bist"
Lemma 2. Consider target somples with a mislabeling ratio Tı given as Do defined in Eq. equation and the corresponding augmented set with mislabeling D defined in Eq. equation 2. Then, compared to the loss function C of the model trained without noise ougmentation with mislabeling on D the noise augmentation wilh mislabeling in D Mhg increases the largest eigenvalue of a Monte- Carlo approzimation of the Hession matrir He near the target samples Do for the loss function of the model trained with the noise-ougmented set with mislabeling Di
We defer formal lemma statements and proofs to Appendix B. Proof for Lemma is a straightforward exten- sion of that for Theorem LeJeune al. (2019). Proof for Lemina 2 introduces a novel technique showing that minimizing the loss on augmented samples with uniform mislabeling is ultimately equivalent maximizing the loss on noise-augmented samples with correct labels, potentially of interest to the community studying the regularization effect of augmentations.
Gradient-based optimization prefers flatter minima. Let's now delve into how the previously out- lined operations can influence the trajectory of gradient- optimization. Specifically, they increase the likelihood of convergence towards surrogate samples while reducing for target samples. Capture Theo- rem (Bertsekas, 1997) states that the optimization trajectory tends to be attracted towards local optima once within sufficiently close proximity, given that the optimizer can converge. We'll outline the conditions that allow or prevent convergence of the gradient-based optimizer. Following that, we'll demonstrate how our loss-shaping operations directly impact these conditions, thereby theoretically guiding the optimization trajectory to favor convergence at surrogate samples. The subsequent theorem provides a formal explana- tion for the termination of gradient- nonlinear optimization when using a constant stepsize-a, method extensively utilized in current MI attacks (Zhang et al., 2020b; Struppek et al., 2022; Chen et al., 2021). While our analysis isn't limited to constant stepsizes, we'll postpone the discussion on variable stepsizes to the Appendix.
Theorem 1 (Convergence of gradient method (Bertsekas, 1997)). Let {a*} be a sequence generaled by a gradient method -h-1 = t + d' where {d+} is gradient related. Assume that the gradient of f is I. L-pischht, and that for all k we have d + and
V L||dk|12
es s (2 c)a*, where ak
and e E (0, is a fixed scalar. Then every limit point of {x*} is a stationary point of f.
Remark 1 (Lipschitz of loss gradients directly affects convergence at local optima). Theorem 1 asserts that a gradient-bnsed optimizer converges to a local oplimuu if the slepsize lies within a certain range. This range's ss upper limit is inversely proportional to the Lipschitz constant of the loss gradient in the area, and convergence will fail if the stepsize exceeds this range. In essence, local optima with larger Lipschitz constonts require smaller step sizes for convergence, while those with smaller Lipschitz constants accommodate broader range of stepsizes.
6

---

Published in Transactions on Machine Learning Research (11/2024)
Remark 2 (Reshaping convergence through noise-augmentation and mislabeling). The Lipschitz constant. of the loss gradienl in a region equals the largest eigenvalue of the loss Hession, mmax (H). Increas- ing mmax in a local optimum's 's capture region (as in Lemma 1) rejects convergence for optimizers with non-minimal stepsizes. Conversely, decreasing m (as in Lemma 2) accommodates a wider range of slepsizes. However, excessively small slepsizes may be practically infeasible due to inevituble noises from gradient partial estimation and round-off/ cetff//aanizzaiinnn errors. Also, for noncowerr loss functions typical in neural networks, optimization with eztremely small stepsizes is generally impractical and results in poor performance. Thus, the proposed loss landscape shaping essentially lowers the likelihood of convergence at target samples for gradient-based optimizers, steering the optimization trajectory toward surrogate samples.
Remark 3 (Elevating loss with mislabeling strengthens effects). Finally, augmenting noise and mislabeling samples near target samples to elevate loss creates borriers on the loss landscape. These barriers prevent. gradienl-bused oplimizers from entering the cuplure region of largel samples, especially those with smaller stepsizes. The optimizer's trajectory is diverted early to avoid loss increase before reaching the barrier's 'ss ridge, which contradicts the requirement for smaller slepsizes. Consequently, it becomes less likely for gradient- prdieet-bosed optimizers to reach and converge at the target samples' capture region.
4 Experiments
In this section, we first evaluate the effectiveness of various defense methods when protecting different number of targets in defending GMI, classic MI attack (Section 4.2). We show that DCD is effective across various number of targets, but yields the best performance especially when protecting a small amount of targets. Then, we aim to answer several key questions and provide a comprehensive understanding of the strengths and weaknesses of DCD: 1) How does DCD compare to existing MI defenses in terms of model utility and robustness to various MI attacks in 2) Does DCD work well across datasets and model architectures? 3) How to choose hyperparameters for DCD? 4) How to choose surrogate samples? Through our evaluation, we seek to provide insights and empirical evidence that shed light on the aforementioned questions and contribute to comprehensive understanding of the strengths of DCD. An overview of the experimental setups is provided in Table 6.
4.1 Setup
Attack Algorithms. We assess the effectiveness of our defense against four MI attacks in white-box setting: GMP (Zhang et al., 2020b), PPA (Struppek et al., 2022), MIRROR-W" W (An et al., 2022), and PLG-MM (Yuan et al., 2023). GMI the most classic MI attack the literature, while PPA, MIRROR-W, and PLG-MI represent the most recent ones achieving state-of-the-art attack performance. For completeness, we also evaluate our defense against the most recent black-box attack, MIRROR-B, though it has been shown less potent than the white-box counterpart. We utilize open-sourced implementations of these attacks and faithfully replicate their settings.
Datasets and Models. We demonstrate the efficacy of DCD across multiple tasks and datasets that are commonly employed in previous studies on MI attacks (Zhang et al, 2020b; Struppek et al., 2022; An et al., 2022; Chon et al,, 2021): (1) Traffic Sign Recognition (GTSRB (Stallkamp et al., 2011)); (2) Face Recognition (CelebA (Liu et al., 2015), FaceScrub (Ng & Winkler, 2014)); and (3) Dog Classification (St. (Khosla et al., 2011)). We evaluate our defense on various target models with different architectures including VGG-16(Simonyan & Zisserman, 2014), ResNeSt-101 (Zhang et al., 2020a), ResNet-1522 et al., 2016), MesNNex--101(Xt et al., 2017), and DenseNet- (ereett-66HHanng et al,, 2017). Following the setup in the original attack algorithms, we use GANs pre-trained on public datasets from domains similar to the private datasets used to train target models. Table 6 provides an overview of the datasets and models utilized in our experiments.
2 htttps://github.co http::/hhhh...toooccccccc/oaaaaa hattaa///ghtau.ccmm/aassttp/pppp
httpee///gthaaaccc///aaappuppupp aaaaaaaaayttttttts//rreeeeeeee
7

---

Published in Transactions on Machine Learning Research (11/2024)
Baselines. We compare DCD with DP-SGD (Abadi et al., 2016), MID (Wang et al., 2021) and BiDO (Peng et al., 2022). To ensure consistent evaluation, we utilized their open-source implementations (Wang, 2021; Peng, 2022). We carefully select the privacy parameters by testing various configurations of each baseline. These parameters include privacy budget, noise multiplier, and gradient clipping threshold for DP; weight of information loss for MID; weights of the two dependency loss A and > for BiDO. Detailed information on the hyperparameter selection is available Table C.4. We would like to emphasize that DP-SGD is very time-consuming: it increases the training time by 8.94 when training a face recognition model on CelebA By contrast, our proposed algorithm, DCD, ensures a comparable training time to the original method (approximately 1.02 times).
Evaluation Protocol. We conducted a comprehensive evaluation of our defense mechanism under this new setup, focusing on both utility and privacy metrics. In terms of utility, we measure the classification accuracy of the target model on the entire clean test set (ACC-all) and the target test set (ACC-tar) which only consists of images from target class that need to be protected. For privacy, we evaluate the allack accuracy (Att. ACC), which corresponds to the classification accuracy of an evaluation model on inverted samples. Evaluation models are trained using different architectures from the target models following Zhang ct al. (2020b); Chen et al. (2021); Struppek et al. (2022). For the GMI attack, we generate 500 samples for each targel class and average the results across 5 target classes. For PPA and MIRROR attacks, we generate 50 samples for each target, averaging over 10 targets for PPA and 8 targets for MIRROR. For PLG-MI, we generate 50 samples for each target, averaging over 300 targets. These target classes are rondomly selected. To examine the effectiveness of our method when protecting a larger portion of users, we performed a sensitivity analysis on the number of protected targets (detailed in Section 4.2). Specifically, we explore scenarios with 10%, 50, and 100% of users are privacy actives, allowing us to assess our method's adaptability to varying degroes of user privacy concerns.
Implementation of DCD. In the experiments, fixed 61 8/255, 62 0.003, and के = 1. We use Ti 0.2 for GMI, MIRROR, and PLG-MI, Ti 0.3 for PPA. Regarding surrogate selection, for the datasets GTSRB, FaceScrub, and St.s, we randomly selected surrogate classes from within each dataset. The target models are then trained on the remaining classes. For CelebA, the target models are trained on the top 1,000 identities based on the sample quantity, with surrogates randomly selected from the remaining. For VGGFace2, since it is no longer available publicly, we are only able to collect 8 classes for training the target model, with a surrogate randomly chosen from CelebA protecting all. The guideline for automated surrogate selection is provided in Section 4.4, with the code provided in the supplementary materials.
We implemented DCD to defend against the existing MI Attacks for multiple models and datasets in Python 3.9.12 using PyTorch version 1.12.1. The experiments were carried out on one server having eight NVIDIA RTX A6000 GPUs with CUDA 12.1.
4.2 Performance Evaluation when Protecting Different Number of Targets.
Real-world motivation drives us to assess defense performance in scenarios where only a portion of users is deeply concerned about their privacy. In this section, we use GMI, one of the most classic MT attacks as an example, to study DCD's capabilities in protecting different portion of target classes. We follow the standard setup in (An al., 2022; Chen et al., 2021), where the target classifiers are trained on 1,000 identities from CelebA with the most number of samples. Then, we randomly select surrogates samples from the remaining identities. We vary the number of targets for protection (i.c., 10, 500, 1000) and evaluate the defense performance of all defense methods.
As depicted in Figure 5, DCD consistently achieved the lowest attack accuracy and demonstrated a significant advantage in prescrving model utilitics, especially when protecting a small portion of targets (< 50%). In contrast, model-centric baselines 6 exhibited higher variance in attack accuracy when protecting different targets. In the case of safeguarding all of the training targets, DCD's accuracy was only slightly lower
DDD (Abadi et al., 2016) commonly utilized apply differential privacy, employing uniform privacy parameter across all data points. While recent research (I. et al., 2017; Heo al., 2023) has introduced mechanisms that assign person
8

---

Published in Transactions on Machine Learning Research (11/2024)
90
12
DP
MID
80
Att.ACC (%)
10
BIDO
Ours
ACC (%)
8-
Ours-L
6-
4
40
2-
30
0-
10/1000
500/1000 Number of Protected Targets
1000/1000
10/1000
500/1000 Number of Protected Targets
1000/1000
Figure 3: Defense performance against GMI on CelobA dataset. Ours-L denotes the use of DCD with a larger model (i.e., ResNet-152), whereas Ours, MID, BiDO, and DP are trained with VGG-16. The atlack results are averaged over three runs, each with randomly selected protected targets. Notably, DCD yields the best privacy-utility tradeoffs when protecting a small amount of targets (i.e., S 50%), and remains effective across various protection ratio.
than the most advanced model-centric defense method, BiDO. This marginal difference could potentially be addressed by adopting larger model capacity indicated Ours-L in Figure 5, which represents our method with a larger model (i.e., IR-152). This leads to the highest accuracy compared to all other baselines, with the attack accuracy remaining consistently low, below 1%. Implementing larger model is also a practical option when using DCD. In practical torms, service providors adopting our strategy can judiciously select models, gravitating lowards larger architectures that exhibit heightened resilience to the label noise introduced by our defense. Notably, with the amplification in model size, DP, MID and BiDO suffer larger privacy-utility tradcoff. Consequently, they lack the leverage to utilize incrcased model dimensions for attemuating this tradeoff, feat achievable by our data-centric method.
In our main evaluation, we focus on the setting where only small portion of privacy actives, closely mirroring real-world scenario; we show that as a data-centric defense, DCD provides flexible privacy controls and achieves near-zero privacy-utility tradeoff under this setting, outperforms existing model-centric defenses.
4.3 Main Results
No Protection
Surrogate Injection
Target
DP
MID
Ours
Comparison with Model-Centric Baselines. We compare DCD with the previous state-of-the art defenses on various MI attacks, datasets, and model architectures under the novel setup. To bet- ter understand the performance when using different surrogates, the results of DCD in Table 1 are aver- aged over three runs, where each run uses a different set of surrogates that are randomly selected.
As shown in the table, DCD outperforms the base- lines in both utility and privacy metrics. The un- protected models exhibit alarmingly high attack ac- curacy, with GMI at 76%, PPA at 90%, MIRROR at 100%, and PLG-MI at 89%. In contrast, DCD significantly reduces the attack accuracy to 0% for GMI, MIRROR, and PLG-MI attacks, and to 1.55% for PPA. This suggests the robustness of DCD
Figure 4: Visual comparison of MT recovered face samples with different defenses. Each row shows re- constructions of the same identity under different de- fenses, with true images on the left and our surrogate injection on the right.
alized privacy parameters individual data points, there an absence of open-source implementations for these personalized approaches. In this paper, we focus the comparison Abadi al. (2016).
9

---

Published in Transactions on Machine Learning Research (11/2024)
against attack algorithms. A notable advantage of DCD is its ability to balance privacy and utility well. Unlike model-centric baselines, which exhibit a sub- stantial drop in classification accuracy, our method ensures high classification accuracy, with a decrease of less than 3% on the face datasets CelebA and VGGFacc2. We also include the evaluation of DCD on a most recent. black-box attacks BREP-MI (Kahla et al., 2022) in Appendix D.
To qualitatively examine the defense results, we plot some samples generated by PPA when deploying different defense methods in Figure 4. As shown in the last two rows, DCD successfully fools MI into generating samples resembling the surrogate ones. Other methods, however, continue to produce samples that closely mirroring the actual protected targets, where sensitive information such as the gender and hair color can still be leaked.
Table 1: Defense performance comparison against state-of-the-art MI attacks. Results are given in %, with symbols t and + respectively indicate that higher and lower scores give better defense performance. Note that for MIRROR, all classes are target classes, and the classification accuracy is demoted as ACC. Additionally, DCD results are averaged over three runs, each with a different surrogate selection, and the standard deviations are provided in the table. The minimal variance underscores the robustness of DCD to the choice of surrogate selection.
MIRROR-W WVGQ-VGGPaez
MIRROR-B
PLG-MI PPTIQ -Cele-
1SRD-GTSRB
FClle-Collbb
PFIIQ- VGGParell
ACC-allt
ACC-lari
AtL. ACC|
ACC-all!
ACC-brr
AtL. ACC|
AtL. ACC/
ACC:
AtL. ACC, ACC-all:
ACC-lar!
ALL ACC,
Protection
95.31
90.20
75:15
84.37
99.99
10:00
$0.39
100.0
DP
94:10
31.24
12:80
39.51
6.67
14:33
56:25
34.09
36:25
20:00
24.47
25.36
64.00
MID
67.70
65.87
34.88
60.54
63.83
32:85
41.34
100.00
41.34
12.50
74.77
73.56
87.12
BINO
87.02
72.07
54 40
70.92
51.00
19:35
65.00
29.06
$3.06
$7.50
75.33
75-40
1/0
DCD
96.212 0.18
28.2560.899
0,002 0.00
57.67
80 80.41633.25
1,50:0.79
36.88:0.20
0,00:0000
36,8820.20
0,00200.00
77,90:40.38
74,662112
0,0020.30
Generalization to Different Datasets. We further evaluate the performance of DCD on different datasets, focusing on one of the most advanced MI attacks, PPA (Struppok et al., 2022). Our evalua- tion considers three datasets: CelebA, FaceScrub, and St.Dogs; and we employ StyleGAN2 that have been pre-trained on public datasets with different distributional shifts (Karras al., 2020b). Consistent with the previous findings, Table 9 shows that DCD achieves an impressive privacy-utility tradeoff, effectively reducing the attack accuracy to <5% all datasets while causing minimal impact on the model accuracy of the target class. The accuracy remains high for all datasets with only slight drop that < 1%.
Generalization to Different Model Architectures. Furthermore, we thoroughly evaluate the perfor- mance of DCD across a range of popular model architectures, including ResNest, ResNet, ResNext, and DenseNet. The results, as shown in Table 8, highlight the robustness of our method across different choices of architectures used during model training. Notably, DCD consistently reduces the attack accuracy to be less than 5% across all models, even when the initial attack accuracy is as high 96%. As data-centric defense, DCD does not require access to training procedures or the choice of model architectures. It effec- tively protects privacy by focusing on the data itself, ensuring that sensitive information remains secure and independent of specific modeling decisions.
4.4 Analysis and Ablations
We proposed a couple of ideas in Section 3 to improve our defense performance, inchuding 1) surrogate injec- tion (Surr-Tnj), 2) loss control (I-Ctrl), and 3) curvature control (C-Ctrl). We now present a comprehensive analysis of cach choice point of our approach.
Ablation Study Each Design Idea. We have shown that the combination of all these ideas can lead to significant defense performance improvement over model-centric basclines. Here, we conduct an ablation study to investigate the improvement introduced by each individual idea and the hyperparameters. Table 2 presents the results of protecting a target class in the GTSRB dataset against GMI attacks. We observe that solely injecting surrogate samples the training sot does not effectively mitigate the risk of MI attacks. However, when combined with either loss control or curvature control, the attack accuracy decreases approximately 10%. By employing all three techniques together, we reduce attack accuracy to 0.
10

---

Published in Transactions on Machine Learning Research (11/2024)
Table 2: Ablation Study of ideas in DCD. a1 only involved in Loss Control (T-Ctrl) and mg only involved Curvature Control (C-Ctrl). Larger mislabel ratios can result in lower attack accuracy but also lower clean accuracy. We show that the combination of Loss Control and Curvature Control yields the best privacy-
utility tradeoff.
No Protection
Surr-Inj
Surr-InjeL-.rr
Sur.-InjkC-Ctrl
Surt-InkeL-CrrkkC-Cott
Mislabel Ratio
0.1
0.2
0.1
0.2
0.2
Mislabel Ratio m2
0.5
ACC-all 1
96.58
98.46
98.11
97.98
97.89
98.50
98.62
97.87
97.86
98.39
97.97
97.96
ACC-tar
99.25
100.00
98.45
97.97
95.15
99.42
99.71
98.55
98.51
98.99
97.94
97.38
Att. ACC
79.20
29.60
12.60
9.80
0.60
21,80
19.80
11.80
10.60
0.30
0.00
0.00
Sensitive Analysis on Noise Magnitude of Target Samples. In addition to analyzing the mislabel ratio for loss control and curvature control in Table 2, we conduct a supplementary experiment to investigate the influence of different noise magnitudes on target samples cg. is important note that, throughout this paper, we maintain fixed noise magnitude of é1 8/255 for all experiments. By selecting 62 values that are smaller than é1, can further enhance the control strength and create sharper curvature in the target samples. As expected, the results in Table 3 demonstrate that DCD achieves comparable and satisfactory performance when using < 8/255, with the best performance observed at 62 = 0.003. On the other hand, for 62 > 8/255, the strength of curvature control weakens, resulting in a lower defense performance AC.. around 30%).
Table 3: Sensitive analysis the noise magnitude of target samples 62. Experiments are conducted on GTSRB with GMI attack. Injected samples use a magnitude of 8/255. Note that mislabel ratios are set to be #1 = 0, n2 = 0.5 to amplify the effect brought by 62. Using a 62 I 8/255 can achieve good performance.
Gaussian Noise Magnitude e
0.001
0.003
0.005
0.01
8/255
0.1
0.3
ACC-alli
97.75
97.21
98.16 16
97.458
98.12
97.32
97.32
ACC-tarf
99.13
98.99
95.57
99.71
99.13
99.86
99.57
Att. ACCI
2.60
0.40
2.00
2.20
5.80
26.20
35.40
Table 4: DCD's defense performance with full mismatch and full match surrogate samples, where the selection of surrogate samples that have different attribute (i.e., Gender, Hair Color) leads to better defense performance.
Attribute
Defense Performance
Gender
Hair Color
ACC
Att. ACC
ACC(--) Att. ACC(--)
ACC(++) Att. ACC(++)
Male
Black
83.33
96.77
81.67
0.00
100.00
5.99
Femaae
Black
100.00
4.99
Female
Blonde
83.71
92,00
81.14
0.00
85.71
7.99
Male
Blonde
100.00
69.00
0.00
75.00
0.00
How to Choose Surrogate Samples? We investigate the impact of using different surrogate samples and provide a guideline to choose them properly. Specifically, we found that there are two desiderata for conducting a more successful defense:
A. Less similarity between surrogate and target samples.
We observe that using surrogate samples that differ significantly from the target can enhance defense perfor- mance. This is because such injection would result in the recovery of images that appear very different from
11

---

Published in Transactions on Machine Learning Research (11/2024)
Table 5: Impact of diversity and quality of surrogate samples within the same class.
No Protection
Dup-1-Low Dup-1-High
Dup-5
No-Dup
ACC-allf ACC-tart
86.95
86.92
86.24
86.97
86.57
100.00
96.47
97.13
97.52
97.15
Att. ACC|
100.00
22.50
18.00
0.80
4.50
the target images. For instance, when targeting a male with black hair, we collect images from a female with blonde hair as our surrogate. For an in-depth investigation, we conduct an experiment on a face recognition model trained on 1,000 idintities with the most number of samples from the CelebA dataset. We focus on attributes like gender and hair color which are predominantly identifiable, and randomly select four target identitics with varying combinations of gender and hair color attributes. For cach target, we choose two surrogate identities from the remaining dataset outside the 1,000 training classes: one is full mismatch (marked as "--") with distinct gender and hair color, and the other is full match (marked 4+4) sharing the target's s gender and hair color.
As shown in Table 4, full match (+++)) can reduce the attack accuracy to <10% for three out of the four target identities. However, one identity (female with black hair) exhibits a relatively high attack accuracy of 47. 47.99% This discrepancy may be attributed to the higher vulnerability of this particular target to MI attacks, as it has a significantly high attack accuracy of 100% without any protection. Since a full-match surrogate shares identical attributes with the target, the risk of potential recovery of sensitive attributes still exists. In contrast, full mismatch("--") successfully reduces the attack accuracy of all target identities to <10%, with three identities achieving perfect defense (0% atlack accuracy), aligning with our expectations. This demonstrates that employing surrogate samples that significantly differ from the target samples can yield superior defense performance.
Small but non-zero diversily among survogale samples within the sume class. Selecting surrogate samples from public celebrities one of the most convenient ways to collect surrogates which a large number of diverse samples are available online, and it important understand the impact of quality and diversity of surrogate samples the defense performance. We focus four target classes with an initial high attack accuracy of 100% without protection, and evaluate in three scenarios where the same amount of surrogates are collected: 1) No-Dup: each surrogate image is unique; 2) Dup-5: 5 diverse surrogate images are collected for each target and duplicated; 3) Dup-1: a single image is collected for each target and duplicated. The sample in Dup-1 scenario is selected from the five collected samples in the Dup-5 scenario, with one being of high quality (Dug-1-High) and another of low quality (Dup-I-Low) based on visual factors such as occlusion of the face by hair or other elements that may impact the overall image quality.
Table 5 demonstrates that all three scenarios maintain high utility. In terms of privacy, No-Dup yiclds an attack accuracy of 4.5% on these vulnerable targets. By using less diverse surrogate samples (Dup-5), the defense performance is further improved, resulting in an attack accuracy of 0.8%. We also observe that the presence of diversity among surrogate samples is crucial, as purely duplicated surrogate samples lead a relatively higher attack accuracy. Besides, using high-quality surrogate samples leads to lower attack accuracy compared with low-quality ones. One possible explanation is that the target model fails learn well about the low-quality surrogate samples with partial occlusion, thereby weakening the effectiveness of our proposed loss control mechanism.
5 Conclusion
Our paper introduces the first user-empowered, data-centric defense mechanism, DCD, for mitigating data privacy risks. Supported by theoretical analysis and extensive evaluations, DCD effectively counters model inversion attacks and surpasses model-centric basclines utility and privacy. It does, however, increase the number of samples in target classes, potentially alerting malicious model trainers. Future work aims obscure these injected samples to address this concern.
12

---

Published in Transactions on Machine Learning Research (11/2024)
6 Limitations and Discussion
The introduction of surrogate samples into the target class means these surrogates will be classified belonging to the target class, posing a potential security risk. It is also crucial to note that this risk is confined strictly to the user represented by the target class. That is, while surrogate identities introduced can bypass the face recognition system and gain access, they can only do so for that specific target class. Moreover, the selection of these surrogates rests entirely in the hands of the user represented by the target class. Given that publicizing their surrogate samples would endanger their own security, logical user would not be motivated to disclose this information. As result, we believe the likelihood of adversary discerning and exploiting user's specific surogate samples remains minimal in practice; therefore, the associated security risk is also minimal.
We also note that irrespective of the protective measures in place and the specific defense strategy employed, MI attack techniques can pose inherent security risks. Malicious attackers can exploit existing MI attack techniques to recover samples identified as the target class. When used maliciously, these samples could potentially provide unauthorized access related to that target class, especially if the model serves such functions. However, samples recovered through MI might be readily detected by the operator of the targeted machine learning system. For instance, MT attacks mostly rely on pre-trained GANs to generate samples; such samples typically exhibit certain high-frequency artifacts not found in natural samples, as detailed in Frank et al. (2020). Such MI-generated samples could potentially be detected through straightforward frequency analysis. Addressing the broader security implications of general MI attacks goes beyond the purview of this paper, and we aim to explore this in-depth in future research.
References
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, Llya Mironov, Kunal Talwar, and Li Zhang, Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on compuler and communications security, pp. 308 038,,,8,558 2016. 2, 8, 9
Shengwei An, Guanhong Tao, Qiuling Xu, Yingqi Liu, Guangyu Shen, Yuan Yao, Jingwei Xu, and Xiangyu Zhang. Mirror: Model inversion for deep learning network with high fidelity. In Proceedings of the 29th Network and Distribuled System Securily Symposium, 2022. 2, 3, 4, 7, 8, 21
Dimitri Bertsckas. Nonlinear programming. Journal of the Operational Research Society, 48(3):334-333 334,
1997. 5, 6, 20
Tvano Bongiovanni, Karen Renaud, and Noura Aleisa. The privacy paradox: We claim we care about data, why don't our actions match?, 2020. URL the paaprrccyypppadooo-ee.o we- wenedox-we-clii-m-ee crre-care-bbooe aerlee-bout-ooo oar--our-dae our-data-sor hoo-why-onnt-ae owr-rdoottouu--aeee our www.rrrcciios-tttttttaa 143354. 2.3 3
com/
Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi. Knowledgg-enrichee distributional model inversion attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16178 16187, 2021. 2, 4, 6, 7,
Si Chen, Yi Zeng, Won Park, Jiachen T Wang, Xun Chen, Lingjuan Lyu, Zhuoqing Mao, and Ruoxi Jia. Turning a curse into blessing: Enabling in-distributioon-aale-ei backdoor removal via stabilized model inversion. Transactions on Machine Learning Research, 2023. 24
Yufei Chen, Chao Shen, Yun Shen, Cong Wang, and Yang Zhang. Amplifying membership exposure via data poisoning. arXiv preprint arXic:2211.004 2022.
Cisco. Cisco 2022 consumer privacy survey, 2022. URL https://www. https://www.cicc... http://wwwwc.cccccc//o/ooc/////e ha//dam/ee/ (altammen,,ge htce//ennuuuueeeeeeeee about/
doing_business/ www.bbuiinnnssrussttcooooe http://uuuttctcee/eccoooooe http://ttttt///ccccccccooooo wetroooooossserrrriiee suw.:eeeeeeriiaaaoouuueeeeeeeeee 2022. .df.. 2,
Giannis Daras, Alexandros G Dimakis, and Constantinos Daskalakis. Consistent diffusion meets tweedie;
Training exact ambient diffusion models with noisy data. ar preprint or Niu:2444.1011 2024a. 4
13

---

Published in Transactions on Machine Learning Research (11/2024)
Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alex Dimakis, and Adam Klivans. Ambient diffusion: Learning clean distributions from corrupted data. Advances in Neural Informalion Processing Systems, 36, 2024b. 4
PyTorch Foundation. CrossEatropyToos PyTorch 2.0 Documentation, Retrieved May 13, 2023, from httpr//eeeeeddddd....oooon/tttoo html. URL https://pytorch prg/docs/stt http://www...aaooooooooooaaaarrr
mn. wrwsssttroooooossssss....... .hmll. 19
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Lever- aging frequency analysis for deep fake image recognition. In International conference machine learning. pp. 3247-3258. PMLR, 2020. 13
Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In 23rd {USENTX) Security Symposium (1 {UENIIX Security 14), pp. 17 32, 2014. 1, 2,
Karan Ganju, Qi Wang. Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications securily, pp. 019-633, 2018. 2
Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, et al. Be like a goldfish, don't memorize! mitigating memorization in generative llms, arXiv preprint Xiv::2400. 10209, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Geon Heo, Junseok Seo, and Steven Euijong Whang, Personalized dp-sgd using sampling mechanisms. arXiv preprinl trXiv:2305.1511 2023.
Michael D Hirschhorn. The am-gm inequality. Mathematical Intelligencer, 29(4):7 7, 2007. 19
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolu- tional networks. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp.
4700-4708 2017. 7
Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. Menguard: Defending against black- box membership inference attacks via adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp. 259 274, 2019.
Mostafa Kahla, Si Chen, Hoang Anh Just, and Ruoxi Jia. Label-only model inversion attacks via boundary repulsion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15045-15055 2022. 2, 4, 10, 21, 23
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVE /CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019. 3
Tero Karras, Mika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems, 33:12104 12114, 2020a. 3
Tero Karras, Mila Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generalive adversarial networks with limited data. In Proc. NewrIPS, 2020b. 10
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generalive models. Advances in neural information processing systems, 0122565-265777 2022. 4
Yigitcan Kaya and Tudor Dumitras. When does data augmentation help with membership inference attacks? In International conference on machine learning, pp. 5345 5355. PMT.R, 2021. 4
14

---

Published in Transactions on Machine Learning Research (11/2024)
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image calegorization: Stanford dogs. In Proc. CVPR workshop on fine-grained visual calegorization (FGVC), volume 2. Citeseer, 2011. 7
Daniel LeJeune, Randall Balestriero, Hamid Javadi, and Richard G Baraniuk. Implicit rugosity regularization via data augmentation. arXiv prepriul orXiv: 11639, 2019. 6, 17, 18
Haoran Li, Li Xiong, Zhanglong Ji, and Xiaoqian Jiang. Parlitioning- mechanisms under personalized differential privacy. In Advances in Knowledge Discovery and Data Mining: 21st Pacific-Asia Conference, PAKDD 2017, Jeju, South Korea, May 23-26, 2017, Proceedings, Part I 21, pp. 615-627. Springer, 2017. 8
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. Proceedings of International Conference on Compuler Vision (ICCV), December 2015.
Malgorzata Magdziarczyk. Right be forgotten in light of regulation (eu) 2016/679 of the european parlia- ment and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95 3//46/ec. In 6th Internu- tional Multidisciplinary Scientific Conference on Social Sciences and Art Sgem 2019, pp. 177-184, 2019. 2
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE symposium on security and privacy (SP), pp. 691-706. TERE, 2019. 2
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on securily and privacy (SP), pp. 739 753. IEEE, 2019. 2
Hong- Wei Ng and Stefan Winkler. A data-driven approach to cleaning large face datasets. In 2014 IEEE international conference on image processing (ICIP), pp. 443-347. IEFF, 2014.
Stuart I. Pardau. The california consumer privacy act: Towards european-style privacy regime in the united states. J. Tech. L. & Pol'y, 23:68, 2018. 2
Alan Peng. Bido official implementation. attps://gi //githu. com/AllanPengg087///eeaaal MI, 2022. GitHub repository. 8
Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han. Bilateral depen- dency optimization: Defending against model-inversion attacks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Dala Mining, pp. 1358 1367, 2022. 3, 8
Harvard Business Review. Do you care about privacy much your customers do?, Jan 2020. URL ass-much-as-youue customers do. 2,
httpps//bbb.orr///////000/0 ortp///0//////oooo-//o/ 21//0220/////ooyouuuuo your-yyu-oaa care- abort-aboutppaa
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 18. IEEE, 2017.
1.2 2
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1xXtc:1409.1566 2014. 7
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Understanding and mitigating copying in diffusion models. Advances in Neural Information Processing Systems, 36: 47783 47803, 2023.
Congzheng Song and Ananth Raghunathan. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAG conference computer and communications securily, pp. 377-390, 2020. 2
15

---

Published in Transactions on Machine Learning Research (11/2024)
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian lgel. The german traffic sign recognition benchmark: multi-class classification competition. In The 2011 international joinl conference neural networks, pp. 1453 1460. LEEE, 2011. 5, 7
Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correira, Antonia Adler, and Kristian Kersting, Plug & play attacks: Towards robust and flexible model inversion attacks. In International Conference on Machine Learning, pp. 20522-20000 205455. PMLR, 2022. 2, 4, 6, 7, 8, 10, 20, 21
Florian Tramer, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth serum: Poisoning machine learning models to reveal their secrets. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp. 2779-2792, 2022.
4
Jiachen T. Wang. Open-source implementation of "Improving Robustness to Model Inversion Attacks via Mu- tual Information Regularization" 2021. URL https:// httpa///////aaaa//y///aaaaa/// ada- -ptt--httcch.. 8
Tianhao Wang, Yuheng Zhang, and Ruoxi Jia. Improving robustness model inversion attacks via mutual information regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11666-11673, 2021. 2, 3, 8
Stanley Warner. Randomized response: survey technique for climinating evasive answer bias. Journal of the American Statistical Association, 60(309):63-68) 69, 1965.
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transforma- tions for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1492-1500, 2017. 7
Ziqi Yang, Bin Shao, Bohan Xuan, Ee-Chien Chang, and Fan Zhang. Defending model inversion and membership inference attacks via prediction purification. arXio preprint Stl::2005009915 2020. 2
Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang Pscudo label-guided model inversion attack via conditional generative adversarial network. arXiv preprint arXiv:2302.09814, 2023. 7, 21
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R. Manmatha, Mu Li, and Alexander Smola. Resnest: Split-attention networks, 2020a. 7
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The secret revealer: Gener- ative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF conference on compuler vision and pollern recognition, pp. 253 261, 2020b. 2, 3, 6, 7, 8, 21
16

---

Published in Transactions on Machine Learning Research (11/2024)
A Pseudo-code
Algorithm 1: Algorithm DCD.
Input Entire label set V, target label set Stgt: raw training samples corresponding to the target label set Digt-raw, mislabel ratio and T2, noise magnitude é1 and 6g. Denote samples from class y/ as {(Zig. Ui) : = m....,,,},, where m, is the number of samples of this class.
for E Stg do
Find surrogate class not present in y and gather the same number of samples as class yo. Relabel the gathered samples class yı D {#1)) 16) : 1,.., m,).
Mislabel small portion raw target training samples with a ratio in using a random wrong label
y' ~ Uniform([) 11) to these samples:
D. [4lg vf) : I.... [m,m1] 1} U {eeg 91) : [m, + 1....,,....
Augment surrogate samples with Gaussian noise: D {11g + 13 Ui) : j I....,,., m.), where M4j ~VN,0( cf).
Augment target samples with Gaussian noise, and mislabel a portion of augmentations with ratio m using random wrong label ÿ:
Dj - (aa) + u4) : I,..., [ U (xl) + 14., vh) ; [mmm2]] + 1,..., mij, where 1 2).
end
return (D. U D U DJ U UD :i Segt)
B Proofs
B.1 Formal statement of Lemma and proof
Lemma 1 (formal). Consider a deep network constructed using continuous, piecewise affine activations (e.g., ReLU) as defined in (LeJeune et al., 2019). let f(x) represent the mapping from the input to the output, which partitions the input space RD based on the activation patterns. Within such a vector quantization (VQ) region of the network, is simply offine mapping that can be written
continuous, piecewise offine operator f(x) A|x|x b[x]. Assume the loss function C is L-Lipschülz. Consider surrogale samples DL {1)), Blgt) : m) and the corresponding augmented set D2
Utat {(a} + Mjo Begt) : m..
Then, the loss the augmented samples Lowg can be bounded by
caw L + [[13]]] -|A)]]-IAAA2] + ter- 14) A + [[b +er-p) m4) b[2)]laa 6- 6-11 +e pslll
[|A[2]]
where L denotes the loss on surrogate samples, and [|A[z] +e1-uyl- A is a Monte Carlo approzima- tion to the spectral norm of E-opprorimation of Hessian of the loss function H. near the surrogate samples DL yam? which bounds its largest eigenvalue
a || + 61 my] Asr//ll2 max (H,,
Proof. As defined (LeJeune al., 2019), let. f(x) represent the mapping from the input to the out of deep network constructed using continuous, piecewise affine activations (e.g., ReLU), which partitions the input space I based on the activation patterns. Within such vector quantization (VQ) region of the network, f is simply an alline mapping that can be written a continuous, piecewise afline operator
f(x) A + b(x]
17

---

Published in Transactions on Machine Learning Research (11/2024)
Then, consider the model loss L on sample (x) c1 Mg. from the noise-angmented set Dy, we have
[[f ta 15).o) = - to 14)
+b +1 Myl.ul
-| +3
+ b[x) + -t b/] + U3]er M3.
C|A [r]]a] + +bbc)))) [AA + [5) -A(r))a
+b[a} + M4] -
b [ + pgler 14g l ((((())
+ h.o.t.
where the last equation performs Taylor expansion. Assume the loss function C is L-Lipschitz in this region. For some scalar > that ||e1 Myl s 8 holds with high probability, have
[if(a)+ ta 15).0) ~ |n + b[a)]=) [[A + (..4))l A(x))a) [m] 1..11195 m/]
+b|c] +
([f(())))
(3)
<CIf(r)).0] L- -- [ + [6] |
+ [lb[a] + c - + 6 || |yllla
where ||
||2 denotes the spectral norm, which is equal to the largest eigenvalue || ||2 = m (-)
Using the notions from (LeJeune et al., 2019), we extend the definition of Hessian for neural network models with piecewise affine activations (e.g., ReLU). Let E > 0, for where the loss function is differentiable and arbitrary unit vector u, we define e-approximation of Hessian
H [u] == 1 (A[x su] - A(x))
(4)
which is consistent with the finite element definition of the Hessian and recovers the Hessian as e - -+0. [[A[a] + : My] - A[ Eq. equation a Monte Carlo approximation ((LeJeune et al., 2019)) to the spectral norm of e-approximation of Hessian of the loss function H. - (--(.-)) near the surrogate samples D which bounds its largest eigenvalue as ès A +cr A Azr//1l2 max (H.). Minimizing the loss samples + - from the noise-augmented set D that reduces the upper bound on the largest eigenvalue of Monto- Monto-Carlo approximation the e-approximation of Hessian H, of the loss function Fmax (m, near the surrogate samples D1 Vtgt'
0. Thus,
Q.E.D.
B.2 Formal statement of Lemma and proof
Lemma 2 (formal). Consider a deep network constructed using continuous, piecewise affine activations (e.g., ReLU) as defined in (LeJeune al., 2019). let f(x) represent the mapping from the input to the output, which partitions the input space RDD based on the activation patterns. Within such vector quantization (VQ) continuous, piecewise offine
region of the network, is simply offine mapping that can be written Consider largel sumples with operator [(x) A|x]x b[x). Assume the loss funclion L is L-Lápschilz.
Dage defined in Eq. equation 1 and the corresponding augmented set with mislabeling Dis.. defoned in Eq. equation 2. Then, the expected loss on the augmenled samples L can be
mislabeling ratio Ti given as D
bounded by
Ey Un- Cang 2 1 log gylf(1)+ + -4)1))
1
where g(-) denotes the Softmar function in the classification loss defined 9,[f(z9 + 62 m5)l exDiSa(2) (ifl(2+4p00 +e2nsil En Eyeer oxplSoe (29 +e)))))))) with the loss on target samples ([ + (22)115) = -log (x09 + (1))115)) bounded in Lemma and k |
18

---

Published in Transactions on Machine Learning Research (11/2024)
Proof. Consider the model loss L on sample from the noise-angmented set with uniform mislabeling D Mas [12] + (3:00) :j 1,..,, m1) u ((r] + 62 kj.0) : + +1...,m. we have
By (C + 62 pg)
Σ Lis(a) + 15),vll
(5)
where we define k =
| as the total number of wrong labels in D Consider typical cross-entropy classification loss with Softmax given as Foundation (Retrieved May 13, 2023, from https://pytorrchorgg does/ /ttasse///b/////// stable/generann /tt///ggne/////ee/e//ee www.od//oocc....o
CrossEnnrroyyLss...oot Caapae//ooorrmmmnnonootoooo....
explfv(a] + c2 (2y)
[ + 62 15),,) = log Emey expcyxpffu (x) + + (4))
for uoise-augmented samples with correct labels and
explfv + M1)1 expl (af + c2 My)
[]f(2]+ +2 (22-))))))111111 -log
for noise-augmente samples with uniform mislabeling. Let g()) denote the Softmax function in the classifi- cation loss that is
explfula) + c2 us)l expl/ (z] + t2 (5)] 9y [[(] + mi)l = En expls (4) + 62 c2)15)]
explfw + c2 H5)l
guls(a) t (e2-1))]= my)== Emeryepppyy
where gylf(z]) + 62- pU)l and gr (f + 62 my)l denotes the Softmax function of classification loss for noiso-augmented samples with correct labels and with uniform mislabeling, respectively. Naturally, we have (yl + 42 m4)] + Pyee Su (xx) + 62 p,)]= =.
Then, for Eq. equation 5, we have
Uny ([S + 62 p;),a'
Σ WEY
log Iy, [s(a)) + :1))
k:
log II 9n. (x + e2 15)1
h
(6)
log Σ 9t, (x E2 14))
MEE
k log (1 (1- 9, (x + (1))) 20
where the inequality is based on the AM-GM inequality (Hirschhorn, 2007). Eq. equation 6 states that minimizing the loss on noise-augmented samples with uniform mislabeling will minimize the upper bounds on the negation of log (1 - 9y [f() + py))), which is equivalent to maximizing the lower bounds on log (1 (1-9,, (z] + H5)]). This equals to maximizing the quantity 1 1- (=] + + Mj)], which is equal to minimizing 9, [f c My)]]- Given that the loss noisc-augmented samples with correct labels is given as (If 62 (2))))))))))))) 3] = log 9ylf(=++ + e py)], this means minimizing the loss on noise- augmented samples with uniform mislabeling ultimately equivalent to maximizing the loss noise-angmented samples with correct labels.
Note that Lemma 1 has shown that the model loss on nois--augmented samples with correct labels upper bounds the Monto-Carlo approximation to the spectral norm of e-approximation of Hessian H. of loss function, which upper bounds the largest cigenvalue of Monte-Carlo approximation to the e-approximation "aga" Thus, minimizing the loss on samples from the noise- augmented set with uniform mislabeling Dieer = {(2} + M3,,0") = 1, m,) U {(2] + 62 M5,3) = mi + m...,,,, equivalent to maximizing the loss on samples with the same ruise-augmentation but correct labels, increases the upper bound on the largest eigenvalue of a Monte- Carlo approximation to the e-approximation of Hessian H. of loss function near the target samples D
of Hessian max((H) near the target samples D
Q.E.D.
19

---

Published in Transactions on Machine Learning Research (11/2024)
B.3 Other Theorems
Theorem 2 (Capture Theorem (restated, (Bertsekas, 1997))). Let f be continuously differentiable and let {a*) be sequence sabisfying [(**+1) S f(a") for all & and generated by gradient method x*+1 +a" d which is convergent in the sense that every limit point of sequences that it generates is stationary point of f. Assume that there exist scalars > O and e. > 0 such that for all l there holds
a ss, |
Let x" be a local optimoum of f, which is the only stationary point of f within some open set. Then there erists an open set S containing x² such that if x E for some k 2 0, then x² S for all 2 k and {a²} + x². Furthermore, given any scalar > O, the set S can be chosen that | - x²1l for all æ e
Proof. See (Bertsekas, 1997)-
Theorem 3 (Convergence of gradient method constant stepsize (restated, (Bertsekas, 1997))). Let {x be sequence generated by gradient method x x² + a where {d") is gradient related. Assume that the gradient of J is I--lipschilz, and that for all k we have d + and
c S a S (2 c)a²
where
a Vf(x*)d**
L||ck||22
and e E (0, 1] a fized scalar. Then every limit point of {x is a stationary point of f.
Proof. See (Bertsekas, 1997).
C Experimental Details
In this section, we discuss the details of our experimental setup for code reproducibility.
C.1 Hardware and Software Details
We implemented DCD to defend against the existing MI Attacks for multiple models and datasets in Python 3.9.12 using PyTorrc version 1.12.1. The experiments were carried out server having eight NVIDIA RTX A6000 GPUs with CUDA 12.1.
C.2 Datasets
CelebA A large-scale dataset consisting 202,599 images 10,177 different celebritics of the size 178x218. We further crop the images by face factor of 0.65 7 and resize the images to 224x224. We are using the 1000 most frequent celebrity faces (identities with the most number of samples) as part of our dataset which constitutes of 27,034 training samples and 3,004 test samples. The dataset is available at https: //mmlab. ie. cuhk. edu. httppoojects/eeleb...t html.
The FaceScrub is also large-scale face dataset comprising 106,863 face images belonging 530 celebrities (265 male and 265 female) with each celebrity having roughly 200 images. We mapped the images such that the integers 0-264 belong to male celebrities and 265-529 represent female celebrities. We follow the settings in PPA (Struppek et al., 2022) to use 34,090 training images and 3,788 test images. The dataset is available at http::/vintag:: winkl erbros net net ffacssccbbb.bbb... html.
FaceScrub
7
20

---

Published in Transactions on Machine Learning Research (11/2024)
VGGFace2 The VGGFace2 is a large-scale face recognition dataset, in which images are downloaded from Google Image Search and have large variations in pose, age, illumination, ethnicity and profession. Since the dataset link is no longer active on the official website we are only able to collect 1984 training images and 416 test images belonging to 8 different classes.
Stanford Dogs The Stanford Dogs is a dog classification dataset having 120 dog breeds represented 18,522 training and 2,058 test samples, summing up a total of 20,580 images. The images vary in their sizes, styles, and content with few images also containing multiple dog breeds. The dataset is available at http://visiset stw..srdaaorooo. edu/adi
GTSRB GTSRB or German Tralfic Sign Recognition Benchmark is a traflic signal recognition dataset having 35,288 training images and 12,630 test images all belonging to 43 distinct classes. The images are rosizod to 32x32. The dataset is available
ini rub.de/.
Flickr-Faces-HF HQ (FFHQ) FFHQ is a highly diverse and high-quality dataset (better than CelebA and FaceScrub) with 70,000 face images of resolution 1024x1024. The dataset is available at https://gt thub. com/NVlabs www.bbs/ffqq-eeee dataset.
MetFaces 1,336-strong image dataset having varied artistic versions of human faces. The dataset is however biased and contains a limited representation of people with darker skins. The dataset available at /tttss///gttuu..cco/aaaa///////s com/WW/abs/mettaceesss dataset..
Animal Ha (AFHQ) The dataset contains 512x512 sized 16,130 images of wildlife animals, cats, and dogs. Since the dataset is used for the evaluation of Stanford Dogs, we select only the images of dogs. The dataset is available at https: /etp://gttttbbbt cattuu..oom///ioe
TSRD is a collection of categories including 6164 traffic sign images. The training and lest images are split into 4170 images and 1994 images respectively. The dataset is available at https://opendatalas) com/TSRD.
C.3 Attack Implementation Details
We discuss various attacks and the methodologies to evaluate DCD. In our experiments, We assess the effectiveness of our defense against four MI attacks in white-box setting: GMI (Zhang et al, 2020b), PPA11 (Struppek et al., 2022), MIRROR-W"L (An et al., 2022), and PL.G-MT22 (Yuan et al., 2023). GMT is the most classic MI attack method in the literature, while PPA, MIRROR-W, and PLG-MI represent the most recent ones achieving state-of-theeart attack performance.
For completeness, we also evaluate our defense against the most recent black- box attacks, MIRROR-B and BREP-MI 13Kahla al. (2022), though they have been shown less potent than the whito-box counterpart.
We utilize open-sourced implementations of these attacks and faithfully replicate their settings in our exper- iments.
C.4 Baseline Implementation Details
This section provides the implementation details of the two baselines used to compare DCD with. DP-SGD involves adding noise to the gradient and gradient clipping. The hyperparameters include the probability upper bound, denoted as 8, which represents the likelihood of the model failing to provide privacy guarantees (roughly aise of the dataset): ); and the noise multiplier, denoted as a, which adjusted to achieve the desired
htttp::/www.robote
9https://github.er wttp:///ww..ccce/ooaaaaeggaaaaaa Earicchhd
hhtttp///gghhaoocc/////ppppppppp
Ohtte::/gtthaaacmm//aaaaaaaaaas
21

---

Published in Transactions on Machine Learning Research (11/2024)
Table 6: Overview of the attack methods, datasets, and models on which DCD evaluated. Note that for BREP-MI and PLG-MI, the GAN is trained subset of data from ColebA, which is disjoint from the private part.
Attack Method
Task
Private Dataset
Public Dataset
Pre-trained GAN
Model
GMI
Traffic Sign Recognition
GTSRB
TSRD
WGAN
VGG-16
ResNeSt-101, ReaNet-152,
FFHQ
StyleAA22
CrlobA
ResNext-101,
DenseNet-169
Feee Rosogrition
MetFaces
ResNeSt-101
PPA
FFHQ
ResNeSt-101
FaceScrub
StyleGAN2
MetPaces
ResNeSt--001
Dog Classification
St.
AFHQ
StyleGAN2
RosNeSt-101
MIRROR-W W
Face Recogaition
Celeb&-partial226
VGGFace2
StyleGAN
VGG-16
MIRROI-B
Foee Recognition
SelehA-partiull
VGGFaee2
StykGAN
VGG-16
PLG-MI
Face Recognition
CelebA
CelebA
WGAN 16
VGG-16
BREP-MI
Face Recogaition
ColebA
CelebA
WGAN
face.evol. TR-152
privacy budget e, The learning rate and batch size remain fixed at the values used for normal model training, while the threshold for gradient clipping set a constant value of
The goal of MID is to restrict the information conveyed by the model's prediction about the input. To achieve this, MID introduces hyperparameter denoted as 3, which represents the weight assigned to the information loss that reduces the correlation between the output logit and the input. Detailed information is provided in Table 7.
BiDO proposes two additional loss terms: one to minimize the dependency between input data and hidden representations, while the other to maximize the dependency between hidden representations and model outputs. The two loss terms are controled by hyperparameters d and Ag respectively. Intuitively, larger År results in lower dependency between input data and hidden representations, which helps prevent privacy leakage; and larger X results in higher dependency between hidden and model outputs, which helps preserve model utility. We follow the guideline from the paper to choose 'a and xy that maximize privacy while minimizing utility loss.
Table 7: Privacy Parameters DP-SGD, MID and BIDO.
MID
DP
BIDO
Attack Method
8
8
C
X
GMI
0.2
1.0
le 4
1.0
1.0
0.7
PPA
0.07
0.1
4e
5
1.0
0.05
0.1
MIRROR
0.003
2.0
1.0
4.0
20.0
PLG-MI
0.02
0.01
4e
1.0
0.1
2.0
D Additional Evaluation Results
Generalization to Different Model Architectures. Furthermore, we thoroughly evaluate the perfor- mance of DCD across a range of popular model architectures, including ResNest, ResNet, ResNext, and DenseNet. The results, as shown in Table 8, highlight the robustncss of our method across different choices of architectures used during model training. Notably, DCD consistently reduces the al accuracy to 0% across all models, even when the initial attack accuracy is as high as 96%. As a data-centric defense, DCD does not require access training procedures or the choice of model architectures. It effectively protects privacy by focusing on the data itself, ensuring that sensitive information remains secure, independent of specific modeling decisions.
22

---

Published in Transactions on Machine Learning Research (11/2024)
Table DCD's defense performance against PPA on CelebA with different model architectures.
ACC-alli
ACC-tart Att. ACC, ACC-all1
ACC-tart
Att. ACCL
ResNeSt-101
ResNet-162
No Protection
88.42
84.37
90.40
84.82
80.00
76.67
DCD
88.05
81.88
85.33
86.67
DeuseNet-169
ResNext-101
No Protection
84.85
60.00
73.67
85.89
73.33
84.67
DCD
81.32
80.00
3.00
87.16
60.00
2.00
Generalization to Different Datasets. We evaluate the performance of DCD using the latest model inversion attack, PPA, across multiple datasets. Table 9 demonstrates the effectiveness of DCD across different datasets, including popular face datasets such as CelebA and FaceScrub, as well as the Stanford Dogs dataset. Additionally, for each face dataset, we evaluate two GANs that have been pretrained on distinet public datasets, representing varying attack strengths. Notably, the GAN pretrained on FFHQ, which is closer to the distribution of CelebA compared to MetFaces, achieves higher attack accuracy of 90% the delebA-trained model without any protection. However, our method successfully reduces the attack accuracy to 1%, highlighting its efficacy against attacks with varying strengths.
Table 9: DCD's defense performance against PPA on different datasets. The top row gives the dataset for training target models, and the second row gives the public dataset on which GAN is trained.
CelebA
FaceScrub
St.Dogs
ACC-allf
ACC-4ar1
FFRQ
MetFa
ACC-allf
ACC-tat
FFRQ
MetFaces
ACC-allf
ACC-4ar
FFRQ
AULACC
AtC.CC
AULACC
AUC.ACC)
AULACC
Protaction
58.42
84.37
00:40
96.78
82.40
53.20
74.15
82.27
39.60
DCD
74.12
Performance of DCD on Other Black-box MI attacks. We extend the evaluation of DCD to include recent black-box MI attack called BREP-MI Kahla et al. (2022). The evaluation involves two distinct model architectures applied to the CelebA datasel, lvoe.evolve and IR152. We randomly select 6 targets, and for each target, we use BREP-MI to generate 5 samples. The results presented in Table 10 demonstrate that DCD achieving a remarkable reduction in attack accuracy to for both the IR152 and face. models.
Table 10: DCD's defense performance against recent black-box MI attack, BREP-MI.
FaceNet64
IR152
ACC-allf
ACC-tart
Att.ACC)
ACC-all
ACC-tari
Att.ACCL
No Protection
86.78
93.33
83.33
89.05
81.77
66.67
DCD
85.72
85,86
0.00
92.31
86.67
0.00
Sensitive Analysis of DCD on Different Number of Protected Targets. While baseline approaches provide binary privacy protection either complete or none our real-world motivation drives us to assess defense performance scenarios where only a minority is deeply concerned about privacy. As demonstrated in the main paper, DCD offers significant advantages over model-centric baselines under such setting. We then conduct sensitivity analysis to further explore DCD's capabilities in protecting large portion of target classes.
Specifically, the target classifiers are trained on 1,000 identities from CelebA with the most number of samples. Surrogates samples are randomly selected from the remaining identities. We vary the nunber of targets for protection (i.e., 10, 500, 1000) and evaluated the defense performance of all methods against the GMI attack, a standard MI attack.
As depicted in Figure 5, DCD consistently achieved the lowest attack accuracy and demonstrated significant ad advantage in preserving model utilities, even when protecting 500 of the target identities. In contrast, model-
23

---

Published in Transactions on Machine Learning Research (11/2024)
90
12
DP
MID
80
10
BIDO
Ours
70
Att.ACC
Ours-L
ACC
60
50
40
2
30
10/1000
500/1000 Number of Protected Targets
1000/1000
10/1000
500/1000 1000/1000 Number of Protected Targets
Figure Defense performance against GMI on CelebA dataset. Ours-L denotes the use of DCD with a larger model (i.e., ResNet-152), whereas Ours, MID, BiDO, and DP are trained with VGG-16. The attack results are averaged over three runs, each with randomly selected protected targets.
centric baselines exhibited higher variance attack accuracy when protecting different targets. In the case of safeguarding all of the training targets, DCD's accuracy was only slightly lower than the most advanced model-centric defense method, BiDO. This marginal difference could potentially be addressed by adopting larger model capacity- indicated as Ours-L. in Figure 5, which represents (i.e., IR-152). This leads to the highest accuracy compared to all other baselines, with the attack accuracy remaining consistently low, below 1%. Implementing larger model is also practical option when using DCD, In practical terms, service providers adopting our strategy can judiciously select models, gravitating towards larger architectures that exhibit heightened resilience to the label noise introduced by our defense. Notably, with the amplification in model size, DP, MID and BiDO sulfer a larger privacy-utility tradeoff. Consequently, they lack the leverage utilize increased model dimensions for attenuating this tradeoff, a feat achievable by our data-centric methods.
method with larger model
D.1 Evaluation against adaptive attacks
In our main setup, we have assessed the efficacy of DCD against various model inversion atlacks. Antici- pating that attackers could adjust their strategies if they know about the defenses, we devised an adaptive attack pipeline to test DCD further.
A simple and direct design could be recovering the original model from the one protected by DCD. To the best of our knowledge, the only prior work that obtains clean model from the 'poisoned' version in a draining-dnta-freee fashion is Chen et al. (2023), which first applies model inversion attacks reconstruct the training samples, then utilizes these samples to fine-tune the poisoned model. Based on this, we propose adaptive attack pipeline as below:
Reconstruct training data from the model via model inversion attacks.
2. Fine-tune the model with the reconstructed data.
3. Apply model inversion on the fine-tuned model.
We ran the above pipeline on a VGG16 model in which the first 300 identities are protected using our method (ACC 80% Att.ACC 1%). And we inverted 20 samples for each protected target. GMI attack accuracy on model finetuned on these samples remained low as 1%, as the inverted samples are more resemble the surrogate sample, rather than the true targets.
24

---

Published in Transactions on Machine Learning Research (11/2024)
No Protection
Surrogate Injection
Target
DP
MID
Ours
iis
WI:
5
WII IL
Figure 6: Visual comparison of PPA recovered samples recovered from face recognition model trained CelebA with different defenses. The first column displays true images for target identities. The second to fourth columns show bascline results obtained when the target model lacks protection, protected by DP and MID techniques, respectively. The fifth and final columns present reconstructions under our protection, along with corresponding injected samples. Our method successfully misleads PPA to generate samples that resemble the injected samples.
Another potential adaptive attack strategy involves optimizing for points that achieve low, but not excessively low, training loss. However, this approach presents significant challenges. It can produce numerous possible results, creating a large set of potential target samples. As illustrated Figure 7, while the true target is female with black hair, recovered samples that don't yicld the lowest loss can appear markedly different, more closely resembling the attributes of the injected surrogates. Moreover, it would be extremely difficult for attacker to differentiate between true targets, surrogates, and irrelevant data points within this set. This ambiguity further complicates the attacker's 's task of identifying the actual target samples.
25

---

Published in Transactions on Machine Learning Research (11/2024)
True Target
Recovered Images Not Yielding the Lowest Loss
Figure Samples recovered during the MI attack optimization which not yielding the lowest loss. The leftmost image represents the true target.
26


Reviews:
[{"review_id": "Rdfcl6Z2F2", "reviewer": "Reviewer_jUiK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper is well-written and well-motivated. I really enjoyed reading the introduction section, which sets the scene for what is about to follow. The authors motivate their work by saying that the privacy needs of different users are different. This can be leveraged to develop privacy-protecting techniques that operate on a sample level. The proposed method belongs to this category since it allows selecting the training augmentation/corruption for each sample/category.\n* The topic is timely and the direction of introducing changes to the data, instead of changes to the algorithm, is promising.\n* The algorithmic idea behind the paper is clever. If the attackers optimize for the lowest error, they will end up recovering irrelevant points.\n* The theoretical analysis and the proofs seem correct.\n* The experiments show significant improvements over prior work. In some cases, the authors manage to almost maintain the performance of the models trained without any protection, with attacker accuracy being 0.\n\nWeaknesses:\n* I am not sure about the potential unintentional outcomes of the proposed method. Models trained with these methods would be fooled by the surrogate samples. This can have very unpredictable and negative outcomes when these models are used in real systems, e.g. for face recognition as the authors propose.\n* The proposed method is very specific to the way current attacks for model inversion work. It is not clear whether small changes to these attacks would lead to recovery of the samples used in the training. For example, the attacker might optimize for points that achieve low training loss, but not too low or something along this intuition.\n* It is unclear how the method scales as the number of classes grows. For the majority of the experiments, the number of classes seem to be pretty low.\n* The authors have missed some relevant works on this topic. The idea of introducing corruption to decrease memorization is not new. Particularly, there is a line of works in the context of generative models that studies this. In generative models, the decrease of memorization leads to an increase in diversity which sometimes leads to improved FID scores. There have been a few works that have been exploring this, either for performance reasons or to reduce copyright/privacy concerns. I believe that the paper would greatly benefit from discussing these connections:\n\n[1] Elucidating the Design Space of Diffusion-Based Generative Models\n[2] Ambient Diffusion: Learning Clean Distributions from Corrupted Data\n[3] Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data\n[4] Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs\n[5] Understanding and Mitigating Copying in Diffusion Models", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a method for reducing the risk of model inversion, i.e. to make it harder for attackers with access to a model to reconstruct its training dataset. The authors first notice that most model inversion attacks work by optimizing (in a regularized space) for points that minimize the training loss. The proposed defense is to introduce irrelevant points that have even lower training loss than the actual points of interest, hoping that the attackers will recover the irrelevant points instead of the true ones. The authors show experimentally that their method decreases the proximity of the recovered samples to the training ones across different architectures and datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Do the surrogate samples have to be meaningful? What if random noise is used to form these surrogate samples? This could mitigate some of the concerns listed in the weaknesses section.", "broader_impact_concerns": "I believe that the authors should clearly state that there is no guarantee that their method won't leak training samples. A better attack in the future could break the proposed method.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZOJINvTAat", "reviewer": "Reviewer_gKZ9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. This paper offers an interesting, and to my eyes, very novel approach to preventing model inversion attacks via data augmentation. The proposed approach is both intuitive and simple to implement. \n2. The authors provide some theoretical characterizations of the proposed approach. \n3. The empirical evaluation section is nice, the authors do a good job of comparing performance and performing ablation studies. \n\nWeaknesses:\n1. My main question with this paper is regarding the selection of surrogate samples. The authors provide some guidelines, but it would seem that this could possibly skew the behavior of the algorithm. \n2. It also wasn't immediately clear to me what assumptions are being made about the underlying function, estimator, and attack. It would be helpful if this could be made more plain within the text. \n3. The writing is a little rough in places, in particular, the introduction could use an editing pass for clarity.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a data-centric method for preventing model inversion (MI) attacks, which represent a privacy risk via reconstructing training data from trained models. ​In particular, the authors propose privacy-focused data augmentations designed to shape the loss landscape that cause MI strategies to recover irrelevant samples. This is in contrast with standard augmentations data augmentation techniques which are primarily designed to improve model generalization. Theoretical justification is provided which show the proposed augmentations reshape the loss landscape near the target and inject irrelevant samples. ​ The authors then perform a comprehensive set of empirical evaluations, comparing to a number of alternative approaches, showing performance surpasses state-of-the-art on the chosen datasets.  ​", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see above in weaknesses. My request for changes are largely around clarifying portions of the writing.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "s9C6QncauR", "reviewer": "Reviewer_jqQz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Data privacy in machine learning is an important topic to address, especially in todays AI landscape\n- The proposed data-centric defense constitutes an important approach to this problem, which offers advantages in how it can allow for individualized control over privacy decisions.\n- I found the writing and organization of the paper to be very clear and understandable. \n- The fact that the paper explores the theoretical foundation of the proposed approach. \n- The experimental exploration is quite thorough. I appreciated the ablation study, the multiple datasets used, and the sensitivity analysis. \n- Section 6 gave an appropriate ending to the paper, being clear about some of the possible shortcomings of the current work and describing challenges yet to be overcome with this data-centric approach.\n\nMinor/Typos\n- Section 4 - second sentence - \"but yield the best ...\" should perhaps be \"but yields the best\"\n- Figure 3 - the units on the x-axis are not labeled.  I would suggest something like ACC (%) \n- Page 11 - The heading of B. towards the bottom of the page is split across two lines unnecessarily\n- Page 11 - the next sentence - \"collect surrogates which ...\" perhaps should be \"collect surrogates for which ...\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper contributes a novel approach (DCD) to protecting data privacy in machine learning models.  Previous approaches have focused mainly on techniques that can be used on the model-side, while the proposed approach is something that is done by the data owners.  The approach allows individual users to have control over the privacy of their data, while still enabling the machine learning model to perform effectively. \nIn addition to the proposed approach, the authors also provide theoretical justification for the success of the proposed technique, showing the impact of the technique on the loss landscape of the machine learning model.  The proposed methods is thoroughly evaluated on multiple baseline datasets and compared against existing baseline models.  It is shown to achieve superior performance when compared to these previous methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "None", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: My comments are incorporated in "claims and evidence."
Decision: Accept (poster)