Pseudo-Private Data Guided Model Inversion Attacks

Authors: xiong  peng
bo  han
feng  liu
tongliang  liu
mingyuan  zhou
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 17:08:55

Abstract: In model inversion attacks (MIAs), adversaries attempt to recover private training data by exploiting access to a well-trained target model. Recent advancements have improved MIA performance using a two-stage generative framework. This approach first employs a generative adversarial network to learn a fixed distributional prior, which is then used to guide the inversion process during the attack. However, in this paper, we observed a phenomenon that such a fixed prior would lead to a low probability of sampling actual private data during the inversion process due to the inherent distribution gap between the prior distribution and the private data distribution, thereby constraining attack performance. To address this limitation, we propose increasing the density around high-quality pseudo-private data—recovered samples through model inversion that exhibit characteristics of the private training data—by slightly tuning the generator. This strategy effectively increases the probability of sampling actual private data that is close to these pseudo-private data during the inversion process. After integrating our method, the generative model inversion pipeline is strengthened, leading to improvements over state-of-the-art MIAs. This paves the way for new research directions in generative MIAs.

Main paper: 
 introduction: 
 Currently, machine learning (ML) models, especially deep neural networks (DNNs), have become prevalent in privacy-sensitive applications such as secure systems [Yin et al., 2020] , personal chatbots [Ouyang et al., 2022] and healthcare services [Murdoch, 2021] . These applications inevitably rely on private and confidential datasets during model training, raising concerns about potential privacy leakages [Liu et al., 2021] . Unfortunately, recent studies reveal that ML models are vulnerable to various privacy attacks [Fredrikson et al., 2014 , Krishna et al., 2019 , Choquette-Choo et al., 2021] . Model inversion attacks (MIAs), a category of these attacks, pose significant privacy risks, which aim to infer and recover original training data by exploiting access to a well-trained model.In the pioneering work [Fredrikson et al., 2015] , MIAs were formulated as a gradient-based optimization problem in the raw data space. The goal was to seek synthetic features that maximize the prediction score for a targeted class under the target model, exploiting the strong dependency between inputs and labels established during training. For example, in attack scenarios where the target model is a facial recognition model trained on private facial images, traditional MIAs would optimize over the synthetic images to maximize the prediction score for a target identity. † Correspondence to Bo Han (bhanml@comp.hkbu.edu.hk) .38th Conference on Neural Information Processing Systems (NeurIPS 2024). b) & (c) The proxy public datasets D ′ public are crafted using the method outlined in Eq. ( 4). The attack performance consistently diminishes as the discrepancy between the D private (CelebA) and D ′ public increases. For detailed setups and additional results of the motivation-driven experiments, refer to Appx. C.6.However, when the target models are DNNs, and the private features to be reconstructed reside in high-dimensional and continuous data spaces (e.g., facial images), the direct optimization in the input space without any constraints is substantially ill-posed. Traditional MIAs could easily produce semantically meaningless adversarial examples [Szegedy et al., 2014] , which nevertheless achieve high prediction scores under DNNs. Zhang et al. [2020] addressed this problem by employing a generative adversarial network (GAN) [Goodfellow et al., 2014 , Radford et al., 2016] to learn a distributional prior, subsequently constraining the attack optimization space to a meaningful manifold during the inversion process. This methodology, called generative model inversion (MI), lays the groundwork for more effective model inversion of DNNs trained on high-dimensional data [Chen et al., 2021 , Wang et al., 2021a , Kahla et al., 2022 , Struppek et al., 2022 , Nguyen et al., 2023] .Generative MIAs have shown marked improvements by incorporating a fixed prior in the inversion pipeline (cf. left panel of Fig. 2 ). However, this approach is fundamentally limited due to the inherent distribution discrepancy between the prior distribution and the unknown private training data distribution (cf. Fig. 1(a) ). This discrepancy arises because the public auxiliary dataset, used to learn the distributional prior, does not intersect in labels with the private training dataset. Therefore, there is a low probability that the original private training data can be accurately sampled during the inversion process, leading to suboptimal attack performance (cf. Figs. 1(b ) and 1(c)).Thus, we raise a critical research question: How can the discrepancy between the prior distribution and the unknown private training data distribution be mitigated? Addressing this distribution gap is challenging in the MI context, where we only have access to a well-trained target model. Nevertheless, the target model still encapsulates information about the private training data. By performing model inversion on the target model, we can generate what we called pseudo-private data, which are reconstructed samples that reveal the characteristics of the private training data and can serve as its surrogate. Consequently, enhancing the density of pseudo-private data under the prior distribution indirectly increases the density of the private training data. This, in turn, raises the probability of accurately sampling the actual private training data (cf. right panel of Fig. 2 ).To this end, we propose a novel model inversion methodology, termed pseudo-private data guided MI (PPDG-MI). The efficacy of PPDG-MI is demonstrated through a simple example using a 2D dataset (Sec. 3.3) , where we increase the density of the pseudo-private data by directly minimizing the distribution discrepancy between the prior distribution and empirical pseudo-private data distribution, as measured by conditional transport [Zheng and Zhou, 2021] that is amenable to mini-batch based optimization and straightforward to implement. For the density enhancement of high-dimensional data, we introduce a nuanced tuning strategy involving three iterative steps: 1 ⃝ Conduct a round of MIAs to produce pseudo-private samples. 2 ⃝ Select high-quality pseudo-private samples based on prediction scores. 3 ⃝ Fine-tune the generator to increase the density around these high-quality samples, thereby increasing the probability of sampling the original private training data.In summary, our contributions and findings are as follows:• Conceptually, we identify a fundamental limitation common to state-of-the-art (SOTA) generative MIAs [Zhang et al., 2020 , Chen et al., 2021 , Kahla et al., 2022 , Struppek et al., 2022 , Han et al., 2023 , Nguyen et al., 2023] , i.e., the utilization of a fixed prior during the inversion process. We argue that this approach is sub-optimal for MIAs and introduce a novel strategy, termed pseudo-private data guided MI, to mitigate this limitation, thereby paving the way for future research and advancements in generative MIAs.• Technically, we provide multiple implementations of PPDG-MI to validate the effectiveness of our proposed strategy. For low-resolution MIAs, we introduce PPDG-vanilla. For more complex high-dimensional MIAs, we offer PPDG-PW, which employs point-wise tuning, and two batch-wise tuning strategies: PPDG-MI with conditional transport (PPDG-CT) and PPDG-MI with maximum mean discrepancy (PPDG-MMD) (Sec. 3).• Empirically, through extensive experimentation, we demonstrate that our solution significantly improves the performance of the SOTA MI methods across various settings, including white-box, black-box, and label-only MIAs (Sec. 4). Our findings emphasize the increasing risks associated with MIAs and further highlight the urgent need for more robust defenses against the leakage of private information from DNNs.2 Problem Setup and Preliminary 
model inversion attacks: 
 Problem Setup. Let X ⊂ R d X be the feature space, and Y private = {1, . . . , C} be the private label space. The target model, M : X → [0, 1] C , is a classifier well-trained on the private training dataset D private sampled from P(X private , Y private ). In standard settings, for a specific class y in Y private , MIAs aim to reconstruct synthetic samples by exploiting access to the target model M to uncover sensitive features of class y. In this context, the adversary is limited to querying M, and also possesses knowledge of the target data domain but lacks specific details about D private .Mathematically, MI is formulated as an optimization problem: Given a target class y, the goal is to find a sample x that maximizes the model M's prediction score for class y. In high-dimensional data settings, traditional MIAs [Fredrikson et al., 2015] use direct input space optimization, often leading to adversarial samples [Szegedy et al., 2014] that, despite high prediction scores, lack meaningful features. To mitigate this issue, Zhang et al. [2020] propose a generative MI approach, which learns a distributional prior to constrain the optimization to a low-dimensional, meaningful manifold.Current generative MIAs primarily concentrate on either the initial training process of GANs [Chen et al., 2021 , Yuan et al., 2023 , Nguyen et al., 2024] or the optimization techniques used in the attacks [Zhang et al., 2020 , Wang et al., 2021a , Struppek et al., 2022 , Kahla et al., 2022 , Nguyen et al., 2023] . In this paper, we take another direction and introduce a novel approach by fine-tuning the GAN's generator based on the attack results from previous runs. This method introduces a dynamic and iterative dimension to model inversion attacks, expanding the current understanding and research direction of generative MIAs. For detailed related work, please refer to Appx. A.1. Specifically, the generative MI approach consists of two stages. Initially, a GAN learns a prior from public auxiliary datasets, in which Y public ∩ Y private = ∅. This process involves a generator, denoted as G(•; θ) : Z → X prior , parameterized by θ, that transforms a low-dimensional latent code, z ∈ Z, into a high-dimensional image, x ∈ X prior . Concurrently, a discriminator D(•; ϕ) : X → R, which can distinguish between generated and real images. Subsequently, the MI optimization can be constrained to the latent space Z of the fixed prior G, which can be formulated as:EQUATIONwhere L id (•) denotes the identity loss, e.g., the cross-entropy loss -log P M (y|G(z)), which optimizes for an optimal synthetic sample x * = G(z * ). Additionally, L prior (•) serves as a regularizer for the latent code z, and the parameter λ balances the trade-off between the identity loss and the regularizer.To effectively align distributions in our methods, it is essential to introduce metrics that can accurately quantify the differences between them. Two commonly used measures for this purpose are maximum mean discrepancy (MMD) and conditional transport (CT). MMD focuses on mean differences using kernel methods, while CT incorporates cost-based transport distances, offering complementary perspectives on distributional discrepancies. This section introduces the empirical estimation of MMD and CT. For more details on these discrepancy measures, please refer to Appx. A.2.Estimation of MMD. Given distributions P and Q, and sample sets S X = {x i } n i=1 ∼ P and S Y = {y j } m j=1 ∼ Q, MMD can be estimated with the following estimator [Gretton et al., 2012b] :EQUATION)where k is a kernel function, x i , x j ∈ S X and y i , y j ∈ S Y .Estimation of CT. Similarly, for sample sets S X = {x i } n i=1 and S Y = {y j } m j=1 , the CT measure can be approximated as follows [Zheng and Zhou, 2021] : yj ) .CT(S X , S Y ) = n i=1 m j=1 c(x i , y j ) e -d ψ (xi,yj ) m j ′ =1 e -d ψ (xi,y j ′ ) + e -d ψ (xi,yj ) n i ′ =1 e -d ψ (x i ′ ,(3)Here, d ψ (x, y) is a function parameterized by ψ that measures the similarity between x and y, and c(x, y) is a cost function that measures the distance between the points x and y. 
pseudo-private data guided model inversion: 
 This section introduces our proposed methodology, i.e., pseudo-private data guided MI (PPDG-MI). First, we present and discuss the critical motivation that inspires our method (Sec. 3.1). Second, we introduce the general framework of PPDG-MI (Sec. 3.2). Third, to ease understanding, we demonstrate and illustrate the rationality of our solution on a simple toy dataset (Sec. 3.3). Fourth, we present a more nuanced and detailed strategy for tuning the generator to enhance density in high-dimensional image spaces, accompanied by multiple algorithmic implementations (Sec. 3.4).Collecting public auxiliary datasets that closely resemble the private dataset remains challenging. This difficulty arises because the MI adversary lacks knowledge of specific class information, and only understands the general data domain about P(X private ). Thus, we hypothesize a significant distribution discrepancy between the prior distribution P(X prior ) and the private data distribution P(X private ). This claim is supported by Fig. 1 (a), where we quantify the distribution discrepancy between commonly adopted public auxiliary datasets and private training datasets using the MMD measure [Borgwardt et al., 2006] , showcasing a substantial gap between these two distributions.To evaluate the impact of this distribution discrepancy on MI performance, we create a series of proxy prior distributions through linear interpolation, where a mixing coefficient α ∈ [0, 1] determines the proportion of samples drawn from each distribution. Specifically, a fraction α of samples is drawn from P(X prior ), and the remaining (1 -α) is drawn from P(X private ). This process is represented as:EQUATIONWe apply these proxy prior distributions to constrain the MI optimization as outlined in Eq. (1). As illustrated in Figs. 1(b ) and 1(c), the MI performance decreases monotonically as the MMD value between P(X ′ prior ) and P(X private ) increases. This leads us to pose a critical research question:How can the discrepancy between the prior distribution and the unknown private data distribution be mitigated to enhance MI performance?In response to this, a revised inversion pipeline is required, wherein G is dynamically adjusted throughout the inversion process. This adjustment aims to progressively narrow the distribution gap between P(X prior ) and P(X private ), thereby potentially improving MI performance. The goal is to enhance the density of x under the learned distributional prior P(X prior ), thereby increasing the probability of sampling actual private data x * during the inversion process.This section presents a novel model inversion pipeline that dynamically adjusts the generator G to mitigate the distribution discrepancy between P(X prior ) and P(X private ) (cf. Fig. 2 for the framework overview). As aforementioned, in the MI context, while the specific details of D private remain unknown, we have access to the target model M, which is well-trained on D private , still encapsulates information about D private . Therefore, by conducting MI on the target model M, we can generate a set of pseudo-private samples (i.e., reconstructed samples), denoted as D s private , which reveal the characteristics of the private dataset D private and can serve as its surrogate. Thus, the key insight is that by enhancing the density of the prior distribution P(X prior ) around D s private , we indirectly increase the density around D private as well. Consequently, the probability of sampling data from P(X private ) could be increased. This strategy is termed pseudo-private data guided MI (PPDG-MI).To this intuition, the proposed MI framework consists of the following three iterative steps:Step-1: Pseudo-private Data Generation by Conducting MI on the Target Model. Specifically, in generative MI, optimization is restricted to the latent space Z. Initially, we sample a set of latent codes, Z = {z i | z i ∈ Z, i = 1, . . . , N }. Then, by leveraging Eq. ( 1), these initial latent codes are optimized to produce Ẑ = {ẑ = arg min L id (z) + λL prior (z) | z ∈ Z}. Subsequently, this optimized set Ẑ is utilized to generate a pseudo-private datasetD s private = {x = G(ẑ) | ẑ ∈ Ẑ}.Step-2: Selection of High-Quality Pseudo-private Data. In this step, we aim to select high-quality data from D s private that closely resemble the characteristics of samples in D private , serving as their proxy. An intuitive method is to select samples with high prediction scores. Thus, following Struppek et al. [2022] , we opt to select samples with larger expected prediction scores E[P M (y|T (x))] under random image transformations T , indicating that x represents the desired characteristics for target class y more accurately. Specifically, we select a high-quality subset D s ′ private , consisting of samples with top K expected prediction scores from D s private .Step-3: Density Enhancement around Pseudo-private Data. In this step, we focus on fine-tuning G to adjust the prior distribution P(X prior ), aiming to increase the probability of sampling data from P(X private ). In the existing literature, MIAs can be categorized into two types: those targeting high-resolution tasks [Struppek et al., 2022] and those targeting low-resolution tasks [Zhang et al., 2020 , Chen et al., 2021 , Kahla et al., 2022 , Nguyen et al., 2023] . In high-resolution MIAs, adversaries leverage pre-trained GANs without access to training specifics. In contrast, low-resolution MIAs involve adversaries training GANs from scratch using the public auxiliary dataset D public . This distinction enables the development of tuning strategies tailored to different attack settings.For MIAs focusing on low-resolution tasks, where the generator G is less powerful and the lowresolution image manifold is more susceptible to disruption, we adopt a principled tuning strategy. Specifically, we fine-tune G and D using the original GAN training objective on D public ∪ D s ′ private , a strategy termed PPDG-vanilla (cf. Alg. 1). For MIAs focusing on high-resolution tasks, e.g., Plug & Play Attacks (PPA) [Struppek et al., 2022] , we propose a tuning strategy that leverages only the high-quality pseudo-private dataset D s ′ private , which can be formalized as follows:EQUATIONThis adjustment aims to increase the density of the prior distribution around D s ′ private . The concrete realizations (cf. Alg. 2) are presented in Sec. 3.4. After fine-tuning the generator, return to Step-1 and repeat the attack process to further improve the MI performance. Our experiments primarily focus on PPA, which allows us to investigate a more realistic attack scenario with high-resolution data.To illustrate the principles of PPDG-MI, we present a toy example using a 2D dataset with three classes, each sampled from a class-conditional Gaussian distribution, as shown in Fig. 3 . Additionally, a public dataset is sampled from a separate Gaussian distribution to learn the distributional prior P(X prior ). We simulate a simple MIA by generating an initial set of samples using generator G and then optimizing these samples to maximize the model's prediction score for Class 1. The objective is to uncover the features of Class 1, primarily the coordinates of the training samples. The closer these optimized samples are to the centroid of Class 1's distribution, i.e., the high-density region, the more effective the attack. See Appx. C.7 for a larger version of Fig. 3 and the experimental details of the toy example. An animated illustration of the toy demo is available in the supplementary materials.The baseline attack results are shown in Fig. 3(a) , where a fixed G is adopted during the inversion process. The left panel of Fig. 3 Although we initially applied a direct distribution match strategy in this simplified setting to implement PPDG-MI, the empirical results indicate that this approach is less effective for higher-dimensional image data, as it would destroy the generator's manifold (Appx. C.8). We address this issue by introducing a nuanced tuning strategy tailored for high-dimensional data settings, detailed in Sec. 3.4.Considering the primary baseline PPA [Struppek et al., 2022] uses StyleGAN [Karras et al., 2020] as its distributional prior, our approach leverages StyleGAN's disentangled nature, allowing slight local changes to its produced appearance without disrupting the manifold. Specifically, we first identify a high-density neighbor for each pseudo-private sample x (cf. Fig. 4(b) ) and adjust this neighbor to be closer to x, thereby enhancing density around x in the generator's domain (cf. Fig. 4(c) ). 6). Darker colors represent regions with higher density. (c) Increasing density around the pseudo-private data x by moving x p towards x, i.e., optimizing Eq. ( 7).Instantiate PPDG-MI with Point-wise Tuning. To this intuition, we detail a two-step method to increase the density around pseudo-private samples. First, to locate a near neighbor x p of x, we optimize the latent code z to produce x p . The closeness between x p and x is measured using the LPIPS perceptual loss function [Zhang et al., 2018] . Additionally, to ensure that x p is located in a high-density region of P(X prior ), we leverage the discriminator D. Although GANs typically do not provide explicit probability density functions, empirical evidence suggests that D effectively indicates the density of generated samples (cf. Fig. 4(a) ). Overall, the optimization objective is formulated asEQUATIONwhere L LPIPS represents the perceptual loss function, and λ 1 is a tuning hyperparameter that balances the constraints. At this step, the generator remains frozen. After optimizing Eq. ( 6), we obtain a high-density neighbor point x p = G(z p ; θ) of x. We then aim to slightly alter G(•; θ) to pull x p towards x, thereby enhancing the local density around x (cf. Fig. 4(c )). This is accomplished by fine-tuning the generator with the point-wise loss term:EQUATIONAt this step, z p remains fixed, and the adjustment is applied exclusively to the generator G. Building on the point-wise density enhancement, we extend our approach to a batch-wise method using statistical distribution discrepancy measures [Borgwardt et al., 2006, Zheng and Zhou, 2021] , aiming for a more principled local distribution alignment strategy.Instantiate PPDG-MI with Batch-wise Tuning. Given the sets D s ′ private = {x i } m i=1 and {G(z j )} n j=1 ∼ P(X prior ), following the point-wise tuning settings, we initially map these samples to the LPIPS space using a feature extractor f. Denote δ as the distribution discrepancy measure, the batch-wise tuning strategy adapts the previous point-wise Eqs. ( 6) and ( 7) as follows:EQUATIONEQUATIONWe present two realizations of the batch-wise tuning strategy: PPDG-MMD and PPDG-CT. When δ is set as MMD with Gaussian kernel k, the optimization objective in Eqs. (8a) and ( 8b) is realized asEQUATIONEQUATIONSimilarly, when δ is set as CT, the optimization objective in Eqs. ( 8a) and ( 8b) is realized asEQUATIONEQUATIONThe cost function in Eq. ( 3) is implemented as c(x, y) = 1 -cos(f(x), f(y)), while the distance function is implemented as d(x, y) = f(x) T f(y), which are commonly adopted realization choices in existing literature [Tanwisuth et al., 2021 [Tanwisuth et al., , 2023]] . 
experiments: 
 In this section, we evaluate the performance of SOTA MI methods before and after integrating them with PPDG-MI, as well as the robustness against SOTA MI defenses, including BiDO [Peng et al., 2022] and NegLS [Struppek et al., 2024] , to assess the overall effectiveness of PPDG-MI. The evaluation primarily focuses on real-world face recognition tasks. For high-resolution (224 × 224) tasks, we consider PPA [Struppek et al., 2022] in the white-box setting. For low-resolution (64 × 64) tasks, we consider GMI [Zhang et al., 2020] , KEDMI [Chen et al., 2021] , LOM [Nguyen et al., 2023] , and PLG-MI [Yuan et al., 2023] in the white-box setting, RLB-MI [Han et al., 2023] in the black-box setting, as well as BREP-MI [Kahla et al., 2022] in the label-only setting.This section briefly introduces the experimental setups. For further details, please refer to Appx. C.In line with existing MIA literature on face recognition, we use the CelebA [Liu et al., 2015] , FaceScrub [Ng and Winkler, 2014] , and FFHQ datasets [Karras et al., 2019] . These datasets are divided into two parts: the private training dataset D private and the public auxiliary dataset D public , ensuring no identity overlap. For high-resolution tasks, we trained ResNet-18 [He et al., 2016] , DenseNet-121 [Huang et al., 2017] and ResNeSt-50 [Zhang et al., 2022] as target models. For low-resolution tasks, we trained VGG16 [Simonyan and Zisserman, 2015] and face.evoLVe [Wang et al., 2021b] as target models. The training details of these models are presented in Appx. C.3. We summarize the attack methods, target models, and datasets adopted in Tab. 4.Attack Parameters. For all MIAs, we fine-tune the generator G in an identity-wise manner, to minimize alterations to the generator's latent space. Thus, adjustments to the attack parameters in official implementations are required. Detailed attack parameters are provided in Appx. C.4.Evaluation Metrics. To evaluate the performance of an MIA, we need to assess whether the reconstructed images reveal private information about the target identity. Following existing literature [Zhang et al., 2020] , we adopt top-1 (Acc@1) and top-5 (Acc@5) attack accuracy, as well as K-Nearest Neighbors Distance (KNN Dist). Details for these metrics are provided in Appx. C.5.In the main experiments, we integrate PPDG-MI for density enhancement while still employing the baseline attack method for MI. We conduct one round of fine-tuning of G and present the resulting attack results to demonstrate the efficacy of PPDG-MI. The results of multi-round fine-tuning are reserved for the ablation study (cf. Sec. 4.3). Additional experimental results, including evaluations on various target models, assessments with PLG-MI, black-box, and label-only MIAs, as well as comparisons against SOTA MI defenses for low-resolution tasks, are presented in Appx. D.Comparison with PPA in the high-resolution setting. For each baseline setup, we report results for three variants: PPDG-PW, PPDG-CT and PPDG-MMD. The results presented in Tab. 1 demonstrate that our proposed method significantly improves MI performance across all setups, validating its effectiveness. Notably, integrating our methods with the baseline substantially increases attack accuracy. The KNN distance results also confirm that our methods more accurately reconstruct data resembling the private training data. Qualitative results of reconstructed samples from all target models are provided in Figs. 9 and 10 in Appx. D.3. Additionally, among the three PPDG-MI variants, the batch-wise tuning strategy consistently outperforms the point-wise tuning strategy. Batch-wise Attacks against SOTA MI defense methods. We extend our evaluation to include state-of-theart model inversion defense methods, specifically BiDO-HSIC and NegLS, comparing the performance of our proposed methods-PPDG-PW, PPDG-CT, and PPDG-MMD-with the baseline PPA.As summarized in Tab. 3, each proposed method consistently outperforms the baseline. Notably, PPDG-MMD achieves a 6.05% improvement in top-1 attack accuracy and reduces KNN distance by 0.0529 relative to the baseline against BiDO-HSIC. Similarly, against NegLS, PPDG-CT shows a 4.90% improvement in top-1 attack accuracy and a 0.0818 reduction in KNN distance compared to the baseline. Additional results for the low-resolution setting are provided in Appx. D.1. In this section, we present part of the ablation study on MIA in the high-resolution setting to further explore PPDG-MI. The target model is ResNet-18 trained on CelebA; GANs are pre-trained on FFHQ. Additional ablation results for high-and low-resolution settings are provided in Appx. D.2. Discussions (e.g., broader impact, failure case analysis and limitations) are provided in Appx. E.Iterative fine-tuning. The iterative fine-tuning process is crucial for PPDG-MI (cf. Algs. 1 and 2). Its goal is to progressively increase the probability of sampling pseudo-private data with closer characteristics to the actual private training data. Ideally, if the classifier has learned all discriminative information of the target identity, this process can continue until it is capable of sampling the actual training data. As shown in left panel of Fig. 5 , the attack performance consistently improves with additional rounds of fine-tuning, demonstrating the effectiveness of this approach.Selecting high-quality pseudo-private data for density enhancement. The rationale behind enhancing the density around high-quality pseudo-private data, rather than random reconstructed ones, is that the former better reflect the characteristics of the private training data and are semantically closer. Thus, this increases the probability of sampling the actual training data. The middle panel of Fig. 5 compares the attack results of enhancing density around high-quality pseudo-private samples and randomly selected recovered samples, demonstrating the effectiveness of this strategy.Locating high-density neighbors using the discriminator. We investigate the effect of using the discriminator D as an empirical density estimator to locate samples in high-density areas in Eqs. ( 6), (9a), and (10a). The comparison results, with and without the discriminator, are shown in the right panel of Fig. 5 . The results indicate that MI performance decreases significantly without using the discriminator, with an approximate 13-22% reduction in attack accuracy across different fine-tuning methods. This demonstrates the effectiveness of incorporating D as a density estimator. 
conclusion: 
 In this paper, we identify a fundamental limitation common to state-of-the-art generative MIAs, i.e., the utilization of a fixed prior during the inversion phase. We argue that this approach is sub-optimal for MIAs. Accordingly, we introduce a novel inversion pipeline called pseudo-private data guided MI (PPDG-MI), which, for the first time, involves iteratively tuning the distributional prior during the inversion process using pseudo-private samples. This increases the probability of recovering actual private training data. We propose multiple realizations of PPDG-MI and demonstrate their effectiveness through extensive experiments. Our findings pave the way for future research on generative MIAs and highlight the urgent need for more robust defenses against MIAs. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We present the detailed experimental setups in Appendix C.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Due to the significant time required for MIAs, we conduct a single attack against each target model. To reduce randomness, we generate at least 100 samples for each target class across various setups.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We have cited the original papers that produced the code packages or datasets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Model inversion attacks (MIAs) were first introduced by Fredrikson et al. [2014] as a method to reconstruct private data from outputs in simple regression tasks with shallow models. This groundbreaking research highlighted the privacy risks of exposing sensitive data via model predictions.Based on this foundation, Zhang et al. [2020] extended MIAs to more complex DNNs. They developed a methodology that involved learning a distributional prior from a publicly available auxiliary dataset, allowing for effective MIAs in a constrained latent space of the generator. Since these foundational studies, MIAs have received increased attention, particularly in the realm of high-dimensional image data. Recent research divides MIAs into three types based on the attacker's access to the model: white-box, black-box, and label-only settings. Each category represents varying levels of accessibility and potential risk, which informs the ongoing development of defensive strategies in this area.In the white-box setting, attackers have full access to the model, including its architecture and weights.The first white-box attack on DNNs, generative model inversion attack [Zhang et al., 2020] , utilized generative adversarial networks (GANs) to learn a distributional prior and optimize within the latent space. Subsequently, the knowledge-enriched distributional model inversion (KEDMI) attack [Chen et al., 2021] employed a specialized GAN with an advanced discriminator that leverages information from the target model. Wang et al. [2021a] proposed variational model inversion (VMI), which uses a probabilistic approach with a variational objective to ensure both diversity and accuracy. This evolution culminated in the Plug & Play Attack (PPA) [Struppek et al., 2022] , which enhances the recovery of images ranging from low to high resolution. Additionally, Nguyen et al. [2023] introduced logit maximization (LOM) loss as an alternative to the cross-entropy (CE) identity loss previously used in [Zhang et al., 2020 , Chen et al., 2021 , Wang et al., 2021a] and addressed issues of model overfitting with the model augmentation technique. Moreover, Yuan et al. [2023] advanced MIAs with the pseudo label-guided model inversion (PLG-MI), employing a conditional GAN (cGAN) and max-margin loss, and using pseudo-labels to decouple the search space for different classes.In the black-box setting, attackers lack direct access to the model's internals but can still query the target model a predetermined number of times to observe outputs for specific inputs, using this data to infer sensitive information indirectly. An et al.[2022] adopted a genetic search algorithm as an alternative to gradient descent in the black-box setting. While Han et al. [2023] developed the reinforcement learning-based model inversion (RLB-MI) algorithm, formulating the latent vector optimization as a Markov decision process (MDP) using reinforcement learning.The label-only setting poses the greatest challenge in model inversion attacks, where attackers only have access to the hard labels produced by the model, without any confidence scores or other related information. Kahla et al. [2022] tackled this issue using the boundary-repelling model inversion (BREP-MI) algorithm, which effectively utilizes the labels to simulate a gradient through max-margin loss. This method effectively navigates toward the concentrated areas of the target class in the data distribution by estimating model predictions across a conceptual sphere. Inspired by transfer learning (TL), Nguyen et al. [2024] proposed the label-only via knowledge transfer (LOKT) method for label-only model inversion, transferring knowledge from the target model to a target model-assisted ACGAN (T-ACGAN), effectively turning the label-only scenario into a white-box setting.This section introduces two measures used to evaluate the closeness between distributions P and Q: maximum mean discrepancy (MMD) and conditional transport (CT).Maximum Mean Discrepancy (MMD). Given two random variables X ∼ P, Y ∼ Q, the MMD measure is defined as follows:EQUATION)where F is a class of functions [Gretton et al., 2012a] . This class is often restricted to a unit ball in a reproducing kernel Hilbert space (RKHS) to facilitate analytical solutions [Gretton et al., 2012a] ,leading to the kernel-based MMD defined in the following,EQUATION)where k is a bounded kernel chosen based on the specific properties of the RKHS H k . The use of RKHS allows for the effective computation of MMD, leveraging kernel functions to measure the distance between the distributions in a high-dimensional feature space.Estimation of MMD. Given sample sets S X = {x i } n i=1 ∼ P and S Y = {y j } m j=1 ∼ Q, MMD (Eq. ( 12)) can be estimated with the U -statistic estimator, which is unbiased for MMD 2 [Gretton et al., 2012a] :EQUATION)where x i , x j ∈ S X and y i , y j ∈ S Y . This estimator efficiently computes the MMD by aggregating kernel evaluations over pairs of samples from both distributions.The CT measure provides a complementary approach to MMD by focusing on the transport cost between distributions. It consists of two components:EQUATIONwhere L X→Y and L Y →X represent the transport costs for the forward and backward CT, respectively. They are defined as follows:EQUATIONEQUATIONwhereΠ(Y | X) = e -d ψ (X,Y ) Q(Y ) e -d ψ (X,Y ) Q(Y )dY, andΠ(X | Y ) = e -d ψ (X,Y ) P (X) e -d ψ (X,Y ) P (X)dXrepresent the conditional distributions of Y given X and X given Y , respectively. Here, d ψ (X, Y ) is a function parameterized by ψ that measures the distance between X and Y , and c(x, y) is a cost function that measures the distance between the points x and y.Estimation of CT. Given sample sets S X = {x i } n i=1 ∼ P and S Y = {y j } m j=1 ∼ Q, the CT measure can be approximated as follows:EQUATION)The CT measure evaluates the cost of transporting samples from one distribution to another, providing a detailed assessment of how closely the distributions align. Initialize latent codes:Z = {z i | z i ∈ Z, i = 1, . . . , N }; 8:Obtain optimized latent codes Ẑ using Eq. (1); 9:# Step-2. Select high-quality pseudo-private samples 10:Select pseudo-private samples with top-K stable prediction scores: Initialize latent codes:D s ′ private = TopK{E[P M (y|T (x))] | x ∈ DZ = {z i | z i ∈ Z, i = 1, . . . , N }; 8:Obtain optimized latent codes Ẑ using Eq. (1); 9:Generate pseudo-private dataset:D s private = {x = G(ẑ) | ẑ ∈ Ẑ};This section presents the detailed algorithmic realization of the pseudo-private data guided model inversion (PPDG-MI) method. We describe two variants of the PPDG-MI method, each tailored for different MI scenarios. The first variant utilizes vanilla tuning (cf. Alg. 1), applicable for lowresolution MIAs where the adversary trains a GAN from scratch. The second variant employs nuanced point-wise or batch-wise tuning (cf. Alg. 2), suitable for high-resolution MIAs (i.e., PPA) where pre-trained generators are provided without access to the original training details. These methods are designed to enhance the density of pseudo-private samples under the prior distribution, thereby increasing the probability of sampling from the private data distribution. In our experiments with Plug & Play Attacks (PPA), we conducted all of them on Oracle Linux Server 8.9 using NVIDIA Ampere A100-80G GPUs. The hardware operated under CUDA 11.7, Python 3.9.18, and PyTorch 1.13.1. For MIAs targeting low-resolution facial recognition tasks, we executed these experiments on Ubuntu 20.04.4 LTS, equipped with NVIDIA GeForce RTX 3090 GPUs. This setup utilized CUDA 11.6, Python 3.7.12, and PyTorch 1.13.1.For experiments based on PPA, we train Inception-v3 evaluation models, following the code and guidelines available at the repository https://github.com/LukasStruppek/ Plug-and-Play-Attacks. For training details, please refer to [Struppek et al., 2022] . These models achieve test accuracies of 96.53% on FaceScrub and 94.87% on CelebA. In addition, We use the pretrained FaceNet [Schroff et al., 2015] from https://github.com/timesler/facenet-pytorch to compute the K-nearest neighbors distance, providing a measure of similarity between training samples and the reconstructed samples on the facial recognition tasks.In experiments involving classifiers trained on the 64 × 64 resolution CelebA dataset, we utilize an evaluation model available for download at repository https://github.com/ sutd-visual-computing-group/Re-thinking_MI. This model is built upon the face.evoLVe model [Wang et al., 2021b] , incorporating a modified ResNet50 backbone, and achieves a stated test accuracy of 95.88%. For training details, please refer to Zhang et al. [2020] .For training target models on 224 × 224 resolution CelebA and FaceScrub images, we adapt the training scripts and hyperparameters provided in the corresponding code repository and described in [Struppek et al., 2022] . The only training parameter we modify is the smoothing factor of the label smoothing loss. All models are trained for 100 epochs using the Adam optimizer [Kingma and Ba, 2015] , with an initial learning rate of 10 -3 and β = (0.9, 0.999), and a weight decay of 10 -3 . We reduce the learning rate by a factor of 0.1 after 75 and 90 epochs. For training target models on 64 × 64 resolution CelebA images, we use the training script provided at https://github.com/sutd-visual-computing-group/Re-thinking_MI. These models are trained for 50 epochs using the SGD optimizer with an initial learning rate of 10 -2 , a momentum term of 0.9, and a weight decay of 10 -4 . The batch size is set to 64. The learning rate decay schedule varies depending on the model; please refer to the training script for details.PPA consists of three stages: pre-attack latent code selection, MIA optimization, and final results selection. During pre-attack latent code selection, we choose 100 candidates for each target identity from a search space of 500 latent codes for both CelebA and FaceScrub. In the MIA optimization phase, we maintain an equal number of queries to the target model M to ensure a fair comparison for both the baseline attack and PPDG-MI. Thus, samples are optimized for 70 steps for both CelebA and FaceScrub in the baseline attack and 35 steps for each round of MIA in PPDG-MI. The final results selection stage is omitted. We target the first 100 identities and generate 100 samples per identity. Nonetheless, after some consideration, we believe that maintaining an equal number of queries to the target model M in each round of MIA for both PPDG-MI and the baseline attack may be a more reasonable approach to evaluate the performance of the proposed method.For low-resolution attacks, we generate 1, 000 samples per identity for CelebA and 2, 000 samples per identity for FFHQ, as training GANs typically require more samples. Specifically, for low-resolution white-box attacks, we target the first 100 identities for CelebA and the first 50 identities for FFHQ. Samples in GMI and LOM (GMI) are optimized for 2, 400 steps on the VGG16 target model and 1, 200 steps on face.evoLVe, while KEDMI and LOM (KEDMI) optimize samples for 1, 200 steps on both target models. For PLG-MI, we target 50 identities of the CelebA dataset, generating 200 samples per identity. Samples are optimized for 80 iterations on VGG16 and face.evoLVe target models. For black-box and label-only attacks, we limit our selection to the first 10 identities due to the extremely time-consuming nature of point-wise optimization in these settings. In the black-box attack (i.e., RLB-MI), samples undergo 10, 000 optimization steps in the baseline attack and in each round of PPDG-MI. For the label-only attack (BREP-MI), the maximum number of optimization steps is set to 1, 000 for both the baseline and each round of PPDG-MI.Due to the significant time required for MIAs, we perform a single attack against each target model.To reduce randomness, we generate at least 100 samples for each target class across various setups.Attack Accuracy (Attack Acc). Following Zhang et al. [2020] , we use an evaluation model (typically more powerful than the target model) trained on the target model's training data to identify reconstructed images (cf. Tab. 4). Intuitively, it can be viewed as a proxy for a human evaluator. Attack accuracy is calculated as the proportion of predictions matching the target identity; top-1 (Acc@1) and top-5 (Acc@5) attack accuracy is adopted.K-Nearest Neighbors Distance (KNN Dist). KNN Dist represents the l 2 distance between reconstructed images and the nearest samples from the target model's training data in the embedding space, indicating visual similarity between faces. For PPA [Struppek et al., 2022] , we use the penultimate layer of pre-trained FaceNet [Schroff et al., 2015] , whereas for other low-resolution MIAs, we adopt the penultimate layer of the evaluation model. Lower distances indicate a closer resemblance between the reconstructed samples and the training data.C.6 Experimental Details for Fig. 1 We present the experimental details for generating Fig. 1 . In the motivation-driven experiments, we evaluate the distribution discrepancy between commonly adopted private training datasets and public auxiliary datasets, and investigate the impact of distribution discrepancy on attack performance of MIAs using various public auxiliary datasets and target models. In Fig. 1 (a), we employ kernel-based two-sample tests to evaluate the distributional discrepancy between public auxiliary datasets (FFHQ and FaceScrub) and the private dataset (CelebA). A p-value of 0 signifies no statistical basis to reject the null hypothesis, indicating no discernible distribution difference between the datasets. Conversely, a p-value of 1 implies definitive evidence to reject the null hypothesis. We utilize a Gaussian Kernel-based test to calculate p-values at the feature level, extracting a 512-dimensional feature vector from each image using the face.evoLVe feature extractor. We analyze subsets of N feature vectors, with N varying from 1, 000 to 10, 000. Each subset is sampled 20 times from both datasets, and the results represent the average of these samples.In Figs. 1(b) and 1(c), we investigate the impact of distribution discrepancy on the attack performance of MIAs (LOM and KEDMI) using VGG16 as the target model and face.evoLVe as the evaluation model. The private dataset is CelebA, and the public auxiliary datasets are FFHQ and FaceScrub. We construct the proxy public auxiliary dataset by incrementally integrating the private data into the public auxiliary dataset, increasing the private data ratio by 20% at each interval, from 20% to 80%.To manage computational demands, we measure MMD across batches of 250 images each, using Gaussian kernels with a bandwidth of 1024. The final MMD value is the average result across all batches. Additional results involving IR152 and face.evoLVe target models are shown in Fig. 6 .The target model used in the toy example is a simple 3-layer multilayer perceptron (MLP). This MLP comprises two hidden layers, with the first containing 100 neurons and the second 200 neurons. Each hidden layer is followed by a rectified linear unit (ReLU) activation function to introduce non-linearity. The model is trained on a dataset sampled from a 3-class class-conditional Gaussian distribution, employing standard cross-entropy for loss calculation. Training is performed for 6, 000 epochs using standard stochastic gradient descent (SGD) with an initial learning rate of 0.5, enhanced by a linear warm-up schedule to increase the learning rate gradually.The public auxiliary dataset, distinct from the training dataset to prevent distribution overlap, is generated from a separate Gaussian distribution. The distributional prior is learned using a GAN, where both the generator and discriminator are structured as MLPs with three hidden layers of 400 neurons each. All layers in both models are followed by ReLU activations. The GAN is trained under the Wasserstein GAN with gradient penalty (WGAN-GP) [Gulrajani et al., 2017] framework to ensure a stable training process and reliable generation of new data samples.We employ the learned prior to guide the attack targeting Class 1. Initially, we randomly generate 1, 000 initial points and iteratively update them to maximize the model's prediction score for Class 1 through minimizing an identity loss (i.e., cross-entropy loss), following the approach proposed by Zhang et al. [2020] . We optimize this process using SGD with a learning rate of 0.1.To ensure a fair comparison between the baseline and PPDG-MI, we ensure that both attacks make the same number of queries to the model. Specifically, we query the model 1, 000 times in the baseline attack. For PPDG-MI, we first conduct one round of model inversion, making 500 queries to the target model to generate 1, 000 pseudo-private data points. Subsequently, we enhance the density of the pseudo-private data under the prior distribution by directly aligning the distribution between the prior distribution and the empirical pseudo-private data distribution by minimizing the distribution discrepancy between them. Then, we apply the fine-tuned prior to guide the second round of model inversion, making 500 queries to the model.The baseline attack results are shown in Fig. 7 (a) and the final attack results of PPDG-MI are shown in Fig. 7(d) . It is evident that, in comparison to the baseline where only a small fraction of the reconstructed samples fall within the high-density regions of the training data distribution, all reconstructed samples from PPDG-MI are located in these high-density regions. Quantitatively, the attack performance is evaluated by measuring two metrics: the average distance between the reconstructed samples and the mean of the target class distribution, and the proportion of reconstructed samples that lie in three standard deviations (3σ) of the mean. In comparison, the baseline achieves an average distance of 0.34 and attack accuracy of 22.60%, and PPDG-MI achieves an average distance of 0.04 and attack accuracy of 100.00%. conducting MIA on target identities 1-100, recovering 100 samples per identity, resulting in a total of 10,000 samples in D s private . We then increase the density around D s private under the prior distribution P(X prior ) by fine-tuning G(•; θ) to align with the empirical distribution of D s private , using the CT measure. The results are shown in Tab. 5, where we observe a dramatic decrease in MI performance after fine-tuning the generator. This indicates that direct distribution alignment is less effective for higher-dimensional image data, as it disrupts the generator's manifold. Therefore, we need to employ a nuanced tuning strategy with smaller perturbations to the image manifold.Comparison with white-box MIAs in the low-resolution setting. In this experiment, we utilize the face.evoLVe as the target model. The results are presented in Tab. 6, where PPDG-vanilla consistently outperforms various baseline white-box attacks, offering notable improvements in both attack accuracy and KNN distance metrics across two public auxiliary datasets, CelebA and FFHQ. For instance, in the LOM (GMI) setup, adding PPDG-vanilla results in an increase in top-1 attack accuracy from 68.09% to 71.39% for CelebA, and top-5 attack accuracy from 87.31% to 88.12%. Additionally, there is a reduction in KNN distance from 1417.23 to 1385.10 for CelebA. Similarly, significant improvements are observed for FFHQ, where attack accuracy increases and KNN distances decrease, indicating enhanced data density around pseudo-private samples. However, there is a failure case involving the setup where the target model is face.evoLVe trained on CelebA, with the public dataset as FFHQ. We attribute this failure to multiple factors, which are analyzed in Appx. E. Generally, these results suggest that even with substantial distribution shifts between the private dataset CelebA and the public dataset FFHQ, our principled vanilla fine-tuning strategy, which retains the original GAN training objectives, can effectively improve MIA performance.Comparison with PLG-MI in the low-resolution setting. In this setting, we use the state-of-the-art white-box attack PLG-MI [Yuan et al., 2023] as the baseline for comparison. PLG-MI leverages the target model to generate pseudo-labels for public auxiliary datasets, enabling the training of a cGAN (conditional GAN) on this labeled data. This further decouples the search space across different classes. However, the poor visual quality of samples generated by PLG-MI results in failure cases (refer to Appx. E), limiting the effectiveness of PPDG-MI. Therefore, we manually select 50 identities with high-quality samples from 100 identities and present the quantitative results in Tab. 7. Integrating PPDG-vanilla significantly improves MI performance. This improvement can be attributed to the robust selection strategy, which identifies high-quality samples from all pseudo-private data, along with fine-tuning using the original GAN training objectives to enhance density around high-quality pseudo-private samples. These results indicate that even with a different GAN prior (i.e., cGAN), PPDG-MI shows effectiveness with strong compatibility and outperforms SOTA white-box attack.Comparison with Black-box MIAs in the low-resolution setting. In this setting, we use the SOTA RLB-MI [Han et al., 2023] as the baseline for comparison, the results are shown in the upper part of Tab. 8. Han et al. [2023] formulate the latent space search problem as a Markov decision process and solve it using reinforcement learning, which requires tens of thousands of iterations for optimization, leading to inefficiency and a low probability of sampling representative samples. Integrating PPDGvanilla significantly improves MI performance. This improvement can be attributed to the fine-tuning of the GAN, which effectively enhances the density around the pseudo-private samples, thereby increasing the probability of sampling representative samples. Our experiments demonstrate that PPDG-vanilla boosts the top-1 attack accuracy from 38.50% to 61.60% and the top-5 attack accuracy from 65.10% to 85.30% in the black-box setting, with a substantial reduction in KNN distance from 1431.93 to 1337.09. Additionally, the running time ratio indicates a minimal overhead of just 0.02. These results highlight the effectiveness of our method in improving MI performance in the black-box setting with minimal additional computational cost.Comparison with Label-only MIAs in the low-resolution setting. In this setting, we use the state-of-the-art BREP-MI [Kahla et al., 2022] as the baseline for comparison. The quantitative results are presented in the upper part of Tab. 8. Kahla et al. [2022] introduces a boundary-repelling algorithm to search for representative samples. This algorithm estimates the direction towards the target class's centroid using the predicted labels of the target model over a sphere. Typically, under a radius threshold where the gradient estimator still works reliably, a larger sphere radius indicates a higher-likelihood region around the sphere's centroid. A qualitative illustration of the progression of the reconstructed images towards the actual training images is depicted in Fig. 8 . The upper part of Fig. 8 shows the results of BREP-MI, while the lower part shows the results after integrating BREP-MI with PPDG-vanilla. It is evident that with PPDG-vanilla, BREP-MI can find more representative samples at a smaller radius. This demonstrates that our method effectively increases the density of these regions, leading to faster convergence of the search process.Attacks against SOTA model inversion defense methods. In the low-resolution setting, we evaluate SOTA white-box attacks LOM (GMI), KEDMI, and LOM (KEDMI) against SOTA model inversion defense methods, such as BiDO-HSIC [Peng et al., 2022] and NegLS [Struppek et al., 2024] . The results, summarized in Tab. 9 demonstrate that our PPDG-MI consistently outperforms baseline against BiDO-HSIC and NegLS. For instance, when using NegLS as the defense, PPDG-vanilla significantly enhances both the top-1 attack accuracy and the KNN distance metrics. For the LOM (GMI) attack, PPDG-vanilla improves the Acc@1 from 25.40% to 45.44%, and for KEDMI, it increases it from 27.15% to 30.82%. Additionally, PPDG-vanilla shows improvements compared with LOM (KEDMI) by enhancing Acc@1 from 64.73% to 68.94%, and decreasing KNN Distance from 1320.38 to 1308.09. The enhanced MI performance on the target model trained with SOTA defense methods further underscores the effectiveness of PPDG-MI. The choice of the number K of high-quality pseudo-private samples affects the tuning intensity of the generator G, the informativeness of the empirical local distribution, and the computational cost. In Tab. 10, we illustrate the MI performance for different choices of K. As K increases, MI performance initially improves, reaching an optimal point before it starts to decline. This trend can be attributed to two main factors. First, A higher number of samples provides more detailed information about the local data distribution, allowing the generator G to better capture the underlying characteristics of the data. This leads to an initial improvement in MI performance. Second, As K becomes larger, the generator requires more extensive tuning to accommodate the additional samples. This can result in significant changes to the manifold, potentially disrupting the learned data structure and decreasing MI performance. Additionally, larger K values substantially increase computational cost. Thus, considering all these factors, we highlight the K values chosen for various fine-tuning methods in Tab. 10.Fine-tune different layers. Given that the synthesis network in the StyleGAN generator consists of 18 layers-two for each resolution 4 2 -1024 2 -with earlier layers controlling higher-level features (e.g., general hairstyle, face shape), and later layers controlling more fine-grained features (e.g., finer hairstyle details), we investigate the effect of tuning subsets of these layers. In Tab. 11, we present the results of incrementally adding layers for fine-tuning. Our study reveals that tuning layers with spatial resolutions from 4 2 -128 2 achieves comparable results to tuning all layers, i.e., spatial resolutions from 4 2 -1024 2 . This finding suggests that successful MIAs rely more on inferences about high-level features (e.g., face shape) rather than fine-grained details, aligning with the main goals of MIAs. In this section, we present all ablation studies on MIA in the low-resolution setting to further explore PPDG-MI. The target model, VGG16, and the evaluation model, face.evoLVe, are both trained on CelebA private dataset. GANs are also trained from scratch on CelebA public dataset.Unless otherwise specified, we use KEDMI as the attack method and perform one round of GAN fine-tuning. The size of the pseudo-private dataset is 1, 000, and we fine-tune the GAN for 10 epochs.Iterative fine-tuning. Fine-tuning the GAN with generated pseudo-private data increases the probability of sampling data with characteristics closer to the actual private training data. We examine the impact of iterative fine-tuning on MI performance through experiments with different fine-tuning rounds. We use LOM (GMI) as the attack method and present the results in Tab. 12. The results show that MI performance improves with more fine-tuning rounds, suggesting that pseudo-private data increasingly approximates private training data. However, after the third round, further fine-tuning does not further improve MI performance as observed in earlier rounds. This decline could be due to excessive fine-tuning, which may distort the image manifold and degrade MI performance. Impact of the size of the pseudo-private dataset in GAN fine-tuning. In iterative GAN fine-tuning, the pseudo-private data generated in each round can potentially increase the probability of sampling data with characteristics similar to the real private data in subsequent rounds. This process highlights the importance of determining an appropriate size for the pseudo-private dataset to achieve improved MI performance with an acceptable level of fine-tuning overhead. We investigate how MI performance is affected by varying the size of the pseudo-private dataset, with the results presented in Tab. 13. The results indicate a trend where the MI performance initially improves and then degrades. This suggests that increasing the size of the pseudo-private dataset up to a certain point (e.g., 3, 000) can enhance MI performance. The observed decline in performance could be attributed to the large amount of pseudo-data adopted, which increases the intensity level of fine-tuning and thus disrupts the image manifold. Impact of the number of epochs in GAN finetuning. During iterative GAN fine-tuning, effectively utilizing pseudo-private data while reducing computational overhead is essential. Therefore, the number of tuning epochs is a crucial hyperparameter. This experiment examines the impact of fine-tuning epochs on MI performance. Tab. 14 shows that MI performance consistently improves as the number of GAN fine-tuning epochs increases, indicating that the GAN effectively learns and utilizes pseudo-private data. However, while MI performance improves, the computational overhead (i.e., fine-tuning time for each identity) increases linearly with the number of epochs.Table 15 : Ablation study on identity-wise finetuning vs. multi-identity fine-tuning.Fine-tuning method Acc@1↑ Acc@5↑ Comparison of identity-wise fine-tuning vs. multi-identity fine-tuning. Compared to finetuning the GAN using a single identity, which is specific to one identity and increases overhead, fine-tuning with multiple identities as a whole can significantly reduce computational costs. Thus, we aim to investigate the MI performance of finetuning the GAN based on both single-identity and multi-identity approaches, respectively. The results, presented in Tab. 15, indicate that the MI performance of multi-identity fine-tuning decreases by 4% compared to single-identity fine-tuning. Despite the reduction in computational overhead, fine-tuning the GAN using multiple identities results in poorer MI performance. This is because fine-tuning with multiple identities induces more intensive changes to the generator.In this section, we provide qualitative evidence to demonstrate the effectiveness of our proposed PPDG-MI. Results for high-resolution settings are illustrated in Figs. 9 and 10. Specifically, Fig. 9 presents a visual comparison of reconstructed samples for the first ten identities from three target models-ResNet-18, DenseNet-121, and ResNeSt-50-trained on the CelebA, using GANs pretrained on FFHQ. Fig. 10 shows a similar comparison for the same target models trained on the FaceScrub, also using GANs pre-trained on FFHQ. For low-resolution settings, the results are shown in Figs. 11 and 12. Fig. 11 illustrates reconstructed samples for the first ten identities from VGG16 trained on the CelebA private dataset, using GANs trained from scratch on the CelebA public dataset. Fig. 12 depicts reconstructed samples for the first ten identities from VGG16 trained on the CelebA private dataset, using GANs trained from scratch on the FFHQ public dataset.Scope and Applicability of Model Inversion Attacks. Model inversion attacks (MIAs) have become a critical area of research in assessing privacy risks, especially for discriminative models that handle sensitive data. While MIAs can be effective in certain data types, tasks, and specific contexts, they also face significant limitations, particularly in cases where the data lacks clear identity markers.A notable limitation arises in their applicability to biomedical imaging, where MIAs face distinct challenges. For instance, unlike identity-rich data like facial images, chest X-rays generally lack identifiable features that can be easily linked to individuals. This absence of clear personal markers complicates privacy risk evaluation, making it harder to assess the impact of MIAs.Moreover, even in experimental settings using datasets like ChestXray8 [Wang et al., 2017] , where the primary goal is classification tasks such as diagnosing medical conditions, challenges persist. One key concern is whether a reconstructed chest X-ray would represent a generic "average" image or contain identifiable features unique to specific training samples. This issue arises from the inherent complexity of chest X-rays, which often require specialized medical expertise for accurate interpretation.In summary, MIAs are more effective in domains where data includes explicit identity markers, leading to higher privacy risks as reconstructed images can closely resemble individuals from the training data. However, in fields like biomedical imaging, where data lacks evident identity characteristics, privacy risks are harder to quantify but remain an important area of concern.Broader Impacts. In this paper, we propose a novel model inversion pipeline to enhance the performance of generative model inversion attacks (MIAs), potentially providing new insights and paving the way for future research. From a social perspective, our research on MIAs reveals significant privacy vulnerabilities in machine learning models that, if misused, could compromise sensitive training data. By revealing these risks, we aim to raise awareness and drive the development of robust defense mechanisms and privacy-preserving algorithms that are crucial for enhancing the security of machine learning systems. Overall, although our findings could be misused, the benefits of raising awareness and improving security practices in machine learning systems far outweigh these concerns.Failure Case Analysis. To better assess the feasibility of our proposed method, we closely examined the reconstructed samples. In high-resolution scenarios, we observed that with a single fine-tuning round, PPDG-MI exhibited minimal failures. However, with continued fine-tuning, the visual quality of certain identities significantly deteriorated, indicating substantial alterations in the generator's manifold. Notably, these reconstructed samples still performed well on standard metrics despite poor visual quality, suggesting potential overfitting to these metrics. This finding underscores a limitation in current metrics, such as attack accuracy, which may lack robustness in such cases. Developing more effective and resilient evaluation metrics is an important direction for future research.In low-resolution scenarios, we observed a decline in visual quality and performance on standard metrics for certain identities after a single round of attack, particularly in the case of LOM (KEDMI) and PLG-MI. We hypothesized that this failure stems from the baseline attack that generates samples with poor visual quality and insufficiently representative features of the target identities. These lowquality samples negatively impact the quality of the samples produced by the fine-tuned generator.With each subsequent fine-tuning round, these negative effects accumulated, progressively degrading the generator's manifold and leading to poorer visualization and MI performance. These findings indicate that the generator's capabilities may be insufficient to produce high-quality samples that positively influence subsequent rounds of GAN fine-tuning, thereby impacting the effectiveness of PPDG-MI. Therefore, utilizing more advanced generators represents a potential direction for better demonstrating the advancements of PPDG-MI.Limitations of PPDG-MI. Conducting model inversion attacks is time-consuming, and the iterative fine-tuning in our proposed PPDG-MI adds further overhead, as shown by the running time ratios between the fine-tuning phase and model inversion phase in our experiments. However, our proposed prior distribution tuning methodology provides a promising solution to mitigate the fundamental distribution discrepancies between private and prior distributions. We hope our work will inspire future research to develop more efficient and effective tuning methods. Justification: Our well-organized source code, complete with a detailed README, is available at: https://github.com/tmlr-group/PPDG-MI.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 


Decision: The paper introduces a method to enhancing model inversion attacks (MIAs), which aim to reconstruct class characteristics from a trained classifier. Typically, MIAs rely on training image priors, such as GANs, on public data that differ in distribution from the target model's training data. This distributional discrepancy reduces the effectiveness of MIAs in accurately reconstructing features of the target class. To address this issue, the authors propose pseudo-private data-guided MIAs. The proposed method initially conducts existing MIAs to gather a set of attack samples that reveal features of the target classes. A subset of high-quality samples is selected based on robust prediction scores under the target model. The GAN's generator is then fine-tuned on this subset to increase the sampling density around the pseudo-private data. Subsequent attacks using the updated generator show a significant improvement in attack results across various attacks and settings. However, the reviewers are concerned about the novelty of the work. In addition, the assumptions on the pseudo data distribution are strong, which could lead to bias and model collapse. The paper should further improve the quality based on the reviews.

Reviews: review: paper_summary: paper_summary: The paper proposes a novel plug-and-play method for current state-of-the-art MI methods to enhance their performance and mitigate the challenge of distribution discrepancy. This method first conducts a round of MI attack to acquire pseudo-private data and then utilizes the data to fine-tune the generative prior following certain strategy. The experimental results show that the proposed method enhances the attacking capability of existing MI methods to some extent.
strength_weakness: strength_weakness: 1. The current state-of-the-art white-box MI method PLGMI [1] should be evaluated.
2. There is no experiment for evaluation on any model inversion defenses. Related experiments are expected to demonstrate how the proposed technique impacts the resistance ability of MI attacks against defense strategies.
3. The middle panel in Fig 5 seems wrong. The attack accuracy of random samples is higher than the one of high-quality samples.
4. In Section 4.3, the citation for Fig 5 appears to be Fig 4.3.

[1]Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang. Pseudo label-guided model inversion attack via conditional generative adversarial network. In AAAI, 2023.
[2]Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correia, Antonia Adler, and Kristian Kersting. Plug & play attacks: Towards robust and flexible model inversion attacks. In ICML, 382 2022.
questions: questions: 1. In Section C.4, why is the final results selection stage removed? This technique is critical to PPA [2]. Omitting this stage might leads to the degraded performance of PPA in the Experiment Section.
2. In Section C.4, why the parameters of pre-attack latent code selection stage are significantly lower than the original paper (200 candidates from a search space of 2000/5000 latent codes). This also contributes to the bad performance of PPA [2] in the Experiment Section.
limitations: limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work.
paper_summary: paper_summary: It is well known that deep learning models are susceptible to model-inversion attacks, which is to say that they can be probed to reveal their training data. The authors design a more powerful method of attack by increasing the density of their prior using “pseudo-private data”, they can increase the probability of sampling actual private data.
strength_weakness: strength_weakness: - There are a number of typographic errors (e.g. “vallina”)
- A brief related work section which situates their contribution in the main body would have been appreciated
- There is verbatim repetition in the problem setup.
- They claim that “all state-of-the-art generative MIAs” are limited due to the utilization of a fixed prior during inversion. This bold claim is not backed up.
- Some of the symbols are not properly defined  (e.g. \lambda is undefined)
questions: questions: - What are the real-world settings that this threat model could be observed with this work?
- How might this approach extend to diffusion-based models?
limitations: limitations: The paper makes the limitations and ethics issues with their work clear.
paper_summary: paper_summary: This work introduces a novel application of a generative model inversion attack utilising dynamic (pseudo-private) priors, improving the existing results of MI.
strength_weakness: strength_weakness: So while the idea is straightforward and makes a lot of sense in the chosen setting, the reliance on priors in MI is a) hardly novel [1,2] and b) comes with caveats not covered in the paper. I appreciate that there are various formulations of MI in literature and the one used in this work is generative MI specific, but the main principle of relying on a dynamic prior has previously been covered in the context of MIs.

With respect to point b): if the generative model is conditioned on the pseudo-generations, I suspect this can lead to the issues of bias and/or mode collapse. What I mean here is that there is no guarantee that the reconstructions obtained as part of the pseudo-generation process would resemble the 'actual' training data. This is problem number one, but lets assume the images generated in this step are valid samples from the training dataset. The fact that you are able to generate them means (similarly to the results of the Secret revealer discussed in this work) that they were 'easier' samples with respect to MI vulnerability. And currently, there is no reason for me to believe that this would help you reconstruct the more 'difficult' or informative samples (often on the tails of the distribution [3]). So to simplify: I am not convinced that by leveraging the pseudo-private samples you would be able to improve the attack results meaningfully, as now you would condition your reconstructions to those that are of high similarity to the pseudo-private ones (and I would argue, making it close to impossible to now reconstruct the samples that do not fall under this category, which you could have done with a frozen decoder which would not discriminate between these).

One step which is mentioned as 'optional' is selection of samples that are meaningful. What does this imply? To me this is a very important (and an incredibly challenging) task, which was not given enough discussion in the manuscript, given that you are conditioning your further inversions based on the quality of the pseudo-generated ones. How do you effectively measure it to avoid collapsing into the region of pseudo-reconstructions alone (i.e. similarly to my comments above, how do you measure and use this quantity to encourage MI diversity)?

While the focus of the work is clearly computer vision, it does limit the impact of the findings to these specific settings, making this more of an incremental improvement than a novel paradigm of MI, which the paper positions itself to be.

Minor: the convention is typically MI for model inversion, rather than MIA (which is often used for membership inference), making it a bit more difficult to follow.

[1] - Hatamizadeh, Ali, et al. "Do gradient inversion attacks make federated learning unsafe?." IEEE Transactions on Medical Imaging (2023).
[2] - Usynin, Dmitrii, Daniel Rueckert, and Georgios Kaissis. "Beyond gradients: Exploiting adversarial priors in model inversion attacks." ACM Transactions on Privacy and Security 26.3 (2023): 1-30.
[3] - Feldman, Vitaly. "Does learning require memorization? a short tale about a long tail." Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 2020.
questions: questions: Could you explain why are you making a connection to adversarial samples in the background? Sure they possess the features you describe, but how is this relevant for MI discussion?

The difference between the low/high dimensional reconstructions sounds rather artificial (it seems this is just pre-trained vs FT GAN), is it really necessary to separate these? Are there are other differences making this separation clearer?
limitations: limitations: As per weaknesses above: to me this seems like an incremental improvement in a relatively niche area, which is more suited to a conference specializing in attacks on ML or PPML, as these results are constrained to specific ML settings.
paper_summary: paper_summary: The paper introduces a novel method to enhancing model inversion attacks (MIAs), which aim to reconstruct class characteristics from a trained classifier. Typically, MIAs rely on training image priors, such as GANs, on public data that differ in distribution from the target model's training data. This distributional discrepancy reduces the effectiveness of MIAs in accurately reconstructing features of the target class. To address this issue, the paper presents pseudo-private data-guided MIAs. The proposed method initially conducts existing MIAs to gather a set of attack samples that reveal features of the target classes. A subset of high-quality samples is selected based on robust prediction scores under the target model. The GAN's generator is then fine-tuned on this subset to increase the sampling density around the pseudo-private data. Subsequent attacks using the updated generator show a significant improvement in attack results across various attacks and settings.
strength_weakness: strength_weakness: - The evaluation is missing some aspects. For the investigated attacks, it would be interesting to compare the results to PLG-MI [1], which takes a similar direction (training a conditional GAN on pseudo-labeled data to decouple the latent space for different target classes) and shows strong attack results. 
- Also, the evaluation includes no investigation of existing defense methods like MID [2], BiDO [3], and negative label smoothing [4]. For MID and BiDO, existing attacks have already (partly) broken the defenses; see [4,5]. However, the authors [4] argue that negative label smoothing limits the information provided by a model's prediction scores, which, in turn, might affect the subset selection process used by the pseudo-private data-guided MIAs proposed method. It would be interesting to see if the proposed attack improvement also breaks this type of defense.
- The evaluation focuses on metrics computed on an evaluation model trained on the same data as the target model has been. While this is a common approach in MIA literature, it might provide some misleading results. For example, given that the attacks might reconstruct adversarial features instead of actual robust and characteristic class features, the fine-tuned GAN might also generate images containing adversarial features. Since we know from the literature that adversarial features can be transferable, the attack results could similarly fool the evaluation model and, therefore, overestimate an attack's success. Therefore, it might be reasonable to include other metrics, e.g., the FaceNet distance used in the PPA paper. Another option to measure the attack's success could be the information extraction score introduced in [4]. Using the FaceNet distance in addition to the Attack Accuracy in Fig. 5 would further support the paper's claims.
- Giving more intuition on why MMD and CT are used instead of, e.g., a KL Divergence would improve the paper. Similarly, why use LPIPS as a similarity metric instead of another type of metric, e.g., FaceNet distance?

Minor remarks:
- The term "MMD" used in the caption of Fig. 1 should be introduced before referring to this figure. Also, "MI" (L39) is not formally introduced. It probably stands for "model inversion". However only "MIA" is introduced, which might lead to confusion of the reader.
- Section 2.2: There should be some motivation for why MMD and CT are introduced here. It makes sense later in the paper, but these concepts appear a bit surprising at the point of the background section. 1-2 introduction sentences would support the motivation here. Also, intuition about what MMD and CT are measuring and how they differ supports understanding these concepts.
- L132: From my understanding, the "linear interpolation" between both distributions means that the sample share $\alpha$ is sampled from X_prior and $(1-\alpha)$ is sampled from X_private. This should be clarified in the writing.
- Fig. 3: The font size in the legend is too small and should be increased. Also, the size of the data points should be increased, particularly the red ones in the final image. Currently, those data points are hard to see in (b).
- To make the approach more clear, think about adding "Step-4: Repeating the attack with the updated generator". Currently, the approach only describes the steps of fine-tuning the generator, and it might not be clear to all readers that the attacks will be repeated after this step.
- L296: The figure reference is broken, it currently states "Fig. 4.3" instead of "Fig. 5".

Overall, I liked the paper and the mentioned weaknesses can probably be addressed during the rebuttal. Therefore, I encourage the authors to participate in the discussion and will increase my score if the mentioned weaknesses are sufficiently addressed.

[1] Yuan et al. "Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network". AAAI 2023
[2] Wang et al. "Improving Robustness to Model Inversion Attacks via Mutual Information Regularization". AAAI 2021
[3] Peng et al. "Bilateral Dependency Optimization: Defending Against Model-inversion Attacks". KDD 2022
[4] Struppek et al. "Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks. ICLR 2024
[5] Nguyen et al. "Re-thinking Model Inversion Attacks Against Deep Neural Networks". CVPR 2023
questions: questions: - There is the risk of a mode collapse in fine-tuning the generator. Does the approach require regularization to avoid this failure case, or is the method already robust enough to avoid it?
- Regarding the Ratio metric: From my intuition, the ratio should always be > 2, because the attacks take at least twice the time compared to the baseline (1x runnning the baseline + 1x repeating the attack with the adjusted generator). A more detailed description of this metric would improve the understanding.
limitations: limitations: Some limitations are discussed in Appx. E. However, I think the limitation section could include some failure cases and additional limitations of the method. In which settings did it fail? Are there any additional requirements for the method to work?

While the paper proposes a novel type of attack, I do not think there will be negative societal impact, given that there already exists a long list of publicly available MIA literature and implementations.

Comments: title: Gentle reminder: discussion period concludes today
A polite reminder about the upcoming discussion deadline
Response to Reviewer qx3K (2/2)
Response to Reviewer qx3K (1/2)
Would you mind checking our responses and confirming whether you have any further questions?
Response to the rebuttal
Thank you for your constructive comments
Remaining Responses to Reviewer eDgc (2/2)
Remaining Responses to Reviewer eDgc (1/2)
Remaining Responses to Reviewer qx3K
comment: Dear Reviewer qx3K,

Thank you for your time and valuable comments. We understand you may be quite busy, but the discussion deadline is rapidly approaching.

Could you kindly review our response and let us know if you have any further questions?

Thank you for your attention.

Best regards,

Authors of Submission #7116
Dear Reviewer CEeM,

Thank you once again for taking the time to review our paper and for your ongoing support. Your thoughtful feedback has been very valuable in improving our work.

Sincerely,

Authors of Submission #7116
I thank the authors for the clarifications / adjustments and have update my score.
Dear Reviewer qx3K,

As the discussion deadline is approaching, we would like to have a detailed discussion with you to address any new or remaining concerns you may have. We have already provided responses to your previous remaining concerns and would appreciate knowing if they have resolved your issues.

We look forward to your prompt feedback.

Best regards,

Authors of Submission #7116
Dear Reviewer 4Zsf,

Thank you for taking the time to review our rebuttal. We are glad to hear that our clarifications and additional results have positively influenced your perspective on our work.

Best regards,

Authors of Submission #7116
The experimental results are positive. Thanks to the authors for addressing all my concerns and I would like to increase my score.
> Re: The issue here is that determining the privacy violation of data which 'closely resembles' the training data is not straightforward.

Regarding the applicability of our methods to 'other domains,' our intention was not to imply that our approach is directly transferable across all data domains. Instead, we aimed to suggest that the use of dynamic priors could be advantageous in other tasks where a frozen generator are utilized, such as in image editing [r10]. We hope this clarification addresses the concerns raised.

We appreciate the reviewer’s insightful comments regarding the application of generative model inversion attacks, particularly in biomedical imaging domains such as chest X-rays. We agree that privacy concerns in these domains are more challenging to assess due to the less distinctive nature of chest X-rays compared to identity-bearing data like facial images.

We emphasize that **while model inversion attacks are not universally applicable for evaluating all types of privacy leakage, they remain a crucial direction for understanding and measuring privacy risks**, especially given that model inversion attacks can potentially reconstruct training data with just access to a well-trained classifier. This has been extensively studied in the literature [r1-r3, r6-r9] and is recognized as a trending area of research due to its significant implications, particularly in applications like facial recognition, where the attacks can achieve highly accurate reconstructions. 

To further clarify the specific case you mentioned:

Regarding the first point—whether a reconstructed sample corresponds to a specific patient—it is indeed true that chest X-rays typically lack strong identity markers that can be directly linked to an individual. In the context of model inversion attacks, it is uncommon to associate a specific chest X-ray with a particular patient, as chest X-rays do not possess distinctive identity characteristics like facial images. Consequently, experimental settings in this field often focus on classification problems across multiple conditions, as seen in datasets like ChestXray8.

Regarding the second point— whether a reconstructed sample is merely an "average-looking chest X-ray" or contains identifiable features from the training data, we acknowledge the validity of this concern. This challenge is further compounded by the inherent complexity of chest X-ray images, which are difficult to interpret accurately without specialized expertise. This is why datasets involving facial images are often preferred in studying model inversion attacks, as they provide clearer identity markers, making the risks and outcomes of such attacks more discernible.

However, we emphasize that **this challenge is inherent to generative model inversion attacks in general and is not a specific weakness of the approach presented in our work**. While the complexity of certain data types, such as chest X-rays, complicates privacy violation assessments, it does not diminish the importance of studying model inversion attacks. These attacks remain a significant concern, particularly in domains where data have clearer identity markers.

To address these concerns, **we plan to add a discussion in the manuscript about the scope and applicability of model inversion attacks**, helping readers better understand the contexts in which these methods are most effective for evaluating privacy leakage.

Lastly, we hope our novel contribution has been clearly articulated in our initial response. If you have any further questions or need additional clarification, please feel free to let us know. We would be happy to discuss and clarify any points further. Thank you again for your thoughtful feedback; we hope this response adequately addresses your concerns.

[r6] Chen et al. "Knowledge-Enriched Distributional Model Inversion Attacks." In ICCV, 2021.

[r7] Kahla et al. "Label-Only Model Inversion Attacks via Boundary Repulsion." In CVPR, 2022.

[r8] Han et al. "Reinforcement Learning-Based Black-Box Model Inversion Attacks." In CVPR, 2023.

[r9] Nguyen et al. "Label-Only Model Inversion Attacks via Knowledge Transfer." In NeurIPS, 2024.

[r10] Abdal et al. "Image2stylegan: How to embed images into the stylegan latent space?" In ICCV, 2019.
We're glad to have addressed some of your concerns, and we would like to take this opportunity to further address the remaining points.

> Re: The dynamic prior.

We appreciate your feedback and **understand that the term 'dynamic prior' in our work may have led to some misunderstanding**. To address this, we will revise the manuscript to use a more precise description of our approach. Specifically, we will clarify that we fine-tune the generator $\mathrm{G}$, which represents the prior, during the model inversion process.

The key difference we emphasize is that, in our approach, the generator is continuously fine-tuned throughout the model inversion process. In contrast, previous works such as [r1-r3, r6-r9] select a prior based on a heuristic but do not fine-tune the generator during the model inversion process. This continuous fine-tuning process distinguishes our method. To further clarify the distinction between our approach and previous generative model inversion attacks [r1-r3, r6-r9], we provide the following step-by-step explanation.

In previous generative model inversion attacks, the generative model $\mathrm{G}$ remains **frozen** throughout the model inversion process:
- Initialize latent codes: $\mathbf{Z}=\\{\mathbf{z\_i} \mid \mathbf{z}\_i \in \mathcal{Z}, i = 1,\ldots, N\\}$;
- Obtain optimized latent codes: $\hat{\mathbf{Z}}=\\{\hat{\mathbf{z}} = \text{argmin}~\mathcal{L}\_{\text{id}}(\mathbf{z};y,\mathrm{M}, \mathrm{G}) + \lambda \mathcal{L}\_{\text{prior}}(\mathbf{z};\mathrm{G},\mathrm{D}) \mid  \mathbf{z} \in \mathbf{Z}\\}$;
- Generate recovered samples: $\mathcal{D}_{\text{private}}^{\text{s}} = \\{\hat{\mathbf{x}} = \mathrm{G}(\hat{\mathbf{z}}) \mid \hat{\mathbf{z}} \in \hat{\mathbf{Z}} \\}$.

In contrast, our pseudo-private data guided model inversion attacks (refer to Section 3.2) involve a more dynamic process that includes the following **three iterative steps**:
- Generate pseudo-private dataset with generative model inversion attacks (**frozen** $\mathrm{G}$): $\mathcal{D}_{\text{private}}^{\text{s}}$;
- Select high-quality pseudo-private dataset from $\mathcal{D}\_{\text{private}}^{\text{s}}$ (**frozen** $\mathrm{G}$): $\mathcal{D}\_{\text{private}}^{\text{s}'}$;
- Density enhancement around high-quality pseudo-private data $\mathcal{D}\_{\text{private}}^{\text{s}'}$ (**unfrozen** $\mathrm{G}$): $\mathrm{G}, \mathrm{D} \leftarrow \texttt{Fine-tune}(\mathrm{G}, \mathrm{D}, \mathcal{D}\_{\text{private}}^{\text{s}'})$.

As observed in our approach, the generative model $\mathrm{G}$ is not static but is fine-tuned iteratively based on the selected pseudo-private data. This differs from the static nature of the generator $\mathrm{G}$ in the previous generative model inversion attacks.
Dear Reviewer 4Zsf,

Thanks very much for your time and constructive comments.

Would you mind checking our responses and confirming whether you have any further questions?
Any comments and discussions are welcome!

Thanks for your attention and best regards.

Authors of Submission #7116
I would like to thank the authors for their comprehensive response. While some issues were clarified, I am still not convinced about a number of points.

> our work is the first to introduce the use of a dynamic prior, whereas previous studies [r1, r2, r3] in generative MIAs have relied on a fixed generative prior.

I understand the message the authors are trying to convey, but I would argue that both the r1 and the r2 pick the prior dynamically (i.e. there is a certain heuristic used to select a prior to optimise for, so its not 'static' in the sense the authors seem to suggest). But this is more of a discussion on the setting and the context, not this work itself.

> generative model inversion attacks have demonstrated the ability to recover samples that closely resemble actual training data

This specific point I do not disagree with per se, but the issue here is that determining the privacy violation of data which 'closely resembles' the training data is not straightforward. Were you to run this attack in, lets say, a biomedical domain - you may be able to reconstruct a chest X-ray scan learnt by the model. But you are likely unable to determine a) if there is a specific patient this scan corresponds to, b) if there are any features which actually belong to the training record that you attempted to reconstruct and are not just an 'average looking chest X-ray'. The same logic, in my view, applies to the setting discussed in this work: while the idea of using priors (and particularly dynamic priors) does make sense in some settings, where potential over-conditioning is not a major issue (e.g. one may argue that many facial features may be similar to one another), this is not the case universally. And while on its own this does not in any way diminish the contributions of this work, I do not believe that this is something that can be easily applied in other domains.

The aforementioned issue could have been eliminated should you demonstrate superior performance of your method in a distinctly different setting, for instance (even within the imaging modality, but ideally beyond that). Therefore I am inclined to keep my score unchanged: I am still not convinced by the scope of the novel contribution proposed in this work.
We would like to thank you again for your time and efforts in reviewing our paper, as well as for your generous support of our work. Your insightful comments have greatly improved the quality of the paper.

Sincerely,

The Anonymous Authors
Dear authors,

after reading all reviews and responses, I decided to slightly increase my score since my main weaknesses (missing evaluations, baselines, defenses, clarifications) have been addressed. I think the authors did a good job at providing additional information requested by the reviewers. However, I think the initial submission is missing too many details and experiments to justify a higher score.
> Q2. Regarding the Ratio metric.

Thank you for your insightful question. To ensure a fair comparison, we maintained **an equal number of queries** to the target model $\mathbf{M}$ during the inversion process for both the baseline PPA and PPDG-MI. For example, in the experiment where $\mathcal{D}_{\text{public}}$ = FFHQ and $\mathrm{M}$ = ResNet-18, the baseline attack's optimization iterations were set to 70, while PPDG-MI was configured with 35 optimization steps per round. Due to space constraints, we have detailed these attack parameters in Appendix C.4. 

Additionally, after considering your perspective, we agree that maintaining the same number of optimization iterations per round as the baseline is a more reasonable approach to validate the effectiveness of PPDG-MI. This setup would only further enhance our experimental results. We appreciate your feedback and will consider this for futher experiments.

> Limitations: The limitation section could include some failure cases and additional limitations of the method.

We appreciate the reviewer's suggestion. We will further examine the experimental results and include a discussion of failure cases and additional limitations of the method in the final version of our paper.

---
**References**:

[r1] Tran et al. "Hierarchical Implicit Models and Likelihood-Free Variational Inference." In NeurIPS, 2017.

[r2] Yin et al. "Semi-Implicit Variational Inference." In ICML, 2018.

[r3] Karras et al. "Analyzing and Improving the Image Quality of StyleGAN." In CVPR, 2020.

[r4] Struppek et al. "Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks." In ICML, 2022.
> W5.2. Section 2.2: There should be some motivation for why MMD and CT are introduced here ... understanding these concepts.

Thank you for your valuable suggestions. We have revised Section 2.2 to include additional context explaining the motivation for introducing MMD and CT at this point in the paper. Specifically, we added 1-2 introductory sentences to clarify the relevance of these measures to our method. Additionally, we included a brief explanation of what MMD and CT measure and how they differ, providing readers with a clearer understanding of these concepts. The specific content added is as follows:

"To effectively align distributions in our subsequent methods, it is essential to introduce metrics that can accurately quantify the differences between them. Two commonly used measures for this purpose are maximum mean discrepancy (MMD) and conditional transport (CT). MMD focuses on mean differences using kernel methods, while CT incorporates cost-based transport distances, offering complementary perspectives on distributional discrepancies."

> W5.3. L132: From my understanding, the "linear interpolation" between both distributions means that the sample share $\alpha$ is sampled from X_prior and $(1-\alpha)$ is sampled from X_private. This should be clarified in the writing.

Thank you for your feedback. We have made the necessary clarifications in the manuscript. The specific content added is as follows:

"To evaluate the impact of this distribution discrepancy on MI performance, we create a series of proxy prior distributions through linear interpolation, where a mixing coefficient $\alpha\in [0,1]$ determines the proportion of samples drawn from each distribution. Specifically, a fraction $\alpha$ of samples is drawn from $\mathrm{P}(\mathcal{X}{\text{prior}})$, and the remaining $(1-\alpha)$ is drawn from $\mathrm{P}(\mathcal{X}{\text{private}})$."


> W5.4. Fig. 3: The font size in the legend ... Currently, those data points are hard to see in (b).
> W5.6. L296: The figure reference is broken, it currently states "Fig. 4.3" instead of "Fig. 5".

Thank you for your detailed observations. We have made the necessary adjustments based on your suggestions. Specifically, we increased the font size in the legend and enlarged the data points in Fig. 3 to enhance their visibility and make them easier to interpret. Additionally, we corrected the figure reference on line 296, updating it from "Fig. 4.3" to "Fig. 5" as appropriate.

> W5.5. To make the approach more clear, think about adding "Step-4: Repeating the attack with the updated generator" ... be repeated after this step.

Thank you for your suggestion. As clarified in Section 3.2, PPDG-MI consists of three **iterative** steps, so adding a separate "Step-4: Repeating the attack with the updated generator" may be redundant. However, to enhance clarity, we will add a note at the end of Step-3 indicating "return to Step-1 and repeat the attack with the updated generator." This addition should make the iterative nature of the process more explicit to all readers.

> Q1. There is the risk of a mode collapse in fine-tuning the generator. Does the approach require regularization to avoid this failure case, or is the method already robust enough to avoid it?

- For MIAs focusing on low-resolution tasks, we adopt a principled tuning strategy, fine-tuning $\mathrm{G}$ and $\mathrm{D}$ using the original GAN training objective on $\mathcal{D}\_{\text{public}} \cup \mathcal{D}\_{\text{private}}^{\text{s}}$. This approach mitigates the risk of mode collapse. 
- However, for MIAs targeting high-resolution tasks, such as PPA [r4], we are unable to apply a principled tuning strategy due to the lack of access to the GAN training specifics. Instead, we employ an empirical strategy to fine-tune $\mathrm{G}$. While this approach may affect the quality of the generated images, it does not lead to mode collapse, as we only make slight alterations to the generator $\mathrm{G}$. Currently, we do not employ regularization to avoid this failure case but instead manage it by controlling the fine-tuning strength (i.e., through hyperparameter adjustment). We believe that incorporating regularization could further enhance the robustness of this process.
> W4. While the focus of the work is clearly computer vision ... which the paper positions itself to be.

Model inversion attacks have garnered increasing attention in the trustworthy machine learning area due to their potential to reveal privacy risks in machine learning models. While current generative MIAs primarily focus on either the initial training process of GANs or the optimization techniques used in the attacks, our paper takes a different direction. We introduce a novel method by fine-tuning the GAN's generator based on the attack results from previous runs.
This approach introduces a dynamic and iterative dimension to model inversion attacks, **expanding the current understanding and application of generative MIAs**. Although our work is primarily focused on computer vision, **the underlying principles and methodologies could potentially be adapted to other domains, making this more than just an incremental improvement**.

> W5. Minor: the convention is typically MI for model inversion, rather than MIA (which is often used for membership inference), making it a bit more difficult to follow.

Indeed, both "MI attacks" and "MIAs" have been used to refer to model inversion attacks in the literature. For example, "MI attacks" is used in works such as GMI [r1] and LOMMA [r3], while "MIAs" is the terminology adopted in PPA [r2] and LS [r4].

> Q1. Reason for making a connection to adversarial samples in the background.

Thank you for highlighting this point. Traditional MIAs [r5] on DNNs trained with image data adopt direct optimization in the input space, which can lead to the generation of adversarial samples. This limitation led Zhang et al. [r1] to propose generative MIAs as a more effective alternative. We have outlined this progression in the introduction section, specifically in lines 26-41. To eliminate any confusion, we will make the necessary clarifications in the problem setup section.

> Q2. The difference between the low/high dimensional reconstructions sounds rather artificial (it seems this is just pre-trained vs FT GAN), is it really necessary to separate these? Are there are other differences making this separation clearer?

We appreciate the reviewer's constructive comments. The distinction between low-resolution and high-resolution settings is based on the typical approach in each scenario: the low-resolution setting usually involves training GANs from scratch using low-resolution public auxiliary data, while the high-resolution setting involves using pre-trained StyleGAN models trained on high-resolution data. This separation was originally introduced by the state-of-the-art model inversion defense LS [r4], and we followed this established setup. We acknowledge your point and will consider your suggestion to categorize the approaches based on whether the GAN is trained from scratch or a pre-trained StyleGAN is used.

---
**References**:

[r1] Zhang et al.  "The Secret Revealer: Generative Model-inversion Attacks Against Deep Neural Networks." In CVPR, 2020.

[r2] Struppek et al. "Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks." In ICML, 2022.

[r3] Nguyen et al. "Re-thinking Model Inversion Attacks Against Deep Neural Networks
." In CVPR, 2023.

[r4] Struppek et al. "Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks." In ICLR, 2024.

[r5] Fredrikson et al. "Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures." In CCS, 2015.