Title: Pseudo-Private Data Guided Model Inversion Attacks
Venue: NeurIPS 2024
Accepted: yes
Keywords: Model Inversion Attacks
Abstract: In model inversion attacks (MIAs), adversaries attempt to recover private training data by exploiting access to a well-trained target model. Recent advancements have improved MIA performance using a two-stage generative framework. This approach first employs a generative adversarial network to learn a fixed distributional prior, which is then used to guide the inversion process during the attack. However, in this paper, we observed a phenomenon that such a fixed prior would lead to a low probability of sampling actual private data during the inversion process due to the inherent distribution gap between the prior distribution and the private data distribution, thereby constraining attack performance. To address this limitation, we propose increasing the density around high-quality pseudo-private data—recovered samples through model inversion that exhibit characteristics of the private training data—by slightly tuning the generator. This strategy effectively increases the probability of sampling actual private data that is close to these pseudo-private data during the inversion process. After integrating our method, the generative model inversion pipeline is strengthened, leading to improvements over state-of-the-art MIAs. This paves the way for new research directions in generative MIAs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel method to enhancing model inversion attacks (MIAs), which aim to reconstruct class characteristics from a trained classifier. Typically, MIAs rely on training image priors, such as GANs, on public data that differ in distribution from the target model's training data. This distributional discrepancy reduces the effectiveness of MIAs in accurately reconstructing features of the target class. To address this issue, the paper presents pseudo-private data-guided MIAs. The proposed method initially conducts existing MIAs to gather a set of attack samples that reveal features of the target classes. A subset of high-quality samples is selected based on robust prediction scores under the target model. The GAN's generator is then fine-tuned on this subset to increase the sampling density around the pseudo-private data. Subsequent attacks using the updated generator show a significant improvement in attack results across various attacks and settings.\n\nSTRENGTHS:\n- Existing MIAs either focused on the initial training of GANs or the attack's optimization method. This paper takes another direction and adds the idea of fine-tuning the GAN's generator on attack results from a previous run. This method adds a novel and exciting dynamic dimension to the literature on model inversion attacks.\n- The approach is well-motivated and supports its claims empirically with toy examples and results on standard MIA benchmarks. This fact makes the proposed improvements convincing and reasonable. Particularly Sec. 3.3 adds valuable insights to the methodology.\n- The evaluation investigates a broad range of model architectures, datasets, and types of MIAs. The proposed method shows improved results across all settings, supporting the paper's claims and the method's efficiency. \n\nSmall remark:\n- The appendix is well-formatted and clearly arranged. It provides valuable additional insights.\n\nWEAKNESSES:\n- The evaluation is missing some aspects. For the investigated attacks, it would be interesting to compare the results to PLG-MI [1], which takes a similar direction (training a conditional GAN on pseudo-labeled data to decouple the latent space for different target classes) and shows strong attack results. \n- Also, the evaluation includes no investigation of existing defense methods like MID [2], BiDO [3], and negative label smoothing [4]. For MID and BiDO, existing attacks have already (partly) broken the defenses; see [4,5]. However, the authors [4] argue that negative label smoothing limits the information provided by a model's prediction scores, which, in turn, might affect the subset selection process used by the pseudo-private data-guided MIAs proposed method. It would be interesting to see if the proposed attack improvement also breaks this type of defense.\n- The evaluation focuses on metrics computed on an evaluation model trained on the same data as the target model has been. While this is a common approach in MIA literature, it might provide some misleading results. For example, given that the attacks might reconstruct adversarial features instead of actual robust and characteristic class features, the fine-tuned GAN might also generate images containing adversarial features. Since we know from the literature that adversarial features can be transferable, the attack results could similarly fool the evaluation model and, therefore, overestimate an attack's success. Therefore, it might be reasonable to include other metrics, e.g., the FaceNet distance used in the PPA paper. Another option to measure the attack's success could be the information extraction score introduced in [4]. Using the FaceNet distance in addition to the Attack Accuracy in Fig. 5 would further support the paper's claims.\n- Giving more intuition on why MMD and CT are used instead of, e.g., a KL Divergence would improve the paper. Similarly, why use LPIPS as a similarity metric instead of another type of metric, e.g., FaceNet distance?\n\nMinor remarks:\n- The term \"MMD\" used in the caption of Fig. 1 should be introduced before referring to this figure. Also, \"MI\" (L39) is not formally introduced. It probably stands for \"model inversion\". However only \"MIA\" is introduced, which might lead to confusion of the reader.\n- Section 2.2: There should be some motivation for why MMD and CT are introduced here. It makes sense later in the paper, but these concepts appear a bit surprising at the point of the background section. 1-2 introduction sentences would support the motivation here. Also, intuition about what MMD and CT are measuring and how they differ supports understanding these concepts.\n- L132: From my understanding, the \"linear interpolation\" between both distributions means that the sample share $\\alpha$ is sampled from X_prior and $(1-\\alpha)$ is sampled from X_private. This should be clarified in the writing.\n- Fig. 3: The font size in the legend is too small and should be increased. Also, the size of the data points should be increased, particularly the red ones in the final image. Currently, those data points are hard to see in (b).\n- To make the approach more clear, think about adding \"Step-4: Repeating the attack with the updated generator\". Currently, the approach only describes the steps of fine-tuning the generator, and it might not be clear to all readers that the attacks will be repeated after this step.\n- L296: The figure reference is broken, it currently states \"Fig. 4.3\" instead of \"Fig. 5\".\n\nOverall, I liked the paper and the mentioned weaknesses can probably be addressed during the rebuttal. Therefore, I encourage the authors to participate in the discussion and will increase my score if the mentioned weaknesses are sufficiently addressed.\n\n[1] Yuan et al. \"Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network\". AAAI 2023\n[2] Wang et al. \"Improving Robustness to Model Inversion Attacks via Mutual Information Regularization\". AAAI 2021\n[3] Peng et al. \"Bilateral Dependency Optimization: Defending Against Model-inversion Attacks\". KDD 2022\n[4] Struppek et al. \"Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks. ICLR 2024\n[5] Nguyen et al. \"Re-thinking Model Inversion Attacks Against Deep Neural Networks\". CVPR 2023\n\nQUESTIONS:\n- There is the risk of a mode collapse in fine-tuning the generator. Does the approach require regularization to avoid this failure case, or is the method already robust enough to avoid it?\n- Regarding the Ratio metric: From my intuition, the ratio should always be > 2, because the attacks take at least twice the time compared to the baseline (1x runnning the baseline + 1x repeating the attack with the adjusted generator). A more detailed description of this metric would improve the understanding.\n\nLIMITATIONS:\nSome limitations are discussed in Appx. E. However, I think the limitation section could include some failure cases and additional limitations of the method. In which settings did it fail? Are there any additional requirements for the method to work?\n\nWhile the paper proposes a novel type of attack, I do not think there will be negative societal impact, given that there already exists a long list of publicly available MIA literature and implementations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a novel application of a generative model inversion attack utilising dynamic (pseudo-private) priors, improving the existing results of MI.\n\nSTRENGTHS:\nThe work is very clear, the idea itself makes sense and the results are well-presented. Authors explicitly target a specific problem and manage to outperform the existing works in the MI field. The method itself is well-motivated and evaluated in a variety of settings.\n\nWEAKNESSES:\nSo while the idea is straightforward and makes a lot of sense in the chosen setting, the reliance on priors in MI is a) hardly novel [1,2] and b) comes with caveats not covered in the paper. I appreciate that there are various formulations of MI in literature and the one used in this work is generative MI specific, but the main principle of relying on a dynamic prior has previously been covered in the context of MIs.\n\nWith respect to point b): if the generative model is conditioned on the pseudo-generations, I suspect this can lead to the issues of bias and/or mode collapse. What I mean here is that there is no guarantee that the reconstructions obtained as part of the pseudo-generation process would resemble the 'actual' training data. This is problem number one, but lets assume the images generated in this step are valid samples from the training dataset. The fact that you are able to generate them means (similarly to the results of the Secret revealer discussed in this work) that they were 'easier' samples with respect to MI vulnerability. And currently, there is no reason for me to believe that this would help you reconstruct the more 'difficult' or informative samples (often on the tails of the distribution [3]). So to simplify: I am not convinced that by leveraging the pseudo-private samples you would be able to improve the attack results meaningfully, as now you would condition your reconstructions to those that are of high similarity to the pseudo-private ones (and I would argue, making it close to impossible to now reconstruct the samples that do not fall under this category, which you could have done with a frozen decoder which would not discriminate between these).\n\nOne step which is mentioned as 'optional' is selection of samples that are meaningful. What does this imply? To me this is a very important (and an incredibly challenging) task, which was not given enough discussion in the manuscript, given that you are conditioning your further inversions based on the quality of the pseudo-generated ones. How do you effectively measure it to avoid collapsing into the region of pseudo-reconstructions alone (i.e. similarly to my comments above, how do you measure and use this quantity to encourage MI diversity)?\n\nWhile the focus of the work is clearly computer vision, it does limit the impact of the findings to these specific settings, making this more of an incremental improvement than a novel paradigm of MI, which the paper positions itself to be.\n\nMinor: the convention is typically MI for model inversion, rather than MIA (which is often used for membership inference), making it a bit more difficult to follow.\n\n[1] - Hatamizadeh, Ali, et al. \"Do gradient inversion attacks make federated learning unsafe?.\" IEEE Transactions on Medical Imaging (2023).\n[2] - Usynin, Dmitrii, Daniel Rueckert, and Georgios Kaissis. \"Beyond gradients: Exploiting adversarial priors in model inversion attacks.\" ACM Transactions on Privacy and Security 26.3 (2023): 1-30.\n[3] - Feldman, Vitaly. \"Does learning require memorization? a short tale about a long tail.\" Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 2020.\n\nQUESTIONS:\nCould you explain why are you making a connection to adversarial samples in the background? Sure they possess the features you describe, but how is this relevant for MI discussion?\n\nThe difference between the low/high dimensional reconstructions sounds rather artificial (it seems this is just pre-trained vs FT GAN), is it really necessary to separate these? Are there are other differences making this separation clearer?\n\nLIMITATIONS:\nAs per weaknesses above: to me this seems like an incremental improvement in a relatively niche area, which is more suited to a conference specializing in attacks on ML or PPML, as these results are constrained to specific ML settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-15"}

comments: SUMMARY:
It is well known that deep learning models are susceptible to model-inversion attacks, which is to say that they can be probed to reveal their training data. The authors design a more powerful method of attack by increasing the density of their prior using “pseudo-private data”, they can increase the probability of sampling actual private data.

STRENGTHS:
The contribution of this work is clear and intuitive. The authors provide empirical evidence for their claims and they propose different algorithms which leverage generated samples. They benchmark their method against existing methods and show how they can build on prior work in this area. The illustration and build up to their core result is well structured and makes the paper easy to follow.

WEAKNESSES:
- There are a number of typographic errors (e.g. “vallina”)
- A brief related work section which situates their contribution in the main body would have been appreciated
- There is verbatim repetition in the problem setup.
- They claim that “all state-of-the-art generative MIAs” are limited due to the utilization of a fixed prior during inversion. This bold claim is not backed up.
- Some of the symbols are not properly defined  (e.g. \lambda is undefined)

QUESTIONS:
- What are the real-world settings that this threat model could be observed with this work?
- How might this approach extend to diffusion-based models?

LIMITATIONS:
The paper makes the limitations and ethics issues with their work clear.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel plug-and-play method for current state-of-the-art MI methods to enhance their performance and mitigate the challenge of distribution discrepancy. This method first conducts a round of MI attack to acquire pseudo-private data and then utilizes the data to fine-tune the generative prior following certain strategy. The experimental results show that the proposed method enhances the attacking capability of existing MI methods to some extent.\n\nSTRENGTHS:\n1. Clarity: The paper is well-structured and easy to follow.\n2. Originality: The proposed method seems novel and interesting.\n3. Intuitive pipeline: The method is easy to understand and implement.\n\nWEAKNESSES:\n1. The current state-of-the-art white-box MI method PLGMI [1] should be evaluated.\n2. There is no experiment for evaluation on any model inversion defenses. Related experiments are expected to demonstrate how the proposed technique impacts the resistance ability of MI attacks against defense strategies.\n3. The middle panel in Fig 5 seems wrong. The attack accuracy of random samples is higher than the one of high-quality samples.\n4. In Section 4.3, the citation for Fig 5 appears to be Fig 4.3.\n\n[1]Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang. Pseudo label-guided model inversion attack via conditional generative adversarial network. In AAAI, 2023.\n[2]Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correia, Antonia Adler, and Kristian Kersting. Plug & play attacks: Towards robust and flexible model inversion attacks. In ICML, 382 2022.\n\nQUESTIONS:\n1. In Section C.4, why is the final results selection stage removed? This technique is critical to PPA [2]. Omitting this stage might leads to the degraded performance of PPA in the Experiment Section.\n2. In Section C.4, why the parameters of pre-attack latent code selection stage are significantly lower than the original paper (200 candidates from a search space of 2000/5000 latent codes). This also contributes to the bad performance of PPA [2] in the Experiment Section.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-14"}