Label-Only Model Inversion Attacks via Knowledge Transfer
Authors: Ngoc-Bao Nguyen
Keshigeyan Chandrasegaran
Milad Abdollahzadeh
Ngai-man Cheung
Venue: NeurIPS 2023 poster (2023)
Keywords: Model Inversion attacks
Generative models
Surrogate models
Knowledge transfer
TL;DR: This work proposes a novel method for Label-Only Model Inversion attacks via Knowledge Transfer (LOKT). LOKT outperforms existing SOTA by more than 15% across all MI benchmarks, highlighting rising privacy threats for machine learning models.

Abstract: In a model inversion (MI) attack, an adversary abuses access to a machine learning (ML) model to infer and reconstruct private training data. Remarkable progress has been made in the white-box and black-box setups, where the adversary has access to the complete model or the model's soft output respectively. However, there is very limited study in the most challenging but practically important setup: Label-only MI attacks, where the adversary only has access to the model's predicted  label (hard label) without confidence scores nor any other model information.  

In this work, we propose LOKT, a novel approach for label-only MI attacks. Our idea is based on transfer of knowledge from the opaque target model to  surrogate models. Subsequently, using these surrogate models, our approach can harness advanced white-box attacks. 
We propose knowledge transfer based on generative modelling, and introduce a new model, Target model-assisted ACGAN (T-ACGAN), for effective knowledge transfer. Our method casts the challenging label-only MI into the more tractable white-box setup. We provide analysis to support that surrogate models based on our approach serve as effective proxies for the target model for MI. Our experiments show that our method significantly outperforms existing SOTA Label-only MI attack by more than 15% across all MI benchmarks. Furthermore, our method compares favorably in terms of query budget. Our study highlights rising privacy threats for  ML models even when minimal information (i.e.,  hard labels) is exposed. Our study highlights rising privacy threats for  ML models even when minimal information (i.e.,  hard labels) is exposed. Our code, demo, models and reconstructed data are available at our project page:
https://ngoc-nguyen-0.github.io/lokt/

Label-Only Model Inversion Attacks via Knowledge Transfer
Ngoc-Bao Nguyen"!
Keshigeyan Chandrasegaran"
Milad Abdollahzadeh' Ngai-Man Cheung
Singapore University of Technology and Design (SUTD) Stanford University
th thibsongoo__nuueee@neeg 1. potd.edu: .g ngaiman_cheuuee cheung@sutd edu. adu.sg
Abstract
a model inversion (MI) attack, an adversary abuses access to machine learning (ML) model infer and reconstruct private training data. Remarkable progress has been made in the white-box and black-box setups, where the adversary has access the complete model the model's soft output respectively. However, there is very limited study in the most challenging but practically important setup: Label- only MI attacks, where the adversary only has access to the model's predicted label (hard label) without confidence scores nor any other model information.
In this work, we propose LOKT, novel approach for label-only MI attacks. Our idea is based on transfer knowledge from the opaque target model to surrogate models. Subsequently, using these surrogate models, approach can hamness ad- vanced white-box attacks. We propose knowledge transfer based on generative mod- elling, and introduce new model, Target model-assisted ACGAN (T-ACGAN), for effective knowledge transfer. Our method casts the challenging label-only MI into the more tractable white-box setup. We provide analysis support that surrogate models based on our approach serve as effective proxies for the target model for MI. Our experiments show that method significantly outperforms existing SOTA Label-only MI attack by more than 15% across all MI benchmarks. Further- more, method compares favorably in terms of query budget. Our study high- lights rising privacy threats for ML models even when minimal information (i.c., hard labels) is exposed. Our code, demo, models and reconstructed data avail- able at our project page:
nguyen O. gi writhub... Lo/lokt/
Introduction
Model inversion (MI) attacks aim to infer and reconstruct sensitive private samples used in the training of models. MI and their privacy implications have attracted considerable attention recently
2
6
7
& 9 [O] []. The model subject MI is referred as targel model. There three
categories of MI attacks: (1) White-box MI, where complete target model information is accessible
by the adversary [122 7 [O]; (2) Black-box MI, where target model's soft labels are accessible
4. [2] 10 13]: (3) Label-only MI, where only target model's hard labels are accessible [6]. This
paper focuses on label-only MI. which is the most challenging setup only limited information
(hard labels) is available (Fig. M.
In most existing work, MI attack is formulated that maximize the likelihood under the target model highly non-linear. When the sensitive private samples are high-dimensional samples (e.g. face
optimization problem to seek reconstructions 6) For DNNs, the optimization problems
These authors contributed equally. Corresponding author.
Work done while SUTD.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

images), the optimizations are ill -oosed even in white-box setups. To overcome such issues, recent MI
and solve the optimization problems over GAN latent space rather than the unconstrained image space. For example, MI attacks on face recognition systems could leverage GANs to learn face manifolds from public face images which have no identity intersection with private training images. White-box attacks based on public data and GANs have achieved remarkable success p 2 3 7 [1].
2 10 IT learn distributional priors from public data via GANs 14
T3
16
17
We follow
existing work and recent label-only MI [6] and leverage public data in our method. Furthermore, similar to existing work, we use face recognition models as examples of target models.
Research gap. Different from white-box attack, study on label-only attack is limited despite its practical importance, e.g.. many practical ML models only expose predicted labels. Focusing on label-only attack and with no knowledge of internal workings of target model nor its confidence score, BREPMI 6 takes black-box search approach to explore the search space iteratively (Fig. Ta)). To seek reconstructions with high likelihood under target model, [6] proposes to query target model and observe the model's hard label predictions, and update search directions using Boundary Repelling in order to move towards centers of decision regions, where high likelihood reconstructions could be found. However, black-box search in the high-dimensional latent space is extremely challenging.
In this paper. we propose a new approach for Label-Only MI attack using Knowledge Transfer (LOKT). Instead of performing a black-box search approach as demonstrated in [6] and directly searching high- high-likelihood reconstruction from the opaque target model (Fig. [T)),, which could be particularly challenging for nigh-dimensional search space, we propose a different approach. Our approach aims to transfer the decision knowledge of the target model to surrogate models, for which complete model information is accessible. Subsequently, with these surrogate models, we could harness SOTA white box attacks to seek high-likelihone reconstructions (Fig, (b)). To obtain the surrogate models, we explore generative modeling [8 20 22). In particular, we propose a new Target model model-assisted ACGAN, T-ACGAN, which extends ACGAN (23) and leverages our unique problem setup where we have access to the predicted labels of the target model as shown in Fig. dd) In particular, by effectively leveraging the target model in discriminator/Classffee training, we can explore synthetic data for decision knowledge transfer from the target model to the surrogate model. With T-ACGAN capturing the data manifold of public samples, synthetic data is diverse and abundant. We hypothesize that such rich synthetic data could lead to improved decision knowledge transfer. Moreover, as training progresses, T-ACGAN generator learns to improve its conditional generative capabilities, enabling it to produce more balanced synthetic data for surrogate model learning. We explore several surrogate model designs. In one configuration, employ the discriminator/ classifier of T-ACGAN as the surrogate model. In an alternative design, we utilize the generator of T-ACGAN to train different surrogate model variants. It's noteworthy that the generator of T-ACGAN can be readily employed for white-box attacks, and its conditional generation capabilities can effectively reduce the search space during inversion. In addition, we perform analysis to support that our surrogate models are effective proxies for the opaque target model for MI. (Fig. [c)). Overall, our T-ACGAN renders improved surrogate models, resulting in significant boost in MI attack accuracy (Fig. (i)) and reduced number of queries compared to previous SOTA approach. Our contributions are:
We propose LOKT, a new label-only MI by transferring decision knowledge from the target model to surrogate models and performing white-box attacks the surrogate models (Sec. Our proposed approach is the first to address label-only MI via white-box attacks.
We propose a new T-ACGAN to leverage generative modeling and the target model for effective knowledge transfer (Sec. 4.
We perform analysis to support that our surrogate models are effective proxies for the target model for MI (Sec. 55)
We conduct extensive experiments and ablation to support our claims. Experimental results show that our approach can achieve significant improvement compared to existing SOTA MI attacks (Sec. Additional experiments/ analysis are in Supplementary.
2
Related work
Model Inversion (MJ) has particularly alarming consequences security-sensitive domains, such as
face recognition [24 25 26 27), medical diagnosis [28] 29] 30 Fredrikson al. [31) introduces the

---

Existing Works on Label-Only MI
(d) Decision Knowledge transter using our 7-ACGAN
Opeque Target Modid
Reconstruct
Private Data
Label Only
D.
pub
MIAAtack
Real' Take
(b) Our Label-Only
via Knowledge
(LOKT)
Xy
Opaque Target
Surogaie
Identity Classtication
Stage
Decision
Knowlledge Transfer
Stage
Surrogate Model
Reconstruct
Whoe-box
Knowledge
Casting Labet-only MI Artack White-box MI Attack
(c) Label-only MI
challenging Attack
(e)
Models Effective Target
Criteria
Architecture/
Soft-labels
Hard-labels
reg.
Parameters
Queries
White-box
189
High
Label-only
X
X
High
Private
Attack
0.5g
Traiuing
Acc.
Data
Existing
a
73.938
0850
Reconstruction Results
93.93$
3.30
1990
Figure Overview and our contributions. (a) Under Label-only model inversion (MI) attack, the Target model T is opaque. (b) Stage 1: As our first contribution, we propose a knowledge transfer scheme to render surrogate model(s). (b) Stage 2: Then, we cast the Label-only MI attack as white-box MI attack on surrogate model(s) S. (c) This casting can case the challenging problem setup of label-only MI attack into a white-box MI attack. To our knowledge, our proposed approach is the first to address label-only MI via white-box MI attacks. (d) We propose T-ACGAN to leverage generative modeling and the target model for effective knowledge transfer render surrogate model(s). Knowledge transfer renders D (Discriminator) as surrogate model, and further generated samples of T-ACGAN can be used train additional surrogate variant (Sec. 4.3) (e) Our analysis demonstrates that S is an effective proxy for T for MI attack (details in Sec 5). In particular, white-box MI attack on mimics the white-box attack on opaque T. (f) Our proposed approach significantly improves the Label-only MI attack (e.g. & 20% improvement in standard CelebA benchmark compared to existing SOTA 66 resulting in significant improvement private data reconstruction. Best viewed in color.
first MI attack for simple linear regression models. Recently, several works extend MI for complex DNNs under different setups. For white-box setup, [ proposes Generative Model Inversion (GMI) 33) to constrain the search space. 2 proposes Knowledge- Enriched Distributional Model Inversion (KEDMI) train an inversion- GAN for the attack.
to leverage public data and GAN 32
3 proposes Variational Model Inversion (VMI) to apply variational objectives for the attack. Very recent work 7 proposes Pseudo Label-Guided MI (PLG-MD) to apply model's soft output to train conditional GAN (cGAN) 34 for white-box attack. LOMMA proposes better objective function for MI and model augmentation to address MI overfitting. For black-box attack, where model's soft output is available, 4 proposes to train inversion model and decoder generate target images using predicted scores of the inversion model. [12] proposes an ad adversarial approach for black-box MI. For label-only attack. [6] proposes BREPMI, the first label-only MI using black-box Boundary Repelling search. See Supplementary for further discussion of related work.
Problem setup
Given a target model T, the goal of MT to infer private training data Dpriv by abusing access to model T.. More specifically, given target class/ identity label y. the adversary aims to reconstruct

---

an image r which is similar to the images of class in D Priie Most MI formulate the inversion as optimization problems to seek the highest likelihood reconstructions for identity y under T. As direct searching for in the unconstrained image space is ill ill-posed, many MI attacks [1 2 3
7 6 leverage
public dataset Puu that is the same domain as D Ppries e.g., Dpris and pub are facial image datasets.
GAN 14 is applied to learn distributional prior from Dpub. and the adversary searches the GAN latent space instead of the unconstrained image space for high- high-likelihodd reconstructions under T::
max log Pr(vlG(2))
(1)
Here, G the generator, and Pr(yl-) the likelihood of an input for identity y under target model T. White-box attacks apply gradient ascent and some regularization [ 3 7 to solve Eq. [ whereas label-only attack BREPMI [ applies black-box search tackle Eq. In this paper, we also tackle
Eq. [ Jnder label-only setup, i.e. only the predicted label is available.
4
Approach
Our proposed label-only MI consists of two stages. In stage we learn surrogate models. In stage we apply SOTA white-box attack on the surrogate models. To learn surrogate models, we explore an approach based on GAN and propose a new Target model-assisted ACGAN (T-ACGAN) for effective transfer of decision knowledge. Our T-ACGAN learns the generator G and the discriminator D with classifier head C. In one setup, we directly take C' o D as the surrogate mode In another setup, we apply G to generate synthetic data to train another surrogate model S or an ensemble of S. Then, we apply SOTA white-box attack on C' o D, S or the ensemble of S. In our experiments, we show that using C a D in white-box attack can already outperform existing SOTA label-only attack. Using S or an ensemble of can further improve attack performance. The G obtained from our T-ACGAN can be readily leveraged in the attack stage.
4.1 Baseline
Before discussing our proposed approach, we first discuss a simple baseline for comparison. Given the public data, one could directly use the target model 7 to label the data and learn the surrogate model S. For Ip E Dpub, we construct (Ip, Ø). where ÿ = T'zp) is pseudo label of private identity. We obtain the dataset Ppub with samples (xp: M). i.e. Dpus is the public dataset with pseudo labels.
We apply Dyub to train S. However, this algorithm suffers from class imbalance. In particular, some private identities could have less resemblance to Zp E Duub. As a result, for some w. there is only small number of Tp classified into it, and Prub is class imbalanced. When using Dpus to train S, minority classes may not gain adequate decision knowledge under T and could perform sub-optimally. In our experiments, we also apply techniques to mitigate the class imbalance in D Ppub- However, the performance of this baseline approach is inadequate will show in the experiments.
4.2 Review of ACGAN
In standard ACGAN 23 we are given a real training dataset with label, i.c., Dreat with samples
(zy, y). The generator G takes random noise vector z and a class label y as inputs generate fake sample y. The discriminator D outputs both probability distribution over sources P(s[z) = D(z). where E { (Real, Fake], and probability distribution over the class labels, i.e., P(c(z) Co (()) and is one of the classes. For real training sample of label and fake sample f = G(z, y) with conditional information y, the loss functions for D, and are:
LD. - EE'log P(s Fake| [ E'log P(s Real|z,
(2)
Ellog P(c = y| E(log P(c = y|2,,]
Le = Flo P(s Fakelæy)) Fllog P(c yl y|æ;)]
(3)
"With slight abuse of notation we use D) to represent the entire discriminator and the discriminator up to and including the penultimate layer in the context of C C D.

---

4.3 Our Proposed T-ACGAN and Learning of Surrogate Model
Unlike standard ACGAN setup where we have access to labelled data Dreat with samples (Zy, y). in our setup, we have access to real public data without label: pub with samples 2p. Importantly, we can leverage the target model T to provide pseudo labels for generated samples xy G(2. y), which are diverse and abundant. Our proposed T-ACGAN aims to take advantage of T to provide more diverse and accurate pseudo labelled samples during the training.
D and C' Learning, Our T-ACGAN leverages T to assign pseudo labels to the diverse generated samples 5 ())z)),,a i... y = Txy)). We apply samples 2p and (xz, 9) to learn D and C::
LD. = EEllog P(s = Fakelæy) - Ellog P(s = Reall [xp]
Eillog P(c = 5|
In Eq. 4 the term Ellog P(c= = و| = Ellog P(c = (i v)) is different from ACGAN and may look intriguing. Instead of using y as class supervision to train D and C as in ACGAN (Eq. - our T-ACGAN takes advantage of I to apply ÿ = (z, y)) to train D and C, as ÿ is more accurate conditional information compared with y especially during the initial epochs. With Eq. our method transfers the decision knowledge T' into D and C' via diverse generated samples. Furthermore, as we can generate diverse pseudo labelled samples (zy, 5) using T and G. pseudo labelled data based on Tp can be omitted. In our experiment, we show that we can achieve good performance using diverse samples (z 9). In T-ACGAN, we utilize public data Tp only for real/fake discrimination.
G Learning. We follow ACGAN training for G, i.e Eq. 3 With and C' trained with decision knowledge of T in the above step, they provide feedbacks to G to improve its conditional generation in the private label space T. In our experiment, we analyze y in xy = G(z, y) and f = T(xr)) As training progresses, G improves its conditional generation, and y and ÿ become more aligned. Note that, as T outputs only hard labels, 7' cannot be readily applied to provide feedback for G learning.
Surrogate Model. With alternating 1 and C' learning and G learning, we obtain D, C' and G. We explore three methods to obtain the surrogate model. * (i) We directly take C' o D in T-ACGAN as the surrogate model and apply white-box attack on Co D. This be justified
C D is trained based on decision knowledge of T to classify a sample into identities of private training data. * (ii)
We apply G of T-ACGAN to generate dataset D) sake with samples (Iy,ğ), where 2 = G(z, y) and V = TTr)). We apply Dakee to train another surrogate model S. (iii) We use the same dataset Dake in (ii) to train ensemble of S of different architectures. As pointed out in using an ensemble of S could improve thite-box attack performance.
White-box Attack. With surrogate model C' o D, S or an ensemble of S, any white-box attack can be applied. In our experiments, we show that our surrogate models are effective across a range of white-box attacks (See the Supplementary). Furthermore, G' in T-ACGAN can be readily leveraged for performing the attack. Particularly, based on G(2, y) obtained in the above steps, we could reduce the search space during inversion to the latent region corresponding to the target identity y, leading to more efficient search and improved attack accuracy
Analysis for justification of surrogate models
In this section, we provide an analysis to justify why our surrogate model could be an effective proxy for T under MI, i.e., the results of white-box MI attack on our surrogate model be good approximation to that of white-box MI attack on T. Note that results of white-box MI on T cannot be obtained directly as T exposes only hard labels. To simplify the presentation, we focus our discussion on S. As discussed in Sec. 3 most MI attacks formulate inversion as an optimization problem of secking reconstructions that achieve highest likelihood under target model. Therefore, when we carry out MI on S with SOTA white-box approaches, we expect to obtain high-likelihood reconstructions under S (or high-likelihood generated samples of GAN under 5, see Eq. T]. We use P3 and Pr to denote likelihood of sample under S and T respectively.
In what follows, we provide analysis to support that S based on our approach would possess an important property of good proxy for T. Property Pl: For high-likelhhodd sumples under S, it is likely that they also have high likelihood under T. See Fig. [e) for distribution of generated samples' Pr conditioning those with high Ps. It can be observed that many have high Pr. Particularly, it is
5

---

uncommon for high- lighllikelihod samples under S to have low likelihood under T (see Fig. Te) only few samples have low Pr)).
With Property Pl, the result obtained by white-hox on S (which is a high likelihood sample under S) is likely have high likelihood under T and could be a good approximation white-box on T' (which is high likelihood sample under 7). In Fig. [e). PI can be clearly observed Therefore, S using our approach would possess P1 and would be good proxy for ' for MI.
the result of
Why would S possess property PI? This could be intriguing. After all, T does not expose any likelihood information. The labels of samples assigned by T are the only information available to S during training of S. It does not appear that can discern low or high-likelihood samples under T.
To discuss why S would possess PI, we apply findings and analysis framework of Arpit et al. [35] regarding the general learning dynamies of DNNs. [35] presents data-centric study of DNN learning with
In [35], "easy samples" are ones that fit better some patterns in the data (and correspondingly "hard samples"). The easy and hard samples exhibit high and low likelihoods in
DNNs resp. as discussed in [35]. Furthermore, an important finding from [35 is that, in DNNs learning, the models learn simple and general patterns of the data first in the training stage to fit the easy samples.
We apply the framework of [35 to understand our learning of S and the reason why S would possess P1. Fig. 2(a) illustrates easy and hard samples in our problem: patterns of face identities can be observed in some samples (easy samples), while other samples (hard samples) exhibit diverse appearance. Similar to 35 Fig. (b)) shows that these easy and hard face samples tend to have high and low likelihood under T. Fig. [(c) shows the learning of S on these easy and hard samples at different epochs. Consistent with the "DNNs Learn Patterns First" finding in [35]. S learns general identity patterns first to fit the easy samples. Therefore, Ps of easy samples improve at a faster pace in the training, and many of them achieve high Ps. As easy samples tend to have high Pr, we observe Pl S. For the hard samples (which tend to have low PT), is uncommon for S to achieve high likelihood them they do not fit easily to the pattern learned by S.
Experiments
In this section, we present extensive experiment results and ablation studies: (i) We show that our proposed T-ACGAN can lead better surrogate models compared to alternative approaches (Sec. 6.2). (ii) We show that our proposed approach LOKT can significantly outperform the existing SOTA label-only MI attack (Sec. 6.3). (iii) We present additional results (Sec. (6.4) to demonstrate the efficacy of LOKT against SOTA MI defense methods. We further show that LOKT compares favorably in terms of query budget compared to existing SOTA, Additional experiments/analysis provided in Supplementary.
6.1 Experimental Setup
To ensure fair comparison, we adopt the exact experimental setup used in BREPMI [6) In what follows, we provide details of the experimental setup.
Table 1: Details of target model
T. To showcase the effectiveness of our proposed method, we conduct a comprehensive set of 30 experi- ments, covering 10 different setups.
40 Face- scrub 41) and Pubfig83 [42). We further study Label- Only MI attacks under distribution shift using FFHQ dataset [43 which contains images that vary in terms of background, ethnicity, and age. Following 2 6), we divide each dataset (CelebA/ Facescrub/ Pubfig83) into two non-overlapping sets: private set Pprie for training the target model T, and public set Dpub for training GAN/T-ACGAN, More details on datasets and attacked identities can be found in Supplementary.
Dataset. We use three datasets, namely CelebA
D Priso
Architecture
classes
FaceNet64 [3]]
IR152 [3]
CelebA
VGG16 38
1,000
BIDO-HSIC
39
MID 9
Facescrub Publig83
FaceNet64 [6
200
FaceNet64
50
Target Models. Following 2 6), we use 3 target models T including VGG16 38 152 37], and FaceNet64 [36]. All target models are provided in
Fig e are Pr and Ps of x G(z, y) from T-ACGAN, More details in Supp.

---

Face Embeddings
(a) Easy & Hard samples
13A
Easy
Hard
10
42.
-122
(b) 7 b'hhaviorr under
(c) Training Dynamics of Easy & Hard samples
Easy
Hard
Easy Hard samples
epoch
epoch 100
epoch 200
63900
hard
33903
15933
0.0
0.2
0.6
3.5
0.5 025 0.2 0.1 05
Figure apply the framework of [35] to analyze learning dynamics S reason why possesses property Pl, and therefore could be an effective proxy for T under MI. We analyze generated samples xy from our T-ACGAN for 3 identities (IDs 20, 16, 36). Note that zy analysis relevant generated samples are used in MI attacks. (a): We analyze face embeddings of æy extracted from publicly available SOTA face recognition model here Different clusters and different distances from cluster centroids can be observed, suggesting patterns of face identities in some samples (easy samples) while diverse appearance other samples (hard samples). We use distances from centroids to identify easy samples 2 and hard samples z²) (easy samples are indicated using transparent blue circle for each ID in the visualization). Visualization of æG and z in image space further demonstrates identity patterns in 27 and diverse appearance in x².. (b): Similar to [35 we observe that 5 and x tend to have high and low likelihood under T (P)-) resp (training data). (c): We track likelihood under S (Ps) for st and x during the training of 5. As training progresses, Ps of 2 and xh improve, and samples move up vertically (note that Pr of samples do not change). Consistent with the "DNNs Learn Patterns First" finding in (35) S learns general identity patterns first to fit the easy samples. Therefore, Ps of x improve at faster pace the training, and many of them achieve high Ps at epoch 200. As z tend have high Pr, we observe property P1 in S. For x (many of them tend to have low Pr). it is uncommon for S to achieve high likelihood on them as they do not easily to the pattern learned by S. See Supplementary for additional details and analysis. Best viewed in color.
the following methods/ models for evaluating the attack performance under SOTA MI defense methods: BiDO-HSIC 3
Additionally,
MID 9 The details are included in Table [
Evaluation Metrics. Following 6 I 21, we the following metrics to quantitatively evaluate the performance of MI attacks. Further, we also conduct user studies to assess the quality reconstructed data (Sec. 6.5).
Attack Accuracy (Attack acc.): Following 2 we utilize an evaluation model, E, which
employs distinct architecture and is trained on D prév E serves as a proxy for human inspection 1 Higher attack accuracy indicates superior performance.
KNN Distance (KNN dt.): The KNN distance indicates the shortest distance between the recon- structed image of specific identity and its private images. Specifically, the shortest distance is computed using the l2 distance in the feature space, using the evaluation model's penultimate layer.
MI
https::/ggttnni wtw.aaa..coomo.mmmoom Saan///aaahheen. WWaagg/////a-- mi defense
Following previous work, E' can also be trained on p and samples from additional identities. This could improve generalization of E' for accurate evaluation.

---

A smaller KNN distance signifies that the reconstructed images are more closely aligned with the private images of the target identity.
6.2 Training surrogate model with different algorithms
In this section, we demonstrate that our proposed T-ACGAN can lead to better surrogate models for MI. We describe set of alternative approaches that can be used to train surrogate models using pun and compare the performance of these approaches with our proposed method. Specifically, we consider a set of five algorithms, which be broadly classified into three categories, for learning the surrogate model S:
Directly use the public dataset Puub- We present two methods to train S: Direct I. We train
S using the public dataset labelled with target model, i.e. Dyub with samples (1pps M), Tp pub.
ý = T(pp): see Sec. 4.1 Direct II. We apply data augmentation to Ppub of Direct I reduce the class imbalance in Direct I, followed by training S using the newly more balanced dataset.
Training an ACGAN. We provide two versions: ACGAN We train an ACGAN model on ACGAN I. We train ACGAN model on augmented DD Pub used in Direct II. As C o D in ACGAN serves as a classifier, we use C o D for MI attacks.
D pub used in Direct
Training proposed T-ACGAN. We use our proposed method described in Section 4.3 to train T-ACGAN. Similar to ACGAN I and II, we use C' o D after training T-ACGAN for the attack.
For this comparison, we utilize the following settings: == FaceNet64, Dprio CelebA, pub CelebA. Both ACGAN and T-ACGAN adopt the SNResnet architecture [34 44]. To ensure a fair comparison, we use the same architecture as Co D in ACGAN and T-ACGAN for the surrogate model S in Direct I and Direct II. Detailed architecture specifications can be found in the Supplementary. After training the models, we employ the widely-used KEDMI [2] the white-box attack on the trained surrogate models. Table presents the results.
Table 2:
We compare different ap- proaches to train surrogate model for MI attacks. We utilize the following settings: T = FaceNet64, 'D Drrie = CelebA, D puub CelebA, and em-
ploy the KEDMI for MI attacks.
Algorithm
Attack acc.
KNN dt.
Direct
5.87 1.65
1936.12
Direct 11
9.60 2.22
1890.16
The effectiveness of T-ACGAN in training surrogate models
ACGANI 1
6.47 2.15
1771.26
for MI attacks can be observed.
ACGAN
7.87 3.10
1785.20
T-ACGAN
42.07 3.46
1473.99
6.3 Comparison against SOTA label-only MI attack
Standard MI attack setup. In this section, we present the results obtained from the standard attack setup on three datasets: CelebA, Facescrub, and Pubfig83, as detailed in Table We evaluate three designs of surrogate: (i) We directly use C' o D from our T-ACGAN the surrogate model. The architecture of T-ACGAN can be found in the supplementary material. (ii) We utilize the synthetic data generated by G of our T-ACGAN and label it using the target classifier T to train another surrogate model, denoted as S = Densenet-161 [45]. (iii) We employ the same data as in (ii) to train ensemble of surrogate models, denoted as Sen. using different architectures including Densenet-121. Densenet- 1611, and Densenet-169.
We compare our results with the state-of-the-arr (SOTA) label-only MI attack BREPMI (6] To conduct our attacks, we utilize white-box PLGMI 7 on the surrogate models. Since PLGMI performs attacks using conditional GAN trained with white-box access of the target classifier, we replace with our T-ACGAN, which becomes available for use after training the surrogate models.
Our proposed method LOKT demonstrates a significant improvement in Attack accuracy and KNN distance compared to the SOTA label-only MI attack BREPMI (6) Our top I attack accuracies are better than BREPMI from from 17.2% to 29.87% across all setups when we utilize the ensemble Sen.
Fig. (f) presents visual comparison of various methods under the setup Ppriv CelebA, Dpub CelebA. More results are available in the Supplementary. Results clearly indicate that LOKT produces images that are closer to the ground truth (private data) compared to BREPMI (6). This outcome provides strong evidence of the effectiveness of our approach in generating realistic images that closely resemble private data, which is critical for conducting successful MI attacks.
MI attacks under large distribution shift. Table compares the MI attack results the large distribution shift setup, where we use Ppus~ FFHQ, Dyriv CclebA/ Facescrub/ Pubfig83, and
8

---

Table 3: We conduct comprehensive experiments to compare our proposed method LOKT and existing SOTA BREPMI [6] across standard MI attack benchmarks. Specifically, we evaluate the performance of our three proposed designs of surrogate, namely C o D. S, and Sen, while BREPMI performs black-box search on T directly. We highlight the best results in each setup in bold.
Setup
Attack
Attack acc. 1 KNN
Setup
Attack
Attack acc.
KNN dt.
T FaceNet64
BREPMI
73.93 + 4.98
1284.41
T FaceNet64
BREPMI
55.60 +:2 4.34
1012.83
D pris CelehhA
Co
81.00 + 4.79
1 1298.63
D proo Pubfig83
C. D 74,80 +: 5.93
924.58
D
Puuh CelebA
LOKT S
92.80 + 2.59
1207.25
put Pubfig83
LOKT 8
61.60 + 3,58
993.44
Sen
93,93 + 2.78
1181.72
Sen
80.00 + 3,16
883.52
BREPMI
71.47 + 5.32
1277.23
BREPMI
72.80 + 3.90
971.51
T
IR152
T
FaceNe164
D Prrcs CelehA
Co 72,07 : 4.03
1358.94 94
D pric
Pubfig83
Cc 85,60 +: 2.61
914.15
D puh CelebA
LOKT S
89,80 + 2.33
1220.00
puh FFHQ
LOKT 8
88,40 +: 2.97
920.99
Sen
92.13 + 2.06
1206.78
Sen
94,40 + 3.85
862.24
BREPMI
57.40 +: 4.92
1376.94
BREPMI
40,20 + 6.60
1236.4
T
VGG16
T
FaceNet64
D Daris CelehA
Co D 71.33 :: 4.39
1364.47
D
Prrtu Facescruh
C.a D. 45,70 +: 4.00
1296.29
D puh CelebA
LOKT
85.60 +: 3.03
1252.09
D puuh Faceserub
LOKT 8
53,20 +: 1280,70
Sen
87.27 + 1,97
1246.71
S.m
58.60 +: 4.86
1225.13
BREPMI
43,00 =: 5.14
1470.55
BREPMI
37.30 + 3,99
1456.59
T
FaceNer64
T
FaceNet64
D préc CelehA
Co D 43.27 + 3.53
1516.18
proo Facescruh
C D
44.50 + 5,98
1403.73
Ppuh FFHQ
LOKT S
59.13
:: 2.77
1437.86
Dpub FFHQ
LOKT $
47.20 +: 4.39
1404.85
62.07
3.89
1428.04
53.70 4.57
1338.67
Table 4: We report Laly MI Attack results
Table 5: The comparison of the number of queries (in millions) used by LOKT and BREPMI (6]. The attacks using S and Sen con- sume additional 500k queries comparing to CoD to label the synthetic images to train S and San.
under SOTA defense models namely BiDO [39]
and MID (Р] We use 2 Pprii CelebA, Dyub
CelebA. We highlight the best results in bold.
Sctup
Attack
Attack acc.
KNN
T= BIDO (39
37.40 = 3.66
1500.45
Our results show that we use fewer number of
Paris CelebA
Co
45.73 + 5.94
1493.48
queries than BREPMI in all setups.
puo =CelebA
LOKT S
S.n
58.53 = 4.87
64.73 + 3.07
1395.93
1427.22
LOKТCOD) LOKT 8/5cn
BREPMI
BREPMI[6
39.20 :: 4.19
1458.61
FaceNe164
12.16
12.66
17.98
T MID M
IR152
12.16
12,66
18.06
D CelebA
D
44.13 + 3,54
1475.73
VGG16
12.16
12.66
18.12
pu6 CelebA
LOKT S
55.33
1 4.40
1393.76
BIDO-HSIC
12,16
12.66
18.39
S.
60.33 : 4.76
1374.34
MID
12.16
12.66
18.25
T = FaceNet64. The attack results of BREPMI drop significantly (by 30.93% for CelebA and 2.9% for Facescrub), while the results for Pubfig83 notably increase, which can be attributed to the small size of the Pubfig83 dataset (6. Our proposed method outperforms BREPMI, with the top 1 attack accuracies increase from 16.40% to 21.600 1..60% for all setups. Moreover, the KNN distance indicates that our reconstructed images are closer to the private data than those reconstructed by BREPMI.
6.4 Additional results
MI attack results using MI defense model. We investigate the attacks on the MI defense model (see Table 4). Specifically, we utilize the SOTA defense model BiDO-HSIC (39] and MID [9. Our results indicate that BiDÓ-HSIC successfully reduce the effectiveness of the white-box SOTA attack, PLGMI, by 9.57% (See the result in the Supplementary). In the label-only setup, the performance of BREPMI becomes relatively low with attack accuracy of only 37.40% for BiDO-HSIC (39 and 39.20% for MID 9 In contrast, our approach achieves a much higher attack accuracy of 60.73% and 60.33%, almost doubling the performance of BREPMI. These results demonstrate that our approach effective in conducting MI attacks on MI defense models, even in scenarios where the adversary has limited information about the target classifier.
High resolution. We conduct the experiment with high resolution images which has not been addressed yet for label-only setup 6 In particular, we train a new target classifier T = Resnet-152 using CelebA setup with the image size 128 128. For fair comparison between BREPMI and our proposed method, T-ACGAN has the same GAN architectures used by BREPMI. The details of the architecture can be found in the Supplementary.
The results are shown in Table 6 LOKT outperforms BREPMI, with top 1 accuracy surpassing BREPMI by 20.27%. Our inverted images are closer to private training samples than BREPMI

---

Table 6: We conduct the experiment with higher resolution images. We use 7' Resnet-152, D prin CelebA, Puub CelebA, image size 128 128.
Table 7: User study results. Our human study reveals that users distinctly favor our approach, with 64.30% user preference for images recon- structed using our proposed approach, LOKT, compared to BREPMI's lower 35. 70% user pref- erence.
The natural accuracy of T is 86.07%. We highlight the best results bold.
Setup
Attack
Attack ncc. 1
KNN
BREPMI6
50.33 4.71
1389.09
Method
User Preference (1)
D. = CelehA
Co
66.87 = 3,93
1356.53
BREPMI
35.70%
Pps CelebA
LOKT
66.80 3,83
1341 0.
S..
70,60 4.43
1320.16
LOKT
64.30%
(smaller KNN distance). We believe our study can provide new insight on the effectiveness of SOTA label-only attack at a higher resolution of 128 x 128, paving the way future label-only model inversion attacks at resolutions beyond 128x 128.
Query budget. In this experiment, compare query budget between our proposed method and BREPMI [2] In the BREPMI, queries to the target classifier are required to identify the initial points for attacking and estimate the gradients during the attack. In method, queries T are required to label the synthetic data during the training of T-ACGAN to obtain C' o D, and additional 500k queries to label generated images of T-ACGAN to train S and the ensemble Sen- For comparison, as shown in Table we use Dprie = CelebA and Pmu CelebA. The results show that our proposed method requires 30% fewer queries compared to BREPMI.
6.5
User study
User study setup. In this section, we go beyond objective metrics and consider subjective evaluation of MI attacks. In particular, we conduct a human study to understand the efficacy of our proposed method, LOKT, compared to BREPMI. We follow the setup by [10] for human study and use Amazon Mechanical Turk (MTurk) for experiments. The user interface is provided in the Supplementary. In this study, users are shown 5 real images of a person (identity) as reference. Then users are required to compare the 5 real images with two inverted images: one from method (LOKT), the other from BREPMI. We use D Prite = CelebA, Dpub CelebA and FaceNet64. Following [10], randomly selected 50 identities with 10 unique users evaluating each task accounting 1000 comparison pairs.
Our human study reveals that users distinctly favor our approach, with 64.30% user preference for images reconstructed using our proposed approach, in contrast to BREPMI's lower 507 user preference. These subjective evaluations further show the efficacy of our proposed method, LOKT, the challenging label-only MI setup.
User study results. We report the user study results in Table 7
Discussion
Conclusion. Instead of performing black-box search approach in existing SOTA, propose new label-only MI approach (LOKT) by transferring decision knowledge from the target model to surrogate models and performing white-box attacks the surrogate models. To obtain the surrogate models, we propose a new T-ACGAN to leverage generative modeling and the target model for effective knowledge transfer. Using findings of general learning dynamics of DNNs, analysis to support that our surrogate models are effective proxies for the target model under MI. We perform extensive experiments and ablation to support our claims and demonstrate significant improvement over existing SOTA.
conduct
Broader Impacts. Understanding model inversion attacks holds significance as Al models continue to see widespread deployment across various applications. By studying and understanding the approaches and methodologies for model inversion, researchers can develop good practices in deploying AI models and robust defense mechanisms for different applications esp. those involving sensitive training data. It is important emphasize that the objective of model inversion research is to raise awareness of potential privacy threats and bolster our collective defenses.
Limitations. While our experiments are extensive compared to previous works, practical applica- tions involve different types of private training datasets such as healthcare data. Nevertheless, our assumptions general, and we believe our findings be applied to broader range of applications.
10

---

Acknowledgements. This research is supported by the National Research Foundation, Singapore under its AI Singapore Programmes (AISG Award No.: AISG2-TC-2002-0001 and SUTD project PIE-SGP-AI-201 8-01. This research work is also supported by the Agency for Science, Technology and Research (A *STAR) under its MTC Programmatic Funds (Grant No. M231.70002),, This material is based on the research/work support in part by the Changi General Hospital and Singapore University of Technology and Design, under the TeahthTech Innovation Fund (HTIF Award No. CGH-SUTD-2021-40 We thank anonymous reviewers for their insightful feedback.
References
[1] Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The secret revealer: Generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 253-261, 2020.
[2] Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi. Knowledge-enrichee distributional model inversion attacks, Proceedings of the IEEE/CVF international conference on computer vision, pages 16178-16188 16187, 2021.
[3] Kuan-Chich Wang, Yan Fu, Ke Li, Ashish Khisti, Richard Zemel, and Alireza Makhzani. Variational model inversion attacks. Advances in Neural Information Processing Systems, 34:9706-997 97099,9999966 2021.
[4] Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 240, 2019.
[5] Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correira, Antonia Adler, and Kristian Kersting. Plug and play attacks: Towards robust and flexible model inversion attacks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pages 10522-20545 PMLR, 17-23 Jul 2022.
[6] Mostafa Kahla, Si Chen, Hoang Anh Just, and Ruoxi Jia. Label-only model inversion attacks via boundary repulsion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pages 15045-15033, 2022.
[7] Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang. Pseudo label- guided model inversion attack via conditional generative adversarial network. Thirty Seventh AAAI Conference on Artificial Intelligence (AAAI (4A)23),200 2023.
[8] Mahdi Khosravy, Kazuaki Nakamura, Yuki Hirose, Naoko Nitta, and Noboru Babaguchi. Model inversion attack by integration of deep generative models: Privacy-sensitive face generation from face recognition system. IEEE Transactions on Information Forensics and Security, 17:357-372, 2022.
[9] Tianhao Wang, Yuheng Zhang, and Ruoxi Jia. Improving robustness to model inversion attacks via mutual information regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11666 111666116673 11673, 2021.
[10] Shengwei An, Guanhong Tao, Qiuling Xu, Yingqi Liu, Guangyu Shen. Yuan Yao, Jingwei Xu, and Xiangyu Zhang. Mirror: Model inversion for deep learning network with high fidelity. In Proceedings of the 29th Network and Distributed System Security Symposiun, 2022.
[11] Ngoc-Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Cheung. Re- thinking model inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16384 -6393,, June 2023.
[12] Ulrich Aivodji. Sebastien Gambs, and Timon Ther. Gamin: An adversarial approach to black-box model inversion. In The AAAI Workshop on Privacy-Preserving Artificial Intelligence, 2020.
[13] Gyojin Han, Jachyun Choi, Hacil Lec, and Junmo Kim. Reinforcement learning- based black- box model inversion attacks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 20504-20513, June 2023.
[14] Ian Goodfellow, Jean Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, 2014.
Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron
11

---

[15] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[16] Tero Karras, Samuli Laine, Miik Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[17] Keshigeyan Chandrasegaran, Ngoc-Trung Tran, and Ngai-Man Cheung. closer look fourier spectrum discrepancies for cnn- -nneereetted images detection. In Proceedings of the IEFFF/CV conference on computer vision and pattern recognition, pages 7200-7209, 2021.
[18] Milad Abdollahzadeh, Touba Malekzadeh, Christopher TH Teo, Keshigeyan Chandrasegaran, Guimeng Liu, and Ngai-Man Cheung. survey on generative modeling with limited data, few shots, and zero shot. arXiv preprint arXiv:2307. 14397, 2023.
[19] Yunqing Zhao, Chao Du Milad Abdollahzadeh, Tianyu Pang. Min Lin, Shuicheng Yan, and Ngai- Man Cheung. Exploring incompatible knowledge transfer in few-shot image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7380-7391, 2023.
[20] Christopher TH Teo, Milad Abdollahzadch, and Ngai-Man Chcung. Fair generative models via transfer learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2429-24222222 2437, 2023.
[21] Yunging Zhao, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Man Cheung. Few-shot image generation via adaptation-aware kernel modulation. Advances Neural Information Processing Systems, 35:1 19419447-19440, 2022.
[22] Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Cheung. On measuring fairness in generative models. Advances in Neurul Information Processing Systems, 36, 2023.
[23] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In International conference on machine learning, pages 2642-2651. PMLR, 2017.
[24] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unitied embedding for face recognition and clustering. In Proceedings of the IEEE conference computer vision and pattern recognition, pages
2015.
[25] Yaniv Taigman, Ming Yang. Mare' Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap human- level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1701-1708, 2014.
[26] Qiang Meng. Shichao Zhao, Zhida Huang, and Feng Zhou. Magface: A universal representation for face recognition and quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14225-14234, 2021.
[27] Yuge Huang, Yuhan Wang. Ying Tai, Xiaoming Liu, Pengcheng Shen. Shaoxin Li, Jilin Li, and Feiyue Huang. Curricularface: adaptive curriculum learning loss for deep face recognition. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5901-5910. 2020.
[28] Benoit Dufumier, Pietro Gori, Julie Victor, Antoine Grigis, Michele Wessa, Paolo Brambilla, Pauline Favre, Mircea Polosan, Colm Medonald, Camille Marie Piguet, et al. Contrastive learning with continuous proxy meta-date for 3d mri classification. In Medical Image Computing and Computer Assisted Intervention MICCAl 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part II 24, pages 58-68. Springer, 2021.
[29] Jiawei Yang, Hanbo Chen, Jiangpeng Yan, Xiaoyu Chen, and Jianhua Yao. Towards hetter understanding and better generalization of few-shot classification in histology images with contrastive learning. 2022.
[30] Jonas Dippel, Steffen Vogler, and Johannes Höhne. Towards fine-grained visual representations by combining contrastive learning with image reconstruction and attention-weighted pooling. arXiv preprint arXiv:2104.0432. 043303, 2021.
[31] Matthew Fredrikson, Eric Lantz Somesh Jha. Simon Lin, David Page. and Thomas Ristenpart. Privacy in pharmacogenetics: An (End-to-End) to-End) case study of personalized warfarin dosing. In 23rd USENIX Security Symposium (USENIX Security 14), pages 17-32, 2014.
12

---

[32] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications the ACM, 63(11):139 144, 2020.
[33] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 314-223. PMLR. 2017.
[34] Takeru Miyato and Masanori Koyama. cGANs with projection discriminator. In International Conference on Learning Representations, 2018.
[35] Devansh Arpit, Stanislaw Jastrzebski. Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville. Yoshua Bengio, and Simon Lacoste-Julic closer look at memorization in deep networks. In /CML, 2017.
[36] Cheng, Jian Zhao, Zhecan Wang, Yan Xu, Karlekar Jayashree, Shengmei Shen, and Jiashi Feng. Know you at one glance: A compact vector representation for low-shot learning. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1924 1934-1932.20 2017.
[37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778 7078,, 2016.
[38] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arXiv:140.
1556, 1556, 2014.
[39] Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han. Bilateral dependency optimization: Defending against model-inwe inversion attacks. In KDD, 2022.
[40] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In P Proceedings of the IEEE international conference computer vision, pages 3730-3738, 2015.
[41] Neng-Wei Ng and Stefan Winkler. A data-driven approach to cleaning large face datasets. In 2014 IEEE international conference image processing (ICIP). pages 343-347. IEEE, 2014.
[42] Nicolas Pinto, Zak Stone, Todd Zickler, and David Cox. Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook. In CVPR 2011 WORKSHOPS, pages 35-42, IEEE, 2011.
[43] Tero Karras, Samuli Laine. and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and patiern recognition, pages 4401-4410 2019.
[44] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. International Conference on Learning Representations, 2018.
[45] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 1700-4708. 2017.
13


Reviews:
[{"review_id": "YSAoxdhPTA", "reviewer": "Reviewer_VBSE", "summary": "This paper proposes a two-step label-only model inversion attack using knowledge transfer. In the first step, a surrogate model is trained via knowledge transfer. In the second step, a white-box model inversion attack is performed on the surrogate model. The authors explore several approaches to train the surrogate including learning-based model extraction using a public dataset labeled by the target model and the use of a classifier head of the discriminator of a GAN trained on a public dataset labeled by the target model. ", "questions": "It will be great if the authors can elaborate more on the difference between T-ACGAN and GAMIN [10]. The authors briefly describe GAMIN as an \"an adversarial approach for black-box MI\". However, the current proposal shares several similarities with GAMIN, namely the use of GANs to learn a surrogate, which is then used to perform the MI attack. Furthermore, the authors classify GAMIN as a black-box MI attack relying on soft labels, while Algorithm 1 in [10] seems to indicate it could be considered a label-only MI.\n\n- How does the intra-class diversity of the dataset affect the result of the attack?\n\n- What explains the difference in performance between ACGAN scenarios described in Section 6.2 and T-ACGAN given they seem to rely on $\\widetilde{D}_{pub}$ which is obtained using the target model\n\n[10] Ulrich Aivodji, Sebastien Gambs, and Timon Ther. Gamin: An adversarial approach to black-box model 364 inversion. In The AAAI Workshop on Privacy-Preserving Artificial Intelligence, 2020.\n", "limitations": "N/A", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Several experiments have been performed to evaluate the performance of the proposed solution against different target models.", "weaknesses": "The main weakness is the lack of comparison with existing works that also rely on GAN (cf. questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> It will be great if the authors can elaborate more on the difference between T-ACGAN and GAMIN [10]. The authors briefly describe GAMIN as an \"an adversarial approach for black-box MI\". However, the current proposal shares several similarities with GAMIN, namely the use of GANs to learn a surrogate, which is then used to perform the MI attack. Furthermore, the authors classify GAMIN as a black-box MI attack relying on soft labels, while Algorithm 1 in [10] seems to indicate it could be considered a label-only MI.\n\nThank you for the comment. There are major fundamental differences between **GAMIN (for black-box attack and requires soft prediction)** and **T-ACGAN (our proposed algo for label-only attack and does not require soft prediction)**. See below Table for comparison.\n\nWe have read GAMIN paper and examined their code very carefully. Kahla et al. [6] also suggest GAMIN is black-box.  \n\n\nTable to compare GAMIN [a] and Ours: \n|  | **GAMIN [a]** | **Ours** |\n|-|-|-|\n|Training objective | Boundary-Equilibrium Adaptive loss [a] | Extended ACGAN loss |\n|Attack type | Black box | Label only |\n|Attack setting | Learn a surrogate for each target identity  | Learn a surrogate for all target identities |\n|Post-processing | Require (filtering etc.), see Fig D in attached PDF. Low-quality raw output | Not required |\n|Performance | Appear to be inadequate under black-box setting; 20% for MNIST [a] , 20% for CNN.  Note that the code of GAMIN has missing library, cannot run. See Fig D in PDF | Reasonable under more restricted label-only setting, e.g. 93.93% for CelebA, see our paper Table 3.|\n\nMore details to compare GAMIN and ours:\n\nCentral to GAMIN is the use of “Boundary-Equilibrium Adaptive loss” [a] as the training objective, which requires soft prediction from the target model $T$ to compute cross-entropy loss $L_H$. GAMIN's training objective:\n\n$L_S = L_H(y_t, Y_S) - k_t * L_H(y_t, Y_G)$\n\nSee Fig. C in the attached PDF, where we highlight training objective in GAMIN algo.\nNote that soft prediction from target model $Y_S$ and $Y_G$ are required for cross-entropy $L_H$ of label $y_t$. The main idea behind GAMIN objective is to learn the distinction between noise inputs and generated samples.\n\nIn our T-ACGAN, we develop an extended AC-GAN loss as training objective (our paper Eq. 4): \n\n$\\mathcal{L}_{D,C} $\n\n$=  - E[\\log P(s = Fake| x_{f})] - E[\\log P(s = Real| x_p)]  - E[\\log P(c = \\tilde{y} | x_f)] $\n\nwhere $\\tilde{y} = T(x_f)$.\nOur main idea is to use hard prediction of target model $T$ to obtain $\\tilde{y}$ to learn the classifier head. Critically, no soft prediction from $T$ is needed.\n\nThe difference between GAMIN's learning objective and ours is fundamental and significant.\n\nWe remark that in their code, they replace cross-entropy loss with L1 loss, soft prediction is still required. We have submitted their code for reviewer easy reference, See Line 82,83 in trainer.py. See also Line 30, 31 in utils.py.\n\n[a] U. Aivodji et al. GAMIN: An Adversarial Approach to Black-Box Model Inversion. \n\n$\n$\n> What explains the difference in performance between ACGAN scenarios described in Section 6.2 and T-ACGAN given they seem to rely on   $D_{pub}$   which is obtained using the target model.\n\nThe main reason is that for T-ACGAN we leverage diverse generated data to achieve rich decision knowledge transfer.\nSpecifically, for ACGAN setups in Sec 6.2, $D_{pub}$ (or augmented $D_{pub}$) is labeled by target model $T$ for decision knowledge transfer to train the ACGAN. However, diversity of samples is limited by $D_{pub}$. Data augmentation only adds superficial variation.\nFor our T-ACGAN, by integrating $T$ into ACGAN, generated samples are labeled by $T$ for decision knowledge transfer (Ln 146). The generated data randomly sampled from $G$ is very diverse, enabling transfer of rich decision knowledge. Furthermore, as training progresses, $G$ improves its conditional generation (see Fig. A.8 in Supp), achieving much class-balance decision knowledge transfer. \n\n$\n$\n>How does the intra-class diversity of the dataset affect the result of the attack? \n\nReviewer's comment is insightful. First, we would like to emphasize that we follow strictly experiment setups in BREPMI [6].  Therefore, our experiment has already included private training data with considerable intra-class diversity, see Fig. E for some examples.\n\nWe hypothesize that it becomes more challenging for all MI attacks (white-box, black-box, label-only) when intra-class diversity increases. It is because, with increased intra-class diversity, the variance of the feature distribution increases, and it becomes more difficult to reconstruct the identity features. However, the investigation needs much effort, e.g. to take into account the effect of reduced natural accuracy (model utility) due to increased intra-class diversity. We will look into this as future work as this is beyond the scope of this work ( new label-only attack)."}, {"review_id": "HauIdpJg8F", "reviewer": "Reviewer_Um8m", "summary": "The paper focuses on the setting where the adversary has only label access to the target model. The authors suggested to use transfer learning and generative models to improve the model inversion attacks. In particular the authors proposed T-ACGAN which is a variation of ACGAN that leverages the access to the target model and uses the label that the target model generates. ", "questions": "It would be interesting to also include structural similarity scores for the evaluations.\n\nIt would be interesting to compare the knn distances of the attacks with the average distance between the images of the same class. The idea here is to see if the attack only gives an average image from the class or can it give more details. Same for the other metrics.\n\n\n", "limitations": "Limitation are addressed in the work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "\nThe paper considers a realistic thread model where the adversary does not have whitebox access to the model.\n The authors also evaluate different datasets with different metrics. The attack also reduces the number of required queries.\nThe authors also evaluated the settings where the public and the private dataset are not the same.\n\n", "weaknesses": "The details about the target labels and the model that is used to compute the attack accuracy metrics is missing. Structure of the paper can be improved to be more self-contained.\n\nWhile the work follows the existing evaluation scheme of the previous works, the results are hard to interpret. The attack accuracy metrics mainly relies on the classification accuracy of the generated images which might not be the cause of the privacy leakage. One suggestion can be instead of using the target model feature space, the authors can use some of the open source image embedding models such as CLIP in their evaluation which might make the results easier to understand. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\n> In Fig 1.c The authors show that the adversary does not have access to the hard labels ? is that correct?\n\nThank you for pointing out this to us. It is a typo, the adversary has access to the hard labels.\n\n$ $\n\n> Looking at the **membership inference attack** literature, the better way of measuring the MIA performance is by looking at false positive true positive curve and not accuracy [R1]. However the work only reports accuracy.\nThe authors did not compare their results with existing approaches that use label-only setting[R2]. In particular I think a better baseline would be to compare the results (preferably ROC) using [R2] to estimate the confidence of labels and then use techniques from [R1] to do a hypothesis testing for membership inference attacks. The current comparison is not detailed.\n\n>[R1] Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., & Tramer, F. (2022, May). **membership inference attack** from first principles. In 2022 IEEE Symposium on Security and Privacy (SP) (pp. 1897-1914).\n\n>[R2] Choquette-Choo, Christopher A., et al. \"Label-only **membership inference attacks**.\" International conference on machine learning. PMLR, 2021.\n\nDear Reviewer Um8m,\n\nOur paper focuses on Model Inversion, not Membership Inference as the reviewer suggested. Therefore, we believe your comments are completely irrelevant and out of context.\n\n **Model Inversion and Membership Inference are totally different** as discussed in many works [a,b,c,d]. \nFor model inversion, the objective is to **infer and reconstruct the training data of a trained model.**\nFor membership inference, the objective is to **determine whether a specific sample was used to train the model or not**.\n\nNevertheless, **we do appreciate a lot Reviewer's valuable time in reading our submission**. Thank you very much.\n\n[a] Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, Dawn Song. The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks. CVPR-2020.\n\n[b] Tanuwidjaja, H. C., Choi, R., Baek, S., & Kim, K. (2020). Privacy-preserving deep learning on machine learning as a service—a comprehensive survey. IEEE Access 2020.\n\n[c] Rigaki, M., & Garcia, S. (2020). A survey of privacy attacks in machine learning. arXiv preprint arXiv:2007.07646.\n\n[d] Yang Ziqi, Shao Bin, Xuan Bohan, Chang Ee-Chien, and Zhang Fan. 2020. Defending model inversion and membership inference attacks via prediction purification. arXiv preprint arXiv:2005.03915 (2020).\n"}, {"review_id": "paZpnE8Iaa", "reviewer": "Reviewer_txcd", "summary": "This paper proposes a label-only model inversion (MI) attack. It focuses on a more challenging scenario where the target network only outputs the label. It trains a class-conditional GAN based on some public data and the label information provided by the target network. A surrogate network can be trained at the same time as training the GAN or using the fake data generated by the trained GAN. Then they use the existing white-box MI attack method on the surrogate network to generate the images. The generated images are used to complete the label-only attack on the target model. Experiments conducted on three datasets and models show its effectiveness in terms of attack accuracy and feature similarity with the ground truth image.", "questions": "1. What are the target labels in the evaluation? How are they selected? \n\n2. How can the method attack target labels that are not sampled during training TACGAN?\n\n3. Are there overlapped identities/labels in the private and public datasets?\n\n4. Can the proposed method successfully attack larger models and datasets?\n\n5. Can humans correctly recognize the inversion results and the target labels?\n\n6. Is the proposed method robust against [MID]?", "limitations": "Possible limitations are 1) high time cost, 2) about 13 million required queries, and 3) uncommon labels.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper targets a more challenging MI attack scenario where the subject model only returns the labels.\n2. Experimental results show the proposed method outperforms one existing baseline.\n3. Code is provided.", "weaknesses": "1. Some critical details of the evaluation are missing. What are the target labels? How are they selected? When splitting the dataset into private and public datasets, is the separation based on identities/labels? That is, are there overlapped labels in the private and public datasets?\n\n2. It's not clear how this method can handle the rare labels (similar to the class imbalance issue mentioned in this paper). If T has 1K classes in the private dataset, I wonder if the synthesized images in eq 4 can equally cover all the 1K. If not, how can the method attack those uncovered labels?\n\n3. The evaluated models and datasets are not large enough. It would be more persuasive if the method can attack models trained on larger datasets with more classes. For example, ResNet50 trained on VGG Face2, and Inception Resnet on CASIA.\n\n4. User studies are necessary to show the leakage of privacy brought by the inversion results. For example, show the inversion result of one target label and images of the target label and other several labels to users and check if they can recognize the target label. Similarly, show one image of the target label and the inversion results of the target label and other labels.\n\n5. Line 188. Based on the current definitions of P1 and P2, they are equivalent. Use $Pr(x)$ denotes the probability of sample $x$ has high $P_T$. P1 means if $x$ has high $P_S$, then $Pr(x)$ is high. P2 means if $x$ has high $P_S$, then $1-Pr(x)$ is low. They are describing the same thing. However, according to line 212, P2 should be it's uncommon for low-likelihood samples under $T$ to have high likelihood under $S$.\n\n6. Another defense method [MID] should be evaluated.\n\n7. Missing related work similar to cite [PLUG] [MIRROR].\n\n[MID] Wang, Tianhao, et al. “Improving Robustness to Model Inversion Attacks via Mutual Information Regularization.” AAAI 2021.\n\n[PLUG] Struppek, Lukas, et al. \"Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks.\" ICML 2022.\n\n[MIRROR] An, Shengwei, et al. \"Mirror: Model inversion for deep learning network with high fidelity.\" NDSS 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> What are the target labels? Are there overlapped identities/labels in the private and public datasets?\n\nPls see Ln 223: “To ensure a fair comparison, when evaluating our method, we use the **exact** same experimental setup as BREPMI [6].” Following [6], each dataset is divided into two distinct/ non-overlapping sets: private/ public dataset. There are no shared identities/ labels.\n\n$~$\n> If T has 1K classes in the private dataset, I wonder if the synthesized images in eq 4 can equally cover all the 1K.\n\nYes, our method can achieve adequate coverage for all 1K classes (see experiment below). Our decision knowledge transfer using target model enables generation of diverse conditional samples using our T-ACGAN (Eqn. 3, 4) adequately covering all 1K classes.\n\n**Experiment setup.** Let $x = G(z, y)$, $\\hat{y} = T(x)$. For each class $y$, $\\gamma$ represents the percentage of samples with y the same as $\\hat{y}$ (Supp. A.2). We use $T$ = FaceNet64, $D_{priv}$ = CelebA, $D_{pub}$ = CelebA setup to demonstrate the $\\gamma$ distribution for all 1K classes using our T-ACGAN. \n\n**Results.** The results are included in rebuttal PDF submission. As one can observe, synthesized images using T-ACGAN renders adequate coverage for all 1K classes with minimum $\\gamma$ value being 6.6%. \n\n$~$\n> The evaluated models/ datasets are not large enough.\n\nWe emphasize we **strictly** follow BREPMI [6] for model/dataset setups. Nevertheless, we conduct additional experiments. We use target classifier $T$ = ResNet50, which is trained on the $D_{priv}$ = VGGFace2 dataset consisting of 3.3 million images / 8631 training classes. We use $D_{pub}$ = FFHQ. We use E= Inception-V1 trained on VGGFace2.\nWe report the result in Table 1 [txcd]. Our method significantly improves attack accuracy and KNN distance compared to BREPMI, with top1 accuracy surpassing BREPMI by 12.6%-19.8%.\nTable 1 [txcd]. Attack results. We use $T$ = ResNet50, $D_{priv}$ = VGGFace2, $D_{pub}$ = FFHQ, $E$ = Inception-V1. \n| Attacks | Top 1 ($\\uparrow$) | KNN dist ($\\downarrow$) |\n|-|-|-|\n| BREPMI | 56.00 $\\pm$ 4.65 | 428.19 |\n| Ours ($C \\circ D$) | 68.60 $\\pm$ 3.98 | 389.28 |\n| Ours ($S$) | 71.20 $\\pm$ 3.03 | 397.37 |\n| Ours ($S_{en}$) | 75.80 $\\pm$ 2.01 | 379.50 |\n\n$~$\n> User studies are necessary to show the leakage of privacy brought by the inversion results. \n\n> Can humans correctly recognize the inversion results and the target labels?\n\nWe conducted a **human study** to reveal serious privacy leakage under our proposed method compared to BREPMI. We follow the setup proposed by An et al. [MIRROR] for human study. Specifically, in this study, • Users are shown 5 real images of a person (identity) as reference. • Then users are required to compare the 5 real images with two inverted images: one from our method, the other from BREPMI. We use $D_{priv}$ = CelebA, $D_{pub}$ = CelebA and $T$ = FaceNet64. Following An et al., we randomly selected 50 identities with 10 unique users evaluating each task accounting to 1000 comparison pairs. More details on human study, user interface, example, experiment details and results are in the PDF submission. \n\n**User study results.** Our human study reveals that users distinctly favor our approach, with **64.30% user preference** for images reconstructed using our proposed approach, in contrast to BREPMI’s lower 35.70% user preference. Detailed results included in PDF submission. This further shows the efficacy of our method.\n\n$~$\n> Line 188. Based on the current definitions of P1 and P2, they are equivalent. \n\nThank you, we will refine this discussion. In fact, P1 could justify our method.\n\n$~$\n> Another defense method [MID] should be evaluated.\n> Is the proposed method robust against [MID]?\n\nFollowing reviewer’s suggestion, we use the official implementation to train MID on CelebA setup (Natural Acc.=79.16%). We use $T$ = MID, $D_{priv}$ = CelebA, and $D_{pub}$ = CelebA. The results are as follows:\n| Attacks | Top 1 ($\\uparrow$) | KNN dist ($\\downarrow$) |\n|-|-|-|\n| BREPMI | 39.20 $\\pm$ 4.19 | 1458.61 |\n| Ours ($C \\circ D$) | 44.13 $\\pm$ 3.54 | 1475.73 |\n| Ours ($S$) | 55.33 $\\pm$ 4.40 | 1393.76 |\n| Ours ($S_{en}$) | 60.33 $\\pm$ 4.76 | 1374.34 |\n\nOur proposed method is robust/ surpasses BREPMI's performance across all setups. These results are comparable to the SOTA defense BiDO-HSIC (Table 4-main paper, Table B.2-Supp).\n\n[MID] Wang, Tianhao, et al. “Improving Robustness to Model Inversion Attacks via Mutual Information Regularization.” AAAI 2021.\n\n$~$\n> Missing related work similar to cite [PLUG] [MIRROR].\n\nWe will update related works.\n\n$~$\n> What are the target labels in the evaluation? How are they selected?\n\nSee Ln 223: “To ensure a fair comparison, when evaluating our method, we use the exact same experimental setup as BREPMI [6].” Following [6], we attack the first 300 out of 1000 labels in the experiments using CelebA dataset.  In cases of Facescrub and Pubfig, we attack all the labels of the target classifier (200 and 50, respectively).\n\n$~$\n> How can the method attack target labels that are not sampled during training T-ACGAN?\n\nThis highlights a critical concern with the SOTA label-only methods. Unlike BREPMI, our T-ACGAN successfully addresses the challenge of covering all the labels enabled by decision knowledge transfer from the target model. Our results demonstrate the effectiveness of T-ACGAN, as it covers all 1,000 classes adequately.\n\n$~$\n> Can the proposed method successfully attack larger models/ datasets?\n\nYes, under the setup with $T$=ResNet50, $D_{priv}$=VGGFace2 and $D_{pub}$=FFHQ, our proposed method outperforms BREPMI, with top 1 accuracy surpassing BREPMI by **12.6%-19.8%**.\n\n$~$\n> Possible limitations: 1) high time cost, 2) ~13 million required queries, 3) uncommon labels.\n\nTable 5 (main paper) shows that our method can reduce required queries by more than 30% compared to SOTA BREPMI. The reviewer's insight is greatly appreciated, and we agree that further reduction is desirable. "}, {"review_id": "ZS0p7Kieho", "reviewer": "Reviewer_5AoF", "summary": "The paper proposes a novel label-only model inversion attack (MIA). The underlying idea is to train a surrogate model and then perform existing white-box MIAs to reconstruct characteristic samples from the target model's training data. Instead of using the target model to label a public dataset, the paper proposes to train a modified ACGAN using the target model to provide hard labels on the generated samples during training. To demonstrate that the surrogate model assigns similar scores to inputs as the target model does, an analysis of the behavior of both models for easy- and hard-to-classify samples is performed. The proposed approach is then compared to BREP-MI as label-only MIA on different facial image datasets. ", "questions": "- BREP-MI uses a different GAN architecture (simple DCGAN) compared to ACGAN. What is the general image quality of the two GANs? Could it be the case that the proposed approach also performs better because of higher image quality?\n- Why use KEDMI for the white-box attack instead of, e.g., LOMMA [Nguyen et al., CVPR 2023], VMI [Wang et al., NeurIPS 2021], or Plug&Play Attacks [Struppek et al., ICML 2022]? Also, [Struppek et al.] found the attack to be rather unstable in other settings.\n- Why choose DenseNet as the surrogate architecture? Does the approach perform comparably to other choices of architecture?", "limitations": "Limitations are briefly discussed at the end of the paper. However, I think should be a larger discussion on the influence of the availability of public data, the influence of the GAN architecture, possible limitations in other domains or with higher resolution, etc.\n\nWhile the discussion on potential negative impact is rather shallow, I think it is sufficient for the paper, also compared to previous work.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper proposes a novel label-only MIA, which is the hardest setting for model inversion. The idea of training a surrogate model to perform white-box MIAs is, at least in the model inversion setting, novel.\n- The results compared to BREP-MI are promising and demonstrate the attack improvement in the investigated setting.\n- The analysis based on [Arpit et al.] for the training behavior and the comparison between the target and surrogate model is quite interesting.", "weaknesses": "- While the evaluation procedure follows the one of BREP-MI, I feel that the setting is a bit outdated, given that rather old/small models are investigated as target models. Also, the image size of 64x64 is quite small. It would, therefore, be interesting to see how the attacks perform on images of higher resolution, e.g., 256x256, and other, more recent architectures, e.g., VITs.\n- While attack accuracy and feature distance are commonly used in MIA literature, I am not sure if the evaluation should only focus on these metrics. I could imagine that the attacks also craft transferable adversarial examples, which might only leak limited information about the training data but rather exploit adversarial patterns to \"fool\" the target model - and possibly also the evaluation model. I think another metric would there be helpful, e.g., a CLIP zero-shot prediction, face distance computed by a face verification system, etc.\n-It is unclear, how the qualitative samples are selected. Are those cherry-picked? For a fair evaluation, the paper should also state unfiltered attack results (e.g., in the appendix).\n- Generally, I think the overall writing should be improved. Whereas the different concepts/ideas are most of the time clearly described, the written text is not always fluently readable.\n\nSmall remarks:\n- Shortly introduce ACGAN in the introduction. Not every ready may be familiar with the architecture/concept.\n- Fig. 1 c) I think there is a small error, the bottom of the third column should also show a green check symbol, right?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">It would, therefore, be interesting to see how the attacks perform on images of higher resolution and other, more recent architectures, \n\nWe appreciate the Reviewer’s suggestion. We report additional experiment results in 2 parts: (A) The MI attack results using recent architecture ConvNeXt [Liu et. al, CVPR2020] and (B) The MI attack results using ConvNeXt on high-resolution images (224x224).\n\n**A. Attacking recent architectures.**\nWe conducted an experiment using ConvNeXt target model (CelebA setup, 1000 classes, 84.85% natural accuracy). Following the exact procedure in the main paper, we perform attacks using BREPMI and our proposed method. Here we use $D_{pub}$ = CelebA. The results are as follow:\n| Attacks | Top 1 ($\\uparrow$) | KNN dist ($\\downarrow$) |\n|-|-|-|\n| BREPMI | 54.27 $\\pm$ 5.36 | 1394.08 |\n| Ours ($C \\circ D$) | 64.13 $\\pm$ 5.01 | 1358.58 |\n| Ours ($S$) | 81.53 $\\pm$ 3.43 | 1270.50 |\n| Ours ($S_{en}$) | 84.00 $\\pm$ 2.37 | 1246.30 |\n\nThe results show that our proposed method achieves better results in both Top-1 attack accuracy and KNN distance compared to the SOTA BREPMI.\n\n**B. Attacking recent architecture with high-resolution images.**\nFollowing PPA setup [Struppek et. al. ICML 2022] for high-resolution images, we train a new target classifier, ConvNeXt, using FaceScrub dataset at 224x224 resolution (natural accuracy = 97.26%).\nGiven PPA's success in high-resolution whitebox attacks, we use PPA attack in this study. For BREPMI, we use the same StyleGAN used by PPA. The results are as follows:\n| Attacks | Top 1 ($\\uparrow$) | Top 5 ($\\uparrow$) |\n|-|-|-|\n| BREPMI | 0.00 | 0.94 |\n| Ours ($S_{en}$) | 4.00 | 12.11 |\n\nBoth BREPMI and our approach face significant challenges in reconstructing private training data under high-resolution setup. \n\n[PPA] Struppek, L. et al. Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks. ICML 2022.\n[ConvNext] Liu, Z. et al. A convnet for the 2020s. CVPR 2020.\n\n$ $\n\n\n> While attack accuracy and feature distance are commonly used in MIA literature, I am not sure if the evaluation should only focus on these metrics.\n\nWe appreciate the reviewer's insightful comment regarding the large focus on objective metrics to evaluate MI attacks.\n\nTo address this, we go beyond objective metrics and consider **subjective evaluation** of MI attacks. Specifically, we conducted a **human study** to understand the efficacy of our proposed method compared to BREPMI. We follow the setup by An et al. [MIRROR] for human study. Specifically, in this study, • Users are shown 5 real images of a person (identity) as reference. • Then users are required to compare the 5 real images with two inverted images: one from our method, the other from BREPMI. We use $D_{priv}$ = CelebA, $D_{pub}$ = CelebA and $T$ = FaceNet64. Following An et al., we randomly selected 50 identities with 10 unique users evaluating each task accounting to 1000 comparison pairs. More details in the rebuttal PDF submission. \n\n**User study results.** Our human study reveals that users distinctly favor our approach, with **64.30% user preference for images reconstructed using our proposed approach**, in contrast to BREPMI’s lower 35.70% user preference. This subjective evaluation further shows the efficacy of our proposed method in the challenging label-only MI setup.\n\n$ $\n\n\n> It is unclear, how the qualitative samples are selected.\n\nIn Supp (line 9), we have already provided the link to **all inverted images** and private images. For each identity/class, we provide the best pair (consisting of an inverted image and a private image) with the smallest KNN distance. We also emphasize that for human study, we **randomly** selected 50 identities and the corresponding samples.\n\n$ $\n\n\n> Generally, I think the overall writing should be improved. \n> Shortly introduce ACGAN in the introduction. \n\nThank you for the feedback, we will improve the writing accordingly.\n\n$ $\n\n\n> Fig. 1 c) I think there is a small error, the bottom of the third column should also show a green check symbol, right?\n\nYes, it is indeed a typo. Thank you for pointing out this. We will fix it in the next version.\n\n$ $\n\n\n> BREP-MI uses a different GAN architecture (simple DCGAN) compared to ACGAN.?\n\nFor a fair comparison with BREPMI, in this part, we provide the experiment results by training a new T-ACGAN using the **same** architectures as the GAN used by BREPMI. For the discriminator (D), we apply max pooling and add a linear layer before the last layer for the classifier head. As for the generator (G), we retain the same architecture and replace batch normalization with conditional batch normalization.\nThe results are shown below:\n| Attacks | Top 1 ($\\uparrow$) | KNN dist ($\\downarrow$) |\n|-|-|-|\n| BREPMI | 73.93 $\\pm$ 4.98 | 1284.41 |\n| Ours ($C \\circ D$) | 86.40 $\\pm$ 3.18 | 1329.86 |\n| Ours ($S$) | 95.53 $\\pm$ 2.42 | 1199.93 |\n| Ours ($S_{en}$) | 97.53 $\\pm$ 1.31 | 1168.20 |\n\nHere, we use $T$ = FaceNet64, $D_{priv}$ = CelebA, and $D_{pub}$ = CelebA. Our results are better than BREPMI when using the same GAN architecture.\n\n$ $\n\n> Why use KEDMI for the white-box attack.\n\nWe have used a range of white-box attacks, including PLGMI [Yuan et. al. AAAI 2023] and KEDMI. Our main results use PLGMI due to its superior white-box MI attack performance compared to other state-of-the-art methods.\nAs for PPA [Struppek et. al. ICML 2022], both our method and BREPMI struggle to effectively reconstruct high-resolution images potentially due to large StyleGAN latent space. We leave it for our future work.\n\n$ $\n\n> Why choose DenseNet as the surrogate architecture? Does the approach perform comparably to other choices of architecture?\n\nIn section B.4 (supplementary), we study the effect of model architectures on model accuracy and MI attack accuracy to empirically justify our use of DenseNet model variants [10] as surrogate models.\n\n$ $\n\n> Limitations:\n\nThank you for the comment. We will include extended discussion on limitations.\n"}]

Author rebuttal:
We thank all the reviewers for their valuable time and effort to review our work. We appreciate the Reviewers' kind comments and recognition.

We would like to highlight that our paper focuses on Model Inversion, not Membership Inference as one of the reviewers suggested. Therefore, **comments of one of the Reviewers are completely irrelevant and out of context.**

 **Model Inversion and Membership Inference are totally different** as discussed in many works [a,b,c,d].

For model inversion, the objective is to **infer and reconstruct the training data of a trained model.**

For membership inference, the objective is to **determine whether a specific sample was used to train the model or not**.


In what follows, we provide comprehensive responses to all questions. We have provided anonymized link to Area Chair for the code of all additional experiments. We could provide more details if there are further questions. We hope that our responses can address the concerns and we sincerely hope that reviewers could consider increasing the ratings if our responses have addressed all the questions.


Thank you very much.


[a] Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, Dawn Song. The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks. CVPR-2020.

[b] Tanuwidjaja, H. C., Choi, R., Baek, S., & Kim, K. (2020). Privacy-preserving deep learning on machine learning as a service—a comprehensive survey. IEEE Access 2020.

[c] Rigaki, M., & Garcia, S. (2020). A survey of privacy attacks in machine learning. arXiv preprint arXiv:2007.07646.

[d] Yang Ziqi, Shao Bin, Xuan Bohan, Chang Ee-Chien, and Zhang Fan. 2020. Defending model inversion and membership inference attacks via prediction purification. arXiv preprint arXiv:2005.03915 (2020).



Meta-review: This paper proposes a method for label-only model-inversion (MI), i.e., where the adversary has only label access to the target model. The method uses transfer learning and generative models within a variation of the ACCGAN approach that uses access to the target model and the label that the target model generates. I thank the reviewers for engaging in the discussions and their authors for their extensive rebuttal. Overall, all the reviewers are positive about this paper being presented at NeurIPS and I agree.
Decision: Accept (poster)