SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ngoc-Bao Nguyen\*1 Keshigeyan Chandrasegaran\*2\*1 Milad Abdollahzadeh $^{1}$  Ngai-Man Cheung $^{1\dagger}$
$^{1}$ Singapore University of Technology and Design (SUTD)  $^{2}$ Stanford University
thibaongoc_nguyen@mymail.sutd.edu.sg ngaiman_cheung@sutd.edu.sg
In a model inversion (MI) attack, an adversary abuses access to a machine learning (ML) model to infer and reconstruct private training data. Remarkable progress has been made in the white-box and black-box setups, where the adversary has access to the complete model or the model's soft output respectively. However, there is very limited study in the most challenging but practically important setup: Label-only MI attacks, where the adversary only has access to the model's predicted label (hard label) without confidence scores nor any other model information.
In this work, we propose LOKT, a novel approach for label-only MI attacks. Our idea is based on transfer of knowledge from the opaque target model to surrogate models. Subsequently, using these surrogate models, our approach can harness advanced white-box attacks. We propose knowledge transfer based on generative modelling, and introduce a new model, Target model-assisted ACGAN (T-ACGAN), for effective knowledge transfer. Our method casts the challenging label-only MI into the more tractable white-box setup. We provide analysis to support that surrogate models based on our approach serve as effective proxies for the target model for MI. Our experiments show that our method significantly outperforms existing SOTA Label-only MI attack by more than  $15\%$  across all MI benchmarks. Furthermore, our method compares favorably in terms of query budget. Our study highlights rising privacy threats for ML models even when minimal information (i.e., hard labels) is exposed. Our code, demo, models and reconstructed data are available at our project page: https://ngoc-nguyen-0.github.io/lokt/
Model inversion (MI) attacks aim to infer and reconstruct sensitive private samples used in the training of models. MI and their privacy implications have attracted considerable attention recently [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]. The model subject to MI is referred to as target model. There are three categories of MI attacks: (1) White-box MI, where complete target model information is accessible by the adversary [1, 2, 3, 5, 7, 10]; (2) Black-box MI, where target model's soft labels are accessible [4, 12, 10, 13]; (3) Label-only MI, where only target model's hard labels are accessible [6]. This paper focuses on label-only MI, which is the most challenging setup as only limited information (hard labels) is available (Fig. 1).
In most existing work, MI attack is formulated as an optimization problem to seek reconstructions that maximize the likelihood under the target model [1, 2, 3, 6]. For DNNs, the optimization problems are highly non-linear. When the sensitive private samples are high-dimensional samples (e.g. face
* These authors contributed equally. ‡ Work done while at SUTD.
images), the optimizations are ill-posed, even in white-box setups. To overcome such issues, recent MI [1, 2, 3, 5, 10, 7, 6, 11] learn distributional priors from public data via GANs [14, 15, 16, 17], and solve the optimization problems over GAN latent space rather than the unconstrained image space. For example, MI attacks on face recognition systems could leverage GANs to learn face manifolds from public face images which have no identity intersection with private training images. White-box attacks based on public data and GANs have achieved remarkable success [1, 2, 3, 7, 11]. We follow existing work and recent label-only MI [6] and leverage public data in our method. Furthermore, similar to existing work, we use face recognition models as examples of target models.
Research gap. Different from white-box attack, study on label-only attack is limited despite its practical importance, e.g., many practical ML models only expose predicted labels. Focusing on label-only attack and with no knowledge of internal workings of target model nor its confidence score, BREPMI [6] takes a black-box search approach to explore the search space iteratively (Fig. 1(a)). To seek reconstructions with high likelihood under target model, [6] proposes to query target model and observe the model's hard label predictions, and update search directions using Boundary Repelling in order to move towards centers of decision regions, where high likelihood reconstructions could be found. However, black-box search in the high-dimensional latent space is extremely challenging.
In this paper, we propose a new approach for Label-Only MI attack using Knowledge Transfer (LOKT). Instead of performing a black-box search approach as demonstrated in [6] and directly searching high-likelihood reconstruction from the opaque target model (Fig. 1(a)), which could be particularly challenging for high-dimensional search space, we propose a different approach. Our approach aims to transfer the decision knowledge of the target model to surrogate models, for which complete model information is accessible. Subsequently, with these surrogate models, we could harness SOTA white box attacks to seek high-likelihood reconstructions (Fig. 1(b)). To obtain the surrogate models, we explore generative modeling [18, 19, 20, 21, 22]. In particular, we propose a new Target model-assisted ACGAN, T-ACGAN, which extends ACGAN [23] and leverages our unique problem setup where we have access to the predicted labels of the target model as shown in Fig. 1(d). In particular, by effectively leveraging the target model in discriminator/classifier training, we can explore synthetic data for decision knowledge transfer from the target model to the surrogate model. With T-ACGAN capturing the data manifold of public samples, synthetic data is diverse and abundant. We hypothesize that such rich synthetic data could lead to improved decision knowledge transfer. Moreover, as training progresses, T-ACGAN generator learns to improve its conditional generative capabilities, enabling it to produce more balanced synthetic data for surrogate model learning. We explore several surrogate model designs. In one configuration, we employ the discriminator/ classifier of T-ACGAN as the surrogate model. In an alternative design, we utilize the generator of T-ACGAN to train different surrogate model variants. It's noteworthy that the generator of T-ACGAN can be readily employed for white-box attacks, and its conditional generation capabilities can effectively reduce the search space during inversion. In addition, we perform analysis to support that our surrogate models are effective proxies for the opaque target model for MI. (Fig. 1(e)). Overall, our T-ACGAN renders improved surrogate models, resulting in a significant boost in MI attack accuracy (Fig. 1(f)) and reduced number of queries compared to previous SOTA approach. Our contributions are:
- We propose LOKT, a new label-only MI by transferring decision knowledge from the target model to surrogate models and performing white-box attacks on the surrogate models (Sec. 4). Our proposed approach is the first to address label-only MI via white-box attacks.
- We propose a new T-ACGAN to leverage generative modeling and the target model for effective knowledge transfer (Sec. 4).
- We perform analysis to support that our surrogate models are effective proxies for the target model for MI (Sec. 5).
- We conduct extensive experiments and ablation to support our claims. Experimental results show that our approach can achieve significant improvement compared to existing SOTA MI attacks (Sec. 6). Additional experiments/ analysis are in Supplementary.
Model Inversion (MI) has particularly alarming consequences in security-sensitive domains, such as face recognition [24, 25, 26, 27], medical diagnosis [28, 29, 30]. Fredrikson et al. [31] introduces the
(a) Existing Works on Label-Only MI
(b) Our Work: Label-Only MI via Knowledge Transfer(LOKT)
Opaque Target Model Surrogate Model
Decision Knowledge Transfer
Stage 2
2
Stage 1
Surrogate Model
White-box
MI Attack
1
一
Casting Label-only MI Attack as a White-box MI Attack
Private Data
Private Data
(C) Label-only MI is the most challenging MI Attack
Criteria	Architecture / Parameters	Soft-labels	Hard-labels	Concern reg. Queries
White-box	✓	✓	✓	Low
Black-box	X	✓	✓	High
Label-only	X	X	✓	High
(f) Private
Training
Data
Existing
SOTA 
On
construction
Results
Figure 1: Overview and our contributions. (a) Under Label-only model inversion (MI) attack, the Target model  $T$  is opaque. (b) Stage 1: As our first contribution, we propose a knowledge transfer scheme to render surrogate model(s). (b) Stage 2: Then, we cast the Label-only MI attack as a white-box MI attack on surrogate model(s)  $S$ . (c) This casting can ease the challenging problem setup of label-only MI attack into a white-box MI attack. To our knowledge, our proposed approach is the first to address label-only MI via white-box MI attacks. (d) We propose T-ACGAN to leverage generative modeling and the target model for effective knowledge transfer to render surrogate model(s). Knowledge transfer renders  $D$  (Discriminator) as a surrogate model, and further generated samples of T-ACGAN can be used to train additional surrogate variant  $S$  (Sec. 4.3). (e) Our analysis demonstrates that  $S$  is an effective proxy for  $T$  for MI attack (details in Sec.5). In particular, white-box MI attack on  $S$  mimics the white-box attack on opaque  $T$ . (f) Our proposed approach significantly improves the Label-only MI attack (e.g.  $\approx 20\%$  improvement in standard CelebA benchmark compared to existing SOTA [6]) resulting in significant improvement in private data reconstruction. Best viewed in color.
Attack (↑) Acc.
73.93%
under Label:
ur first copy
(b) Steps 2:
(b) Stage 2:
$\operatorname {del}(\mathbf{s})S$  (c)  $\mathrm{TH}$
a box MI attached
e-box M1 attach this box with M1
white-box M1
model for effe
(Discrepntion)
(Dischiminate
train additional
proxy for  $T$  :
proxy for 1
white-box att
MI attack (e.g.
(61) resulting in
(b)] resulting in
s. Recently, several works extend MI for complex  $p$ , [1] proposes Generative Model Inversion (GMI) strain the search space. [2] proposes Knowledge to train an inversion-specific GAN for the attack. To apply variational objectives for the attack. Very MI (PLG-MI) to apply model's soft output to train attack. LOMMA[11] proposes a better objective stress MI overfitting. For black-box attack, where train an inversion model and a decoder to generate in model. [12] proposes an adversarial approach for BREPMI, the first label-only MI using a black-box or further discussion of related work.
(d) Decision Knowledge Transfer using our T-ACGAN
Decision Knowledge Transfer
93.93%
(e) Surrogate Models are Effective Proxies for the Target Model for MI
first MI attack for simple linear regression models. Recently, several works extend MI for complex DNNs under different setups. For white-box setup, [1] proposes Generative Model Inversion (GMI) to leverage public data and GAN[32, 33] to constrain the search space. [2] proposes Knowledge-Enriched Distributional Model Inversion (KEDMI) to train an inversion-specific GAN for the attack. [3] proposes Variational Model Inversion (VMI) to apply variational objectives for the attack. Very recent work [7] proposes Pseudo Label-Guided MI (PLG-MI) to apply model's soft output to train a conditional GAN (cGAN)[34] for white-box attack. LOMMA[11] proposes a better objective function for MI and model augmentation to address MI overfitting. For black-box attack, where model's soft output is available, [4] proposes to train an inversion model and a decoder to generate target images using predicted scores of the inversion model. [12] proposes an adversarial approach for black-box MI. For label-only attack, [6] proposes BREPMI, the first label-only MI using a black-box Boundary Repelling search. See Supplementary for further discussion of related work.
Given a target model  $T$ , the goal of MI is to infer private training data  $\mathcal{D}_{priv}$  by abusing access to model  $T$ . More specifically, given a target class/ identity label  $y$ , the adversary aims to reconstruct
an image  $x$  which is similar to the images of class  $y$  in  $\mathcal{D}_{priv}$ . Most MI formulate the inversion as optimization problems to seek the highest likelihood reconstructions for identity  $y$  under  $T$ . As direct searching for  $x$  in the unconstrained image space is ill-posed, many MI attacks [1, 2, 3, 7, 6] leverage public dataset  $\mathcal{D}_{pub}$  that is the same domain as  $\mathcal{D}_{priv}$ , e.g.,  $\mathcal{D}_{priv}$  and  $\mathcal{D}_{pub}$  are facial image datasets. GAN [14] is applied to learn distributional prior from  $\mathcal{D}_{pub}$ , and the adversary searches the GAN latent space instead of the unconstrained image space for high-likelihood reconstructions under  $T$ :
$$
\max  _ {z} \log P _ {T} (y | G (z)) \tag {1}
$$
Here,  $G$  is the generator, and  $P_{T}(y|.)$  is the likelihood of an input for identity  $y$  under target model  $T$ . White-box attacks apply gradient ascent and some regularization [1, 2, 3, 7] to solve Eq. 1, whereas label-only attack BREPMI [6] applies black-box search to tackle Eq. 1. In this paper, we also tackle Eq. 1 under label-only setup, i.e. only the predicted label is available.
Our proposed label-only MI consists of two stages. In stage 1, we learn surrogate models. In stage 2, we apply SOTA white-box attack on the surrogate models. To learn surrogate models, we explore an approach based on GAN and propose a new Target model-assisted ACGAN (T-ACGAN) for effective transfer of decision knowledge. Our T-ACGAN learns the generator  $G$  and the discriminator  $D$  with classifier head  $C$ . In one setup, we directly take  $C \circ D$  as the surrogate model*. In another setup, we apply  $G$  to generate synthetic data to train another surrogate model  $S$  or an ensemble of  $S$ . Then, we apply SOTA white-box attack on  $C \circ D$ ,  $S$  or the ensemble of  $S$ . In our experiments, we show that using  $C \circ D$  in a white-box attack can already outperform existing SOTA label-only attack. Using  $S$  or an ensemble of  $S$  can further improve attack performance. The  $G$  obtained from our T-ACGAN can be readily leveraged in the attack stage.
Before discussing our proposed approach, we first discuss a simple baseline for comparison. Given the public data, one could directly use the target model  $T$  to label the data and learn the surrogate model  $S$ . For  $x_{p} \in \mathcal{D}_{pub}$ , we construct  $(x_{p},\tilde{y})$ , where  $\tilde{y} = T(x_{p})$  is pseudo label of private identity. We obtain the dataset  $\tilde{\mathcal{D}}_{pub}$  with samples  $(x_{p},\tilde{y})$ , i.e.  $\tilde{\mathcal{D}}_{pub}$  is the public dataset with pseudo labels. We apply  $\tilde{\mathcal{D}}_{pub}$  to train  $S$ . However, this algorithm suffers from class imbalance. In particular, some private identities could have less resemblance to  $x_{p} \in \mathcal{D}_{pub}$ . As a result, for some  $\tilde{y}$ , there is only a small number of  $x_{p}$  classified into it, and  $\tilde{\mathcal{D}}_{pub}$  is class imbalanced. When using  $\tilde{\mathcal{D}}_{pub}$  to train  $S$ , minority classes may not gain adequate decision knowledge under  $T$  and could perform sub-optimally. In our experiments, we also apply techniques to mitigate the class imbalance in  $\tilde{\mathcal{D}}_{pub}$ . However, the performance of this baseline approach is inadequate as we will show in the experiments.
4.2 Review of ACGAN
In standard ACGAN [23], we are given a real training dataset with label, i.e.,  $\mathcal{D}_{real}$  with samples  $(x_r, y)$ . The generator  $G$  takes a random noise vector  $z$  and a class label  $y$  as inputs to generate a fake sample  $x_f$ . The discriminator  $D$  outputs both a probability distribution over sources  $P(s|x) = D(x)$ , where  $s \in \{\text{Real}, \text{Fake}\}$ , and a probability distribution over the class labels, i.e.,  $P(c|x) = C \circ D(x)$ , and  $c$  is one of the classes. For real training sample  $x_r$  of label  $y$  and fake sample  $x_f = G(z, y)$  with conditional information  $y$ , the loss functions for  $D, C$  and  $G$  are:
$$
\begin{array}{l} \mathcal {L} _ {D, C} = - E \left[ \log P \left(s = F a k e \mid x _ {f}\right) \right] - E \left[ \log P \left(s = R e a l \mid x _ {r}\right) \right] \tag {2} \\ - E \left[ \log P \left(c = y \mid x _ {f}\right) \right] - E \left[ \log P \left(c = y \mid x _ {r}\right) \right] \\ \end{array}
$$
$$
\mathcal {L} _ {G} = E [ \log P (s = \text {F a k e} | x _ {f}) ] - E [ \log P (c = y | x _ {f}) ] \tag {3}
$$
*With a slight abuse of notation we use  $D$  to represent the entire discriminator and the discriminator up to and including the penultimate layer in the context of  $C \circ D$ .
4.3 Our Proposed T-ACGAN and Learning of Surrogate Model
Unlike standard ACGAN setup where we have access to labelled data  $\mathcal{D}_{real}$  with samples  $(x_r, y)$ , in our setup, we have access to real public data without label:  $\mathcal{D}_{pub}$  with samples  $x_p$ . Importantly, we can leverage the target model  $T$  to provide pseudo labels for generated samples  $x_f = G(z, y)$  which are diverse and abundant. Our proposed T-ACGAN aims to take advantage of  $T$  to provide more diverse and accurate pseudo labelled samples during the training.
$D$  and  $C$  Learning. Our T-ACGAN leverages  $T$  to assign pseudo labels to the diverse generated samples  $x_{f} = G(z,y)$ , i.e.,  $\tilde{y} = T(x_{f})$ . We apply samples  $x_{p}$  and  $(x_{f},\tilde{y})$  to learn  $D$  and  $C$ :
$$
\begin{array}{l} \mathcal {L} _ {D, C} = - E \left[ \log P \left(s = \text {F a k e} \mid x _ {f}\right) \right] - E \left[ \log P \left(s = \text {R e a l} \mid x _ {p}\right) \right] \tag {4} \\ - E [ \log P (c = \tilde {y} | x _ {f}) ] \\ \end{array}
$$
In Eq. 4, the term  $E[\log P(c = \tilde{y} | x_f)] = E[\log P(c = \tilde{y} | G(z, y))]$  is different from ACGAN and may look intriguing. Instead of using  $y$  as class supervision to train  $D$  and  $C$  as in ACGAN (Eq. 2), our T-ACGAN takes advantage of  $T$  to apply  $\tilde{y} = T(G(z, y))$  to train  $D$  and  $C$ , as  $\tilde{y}$  is more accurate conditional information compared with  $y$  especially during the initial epochs. With Eq. 4, our method transfers the decision knowledge of  $T$  into  $D$  and  $C$  via diverse generated samples. Furthermore, as we can generate diverse pseudo labelled samples  $(x_f, \tilde{y})$  using  $T$  and  $G$ , pseudo labelled data based on  $x_p$  can be omitted. In our experiment, we show that we can achieve good performance using diverse samples  $(x_f, \tilde{y})$ . In T-ACGAN, we utilize public data  $x_p$  only for real/fake discrimination.
G Learning. We follow ACGAN training for  $G$ , i.e. Eq. 3. With  $D$  and  $C$  trained with decision knowledge of  $T$  in the above step, they provide feedbacks to  $G$  to improve its conditional generation in the private label space of  $T$ . In our experiment, we analyze  $y$  in  $x_{f} = G(z,y)$  and  $\tilde{y} = T(x_{f})$ . As training progresses,  $G$  improves its conditional generation, and  $y$  and  $\tilde{y}$  become more aligned. Note that, as  $T$  outputs only hard labels,  $T$  cannot be readily applied to provide feedback for  $G$  learning.
Surrogate Model. With alternating  $D$  and  $C$  learning and  $G$  learning, we obtain  $D$ ,  $C$  and  $G$ . We explore three methods to obtain the surrogate model.  $\bullet$  (i) We directly take  $C\circ D$  in T-ACGAN as the surrogate model and apply a white-box attack on  $C\circ D$ . This can be justified as  $C\circ D$  is trained based on decision knowledge of  $T$  to classify a sample into identities of private training data.  $\bullet$  (ii) We apply  $G$  of T-ACGAN to generate dataset  $\tilde{\mathcal{D}}_{\text{fake}}$  with samples  $(x_f,\tilde{y})$ , where  $x_{f} = G(z,y)$  and  $\tilde{y} = T(x_f)$ . We apply  $\tilde{\mathcal{D}}_{\text{fake}}$  to train another surrogate model  $S$ .  $\bullet$  (iii) We use the same dataset  $\tilde{\mathcal{D}}_{\text{fake}}$  in (ii) to train an ensemble of  $S$  of different architectures. As pointed out in [11], using an ensemble of  $S$  could improve white-box attack performance.
White-box Attack. With surrogate model  $C \circ D$ ,  $S$  or an ensemble of  $S$ , any white-box attack can be applied. In our experiments, we show that our surrogate models are effective across a range of white-box attacks (See the Supplementary). Furthermore,  $G$  in T-ACGAN can be readily leveraged for performing the attack. Particularly, based on  $G(z,y)$  obtained in the above steps, we could reduce the search space during inversion to the latent region corresponding to the target identity  $y$ , leading to more efficient search and improved attack accuracy [7].
5 Analysis for justification of surrogate models
In this section, we provide an analysis to justify why our surrogate model could be an effective proxy for  $T$  under MI, i.e., the results of white-box MI attack on our surrogate model be good approximation to that of white-box MI attack on  $T$ . Note that results of white-box MI on  $T$  cannot be obtained directly as  $T$  exposes only hard labels. To simplify the presentation, we focus our discussion on  $S$ . As discussed in Sec. 3, most MI attacks formulate inversion as an optimization problem of seeking reconstructions that achieve highest likelihood under target model. Therefore, when we carry out MI on  $S$  with SOTA white-box approaches, we expect to obtain high-likelihood reconstructions under  $S$  (or high-likelihood generated samples of GAN under  $S$ , see Eq. 1). We use  $P_{S}$  and  $P_{T}$  to denote likelihood of a sample under  $S$  and  $T$  respectively.
In what follows, we provide analysis to support that  $S$  based on our approach would possess an important property of good proxy for  $T$ . Property P1: For high-likelihood samples under  $S$ , it is likely that they also have high likelihood under  $T$ . See Fig. 1(e) for distribution of generated samples'  $P_T$  conditioning on those with high  $P_S$ . It can be observed that many have high  $P_T$ . Particularly, it is
uncommon for high-likelihood samples under  $S$  to have low likelihood under  $T$  (see Fig. 1(e) only a few samples have low  $P_T$ ).
With Property P1, the result obtained by white-box on  $S$  (which is a high likelihood sample under  $S$ ) is likely to have a high likelihood under  $T$  and could be a good approximation to the result of white-box on  $T$  (which is a high likelihood sample under  $T$ ). In Fig. 1(e), P1 can be clearly observed†. Therefore,  $S$  using our approach would possess P1 and would be a good proxy for  $T$  for MI.
Why would  $S$  possess property P1? This could be intriguing. After all,  $T$  does not expose any likelihood information. The labels of samples assigned by  $T$  are the only information available to  $S$  during training of  $S$ . It does not appear that  $S$  can discern low or high-likelihood samples under  $T$ .
To discuss why  $S$  would possess P1, we apply findings and analysis framework of Arpit et al. [35] regarding the general learning dynamics of DNNs. [35] presents a data-centric study of DNN learning with SGD-variants. In [35], "easy samples" are ones that fit better some patterns in the data (and correspondingly "hard samples"). The easy and hard samples exhibit high and low likelihoods in DNNs resp. as discussed in [35]. Furthermore, an important finding from [35] is that, in DNNs learning, the models learn simple and general patterns of the data first in the training stage to fit the easy samples.