SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ziang Li $^{1}$ , Mengda Yang $^{1}$ , Yaxin Liu $^{1}$ , Juan Wang $^{1*}$ , Hongxin Hu $^{2}$ , Wenzhe Yi $^{1}$  and Xiaoyang Xu $^{1}$
$^{1}$ Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University  $^{2}$ Department of Computer Science and Engineering, University at Buffalo, SUNY
Split Inference (SI) is an emerging deep learning paradigm that addresses computational constraints on edge devices and preserves data privacy through collaborative edge-cloud approaches. However, SI is vulnerable to Data Reconstruction Attacks (DRA), which aim to reconstruct users' private prediction instances. Existing attack methods suffer from various limitations. Optimization-based DRAs do not leverage public data effectively, while Learning-based DRAs depend heavily on auxiliary data quantity and distribution similarity. Consequently, these approaches yield unsatisfactory attack results and are sensitive to defense mechanisms. To overcome these challenges, we propose a GAN-based LAtent Space Search attack (GLASS) that harnesses abundant prior knowledge from public data using advanced StyleGAN technologies. Additionally, we introduce GLASS++ to enhance reconstruction stability. Our approach represents the first GAN-based DRA against SI, and extensive evaluation across different split points and adversary setups demonstrates its state-of-the-art performance. Moreover, we thoroughly examine seven defense mechanisms, highlighting our method's capability to reveal private information even in the presence of these defenses.
The emergence of Deep Learning (DL) has brought about a transformative impact on machine learning applications, granting them remarkable capabilities. To cater to the increasing demand for DL models on edge-side devices, various challenges related to performance arise. The growing size of model parameters poses a burden on resource-constrained edge devices. As a result, the concept of Machine Learning as a Service (MLaaS) has gained popularity as a solution. However, deploying DL services in the cloud, where APIs are provided to users and raw data is collected for service provisioning, raises concerns about potential data leakage. In this context, Split Inference (SI) has emerged as a promising alternative [Eshratifar et al., 2019; Banitalebi-Dehkordi et al., 2021; Kang et al., 2017; Matsubara et al., 2022; Hauswald et al., 2014]. SI involves splitting and deploying DNN models between the edge and the cloud, allowing the cloud's extensive computing and storage resources to be leveraged while ensuring that users only need to upload intermediate feature representations to protect the confidentiality of their original data.
However, recent studies have demonstrated that even with the use of intermediate feature representations, a malicious cloud server can still launch privacy attacks. Of particular concern is the Data Reconstruction Attack (DRA) [He et al., 2019; Singh et al., 2021; Yang et al., 2022], which represents the most severe violation of user privacy as it aims to reconstruct the user's inference
*Corresponding author: jwang@whu.edu.cn
Figure 1: Split Inference.
instances. Existing DRAs suffer from critical flaws that significantly diminish their effectiveness. Optimization-based DRAs [He et al., 2019; Singh et al., 2021], for example, fail to effectively leverage public data, while Learning-based DRAs [He et al., 2019] heavily depend on the quantity of auxiliary data and require a high degree of distribution similarity between the auxiliary data and the inference data.
To address these limitations and enhance the impact of the attack, we propose the GAN-based LAtent Space Search attack (GLASS). This attack leverages the power of StyleGAN [Karras et al., 2019, 2020, 2021; Sauer et al., 2022] and fully capitalizes on the valuable prior knowledge embedded in public data. Additionally, we introduce GLASS++, an improved version that enhances attack stability and effectiveness.
We systematically evaluate the reconstruction performance of Optimization-based GLASS and Learning-based GLASS++ on face data at different split points. Additionally, we thoroughly examine and analyze seven advanced defense mechanisms against DRA in SI. These mechanisms are categorized into three types: Clipping (Dropout Defense[He et al., 2020], DISCO[Singh et al., 2021]), Noise Addition (Noise Mask[Titcombe et al., 2021], Shredder[Mireshghallah et al., 2020]) and Feature Obfuscating (Adversarial Learning[Li et al., 2021], NoPeek[Vepakomma et al., 2020] and Siamese Defense[Osia et al., 2020]). Furthermore, we go beyond the traditional assumptions about adversary capabilities and extend our attack to heterogeneous data. Through our extensive experimentation, we achieve superior attack results across various split points and different adversary setups, successfully bypassing the employed defense mechanisms and compromising their effectiveness.
The key contributions of this paper are:
- We propose GLASS and  $GLASS++$ , which are enhanced DRAs combined with pre-trained StyleGAN models. This is the first instance of utilizing the latent space search characteristic of StyleGAN to develop DRAs specifically for SI. Additionally, we expand the practicality of our methods by designing attack strategies for various adversary settings.
- Through the utilization of advanced StyleGAN technologies, we exploit the rich prior knowledge present in public data, resulting in state-of-the-art reconstruction performance across different split points. Our methods outperform existing baseline attacks on multiple evaluation metrics, showcasing their superiority.
- We conduct a systematic evaluation and comparison of various DRAs against seven defense mechanisms. The results demonstrate that our methods effectively reveal sensitive information and undermine the robustness of the defenses.
2.1 Split Inference
Split Inference (SI) and Split Learning (SL)[Gupta and Raskar, 2018; Thapa et al., 2022; Poirot et al., 2019] have emerged as promising alternatives, which split and deploy DNN models on both the edge-side and the cloud side. In SI, a well-trained model  $M$  is split into client model  $M_C$  and server model  $M_S$ . An inference data  $x^*$  is fed to  $M_C$  to get an intermediate feature representation  $z^* = M_C(x^*)$ .  $z^*$  is then transmitted to the cloud to execute  $y^* = M_S(z^*)$ . Finally,  $y^*$  is returned to the edge-side to complete the inference process, as shown in Figure[1]. Collaborative computing across edge-cloud devices facilitates the reduction of computing payload on the edge side. By transmitting
only the smashed data to the cloud side, a certain level of privacy protection can be ensured. Utilizing distributed inference/training protocols, SI and SL achieve an improved trade-off between utility and privacy.
2.2 Data Reconstruction Attacks on Split Inference
Data Reconstruction Attack (DRA) is one of the most powerful privacy attacks that focuses on reconstructing private inference data. The existing DRAs can be broadly categorized as Optimization-based and Learning-based. [He et al., 2019] introduced regularized Maximum Likelihood Estimation (rMLE), firstly treating DRA as an optimization problem. For an intermediate feature representation  $z^{*} = M_{C}(x^{*})$ , they find the optimal sample  $x$  which minimizes the posterior information from feature-level observation by reducing the Euclidean Distance between  $M_{C}(x)$  and  $M_{C}(x^{*})$ . Additionally, they adapt the Total Variation (TV) [Rudin et al., 1992] to represent the prior information derived from the distribution of natural images. Inspired by the deep image prior [Ulyanov et al., 2018] for feature inversion, [Singh et al., 2021] proposed Likelihood Maximization (LM), which took full advantage of a fixed input Autoencoder network  $M_{AE}$  producing  $x = M_{AE}(\cdot)$  and replaced the target optimization by minimizing the loss  $l_{2}(M_{C}(M_{AE}(\cdot)), z^{*})$ , which significantly improved the optimization-based DRA effect. For learning-based DRA, [He et al., 2019] introduced Inverse-Network (IN) that leverages a certain amount of  $(z^{*}, x^{*})$  pairs gained by querying the  $M_{C}$  to train a model  $M_{C}^{-1}$  such that  $x = M_{C}^{-1}(z^{*})$ . Similarly,  $l_{2}$  norm in the pixel space is also used as the loss function. Facing serious threats of existing DRAs, a variety of defense mechanisms have been proposed to greatly mitigate privacy leakage in SI. They are specifically designed to minimize the disclosure of sensitive information from DRA, while still preserving the practical utility of the inference data.
The Model Inversion Attack [Zhang et al., 2020; Chen et al., 2021; An et al., 2022] seeks to extract sensitive features of an individual in the training data, by leveraging the coupled feature information contained in the confidence score of an ID classification model. The Gradient Inversion Attack [Zhu et al., 2019; Geiping et al., 2020; Jeon et al., 2021] aims to recover original training data from shared gradients. Unlike them, DRA focuses on reconstructing private inference data using feature representation output from the split layer of any functional DNN models.
2.3 StyleGAN & GAN Inversion
The StyleGAN generator consists of a mapping network  $f$  and a synthesis network  $G_{Style}$ . In a typical image generation process of StyleGAN, a latent vector  $z$  is sampled from the  $\mathcal{Z}$  space, which follows the Gauss Distribution. Then an intermediate latent vector  $w$  is obtained from  $f(z)$ . The  $f$  is a mapping network implemented by an 8-layer Multi-Layer Perceptron (MLP), making the generation based on a disentangled representation. Finally, the  $w$  is copied  $N$  times ( $N = \log(output\_size, 2) * 2 - 2$ ) and leveraged to control layer-grained adaptive instance normalization (AdaIN)[Huang and Belongie, 2017] operations, as shown in Figure 2.
With the rapid development of the StyleGAN series network, a variety of GAN Inversion methods have emerged [Abdal et al., 2019, 2020; Richardson et al., 2021; Wang et al., 2023], which aim to invert a given image back into the latent space of a pre-trained GAN model. Especially for StyleGAN, several latent spaces  $(\mathcal{W}+, S, \mathcal{P}, \mathcal{P}+)$  [Zhu et al., 2020b] and formulations (learning, optimization, or both) [Xia et al., 2022] are utilized to achieve better inversion results. Different from common GAN Inversion that focuses on distortion-editability trade-off [Zhu et al., 2020a; Tov et al., 2021], in this paper we customize advanced GAN Inversion technologies to DRA in SI, concentrating on raising the quality of reconstruction.
In this section, we first analyze the threat model of DRA in SI. Then, we formulate the design details and corresponding intuitions of our GLASS and  $GLASS++$ .
Figure 2: The framework overview of GLASS & GLASS++.
We assume an honest-but-curious server-side adversary in SI, who receives the client's intermediate feature representations and tries to reconstruct the private inference instances from them. The adversary has the parameters and structure of the whole target model  $M_T$ , as a white-box setting. This is a reasonable assumption in real-world scenarios, where typically the service provider of SI needs to obtain the target model and perform the splitting setup before deploying it. We also take into account the strictest case that the training of the target model is accompanied by defensive purposes. This usually occurs when the target model is provided to the server by a trusted model provider that sets defense mechanisms, or when the training objectives of the target model are forcibly set by the client with privacy protection requirements. In addition, we extend the consideration to scenarios where the adversary has limited capabilities, i.e., having no client model  $M_C$  or being non-queryable, as a black-box setting or a query-free setting. Besides, we assume that the adversary has an auxiliary public dataset  $\mathcal{D}_A$  with a similar distribution to that of the target model training dataset. Later in Section 5 we show that our attacks can be carried out effectively, even using an auxiliary dataset with a certain distribution shift.
3.2 GLASS
Setup. Before launching the GLASS, a pre-trained StyleGAN generator  $G_{Style}$  is necessary, which is trained on a data distribution similar to the private inference data  $\mathcal{D}_P$ . This is easy to obtain because StyleGAN models pre-trained on various data distributions (especially structured data distributions such as faces) are widely released online. We first formalize Optimization-based DRA as:
$$
\min  \mathcal {L} \left(M _ {C} (x), z ^ {*}\right) \tag {1}
$$
, where  $\mathcal{L}$  is the  $l_{2}$ -distance between two intermediate features, and the adversary tries to find an  $x$  closest to  $x^{*}$  through optimization.
$\mathcal{Z}$  space search. The non-convexity of StyleGAN generation makes the optimization problem nonconvex. For the optimization of non-convex functions, an ideal initial point and certain disturbance is extremely significant. The step ① of GLASS is searching in  $\mathcal{Z}$  space for the reason that the entanglement of  $\mathcal{Z}$  space increases the amplitude of positive perturbation (come from optimizer Adam, SGD, etc) in representation space, which avoids the minimization via gradient descent resulting in poor local minima in the same degree. The formal representation of  $\mathcal{Z}$  space search is:
$$
\min  _ {z \in \mathcal {Z}} \mathcal {L} \left(M _ {C} \left(G _ {\text {S t y l e}} (z)\right), M _ {C} \left(x ^ {*}\right)\right) + \lambda \mathcal {R} \left(G _ {\text {S t y l e}}, z\right) + \alpha \mathcal {T V} \left(G _ {\text {S t y l e}} (z)\right) \tag {2}
$$
. Obtaining distorted images is a common occurrence by directly searching according to  $\mathcal{L}$ , as the optimization process may cause  $z$  to deviate significantly from the distribution of  $\mathcal{Z}$  space. Therefore,
we adopt KL-based regularization [Kingma and Welling, 2013] to constrain the optimization process of  $z$  to conform to the normal distribution as:
$$
\mathcal {R} \left(G _ {\text {S t y l e}}, z\right) = - \frac {1}{2} \sum_ {i = 1} ^ {k} \left(1 + \log \left(\sigma_ {i} ^ {2}\right) - \mu_ {i} ^ {2} - \sigma_ {i} ^ {2}\right) \tag {3}
$$
, where  $\mu_i^2$  and  $\sigma_i^2$  represent the element-wise mean and standard deviation. The regularization item  $\mathcal{R}(\cdot)$  reduces the Kullback-Leibler divergence between  $z$  and the standard Gaussian distribution  $\mathcal{N}(0,1)$ , controlled by  $\lambda$ . Furthermore, we adopt Total Variation [Rudin et al., 1992] to bring prior information of the natural image, which encourages the generated image  $x = G_{Style}(z)$  to be piece-wise smooth, controlled by  $\alpha$ . Defined as:
$$
\mathcal {T V} (x) = \sum_ {i, j} \sqrt {\left| x _ {i + 1 , j} - x _ {i , j} \right| ^ {2} + \left| x _ {i , j + 1} - x _ {i , j} \right| ^ {2}} \tag {4}
$$
$\mathcal{W}+$  space search. Based on the  $z$  got in step ①, we perform  $f(z)$  to obtain  $w$ , then copy it 10 times as  $w+$ . In step ②, we search the  $\mathcal{W}+$  space as follows:
$$
\min  _ {w + \in \mathcal {W} +} \mathcal {L} \left(M _ {C} \left(G _ {\text {S t y l e}} (w +)\right), M _ {C} \left(x ^ {*}\right)\right) + \alpha \mathcal {T V} \left(G _ {\text {S t y l e}} (w +)\right) \tag {5}
$$
. For adequate disentanglement of  $\mathcal{W}+$  space, it is efficient to find the extreme point in the representation space. The two-step search algorithm is intuitively efficient. As depicted in Figure 3(A), the search in  $\mathcal{Z}$  space during step ① introduces significant perturbations, thereby preventing the initial point  $z$  from getting trapped in local optima. Subsequently, leveraging the superior editability provided by the highly disentangled representations in  $\mathcal{W}+$  space, the search in step ② further enhances the resemblance between the reconstructed data and the private input data. The iterative optimization process ultimately achieves the global optimum  $w_{+2}$  (the global optimum refers to the optimum attainable by the attacker with existing knowledge). The detailed algorithm of GLASS can be found in Appendix A.1.
Figure 3: (A) Through the large disturbance brought by  $\mathcal{Z}$  space search, GLASS can easily optimize  $z$  to  $w_{+1}$  and further reach the global optimum  $w_{+2}$  by  $\mathcal{W}+$  space search. (B) When the available information for reconstruction is tiny,  $\mathcal{Z}$  space search will inevitably fall into the local optimal point even accompanying disturbance, making GLASS optimize  $z^{*}$  to  $w_{+1}^{*}$  and  $\mathcal{W}+$  space search to  $w_{+2}^{*}$ . While GLASS++ utilizes the mapping relationship between feature space and latent space to obtain an improved initial point  $w_{+3}^{*}$  and subsequently achieves the global optimal  $w_{+4}^{*}$ .
3.3 GLASS++
Intuition. GLASS, this purely optimization-based DRA method could achieve remarkable results at shallow split points. However, when the split point is deep, the (Width, Height) of the intermediate feature representation becomes far less than that of the private input, meaning that the spatial information for reconstruction is gradually transformed into the semantic information needed for the classification task. The less spatial information available makes the optimization process more difficult, causing the results of GLASS to fall into pool local optima. As shown in Figure 3(B), even with a huge disturbance amplitude,  $z_{2}$  eventually falls into a local extreme point  $w_{+1}^{*}$ , due to the lack of information. This results in little improvement in the subsequent  $\mathcal{W} +$  space search. Therefore, we propose  $GLASS++$  to alleviate the above problem.
Setup. Before launching the GLASS++, we introduce pixel2style2pixel Encoder  $E_{PSP}$  and map2styles blocks  $M_{m}$  from [Richardson et al., 2021]. Tensors of the image space are fed to  $E_{PSP}$  to generate three levels of feature maps. These feature maps are subsequently utilized by the map2styles blocks  $M_{m}$  to extract desired styles  $w+$ . We incorporate these technologies into our DRA framework. For more details on these components, please refer to Appendix B.1.
$\mathcal{W}+$  space mapping & search. To tackle the problem of non-convex function optimization tending to fall into local optima, we employ Learning-based methods. These methods involve mapping the intermediate feature representation to  $\mathcal{W}+$  space, resulting in a more advantageous initial point. We specifically design a feature2image module  $M_{f}$ , projecting vectors from high-dimensional feature space into image space. This module serves as a valuable tool for facilitating subsequent style extraction. Then we joint the  $E_{PSP}$  and  $M_{m}$  to extract desired styles  $w+$ . In step ①, these three items collaboratively perform as Encoder  $E = M_{f} \circ E_{PSP} \circ M_{m}$ , mapping the vectors from feature space to  $\mathcal{W}+$  space as:
$$
w + = E \left(M _ {C} \left(x ^ {*}\right)\right) \tag {6}
$$
, with the help of  $E$ , a point near the global optimum is determined as the initial point of  $\mathcal{W}+$  space search in step ②. As shown in Figure 3(B), it is considerably more effortless to reach  $w+_{4}^{*}$  from  $w+_{3}^{*}$  driven by gradient descent. The detailed algorithm of GLASS++ can be found in Appendix A.2
3.4 Approach Analysis
Referring to [Jeon et al., 2021], we formalize the DRA under SI as an optimization problem. Through a pre-trained StyleGAN generator, the problem of (1) can be better solved by transferring from  $\mathbb{R}^m$  to  $\{G_{Style}(l): l \in \mathbb{R}^k\}$ , where  $l$  is a latent code in either  $\mathcal{Z}$  or  $\mathcal{W}+$  space,  $k$  denotes the dimension of  $l$ , and  $m$  refers to the dimension of image space. Hence, GLASS and GLASS++ perform the latent spaces search as follows:
$$
\min  _ {l \in \mathbb {R} ^ {k}} \mathcal {D} \left(M _ {C} \left(G _ {\text {S t y l e}} (l)\right), M _ {C} \left(x ^ {*}\right)\right) \tag {7}
$$
, where  $\mathcal{D}$  represents total loss terms in latent spaces search. When the private inference data is approximated with a sufficient narrow error, the DRA through latent spaces search in (7) aligns with image space search in (1).
We systematically evaluate our proposed attacks in terms of their performance against representative image classification tasks and compare them with existing attack methods. Additionally, we measure the robustness of seven defense mechanisms against various DRAs. We implement GLASS and  $GLASS++$  in Pytorch [Paszke et al., 2019]. Most experiments are carried out on a server equipped with 256 GB RAM, two Intel Xeon Gold 6133, and four NVIDIA RTX 4090 GPUs.
Datasets & Tasks. we use (1) CelebA [Liu et al., 2015] containing 202,599 face images of 10,177 identities, (2) FFHQ [Karras et al., 2019] containing 70,000 face images with considerable variation in terms of age, ethnicity and image background. Both are scaled down to  $64 \times 64$  pixels. We study DRA against models built for the Attractiveness Classification task: Binary attractiveness classification performed on the CelebA, as we consider attractiveness to be a remarkably inclusive facial attribute. We adopt ResNet-18 [He et al., 2016] and split the target model  $M_C$  into different layers, as shown in Appendix B.2.
Attack Setup. We split the datasets into two parts: a private dataset  $\mathcal{D}_P$  for training the target model and a public data used as an auxiliary dataset  $\mathcal{D}_A$  for training our StyleGAN model. For CelebA, we selected 80,525 images belonging to 3,000 identities with the highest number of images as  $\mathcal{D}_P$ , while the remaining images from other identities as public data  $\mathcal{D}_A$ . This scheme ensures that there are no overlapping identities between  $\mathcal{D}_A$  and  $\mathcal{D}_P$  in all experiments. This means that the public data only helps the adversary obtain general information about the features as prior knowledge,
without providing any class-specific information relevant to the private data[Chen et al., 2021]. For practical reasons, we utilize the FFHQ dataset as public data to train our StyleGAN model, as there are numerous StyleGAN models available on the Internet that are trained with FFHQ. Note that there is a certain distribution shift between the FFHQ dataset and the celeba dataset[Kahla et al., 2022]. To be fair, We set the number of iterations for Optimization-based DRA to 20,000 and the number of training epochs for Learning-based DRA to 30, while incorporating Total Variation into each attack loss function. It is worth acknowledging that the influence of hyperparameters varies across different adversarial settings and defense mechanisms. We analyze the hyperparameter selection strategies within different settings to meet the reasonable effectiveness of various attacks. Detailed information regarding the hyperparameters can be found in Appendix B.3.
Compared Baselines & Evaluation Protocol. We set the three existing DRAs as the baseline attacks: regularized Maximum Likelihood Estimation (rMLE)[He et al., 2019], Likelihood Maximization (LM)[Singh et al., 2021] and Inverse-Network (IN)[He et al., 2019]. For the sake of generality, our experiments are conducted on 40 randomly selected and fixed images, and the mean value of each evaluation metric is calculated as the result.
Evaluation Metrics. In addition to visually quantifying reconstruction attacks, we selected five metrics to evaluate the similarity between the original image and the reconstructed image: Learned Perceptual Image Patch Similarity (LPIPS ↓) [Zhang et al., 2018], Structural Similarity Index (SSIM ↑) [Wang et al., 2004], Peak Signal-to-Noise Ratio (PSNR ↑) [Hore and Ziou, 2010], Mean Squared Error (MSE ↓) and Natural Image Quality Evaluator (NIQE ↓) [Mittal et al., 2012]. Note that "↓" means the lower the metric the higher the relative image quality, while "↑" represents the higher the metric the higher the image quality.
4.2 Attack Performance