Measuring Data Reconstruction Defenses in Collaborative Inference Systems
Authors: Mengda Yang
Ziang Li
Juan Wang
Hongxin Hu
Ao Ren
Xiaoyang Xu
Wenzhe Yi
Venue: NeurIPS 2022 Accept (2022)

Abstract: The collaborative inference systems are designed to speed up the prediction processes in edge-cloud scenarios, where the local devices and the cloud system work together to run a complex deep-learning model. However, those edge-cloud collaborative inference systems are vulnerable to emerging reconstruction attacks, where malicious cloud service providers are able to recover the edge-side users’ private data. To defend against such attacks, several defense countermeasures have been recently introduced. Unfortunately, little is known about the robustness of those defense countermeasures. In this paper, we take the first step towards measuring the robustness of those state-of-the-art defenses with respect to reconstruction attacks. Specifically, we show that the latent privacy features are still retained in the obfuscated representations. Motivated by such an observation, we design a technology called Sensitive Feature Distillation (SFD) to restore sensitive information from the protected feature representations. Our experiments show that SFD can break through defense mechanisms in model partitioning scenarios, demonstrating the inadequacy of existing defense mechanisms as a privacy-preserving technique against reconstruction attacks. We hope our findings inspire further work in improving the robustness of defense mechanisms against reconstruction attacks for collaborative inference systems.

Measuring Data Reconstruction Defenses in Collaborative Inference Systems
Mengda Yang', Ziang Li", Juan Wang! Hongxin Hu?, Ao Ren², Xiaoyang Xu and Wenzhe Yi!
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University Department Computer Science and Engineering, University at Buffalo, SUNY CCollege of Computer Science, Chongqing University
Abstract
The collaborative inference systems are designed to speed up the prediction pro- cesses in edge-cloud scenarios, where the local devices and the cloud system work together to run a complex deep-learning model. However, those edge-cloud collab- orative inference systems vulnerable to emerging reconstruction attacks, where malicious cloud service providers are able to recover the edge-side users' private data. To defend against such attacks, several defense countermeasures have been recently introduced. Unfortunately. little is known about the robustness of those defense countermeasures. this paper, we take the first step towards measuring the robustness of those state-of-the-ar defenses with respect to reconstruction attacks. Specifically, we show that the latent privacy features are still retained in the obfus- cated representations. Motivated by such an observation, we design a technology called Sensitive Feature Distillation (SFD) to restore sensitive information from the protected feature representations. Our experiments show that SFD can break through defense mechanisms in model partitioning scenarios, demonstrating the adequacy of existing defense mechanisms reconstruction attacks. We hope our findings inspire further work in improving the robustness of defense mechanisms against reconstruction attacks for collaborative inference systems.
technique against
Introduction
With the rapid advancement of deep neural network technology, the deployment deep learning (DL.) applications on edge devices is becoming more widespread. However, resource-constrained edge devices are often difficult to accommodate computationally intensive DL models. In addition, cloud-based DL inference applications require devices to upload users' private data cloud service providers, which poses a serious privacy threat. Collaborative DL systems have recently emerged as good alternative address these challenges, where the local devices and the remote servers work together to run a complex neural network [Eshratifar et al., 2019; Banitalebi Saniaalebi-eehkoddi et al. 2021 Li al. 2021b]. A DL model can be split such that only its front part executed on the local edge devices while its back part is performed on the remote cloud. Therefore, end users are not required to upload their private data the cloud, and their privacy concerns can be addressed to some extent.
This paper focuses on reconstruction attacks, where existing research has shown that adversaries can easily recover sensitive data with the output of the middle layers of a DL model [He et al.] 2019 Singh et al. 2021]. To address this problem, many defense approaches have been proposed for
"Corresponding author: jwang whu.edu.cn
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Truth
Vanilla
ERA
SFD
Truth
Vanilla
ERA
SFD
Truth
Vanilla
ERA
SFD
2nd conv block
3rd conv block
4th conv block
avgpool
1st layer
a) Adversarial Learning
b) Noise Mask
Dropout Defense
Figure 1: An example showing the superiority of Sensitive Feature Distillation (SFD) compared to Existing Reconstruction Attack (ERA).
collaborative inference systems. Liu et al. 2019 Xiao et al. 2020 Li et al. 2021a demonstrate that adversarial learning improves the robustness of the model against reconstruction attacks. (Titcombe) al. 2021 Mireshghallah et al. 2020 propose learning additive Laplace noise to protect the privacy of inference data. [He et al. 2020 uses dropout to defend against reconstruction attacks. However, none of the existing work provides a systematic security measurement for various defenses. This leads to our main research question: Are those privacy protection techniques against reconstruction atlacks for collaborative inference systems really robust enough?
In this work, we summarise the various defenses, systematically evaluate their performance in defending against reconstruction attacks, and present their trade-offs regarding data utility and privacy leakage. We propose novel technique break down existing defenses, which can achieve improved reconstruction performance for various target models, datasets, and defense algorithms. We focus on the privacy of the inference data [He al. 2019, Singh al.) 202T]. Since protectors obfuscate the intermediate feature representation to limit the redundant information about the model input, our idea to learn to distill obfuscated features into sanitized features for better reconstruction. Specifically, we introduce Sensitive Feature Distillation (SFD) to extract the image structure knowledge of sensitive features and find that the proper formulation of prior knowledge will be helpful to solve the ill- ill-posed reconstruction problem. We show that this vulnerability is inevitable when different defense countermeasures are utilized. Figure shows the reconstruction results using our proposed method and the existing one.
Our main contributions are summarized below:
We are the first experimentally verify the robustness of reconstruction defenses for inference data privacy in collaborative systems.
We reveal the presence of latent privacy features in protected representations and validate the vulnerability existing defense methods.
We devise technique called SFD against the existing defense mechanisms. SFD can achieve better reconstruction by carefully extracting sensitive information to transfer the obfuscated features into the sanitized features.
We conduct comprehensive measurement of the robustness of various state-of-the-art defenses leveraging our proposed technique. Our experiments show that SFD can achieve 74. 24 performance improvement compared with the baseline attack, making the existing defenses significantly less effective.

---

2 Background and Related Work
This paper focuses the most common DL task: image classification. A Deep Neural Network (DNN) model can be characterized as F :X + y which maps an input data x E X into an output result y E V.
Collaborative Inference Systems. Rumning the entire DL inference application on the edge device common scaling approach to partition the DNN model where the earlier layers of the model are executed within the local edge device while the latter layers are deployed the remote cloud [Kang et al., 2017 Eshratifar et al. 2019; Sanifalebi-Dekkone et al.
can be time- and eesource-onsuuming.
2021 Li et all 2021b]. This scheme can reduce the computational overhead of edge devices on the one hand, and can mitigate privacy leaks of inference raw data on the other [Wang and Gong 2018 Osia et al. 2020]. Collaborative inference empowers the deployment of DL workloads on edge-cloud platforms to be productive.
The model is split into two parts in the collaborative inference system: an on-device part and remote cloud-based part. Each part will contain several layers of the model. From now on, we reformulate the model F = C' o Ennc as the Encoder module Enc and the Classifier module C. The edge device deploys the first part Ene, it feeds the users' inference data x to the feature extractor to get an intermediate feature representation = = Enc(x), and then sends to the cloud. The cloud manages the second part C, which calculates the prediction output y = C(z) when the intermediate value is received from the edge device, then it returns the expected result y to the edge side.
Machine Learning Privacy and Reconstruction Attacks. Recent research presented a variety of
privacy threats against machine learning models, such as membership inference attacks (Shokri et al.
2017a Song and Mittal 2021]. model extraction attacks (Tramer et al. 2016], property inference
attacks Ganju al. 2018; Song and Shmatikov 2019) and model inversion attacks (Yang et al..
2019
He et al. 2019 The above threats are significant in collaborative DL systems as well.
This paper focuses the Reconstruction Attacks which are closely related to much previous work on model inversion attacks. They all aim to derive private information about the original data in the context of accessing the target machine learning model. In the model inversion attack, the adversary is asked to use the association of model inputs and outputs to restore feature information from the training data. Although the original model inversion attack [Fredrikson et al. 2015 has limited
performance [Hitaj et al.) (017), the inversion performance is significantly improved by training
transposed CNNs on an auxiliary dataset Yang et al. 2019 Zhang al., 2020 Zhao et al. 2021
Wang et al.
2021
Unlike recovering training data from the target ML models, reconstruction attacks are type of technology designed to reconstruct model input data at the time of inference. [He et al. 2019. Singh et al. 202T] devised various attack approaches for recovering private inputs in the context edge- cloud colaabtrttvve inference systems. In the white-box attack setting, they developed a regularized likelihood maximization algorithm, which intended find the optimal input data for the given deep neural representations and auxiliary information. The underlying idea of the algorithm is to formulate the reconstruction attack
optimization problem in the data space X' to find image & whose prediction intermediate feature representation Enc(C) approximates a given Enc(x). However, in order to compute the gradient to minimize the loss function, it needs white-box permission to access the target model. Furthermore, this method involves optimization for each natural sample at inference time which makes relatively inefficient. The black-box reconstruction attack trains another neural network G to recognize the mapping of the inverse from Ene(x) to x. Specifically, without having the knowledge of the parameters of Enc, the adversary can collect some pairs (2, 2)) by querying the model, and then she trains a Decoder module G to learn to well approximate the mapping between intermediate representation and corresponding input x. In contrast to the white-box optimization- based attack, this approach is more efficient: the adversary only needs one single forward propagation through the decoder module G to reconstruct any given instance. In this paper, we concentrate on the box reconstruction attack, which is a more practical and realistic scenario.
3

---

oiggnal featare
eitucaried Suture
Tnleccation
Zorig
X
Enc
&
Original task
(a) Defense against reconstruction attack
purtkd
M
Shadow task
Ze
Edge
clis
Cloud
(b) Reconstruction attack based on SFD
Figure 2: Reconstruction defense and attack in edge-cloud collaborative inference systems. (a) shows the main idea of defenses. (b) shows the proposed reconstruction attack based on Sensitive Feature Distillation (SFD).
3 Problem Setup: Defenses Against Reconstruction Attacks
3.1 Adversary: Learn to Reconstruct
The goal of the adversary to reconstruct the private input data sent by the user to the edge-cloud DI. inference services. Following existing works on reconstruction attacks [He et al. 2019| Singh ct al. 2021]. we consider a collaborative inference system between a local, edge-device part and remote, cloud-based part C. For partitioned model F' C Enc, E performs the earlier layers Enc and C performs C. We consider E to be trustworthy, and that will properly process arbitrary input data and compute the intermediate output. EE also attempts to prevent the input data from being reconstructed for privacy, so E applies the defenses to the representation and sends to the remote part C. The cloud C, i.e. the adversary, is untrusted try to steal the input.
3.2 Protector: Learn "NOT" to Reconstruct
For privacy, the edge part of the model typically obfuscates the intermediate feature representations before sending them the cloud. The goal of the specially designed obfuscation convert the input r into an intermediate representation that cannot leak sensitive features of but whose expressive power sufficient to infer the intended label y. Several defense strategies have been developed to mitigate the risks of reconstruction attacks. Figure (a) shows the main idea of the defenses.
Adversarial Learning. The most straightforward defense against a reconstruction attack apply adversarial training GGodffeloo et al. 2014 to improve the robustness of the model's intermediate feature representation. [Liu al. 2019] presents the training objective of the target model, which aims at learning feature representation that can protect the privacy and perform normal classification.

---

The protector formalizes the above training objective in the following min max optimization problem:
min (Lrask + A max Gadv )
Enc,C
Dec
optimal inversion
optimal privacy preserving classificaion
where the Enc aims fool the adversary's 's reconstructor model by minimizing the reconstruction gain Gadv. and the performance of C should be maintained. This leads to training the model with optimal grivac--reseevvig properties while maximizing utility.
Noise Mask. Noisy methods have been introduced to protect model inference (Ticcombe et al.
202T
Mireshghallah et al., 2020] by adding random noise the inference data. (Titcombe al.
1021 suggests defending against reconstruction attacks by adding additive Laplacian noise to the feature mapping. Formally, it calculates znoise Enc(x) c, where the random noise is sampled from a Laplace distribution that is parameterized by location and scale (In (Titcombe et al. [021]) f is set 0). This makes the intermediate data transferred between the edge and the cloud obfuscated by randomness every time the partitioned model is used.
Dropout Defense. (He et al., 2020 introduces dropout defense defeat reconstruction attacks. Specifically, the protector generates random masks M to protect the intermediate data during the inference phase of the model:
dropout Fen(()) @ M
where M denotes a randomly generated mask, @ denotes the element-wise multiplication. To allocate each element of M to values random: one of them is with probability r and the other is with probability 1 1-r- r. Note that the dropout defense differs from noise obfuscation that dropout hides partially sensitive information in the feature representation, whereas noise disturbs all the information of the intermediate data.
Applying noise mask dropout defense directly the feature representation in prediction can intro- duce a significant performance loss. As a result, the protector would prefer to perform training with the noise mask dropout defense imposed, enhancing the robustness of the model to randomness.
3.3 Latent Privacy Features
We refer to (Siggh al., 2021 and visualize the original face image and the learned intermediate feature representations using adversarial learning [Liu et al. (211)). as depicted in Appendix A We can note that after obfuscation by the defense mechanism, the learned individual feature mapping retains the latent sensitive features of the input image, especially the redundant spatial feature information. Consequently, we posit that extracting the potential image structure knowledge of sensitive features can help achieve better reconstruction results.
4 Methodology
The main goal this paper is to study the potential privacy leakage points existing defenses in collaborative inference systems. Specifically, we propose anti-defense method and describe how to exploit Sensitive Feature Distillation (SFD) for more aggressive reconstruction attack through the carefully designed algorithm.
4.1 Adversary's Capability
We consider black-box scenario where an adversary is allowed query the target model's encoder module Enc to make inferences from their collected data samples and obtain the corresponding intermediate features. For adversaries, this challenging attack setting [Salem et al., 2018 The Feature Distiller module of our attack model is defined as supervised DL model, which needs ground truth data for training. So we adopt shadow training techniques following previous works (Shokri et al. 2017b Salem et al. 2020) Zhou et al. 202 We assume the adversary trains shadow model M,, which is designed to imitate feature extraction capability of the target model. We also consider that the shadow model with the same structure as the target model. In order to achieve this
5

---

in practice, the adversary can perform model extraction attacks Pal et al. 2020: Sha et al. 2022]. Moreover, to train the shadow model, we further assume that the adversary has an auxiliary dataset, denoted by D.. that is drawn from the same distribution as the training set of the target model. Once the shadow model has been trained, the adversary can rely on to derive the ground truth clean feature to establish her attack model. Note that most of the experiments related to shadow models' training by previous works make the same assumptions. We subsequently show in Section g that the assumptions regarding knowledge of the model structure can be relaxed.
4.2 Anti-defense Reconstruction Algorithm
Our attack scheme consists three stages. In the first phase, the adversary constructs a shadow model locally, which is used to produce training data for the next stage. In the second phase, the Feature Distiller module of our attack model learns to distill obfuscated features into sanitized features. In the last phase, the decoder module of our attack model takes the sanitized features to produce reconstructed images. The second and third steps are performed alternately in the concrete attack flow. Figure (b) provides diagram of our attack scheme.
Shadow Model. The Feature Distiller module of our attack model needs to be trained in supervised manner. Due to our minimal assumptions in Section 4.1 the adversary can only obtain the obfuscated intermediate features by querying the target model. Nevertheless, we also need ground truth data, iee clean intermediate features, to train the Feature Distiller. To overcome this problem, we rely on shadow training, the shadow model is applied imitate an unshielded target model. The adversary first establishes a shadow model M, = C. Enc, with the same structure as the target model and then uses its own dataset to train M,. She can use SGD or ADAM to optimize the cross-entropy loss function of the shadow model. If the adversary's s objective is to discover the inversion of the DNN for diverse tasks, then other more appropriate loss functions can be utilized. As soon as the shadow model M, is established, the adversary can utilize it to generate pure intermediate features that are not obfuscated.
Sensitive Feature Distillation. A straightforward reconstruction attack may not succeed if the intermediate data z obfuscated to reduce the quantity of information it can leak about the input x. To address this issue, we borrow the idea from the feature- based knowledge distillation [Romero ct al.,
2014
Zagoruyko and Komodakis 2016]. In particular, we adapt and extend the techniques Song and Shmatikov 2019 to make it work in data reconstruction attacks. [Romero Zagoruyko and Komodakis 2016 use the intermediate representations of the teacher
proposed in
al.
2014
model to help the student model achieve better results in the original classification task, [Song and Shmatikov 2019] utilizes feature representations of the auxiliary model for de-censoring to leak attributes of interest. But unlike these efforts, we use the teacher model (shadow model) to help the student model (Feature Distiller) distill sensitive information to improve the effectiveness of the reconstruction attack.
To successfully reconstruct a user's private image x, our approach utilizes a convolutional neural network, termed Feature Distiller, to extract missing sensitive regions in the obfuscated intermediate features. In our settings, the Feature Distiller needs to use clean feature data as the training targets. Specifically, the adversary first uses her local training set X (X are sampled from the adversary's s datasets D.) to query both the target model M and the shadow model M., and observes the corre- sponding intermediate outputs Z Enc( X) and Z, Enc. (X). Next, the adversary can directly train her Feature Distiller using 7 as the training inputs and Z, as the training targets. The feasibility of using loss on feature space for training has been demonstrated in [Romero et al. 2014 Zagoruyko
and Komodakis 2016; Song and Shmatikov 2019]. so we take Sensitive Feature Distillation as the following optimization problem to learn the Feature Distiller M4:
La
Σ 2E7,2,E7,
[|Ma(2)
2
(3)
Here, we pair obfuscated features Z and clean features Z, in order to encourage the Feature Distiller Ma to identify the critical mappings in sensitive spatial information. After training, we can use the Feature Distiller to distill the latent vector that is most similar to the clean feature from the target obfuscated feature z.

---

Table 1: Experiment Configurations. We generate intermediate features from the different CNNs where several convolution, batch normalization, and activation layers are assembled under a single
conv block. Dataset
MNIST ConvNet
CIFAR1O VGG-16
CelebA
Target Model
ResNet-50
2nd conv block (16*16)
1tt conv layer (8*8)
1st conv block (16*16)
3rd conv block (8*8)
Split point
2nd conv layer(4*4)
2nd conv block (8*8)
4th conv block (4*4)
1st fe layer(64)
3rd conv block (4*4)
avgpool (1*1)
1st layer (512)
Reconstruction Attack. At this phase, we use the same black-box attack scheme proposed in [He al. 2019 Singh et al. 2021]: the Enverse-Netwold (Supervised Decoder). Notionally, the Inverse-Network is the proximate reverse function Enc. However, our approach differs from the previous one in that the training input to the Inverse-Network is no longer observed data Enc(x), but purified data M2 Enc(()) produced by the Feature Distiller. Meanwhile, the unverse-Network G and the Feature Distiller Ma are jointly trained, which leads to better reconstruction quality.
Similar to the previous phase, the adversary gathers a stack of training samples of labeled (z, x) pairs. She can then use Ma(z) as the sanitized latent vectors of z to train the Inverse-Network. We leverage the L2 norm the pixel space as the loss function of G:
LG= E [IG(M((2)) - x1/2
(4)
2EZ
G is optimized to reconstruct data from sanitized latent vectors. Notably, the structure of G is not necessarily correlated with the target Encoder module Enc. After G and Ma are trained, the adversary can feed any observed internediate-level value = Enc(x*) to the attack model and obtain the reconstructed sensitive input i = G(Ma(z*)).
Experiment
We systematically measure the robustness of existing methods for defending against reconstruction attacks and the effectiveness of our proposed scheme. We carry out extensive experiments three representative image classification tasks, each which uses a different architecture of the target DNN model.
5.1 Experiment Setup
Datasets. To perform experimental evaluation, we use three standard benchmark image recognition
datasets: MNIST [LeCun 1998). CIFARIO KKizhevsky et al. 2009 and CelebA [Liu et al. 2015
These datasets also serve as baseline datasets for a variety of ML security and privacy tasks. In most of our experiments, we split the dataset into three parts, target models' private dataset Dp. adversary's shadow dataset DD and the test set D; with the rest. We detail each dataset in the following: MNIST contains 70,000 grey-scale images. We set Dp,, D, and Dr to 50,000, 10,000 and 10,000 respectively; CIFAR10 consists of 60,000 color images. We separate D,, D. and D; to 40,000, 10,000, and 10,000 respectively; CelebA is dataset composed of 202,599 images of celebrities. For simplicity, the task is to predict gender. The D,,, D. and D; are set to 101,299, 81, 1,040 and 20,260 accordingly.
Models. For MNIST, We experiment with ConvNet which similar to the LeNet LeCun et al. 1998 architecture, only making the network layer wider. For CIFAR1O. apply VGG- 16 [Simonyan and Zisserman 2014]. For CelebA, we adopt ResNet-50 (He et al. 2016). We split each target model into different layers. For Inverse-Network G, following He et al. 2019 Yang al. 2019).
transposed CNN block to decode target intermediate feature representations to pixel images, where each block has transposed convolutional layer followed by a batch normalization layer well an
"The official implementation of He et al. 2019
Collaborative Inferreceeeeeeeeeeeeee
7

---

Truth
Vanilla
ERA
SFD
Truth
Vanilla
ERA
SFD
Truth
Vanilla
ERA
SFD
2nd conv block
3rd conv block
4th conv block
avgpool
1st fe layer
a) Adversurial Learning
b) Noise Mask
c) Dropout Defense
Figure 3: Reconstruction attack qualitative evaluation: We show the reconstructed images on CelebA. Each row corresponds to different split points of ResNet-50. In both (a), (b) and (c). the first column depicts the original images, and the second column depicts the result derived from attacking unprotected features, which is the upper bound for our approach. The third column presents the reconstruction results of ERA, while the fourth column presents the results of SFD.
activation layer. The number of transposed CNN blocks increases with the deepening of the split point. For Feature Distiller Ma, we choose neural network using multiple convolutional layers (when the split layer is conv block) or full connection layers (when the split layer is fc layer), with the number of hidden units set to the dimension of the feature data.
Compared Baselines. In the collaborative inference system scenario, evaluate the effectiveness of the existing defenses by comparing the proposed attack with Existing Reconstruction Attack (ERA) (He et al. 2019 against all three defense methods.
Experiment Details and Evaluation Metrics. Table gives details of the experimental configura- tions. To quantify the privacy risks of reconstruction attacks, follow the setting in most of the previous work (He et al. 2019 Singh et al.) 202T]. We adopt two metrics, Mean Squared Error (MSE) and Structural Similarity Index (SSIM) Wang al., 2004). We use different evaluation metrics between the reconstructed data and the original instance to gather multiple pieces of evidence of how effective the attack recreates the private target sample.
5.2 Empirical Results and Performance Analysis
Reconstruction Performance. Table 2 shows the performance of the proposed approach. We report the difference values (values in brackets) between the MSE loss of reconstruction attack based on SFD and the baseline ERA, and we also report the difference values between the SSIM of the reconstructed image generated from method and from the baseline, both of which represent performance gains. Averagely, the proposed method provides a great improvement in reconstruction results against all three defenses. We owe this to SFD used in our method, which can capture sensitive spatial information for more aggressive attacks. Note that the vanilla results show the reconstruction effect of using ERA when the target model is without defense. Here, we just use vanilla results to show the best possible results for the reconstruction. The observation is that our approach still provides a competitive performance. Specifically, on the CelebA dataset, we improve on decreasing the MSE loss from 0.124 to 0.032 and increasing the SSIM from 0.370 to 0.618 compared to ERA when the split point is set to the 2nd conv block and against adversarial learning, which brings a 74.2% performance improvement (in terms of MSE loss values) while being close to the vanilla SSIM results 0.785. For noise mask and dropout defense, we can achieve 47.7% % and 40.9 % performance improvement, respectively.

---

Table 2: Reconstruction results against all three defenses, the data in brackets represents the attack performance improvement of reconstruction attacks based on SFD compared to the baseline ERA. Vanilla results show the reconstruction effect of using ERA when the target model is without defense. Defense hyper-parameters are set À = 5.0 for adversarial learning, b = 10.0 for noise mask, and 0.9 for dropout defense. Both noise mask and dropout defense perform training with the
r =
defense algorithm imposed. The smaller the MSE and the larger the SSIM indicates the better the reconstruction results.
Dataset
Split Point
MSB
Vanila
8SIM
MSE
Adversara
SSIMT
MSE
Notes
SSIM
MSE
Droposi
SSIM
coar layer
0.002
0.951
0.412-00.10)
0.123-004511
0.323-(0.070)
0.138+0.082)
0.022-10,006)
0.913+00013)
comv layer
0,057
0.838
0.324-(0.199
0.109-(0.94)
0.255-(0.089)
0.146100.319
0.231-10.110)
(.299
1st Eo layer
0.103
0.709
0.199+0.016)
0.369-00.066)
0.140+0.0077
0.585-00.21)
0.235-00.002)
0.246-00.006)
comr bluck
0.006
0.912
0.235-(0.031)
0.309-(0.181)
0001--0:02
0.72340.116
0.075-(0.030)
0.53740.116
CIFARIO
cunv
0,026
0.762
0.233-(0.045)
0.149-00.123)
0:112-(0.024)
0.409-(0.077)
0.173-00.044)
0.11700 106)
3rd cony block
0.100
0.397
0.230-0.021)
0.143-(0.023)
0.212-(0.005)
0.130+00.19)
0.223-0.0016)
0.132+0.033)
2nd conv block
01010
0.785
0.124-
(0.88)
0.172-(0.022
0.2891
0.088-10.0366
0.416 175)
3rd block
0.020
0.698
0.189-(0.045)
0.331-(0.121)
0,206-(0.038
0.332+00083)
0.155-(0,021)
0.370+(0.090
4th conv 0,038
0.625
0,216-(0.023)
0.369-(0.048)
(.219-(0.01))
0.364:(0.034)
0.202-(0.008)
(.375:(0.027)
avgpool
0,067
0.581
0.187-00.003)
0,405 (0.015)
0.231-20.0022
0.230-(0.008) 0.372)
(0.00))
1st fo lnyer
0.124
0.452
0.2164(0.005))
0.599-(0.0022
0.2524(0.002))
0.3764(0.010)
0.231-00.003)
0.3824(00007)
Table 3: Evaluation for the impact of different defense hyper-paramenn on CIFARIO dataset.
1st conv block
2nd conv block
3rd conv block
Defense
Param
Acc
SSIM
Acc
SSIM
Acc
SSIM
1.0
83.1
0.945+(0.0055)
83.5
0.380+(0.117)
83.4
0.153+(0.014)
Adversarial
2.0
79.8
0.457+(0.221)
81.8
0.167+(0.164)
81.5
(.156+(0.011)
5.0
76.5
0.309+(0.181)
79.0
1..49+(0.112) 123)
77.3
0.143+(0.003))
1.0
85.5
0.876+(0.053)
86.9
0.729+(0.019)
86.0
(.289+((.009)
Noise
5.0
83.7
0.796+(0.089)
85.5
0.526+(0.062)
85.9
(.145+(0.015)
10.0
82.3
0.723+(0.116)
84.3
0.409+(0.077)
84.9
0.130+(0.019)
0.5
84.9
(0.08+
85.4
0.594+00.022)
84.9
0.256-(0.015)
Dropout
0.7
83.7
0.732+(0.089)
84.2
0.461+(0.061)
85.8
0.169+000027)
0.9
79.9
0.537+(0.116)
81.9
0.217+(0.106)
84.6
0.132+0.035)
Comparison for Different Split Points. For different split points of the target model, we conduct both quantitative (Table 2 and qualitative analysis (Figure 3. The visual results show that SFD achieves remarkably better reconstruction performance than ERA, and it works better early layers than at later layers. This is confirmed by the quantitative results where SFD obtained MSE improvement of 0.092 (74.2%) and SSIM improvement of 0.248 in the 2nd conv block, compared to 0.003 (1.6% %) and 0.015 in the avgpool. It is consistent with the intuition that features are becoming more and more abstract during the forward propagation process of the neural network, so the quality of reconstructed images of both SFD and ERA deteriorates as the layer goes deeper. See Appendix for more examples and quantitative results.
Besides, we also observe significant deterioration in reconstructed image quality on deeper layers (especially the fully connected layer). This is due to the size of feature mapping on deeper layers is typically smaller than on shallower layers, and the fully connected layer even mixes up previous visual information, keeping only the semantic information. So it relatively harder for SFD to distill sensitive features from small obfuscated representations. When choosing the partition point in Brivacy-Awars collaborative inference system, we suggest that the split layer on the edge device
should be set deep enough (e.g., at least one fully connected layer [He et al. 2019 2020 to reduce the impact of reconstruction attacks.
Effect of Defense llyper-parameterr Table presents the reconstruction performance for varying defense syper-parameters. The results indicate that even if the hyper-parameters of adversarial learning, noise mask, and dropout defense are set to 5.0, 10.0, and 0.9 (These parameter settings have incurred considerable utility loss), SFD can achieve SSIM improvements of O. 0.181, 0.1 16 and 0. 16 respectively. This demonstrates that SFD can still effectively extract sensitive information from feature spaces that are protected by stronger defenses (more heavily obfuscated features).
In addition, we note that the attack performance of ERA decreases (lower SSIM) with increasing hyper-parameters, which will negatively affect the reconstruction results of SFD (more generic reconstructed images). Thus, even against SFD attacks, larger defense haper-parameters could

---

Table 4: Performance of shadow models for different structure. The accuracy of shadow models using VGG-16, VGG- 13, and ResNet- 18 is 72%, 65%, and 61%, respectively.
1st conv block
2nd conv block
3rd conv
block
Shadow Model
MSE
SSIM
MSE
SSIM
MSE
SSIM
None
0.220
0.457
0.221
0.167
0.219
0.156
VGG-16
0.084
0.678
0.152
0.331
0.203
0.173
VGG-13
0.082
0.682
0.154
0.330
0.206
0.202
ResNet-18
0.078
0.671
0.153
0.302
0.194
0.207
potentially provide stronger obfuscation effects in reconstruction, but as a trade-off, this will severely degrade the utility of the target model. For more details on the discussion of the comparison of defense methods, see Appendix B
Discussion
Relaxing Attacker Model Assumption. We explore further relaxing the attacker's knowledge of the target model structure. For simplicity, we employ shadow model with intermediate feature representations of the same size as the target model, but with a different structure. Table f depicts the evaluation results. In our experiments on CIFAR1O, we apply the VGG-16 described previously in Section 5. to the target model, and use VGG- 16, VGG-13, and ResNet-18 for the shadow models respectively. The results of the reconstruction of the intermediate features of different sizes show both worse and better performance, but the variations are small, indicating that SFD is robust against such changes in the architecture of the shadow model.
Possible Defenses. As shown in Table 2 and Figure 3 both quantitative and qualitative mea- surements of the reconstruction images become more generic as the split point becomes deeper. Mcanwhile, increasing the defense hyperparameters illustrated in Table 3 Combining these two strategies may provide better privacy protection for inference data. However, this may cause heavy computational burden the client devices and reduce model accuracy. We have made it our future work to explore mechanisms to effectively defend against reconstruction attacks in collaborative inference systems.
help mitigate the attacks to degree, as
7 Conclusion
In this work, we track the potential privacy leakage points of defense mechanisms against reconstruc- tion attacks collaborative inference systems. We posit that extracting the latent image structure knowledge of sensitive features can help achieve better reconstruction. Therefore, we devise an anti-defense method using the sensitive feature distillation (SFD) technique bypass the existing defense mechanisms. SFD is designed to purify feature representations that are obfuscated by the defense mechanisms, so as to better extract sensitive information. We systematically measure the effectiveness of our method against three state-of tifte-of-th-att the-hattt art defenses. Experimental results show that SFD achieves remarkable performance improvements over existing reconstruction techniques on several datasets, which demonstrates that current defense schemes are not robust enough. We hope our findings encourage critical exploration of defense mechanisms against reconstruction attacks for private collaborative inference.
References
Amin Banitalebi-Dehkordi., Naveen Vedula, Jian Pei, Fei Xia, Lanjun Wang, and Yong Zhang. Auto-split: general framework of collaborative edge-cloud ai. In Proceedings the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2543 2553, 2021.
Amir Erfan Eshratifar, Mohammad Saeed Abrishami, and Massoud Pedram. Jointdn: an efficient training and inference engine for intelligent mobile cloud computing services. IEEE Transactions on Mobile Computing. 20(2):565 576, 2019.
10

---

Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence infor- mation and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1322-1333, 2015.
Karan Ganju, Qi Wang. Wei Yang. Carl A Gunter, and Nikita Borisov. Property inference attacks fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages 619-633, 2018.
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recogmition, pages 770-778, 2016.
Zecheng He, Tianwei Zhang. and Ruby Lee. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference, pages 168-162, 2019.
Zecheng He, Tianwei Zhang, and Ruby Lee. Attacking and protecting data privacy in edge-cloud collaborative inference systems. IEEE Internet of Things Journal, 8(1 8(12)9706-97) 9716,, 2020.
Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 603-618, 2017.
Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. In Proceedings of the Twenty- Second International Conference on Architectural Support for Programming Languagey and Operating Systems, ASPLOS 17, page 615-629 New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450344654, doi: 10. 00.1145//037699333336699
URL https://ddo orgh 1111/3133133999
3037698
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdh/nnist// 1998.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applicd to document recognition. Pruceedings of the IEEE, 86(11):227 23244, 1998.
Ang Li, Jiayi Guo, Huanrui Yang, Flora D Salim, and Yiran Chen. Deepobfuscator: Obfuscating interme- adversarial learning on smartphones. In Proceedings of the
diate representations with International Conference on Internet-of-Thine Design and Implementation, pages 28 39, 2021a.
Min Li, Yu Li, Ye Tian, Li Jiang, and Qiang Xu. Appealnet: An efficient and highly- edge/cloud collaborative architecture for dnn inference. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 409 414. JEEE, 2021b.
Sicong Liu, Junzhao Du, Anshumali Shrivastava, and Lin Zhong, Privacy adversarial network: representation learning for mobile data privacy. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 314) 2019.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730-3738, 2015.
Fatemehsadat Mireshghallah, Mohammadkazem Taram, Prakash Ramrakhyani, Ali Jalali, Dean Tullsen, and Hadi Esmaeilzadeh. Shredder: Learning noise distributions to protect inference privacy. Proceedings of the Swenty-Fithh International Conference on Architectural Support for Programming Languages and Operating Systems, pages 3-18, 2020.
Scyed Ali Osia, Ali Shahin Shamsabadi, Sina Sajadmanesh, Ali Taheri, Klcomenis Katevas, Hamid R Rabice, Nicholas D Lane, and Hamed Haddadi. A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet of Things Journal, 455):400544518. 2020.
Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. Activethief: Model extraction using active learning and unannotated public data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 865-872, 2020.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint 111. 2014.
11

---

Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv: 0en.:0000001,4, 2018.
Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. {Updates Leakk) Data set inference and reconstruction attacks online learning. In 29th USENIX Security Symposium (USENIX Security 20). pages 1291-1308, 2020.
Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Can't steal? cont-steal! contrastive stealing attacks against image encoders. arXiv preprint arXiv:2201.07515 2022.
Reza Shokri, Marco Stronati, Congzheng Song. and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). pages 3-18. IEEE, 2017a.
Reza Shokri, Marco Stronati, Congzheng Song. and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). pages 18.. IEEE, 2017b.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Abhishek Singh, Ayush Chopra, Ethan Garza, Emily Zhang, Praneeth Vepakommaa, Vivek Sharma, and Ramesh Raskar. Disco: Dynamic and invariant sensitive channel obfuscation for deep neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 12125-12135, 2021.
Congzheng Song and Vitaly Shmatikov. 19X.:1995..11422. 2019.
Overlearning reveals sensitive attributes.
arXiv preprint
Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 27), pages 2615-2632, 2021.
Tom Titcombe, Adam Hall, Pavlos Papadopoulos, and Daniele Romanini. Practical defences against model inversion attacks for split neural networks. arXiv preprint arXiv:2104.05743, 2021.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction [APIs). In 25th USENIX security symposium (USENIX Security 16), pages 601-618, 2016.
Binghui Wang and Neil Zhenqiang Gong. Stealing hyperparameters in machine learning. In 2018 IEEE Symposium on Security and Privucy (SP), pages 36-52. IEEE, 2018
Kuan-Chieh Wang. Yan Fu, Ke Li, Ashish Khisti, Richard Zemel, and Alireza Makhzani. Variational model inversion attacks. Advances in Neural Information Processing Systems, 34:9706- 97169,979,9799 2021.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions image processing, 13(4):600-612, 2004.
Taihong Xiao, Yi-Hsuan Tsai, Kihyuk Sohn, Manmohan Chandraker, and Ming-Hsuan Yang. Adversarial and task-oriented representations. In Proceedings the AAAI Conference on Artificial Intelligence, volume 34, pages 12434-13 12441. 2020.
learning of
Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 225- 240, 2019.
Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint 1cXiv:1612.039288 2016.
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxian Wang. Bo Li, and Dawn Song. The secret revealer: Generative model-innvrsion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 253-261, 2020.
Xuejun Zhao, Wencan Zhang, Xiaokui Xiao, and Brian Lim. Exploiting explanations for model inversion attacks.
In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 682 02-6922,6 2021.
Junhao Zhou, Yufei Chen, Chao Shen, and Yang Zhang. Property inference attacks against gans. arXiv preprint arXiv:211..07608, 2021.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [No]
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [No]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of computing and the type of resources used (c.g.. type of GPUs, internal cluster, or cloud provider)? [No]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "R49nm3RZwqn", "reviewer": "Reviewer_HyYW", "summary": "This paper looks into the three existing privacy protection mechanisms for edge-cloud collaborative inference systems and proposes an anti-defense method, dubbed Sensitive Feature Distillation (SFD), which simultaneously uses two existing techniques, shadow model and feature-based knowledge distillation, for attacking the privacy protection mechanisms. The core idea is to develop a shadow model for the attack-targeting model, which can produce the non-obfusticated and labeled training data for the privacy-protecting model, and then feeds this model to the knowledge distillation process to imitate or learn the behavior of the target model, so that the resulting model can produce the non-obfusticated (or purified) feature map, which can lead to the privacy leak. The authors claim that the proposed attack method can effectively extract the purified feature map from the intentionally obfusticated one and recover the private image for popular image datasets such as MNIST, CIFAR10, and CelebA. ", "questions": "* Currently, the manuscript is written in such a way that there are various assumptions that seem to make the proposed technique work. Moreover, the experiments are performed in quite specific setups, which makes me question about the generality and practicality of proposed solution. Could you please elaborate on this?\n* Especially, the model split point should play a significant role to evaluate the effectiveness of proposed solutions and make comparisons with the existing techniques. I believe, to be fair, the split points should be similar to what prior works used. Could you please comment on this?\n* If I understood correctly, the attackers not only have model's behavior (functionality), model architecture, and hyperparameters a priori, but also have the labeled training data for ground truth that includes possible image inputs and corresponding intermediate feature map results. Should the attackers have all these conditions satisfied to try an attack, or are some of these conditions optional?  ", "limitations": "Elaborated above in the weaknesses part. ", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n* This paper tackles an interesting problem, the privacy protection in edge-cloud collaborative systems. \n* This paper identifies limitations of the existing solutions, which seem valid issues. \n* Writing quality is sufficient for nonknowledgable readers to follow and comprehend the idea. \n\nWeaknesses\n* The proposed technique is based on narrow assumptions that much information is already available to the attackers and given. The examples are the model architecture, hyperparameters, and training data for shadow model (Ds). Moreover, the proposed technique narrows the target data type to be images. \n* The experiments focus on very early split points such as 1st or 2nd conv layer or 1st fc layer. This settings are favorable to the proposed technique since the existing techniques would be weaker to the model inverse attacks. On the other hand, prior works do not seem to split the model in such an early point in the model architecture. For instance, Table 2 in [Mireshghallah et al., 2020] reports that they run 4~11 layers in the edge while offloading the rest to the cloud. For fair comparison, I believe the methodology should be consistent among the baselines and proposed system. \n* While Section 5 offers the analysis for the implication of shadow model, the experiments still use quite small, simple, and outdated models such as VGG-13, VGG-16, and ResNet-18, which do not seem adequate for demonstrating the robustness on the different model architectures. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GF6IYvB5Uyd", "reviewer": "Reviewer_d4JB", "summary": "Existing works in ARL have developed different types of techniques for protecting sensitive information like sensitive attribute and sensitive input. For the sensitive input category, this authors develop an attack that can undermine the security of the existing defense techniques. Motivated by this observation, they introduces a new defense technique that perform favorably on their attack algorithms.", "questions": "How is your contribution 1  (L53-55) different from contribution 3 (L59-63)? I believe both refer to the experimental evaluation only.", "limitations": "The authors may wanna highlight limitations of their empirical evaluation but I do not see any major concern", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength -\n- Sensitive feature distillation is an interesting and new idea in the ARL community.\n- The experimental data highlights the vulnerability of the ARL approaches to protect inversion of inference data.\n\nWeakness -\n- The paper shouldn't call the attack techniques as model inversion since the goal is to infer sensitive information from the data used for prediction query and not the training data used.\n- Wrong characterization of early MI attacks by Fredrikson et al. 2014 and Fredrikson et al. 2015 that extract sensitive attributes (Fredrikson et al. 2014) and reconstruction of original data (Fredrikson et al. 2015).\n- While Mireshghallah et al. 2020 does add noise, it does not offer any differential privacy guarantee.\n- Dropout Defense for preventing reconstruction of visual data has been explored in the context of convolution channel obfuscation [1].\n- L249-250 assert that existing techniques do not leverage feature level data. I think this may not be true w.r.t the attacks used in [1].\n- This paper does not compare with other classes of defense techinques such as [2,3,4,5,6].\n\n\n1. Singh, A., Chopra, A., Garza, E., Zhang, E., Vepakomma, P., Sharma, V. and Raskar, R., 2021. DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12125-12135).\n2. Rezaeifar, S., Voloshynovskiy, S., Asgari Jirhandeh, M. and Kinakh, V., 2022. Privacy-Preserving Image Template Sharing Using Contrastive Learning. Entropy, 24(5), p.643.\n3. Osia, S.A., Shamsabadi, A.S., Sajadmanesh, S., Taheri, A., Katevas, K., Rabiee, H.R., Lane, N.D. and Haddadi, H., 2020. A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet of Things Journal, 7(5), pp.4505-4518.\n4. Vepakomma, P., Singh, A., Gupta, O. and Raskar, R., 2020, November. NoPeek: Information leakage reduction to share activations in distributed deep learning. In 2020 International Conference on Data Mining Workshops (ICDMW) (pp. 933-942). IEEE.\n5. Li, A., Guo, J., Yang, H., Salim, F.D. and Chen, Y., 2021, May. DeepObfuscator: Obfuscating intermediate representations with privacy-preserving adversarial learning on smartphones. In Proceedings of the International Conference on Internet-of-Things Design and Implementation (pp. 28-39).\n6. Liu, S., Du, J., Shrivastava, A. and Zhong, L., 2019. Privacy adversarial network: representation learning for mobile data privacy. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(4), pp.1-18.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "-o32IWK0XYk", "reviewer": "Reviewer_ihvh", "summary": "This paper proposes an attack model for edge-cloud collaborative inference systems. More specifically, the authors develop a technique for evaluating defense mechanisms against model inversion attacks, where the attacker uses obfuscated features (computed on a trusted local edge device and to be fed to an untrusted cloud service for prediction) to recover users’ private data (i.e., the raw image). To this end, the authors propose a technique called Sensitive Feature Distillation (SFD) to sanitize obfuscated features generated by a defense technique, which are then fed to an Existing Model Inversion (EMI) attack to restore the original image. The proposed technique is evaluated against three defense techniques (adversarial learning, noise mask, dropout) on three datasets (MNIST, CIFAR10, CelebA), where it is shown to outperform a baseline EMI attack (owing to the SFD model).", "questions": "- I'm interested to see if an SFD trained on one dataset can transfer to another. This would further relax the constraints on the attacker since they don't always have access to data with the same distribution as the training dataset. Have the authors performed any such experiments, or do they have any insight on the robustness of the SFD in this setting?\n- Figure 3b, the reconstructed images look more and more like generic images as you go down the rows. Is this also observed on other examples? Perhaps the authors could show some examples with the same image in all rows in the appendix.", "limitations": "The proposed method has very few constraints as it only requires blackbox access to the target model, and access to data with the same distribution as the training dataset for training the shadow model. This is discussed in good detail in the paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I enjoyed reading this paper. The authors have developed a novel technique that can be used for evaluating defense techniques in edge–cloud collaborative inference systems. The proposed method is well-described and technically sound, and the black box attack setting makes this applicable to a wide range of scenarios. The evaluation results also reveal that current defense mechanisms cannot adequately defend against adversarial attacks, similar to many other domains.\n\nOne weakness of this type of inversion attack is that the recovered image tends to look more like a generic image of the same class (see the Figure 3b, specially the bottom rows) rather than the original image, and the used metrics somewhat fail to capture this distinction. So perhaps the privacy issue is less severe than what the actual numbers suggest. Nevertheless, the proposed technique is novel and succeeds in outperforming the baseline, which makes it a solid contribution for evaluating defense techniques.\n\nMinor comments:\n- Line 315: \"Strong defence does not help, the performance of our attack increases along with the hyper-parameter increases\", isn't the performance of the attack decreasing (lower SSIM) with larger hyperparameters?\n- Line 301: defense -> defence, for consistency! :)\n- Line 328: \"When repeating the experiments on other datasets or other\", typo?\n- Section numbers in the checklist are unresolved.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept