Measuring Data Reconstruction Defenses in Collaborative Inference Systems
Abstract
The collaborative inference systems are designed to speed up the prediction processes in edge-cloud scenarios, where the local devices and the cloud system work together to run a complex deep-learning model.
However, those edge-cloud collaborative inference systems are vulnerable to emerging Reconstruction Attacks, where malicious cloud service providers are able to recover the edge-side users’ private data.
To defend against such attacks, several countermeasures have been recently introduced.
Unfortunately, little is known about the robustness of those defense countermeasures.
In this paper, we take the ﬁrst step towards measuring the robustness of those state-of-the-art defenses with respect to reconstruction attacks.
Speciﬁcally, we show the latent privacy features still retain in the obfuscated representations.
Motivated by such an observation, we propose a novel technology called Sensitive Feature Distillation (SFD) to restore sensitive information from the protected feature representations.
Our experiments show that SFD can break through defense mechanisms in model partitioning scenarios, demonstrating the inadequacy of existing defense mechanisms as a privacy-preserving technique against reconstruction attacks.
We hope our ﬁndings inspire further work in improving the robustness of defense mechanisms against reconstruction attacks for collaborative inference systems.
Introduction
With the rapid development of deep neural network technology, the deployment of deep learning (DL) applications on edge devices is becoming more widespread.
However, on the one hand, resourceconstrained edge devices are often difﬁcult to accommodate computationally intensive DL models, and on the other hand, cloud-based DL inference applications require devices to upload users’ private data to cloud service providers, which poses a serious privacy threat.
Collaborative DL system has recently emerged as a good alternative to address these challenges, where the local devices and the remote server work together to run a complex neural network [Eshratifar et al., 2019; BanitalebiDehkordi et al., 2021; Li et al., 2021b].
The DL models can be split such that only the ﬁrst few layers are executed on the local edge device while the remaining layers are executed on the remote cloud.
And the end user does not need to upload her raw data to the cloud, so that privacy concerns can be addressed to some extent.
This paper focuses on Reconstruction Attacks , where existing research has shown that adversaries can easily recover sensitive data with the output of the middle layers
[He et al., 2019; Singh et al.,
To address this problem, many defense approaches have been proposed for collaborative inference systems.
[Liu et al., 2019; Xiao et al., 2020; Li et al., 2021a] demonstrates that adversarial learning improves the robustness of the model against reconstruction attacks.
[Titcombe et al., 2021;
Mireshghallah et al., 2020] proposes learning additive Laplace noise to protect the privacy of inference data.
[He et al., 2020] uses dropout to defend against reconstruction attacks.
However, none of the existing work provides a systematic security measurement for various defenses.
This leads to our main research question: Are those privacy protection techniques against reconstruction attacks for collaborative inference systems really robust enough?
In this work, we summarise the various defenses, systematically evaluate their performance in defending against reconstruction attacks, and present their trade-offs regarding data utility and privacy leakage.
We propose a novel technique to break down existing defenses, which can achieve improved reconstruction performance for a variety of datasets, target models, and defense algorithms.
We focus on the privacy of the inference data [He et al., 2019; Singh et al., 2021].
Since protectors obfuscate the intermediate feature representation to limit the redundant information about the model input, our idea is to learn to distill obfuscated features into sanitized features for better reconstruction.
Speciﬁcally, we introduce Sensitive Feature Distillation (SFD) to extract the image structure knowledge of sensitive features and ﬁnd that the proper formulation of prior knowledge will be helpful to solve the ill-posed reconstruction problem.
We show that this vulnerability is inevitable even when different defense countermeasures are utilized.
Figure 1 shows the reconstruction results using our proposed method and the existing one.
The contributions of this work are summarized as follows:
• We are the ﬁrst to experimentally verify the robustness of reconstruction defenses for inference data privacy in collaborative systems.
• We reveal the presence of latent privacy features in protected representations and validate the vulnerability of existing defense methods.
• We propose a technique called SFD against the exisitng defense mechanisms.
SFD can achieve better reconstruction by carefully extracting sensitive information to transfer the obfuscated features into the sanitized features.
• We conduct a comprehensive measurement of the robustness of various state-of-the-art defenses leveraging our proposed technique.
Our experiments show that SFD can achieve 74.2% performance improvement compared with the baseline attack, making the existing defenses signiﬁcantly less effective.
This paper focuses on the most common DL task: image classiﬁcation.
A Deep Neural Network (DNN) model can be characterized as F : X 7!
Y which maps an input data x 2 X into an output result y 2 Y .
Collaborative Inference Systems.
Running the entire deep learning inference application on the edge device can be time- and resource-consuming.
A common scaling approach is to partition the
DNN model where the ﬁrst few layers of the model are executed within the local edge device while the remaining layers are ofﬂoaded to the remote cloud [Kang et al., 2017; Eshratifar et al., 2019;
Banitalebi-Dehkordi et al., 2021; Li et al., 2021b].
This scheme can reduce the computational burden of edge devices on the one hand, and can mitigate privacy leaks of inference raw data on the other [Wang and Gong, 2018; Osia et al., 2020].
Collaborative inference makes it feasible and efﬁcient to deploy large-scale intelligent workloads on today’s edge platforms.
In the collaborative inference system, a model is split into two parts: an on-device part and a remote cloud-based part.
Each part will contain several layers of the model.
From now on, we reformulate the model F = C
Enc as the Encoder module Enc and the Classiﬁer module C .
The edge device deploys the ﬁrst part of the model Enc , it feeds the users’ inference data x to the feature extractor to get an intermediate feature representation z
= Enc ( x ) , and then sends z to the cloud.
The cloud hosts the second part C , which calculates the prediction output y
= C ( z ) when the intermediate value z is received from the edge device, then it returns the expected result y to the edge side.
Machine Learning Privacy and Reconstruction Attacks.
Recent research presented a variety of privacy threats against machine learning model, such as membership inference attacks [Shokri et al., 2017a; Song and Mittal, 2021], model extraction attacks [Tramèr et al., 2016], property inference attacks [Ganju et al., 2018] and model inversion attacks [Yang et al., 2019;
He et al., 2019].
These threats are also important in the context of collaborative DL systems.
This paper focuses on the Reconstruction Attacks which are closely related to much previous work on model inversion attacks.
They all aim to obtain sensitive information about the original data given access to the target machine learning model.
In the model inversion attack, the adversary is asked to recover sensitive features of training data by taking advantage of their correlation with the model output.
Although the original model inversion attack [Fredrikson et al., 2015] has limited performance [Hitaj et al., 2017], the inversion performance is signiﬁcantly improved by training transposed CNNs on an auxiliary dataset [Yang et al., 2019;
Zhang et al., 2020; Zhao et al., 2021;
Wang et al., 2021].
Unlike recovering training data from the target ML models, a reconstruction attack is a class of techniques that aims to recreate the input data fed through a predictive model at inference time.
[He et al., 2019; Singh et al., 2021] formalized diverse attack schemes on sensitive inputs in the context of edge-cloud collaborative inference systems.
In the white-box attack setting, they developed a regularized likelihood maximization algorithm, which intended to ﬁnd the optimal input data for the given deep neural representations and auxiliary information.
The basic idea of the algorithm is to formulate the reconstruction attack as a gradient-based optimization problem in the data space
X to ﬁnd an image ˆ x whose prediction intermediate feature representation Enc (ˆ x ) approximates a given Enc ( x ) .
However, it requires white-box access to the target model to compute the gradients to minimize the loss function.
Furthermore, this method involves optimization for each natural sample at inference time which makes it relatively inefﬁcient.
The black-box reconstruction attack trains another neural network G to directly identify the inversed mapping from Enc ( x ) to x .
Speciﬁcally, without having the knowledge of the parameters of Enc , the adversary can collect some pairs ( z, x ) by querying the model, and then she trains a Decoder module G to learn to well approximate the mapping between intermediate representation z and corresponding input x .
In contrast to the whitebox optimization-based attack, this approach is more efﬁcient: (1) calculating the reconstructed input is parameter-free; (2) for each target sample, the adversary only requires just one single forward pass through the Decoder module G .
In this work, we focus on the black-box reconstruction attack, which is a more practical and realistic scenario.
Problem Setup: Defenses Against Reconstruction Attacks 3.1 Adversary: Learn to Reconstruct
The goal of the adversary is to reconstruct the private input data sent by the user to the edge-cloud DL inference services.
Following existing works on reconstruction attacks [He et al., 2019; Singh et al., 2021], we consider a collaborative inference system between a local, edge-device part E and a remote, cloud-based part C .
For a partitioned model F = C  Enc , E performs the earlier layers Enc and C performs C .
We consider the E is trusted, when input is fed into Enc , E correctly processes it and computes a representation.
E also attempts to prevent the input data from being reconstructed for privacy, so E applies the defenses to the representation and sends it to the cloud.
The cloud C , i.e. the adversary, is untrusted to try to steal the input.
3.2 Protector: Learn “NOT” to Reconstruct
For privacy, the edge part of the model typically obfuscates the intermediate feature representations before sending them to the cloud.
The goal of the specially designed obfuscation is to encode the input x into a representation z that does not leak the characteristics of x , but whose expressive power is sufﬁcient to predict the task label y .
Several defense mechanisms have been proposed to mitigate the risks of reconstruction attacks.
Figure 2 (a) shows the main idea of the defenses.
Adversarial Learning.
The most straightforward defense against reconstruction attack is to apply adversarial training
[Goodfellow et al., 2014] to improve the robustness of the model’s intermediate feature representation.
[Liu et al., 2019] presents the training objective of the target model, which aims at learning a privacy-preserving and task-oriented representation.
The protector formalizes the joint privacy and classiﬁcation objectives in the following min-max optimization problem:
[Equation]
, where the Enc aims to fool the adversary’s reconstructor model by minimizing the reconstruction gain G adv , and the performance of C should be maintained.
This results in designing the optimal privacy mechanism, which is also utility-maximizing.
Noise Mask.
Noisy method has been introduced to protect model inference [Titcombe et al., 2021; Mireshghallah et al., 2020] by adding random noise to the inference data.
[Titcombe et al., 2021] suggests defending against reconstruction attacks by adding additive Laplacian noise to the intermediate feature representation.
Formally, it calculates z noise =
Enc ( x ) + ✏ , the random noise ✏ is sampled from a Laplace distribution parameterized by location µ and scale b each time the model is used.
The partitioned model, no longer a one-to-one function, obscures the data communicated between the edge and the cloud.
Dropout Defense.
[He et al., 2020] introduces dropout defense to defeat reconstruction attacks.
Speciﬁcally, the protector generates random masks M to protect the intermediate data during the inference phase of the model:
[Equation]
, where M denotes a randomly generated mask, ⌦ denotes the element-wise multiplication.
To allocate each element of M to two values at random: one of them is 0 with probability r and the other is 1 with probability 1  r .
Note that the dropout defense differs from noise obfuscation in that dropout hides partial sensitive information in the feature representation, whereas noise disturbs all the information of the intermediate data.
Applying noise mask or dropout defense directly to the feature representation in prediction can introduce a signiﬁcant performance loss.
As a result, the protector would prefer to perform training with the noise mask or dropout defense imposed, enhancing the robustness of the model to randomness.
3.3 Latent Privacy Features
We refer to [Singh et al., 2021] and visualize the original face image and the learned intermediate feature representations using adversarial learning [Liu et al., 2019], as depicted in Appendix A. We can note that after obfuscation by the defense mechanism, the learned individual feature mapping retains the latent sensitive features of the input image, especially the redundant spatial feature information.
Consequently, we posit that extracting the potential image structure knowledge of sensitive features can help achieve better reconstruction results.
Methodology
The main goal of this paper is to study the potential privacy leakage points of existing defenses in collaborative inference systems.
Speciﬁcally, we propose an anti-defense method and describe how to exploit Sensitive Feature Distillation (SFD) for a more aggressive reconstruction attack through the carefully designed algorithm.
4.1 Adversary’s Capability
We consider a black-box scenario where an adversary is allowed to query the target model’s encoder module Enc to make inferences from their collected data samples and obtain the corresponding intermediate features.
For adversaries, this is the most challenging attack setting [Salem et al., 2018].
The Feature Distiller module of our attack model is deﬁned as a supervised DL model, which needs ground truth data for training.
So we adopt shadow training techniques following previous works [Shokri et al., 2017b;
Salem et al., 2020;
Zhou et al., 2021].
We consider the adversary trains a shadow model M s , which is designed to imitate the behavior of the target model.
We also assume that the shadow model has the same model architecture as the target model.
In order to achieve this in practice, the adversary can perform model extraction attacks [Pal et al., 2020; Sha et al., 2022].
Moreover, to train the shadow model, we further assume that the adversary has a local dataset, denoted by D s , which comes from the same distribution as the target model’s training set.
Once the shadow model has been trained, the adversary can rely on the shadow model to derive the ground truth clean feature to train her attack model.
Note that most of the experiments related to shadow models’ training by previous works make the same assumptions.
We later show in Section 6 that the assumptions about the knowledge of model architecture can be relaxed as well.
4.2 Anti-defense Reconstruction Algorithm
Our general attack pipeline contains three phases.
In the ﬁrst phase, the adversary constructs a shadow model locally, which is used to generate training data for the next stage.
In the second phase, the Feature Distiller module of our attack model learns to distill obfuscated features into sanitized features.
In the last phase, the decoder module of our attack model takes the sanitized features to produce reconstructed images.
The second and third steps are performed alternately in the concrete attack ﬂow.
Figure 2 (b) provides a schematic view of our attack pipeline.
Shadow Model.
The Feature Distiller module of our attack model needs to be trained in a supervised manner.
Due to our minimal assumptions in Section 4.1, the adversary can only obtain the obfuscated intermediate features by querying the target model.
Nevertheless, we also need ground truth data, i.e. clean intermediate features, to train the Feature Distiller.
To overcome this problem, we rely on shadow training, the shadow model is applied to mimic an unshielded target model.
The adversary ﬁrst establishes a shadow model M s =
C s  Enc s with the same structure as the target model and then uses its own dataset to train M s .
She can use SGD or ADAM to optimize the cross-entropy loss function of the shadow model.
If the adversary’s goal is to ﬁnd the inversion of the DNN for different tasks, then other more appropriate loss functions can be utilized.
Once the shadow model M s is obtained, the adversary can use it to generate pure intermediate features that are not obfuscated.
Sensitive Feature Distillation.
A straightforward inversion attack may not succeed if the intermediate data z is obfuscated to reduce the amount of information it reveals about input x .
To address this issue, we borrow the idea from the feature-based knowledge distillation [Romero et al., 2014;
Zagoruyko and Komodakis, 2016].
But unlike these efforts, which use the intermediate feature mappings of the teacher model to help the student model achieve better results in the original classiﬁcation task, we use the teacher model (shadow model) to help the student model (Feature Distiller) distill sensitive information to improve the effectiveness of the reconstruction attack.
To successfully reconstruct a user’s private image x , our approach utilizes a convolutional neural network, termed Feature Distiller, to extract missing sensitive regions in the obfuscated intermediate features.
In our settings, the Feature Distiller needs to use clean feature data as the training targets.
Speciﬁcally, the adversary ﬁrst uses her local training set X ( X are sampled from the adversary’s datasets D s ) to query both the target model M and the shadow model M s , and observes the corresponding intermediate outputs Z
= Enc ( X ) and Z s =
Enc s ( X ) .
Next, the adversary can directly train her Feature Distiller using Z as the training inputs and Z s as the training targets.
We take
Sensitive Feature Distillation as an optimization problem with feature space L 2 loss:
[Equation]
, where M d is the Feature Distiller that the adversary wants to learn.
The possibility of training with a feature-space loss has been demonstrated in [Nguyen et al., 2016;
Shen et al., 2019].
We match obfuscated features Z and clean features Z s in order to encourage the Feature Distiller to identify the critical mappings in sensitive spatial information.
After training, we can use the Feature Distiller to distill the latent vector
ˆ
z
that is most similar to the clean feature from the target obfuscated feature z .
Reconstruction Attack.
At this phase, we use the same black-box attack scheme proposed in [He et al., 2019; Singh et al., 2021]: the Inverse-Network (Supervised Decoder).
Conceptually, the
Inverse-Network is the approximated inverse function of Enc .
However, our approach differs from the previous one in that the training input to the Inverse-Network is no longer observed data Enc ( x ) , but puriﬁed data M d ( Enc ( x )) produced by the Feature Distiller.
Meanwhile, the Inverse-Network
G and the Feature Distiller M d are jointly trained, which leads to better reconstruction quality.
We show the detailed description of our proposed reconstruction attack using sensitive feature distillation in Appendix D.
Similar to the previous phase, the adversary generates a stack of training samples of labeled ( z, x ) pairs.
She can then use M d ( z ) as the sanitized latent vectors of z to train the Inverse-Network.
We leverage the L2 norm in the pixel space as the loss function of G :
[Equation]
[Table]
, G is optimized to reconstruct data from sanitized latent vectors.
It is worth noting that the structure of G is not necessarily correlated with the target Encoder module Enc .
After G and M d are trained, the adversary can feed any observed intermediate-level value z ?
=
Enc ( x ? ) to the attack model and obtain the reconstructed sensitive input ˆ x ?
=
G ( M d ( z ? )) .
Experiment
We systematically measure the effectiveness of the existing defenses against reconstruction attacks and our proposed method.
We conduct extensive experiments on three representative image classiﬁcation tasks, each of which uses a different architecture of the target DNN model.
5.1 Experiment Setup
Datasets.
To perform experimental evaluation, we use three standard benchmark image recognition datasets: MNIST [LeCun, 1998], CIFAR
[Krizhevsky et al., 2009] and CelebA
[Liu et al., 2015].
These datasets are also benchmark datasets for various ML security and privacy tasks.
In most of our experiments, we split the dataset into three parts, target models’ private dataset D p , adversary’s shadow dataset D s and the test set D t with the rest.
We detail each dataset in the following: MNIST contains 70,000 grey-scale images.
We set D p , D s and D t to 50,000, 10,000 and 10,000 respectively;
CIFAR10 consists of 60,000 color images.
We separate D p , D s and D t to 40,000, 10,000 and 10, respectively; CelebA is a dataset composed of 202,599 images of celebrities.
For simplicity, the task is to predict gender.
The D p , D s and D t are set to 101,299, 81,040 and 20,260 accordingly.
Models.
For MNIST, We experiment with ConvNet which is similar to the LeNet [LeCun et al., 1998] architecture, only making the network layer wider.
For CIFAR10, we apply VGG-16 [Simonyan and Zisserman, 2014].
For CelebA, we adopt ResNet-50 [He et al., 2016].
We split each target model into different layers.
For Inverse-Network G , following [He et al., 2019; Yang et al., 2019], we use transposed CNN blocks 2 to decode target intermediate feature representations to pixel images, where each block has a transposed convolutional layer succeeded by a batch normalization layer and a activation layer.
The number of transposed CNN blocks increases with the deepening of the split point.
For Feature Distiller M d , we choose a neural network using multiple convolutional layers (when split layer is conv block) or full connection layers (when split layer is fc layer), with the number of hidden units set to the dimension of the feature data.
Compared Baselines.
In the collaborative inference system scenario, we evaluate the effectiveness of the existing defenses by comparing the proposed attack with Existing Reconstruction Attack (ERA)
[He et al., 2019] against all three defense methods.
Experiment Details and Evaluation Metrics.
Table 1 gives details of the experimental conﬁgurations.
To quantify the privacy risks of reconstruction attacks, we follow the setting in most of the previous work [He et al., 2019; Singh et al., 2021].
We adopt two metrics, Mean Squared Error (MSE) and Structural Similarity Index (SSIM)
[Wang et al., 2004].
We use different evaluation metrics Inverse between the reconstructed images and the ground truth to gather multiple pieces of evidence of how well the attack reconstructs the private target image.
5.2 Empirical Results and Performance Analysis
Reconstruction Performance.
Table 2 shows the performance of the proposed approach.
We report the difference values (values in brackets) between the MSE loss of reconstruction attack based on SFD and the baseline ERA, and we also report the difference values between the SSIM of the reconstructed image generated from our method and from the baseline, both of which represent performance gains.
Averagely, the proposed method provides a great improvement in reconstruction results against all three defenses.
We owe this to SFD used in our method, which can capture sensitive spatial information for more aggressive attacks.
Note that the vanilla results show the reconstruction effect of using ERA when the target model is without defense.
Here, we just use vanilla results to show the best possible results for the reconstruction.
The observation is that our approach still provides a competitive performance.
Speciﬁcally, on the CelebA dataset, we improve on decreasing the MSE loss from 0.124 to 0.032 and increasing the SSIM from 0.370 to 0.618 compared to ERA when the split point is set to the 2nd conv block and against adversarial learning, Which brings a 74.2% performance improvement (in terms of MSE loss values) while being close to the vanilla SSIM results 0.785 .
For noise mask and dropout defense, we can achieve a 47.7% and 40.9% performance improvement, respectively.
Comparison for Different Split Points.
We perform both quantitative (Table 2) and qualitative analysis (Figure 3) for different split points of the target model.
The visual results highlight that
SFD achieves signiﬁcantly better reconstruction performance than ERA, and it works better at early layers than at later layers.
This is conﬁrmed by the quantitative results where SFD obtained MSE improvement of 0.092 ( 74.2% ) and SSIM improvement of 0.248 in the 2nd conv block, compared to 0.003 ( 1.6% ) and 0.015 in the avgpool.
It is consistent with the intuition that features are becoming more and more abstract during the forward propagation process of the neural network, so the quality of reconstructed images of both SFD and ERA deteriorates as the layer goes deeper.
See Appendix C for more examples and quantitative results.
Besides, we also observe a signiﬁcant deterioration in reconstructed image quality on deeper layers (especially the fully connected layer).
It is because the size of feature mapping on deeper layers is typically smaller than on shallower layers, and the fully connected layer even mixes up previous visual information, keeping only the semantic information.
So it is relatively harder for SFD to distill sensitive features from small obfuscated representations.
When selecting the split point in
[Table] [Table]
169+(0.027) 0.132+(0.035)
a Privacy-Aware collaborative inference system, we suggest that the split layer on the edge device should be set deep enough (e.g., at least one fully connected layer) to hide the information of sensitive input samples.
Effect of Defense Hyper-parameters.
Table 3 presents the reconstruction performance for varying defense hyper-parameters.
The results indicate that even if the hyper-parameters of adversarial learning, noise mask, and dropout defense are set to 5.0, 10.0, and 0.9 (These parameter settings have incurred considerable utility loss), SFD can achieve SSIM improvements of 0.181, 0.116 and 0.116 respectively.
This demonstrates that SFD can still effectively extract sensitive information from feature spaces that are protected by stronger defenses (more heavily obfuscated features).
In addition, we note that the attack performance of ERA decreases (lower SSIM) with increasing hyper-parameters, which will negatively affect the reconstruction results of SFD (more generic reconstructed images).
Thus, even against SFD attacks, larger defense hyper-parameters may also provide a stronger level of obfuscation in reconstruction, but as a trade-off, this will severely degrade the utility of the target model.
For more discussion about the comparison of defense methods, see
Appendix B.
Discussion
Relaxing Attacker Model Assumption.
To further relax the attacker’s knowledge of the target model architecture, and for simplicity, we use a shadow model with the same size intermediate feature representation as the target model, but with a different architecture.
Table 4 depicts the evaluation results.
In our experiments on CIFAR10, we use VGG-16 mentioned previously in Section 5.1 for the target model, and use VGG-16, VGG-13, and ResNet-18 for the shadow models respectively.
The results of the reconstruction of the intermediate features of different sizes show both worse and better
[Table]
performance, but small
the
Possible Defenses.
As shown in Table 2 and Figure 3, both quantitative and qualitative measurements of the reconstruction images become more generic as the split point becomes deeper.
Meanwhile, increasing the defense hyperparameters can help mitigate the attacks to a degree, as illustrated in Table 3.
Combining these two strategies may provide better privacy protection for inference data.
However, this may cause a heavy computational burden on the client devices and reduce model accuracy.
We have made it our future work to explore mechanisms to effectively defend against reconstruction attacks in collaborative inference systems.
Conclusion
In this work, we track the potential privacy leakage points of defense mechanisms against reconstruction attacks in collaborative inference systems.
We posit that extracting the latent image structure knowledge of sensitive features can help achieve better reconstruction.
Therefore, we propose a technique called SFD to bypass the existing defense mechanisms.
SFD is designed to purify feature representations that are obfuscated by the defense mechanisms, so as to better extract sensitive information.
We systematically measure the effectiveness of SFD against three state-of-the-art defenses.
Experimental results show that SFD achieves signiﬁcant performance gain over existing reconstruction techniques on multiple datasets, which demonstrates that current defense schemes are not robust enough.
We hope our ﬁndings encourage critical exploration of defense mechanisms against reconstruction attacks for private collaborative inference