Private Split Inference of Deep Networks
Authors: Mohammad Samragh
Hossein Hosseini
Kambiz Azarian
Joseph Soriaga
Venue:  (2021)
Keywords: ML privacy
split inference

Abstract: Splitting network computations between the edge device and the cloud server is a promising approach for enabling low edge-compute and private inference of neural networks. Current methods for providing the privacy train the model to minimize information leakage for a given set of private attributes. In practice, however, the test queries might contain private attributes that are not foreseen during training. 
We propose an alternative solution, in which, instead of obfuscating the information corresponding to a set of attributes, the edge device discards the information irrelevant to the main task. To this end, the edge device runs the model up to a split layer determined based on its computational capacity and then removes the activation content that is in the null space of the next layer of the model before sending it to the server. It can further remove the low-energy components of the remaining signal to improve the privacy at the cost of reducing the accuracy. The experimental results show that our methods provide privacy while maintaining the accuracy and introducing only a small computational overhead. 

Under review as a conference paper at ICLR 2021
PRIVATE SPLIT INFERENCE OF DEEP NETWORKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Splitting network computations between the edge device and the cloud server is promising approach for enabling low edge-compute and private inference of neural networks. Current methods for providing the privacy train the model minimize information leakage for a given set of private attributes. In practice, however, the test queries might contain private attributes that are not foreseen during training. We propose an alternative solution, in which, instead of obfuscating the information corresponding
set of attributes, the edge device discards the information irrelevant to the main task. To this end, the edge device runs the model up to a split layer determined based on its computational capacity and then removes the activation content that is in the null space of the next layer of the model before sending it to the server. can further remove the low-energy components of the remaining signal to improve the privacy the cost of reducing the accuracy. The experimental results show that our methods provide privacy while maintaining the accuracy and introducing only small computational overhead.
INTRODUCTION
The surge in cloud computing and machine learning in recent years has led to the emergence of Machine Learning as a Service (MLaaS), where the compute capacity of the cloud is used to analyze the data that lives on edge devices. One shortcoming of the MLaaS framework is the leakage of the clients' private data to the cloud server. To address this problem, several cryptography- based solutions have been proposed which provide provable security at the cost of increasing the communication cost and delay of remote inference by orders of magnitude (Juvekar et al. (2018); Riazi et all 2)19)). The hyyptography-baser solutions are applicable use-cases such as healthcare where a few minutes of delay is tolerable, but not in scenarios where millions of clients request fast and low-cost responses such as in Amazon Alexa or Apple Siri applications. A light-weight alternative to cryptographic solutions is to manually hide private information the edge device; For instance, sensitive information in an image can be blurred on the edge device before sending it to the service provider (Vishwamitra et al. (2017)) This approach, however, is task-specific and may not be viable for generic applications.
The objective of split inference framework, shown in Figure is to provide generic and computa- tionally efficient data obfuscation scheme (Kang et al.] (2017): : Chi et al.) (2018)). The service provider trains the model and splits into two sub-models, M, and M2. where Mi contains the first few layers of the model and M2 contains the rest. The client runs M1 on the edge device and sends the resulting feature vector To preserve the privacy, the client desires 2 to only contain information related to the underlying task. For instance, when sending facial features for cell- cell-phone authentication, the client does not want to disclose other information such as their mood. As seen in Figure [ the privacy leakage is quantified by an adversary that trains the model M3 to extract private label pri from feature vector z.
M, (Ix) the server, which computes the public label as pub M2(z).
Current methods of private split inference aim to censor the information corresponding to list of known private attributes. For example, Feettyy et al. (2018) 8) utilize adversarial training to minimize the accuracy of M3 on the private attribute, and Osia et al. (2018) minimize the mutual information between the query and the private label y at training time. The set of private attributes, however, vary from query to another. Hence, it is not feasible to foresee all types of attributes that could be considered private for a specific MLaaS application. Moreover, the need to annotate inputs with all possible private attributes significantly increases the cost of model training.

---

Under review as a conference paper at ICLR 2021
Server
x
Z
M
jobub
Figure 1: Split inference setup. Client M, locally and sends the features 2 = (z) to server. The server pre- diets the intended attribute youb M2(x). (z). An adversary trains separate model M3 to predict private attribute as yor Ma(z).
M
M3
yori
Client
Adversary
In this paper, we propose alternative solution where, instead of censoring the information that is utilized to predict known private attributes, we discard the information that not used by the main model for predicting the public attribute. Our contributions summarized in the following.
We characterize the information that is not relevant to the prediction of the public attribute as part of the content of the feature vector 2 that will be removed by the server-side model. We then define the null content of the feature vector, 2N, the content in that is in the null-space of the following linear layer. The remaining content is called signal content and is denoted by zs. We have M2(=) M2(2s + =x) M2(=s).
We propose to remove ZN from features, z, and show that it reduces the accuracy of the adversary (Mg), while maintaining the accuracy of the main model (M2). To further discard the private information in 2, we propose to remove the low-energy components of zs, through which we achieve higher privacy (lower accuracy of M3) the cost of small reduction in utility (lower accuracy of M2).
We show our methods provide tradeoffs between edge-computation efficiency, privacy, and accuracy. Specifically, with higher edge computation (more layers on the edge device), the client achieves better privacy the same accuracy. Also, with the same edge computation (a given split layer), removing more components from the signal content provides better privacy at the cost of reduced accuracy.
We perform extensive experiments several datasets and show that methods provide better tradeoffs between accuracy and privacy compared to existing approaches such as adversarial training, despite having knowledge of the private attribute training inference times.
BACKGROUND AND RELATED WORK
We consider the supervised learning setting of Figure where the model M2 M2 is trained with set of examples N and their corresponding public labels {y. 1". V.V Ji=1. At inference phase, the client runs Mj their data and sends the intermediate feature vector = M1 (z) to the server. The goal of private inference is to ensure that z does not contain information about private attributes.
2.1 MEASURING PRIVACY
Several methods have been proposed to measure the privacy leakage of the feature vector. One approach is computing the mutual information between the query and the feature vector (Kraskov) al. (2004). In practice, measuring the mutual information not tractable for high-dimensional random variables, unless certain assumptions are made about the probability distribution of the random variables of interest. more practical approach measures privacy based on the reconstruction error, || - z||, where ž is estimated based on (Mahendran & Vedaldi (2015)). Finally, attribute privacy is defined based on the accuracy of an adversary model that takes z as input and predicts the private label.
In this paper, we use the attribute privacy notion as it applies to a wide range of applications. Assume each example ( has one or multiple private labels (v."" pri L. The adversary trains a separate model M3 with (zi, yrrr) where aftermath process to evaluate the privacy of the model M2 o M1. The split learning framework should achieve high utility, i.e., the server should be able infer the public attribute from accurately, while providing privacy, i.e., should not contain information about pri We refer to the accuracy of M2 Mj on pub public accuracy and the accuracy of M3 Mi pri as private accuracy.
M1 (z:)), as shown in Figure Note that M3 is used as
2

---

Under review as a conference paper at ICLR 2021
2.2 THREAT MODEL
Honest-but-curios server. The server performs the inference of the public attribute but will potentially try to extract private information from the features, 2, as well.
Client capabilities. Upon providing the service, the server also provides profile of the utility (aceuracy on the public attributes), privacy (aceuracy on several private attributes), and computation of the edge device. The client then decides on the best tradeoff based the computational resources and also the desired level of privacy. Such mechanisms are already in use in ML-on-the-edge applications. For example, in the application of unlocking the phone by face recognition, the client can specify the required precision in face recognition, where a lower precision will provide higher utility at the cost of lower security (Chokkattu 2019).
2.3 RELATED WORK
Prior work has shown that the representations learned by deep neural networks can be used to extract private information (Song & Shmatikov (2019) or even reconstruct the raw data (Mahendran & Vedaldi (2015)). Current methods for private inference can be categorized as follows.
Cryptography-bast -based solutions. Since the server is not trusted, solutions based on public key encryption (Al-Riyami & Paterson 2 are not applicable. We consider scenarios where the server provides service to millions of users (e.g., in cases of Amazon Alexa or Apple Siri), and users expect low communication and fast response. Therefore, classic two-party cryptographic solutions for secure function evaluation (Juvekar et al. (018); Riazi et al. (2019) are also not applicable to our scenario.
Noise Injection. A line of work suggests obfuscating private attributes by adding noise to the features, i.e., instead of z, the client sends = + to the server, with the noise designed to maintain public accuracy while reducing private accuracy (Mireshghallah al. (2020)). While noise addition improves privacy, it has been shown to reduce the public accuracy sigmificantly (Liu et al. (2019)).
Information Bottleneck. The notion of mutual information can be used to train private models. Let (a,, b) denote the mutual information between random variables a and b. The idea is to train M1 that maximizes [22, yub while minimizing I [(z, yri (Osia et al. 2018); Moyer et al. (018)). The optimization is formulated as follows:
max Ad
E, Exyy yoo [[(M(()) (x). pub) (M( (+))
(1)
The use of mutual information for privacy, however, has been challenged by practical attacks that extract secret information even when [(2, prr) is small (Song & Shmatikov (2019)).
Adversarial Training. This defense solves the following min-max optimization problem:
max min Egg you (y M3 a MI(r)) L(yPub, M2 M1 (x))).
(2)
My My
where L denotes the cross-entropy loss and is scalar. The above objective can be achieved through adversarial training (Edwards & Storkey (2016); HHmm 2017); Xie et al. (2017): Li et al.) (2018);
Feutry al. (2018); Li al. (2019)) At convergence, the trained M, generates z such that M2 is not an accurate estimation of y while M2(2) accurately describes puu
Existing methods for private split inference have several limitations. First, the underlying assumption in above learning-based defenses is that a set of private attributes along with the public label are provided at training time. In practice, however, might not be feasible to foresee and identify all possible private attributes and annotate the training data accordingly. It also contradicts deployment at-scale since whenever new private attribute is identified, the model M1 needs to be retrained and re-distributed to all edge devices that use the service. Second, current approaches for private inference often provide a poor tradeoff between accuracy and privacy. Moreover, the tradeoff of accuracy and privacy with the client-side computation is not well studied in the split learning framework. In this paper, we characterize this tradeoff and propose an alternative approach that, instead of obfuscating the information related the private attributes, the edge device removes the feature content that is irrelevant to the public task. We empirically show our method successfully reduces the accuracy on private attributes at small no cost to public accuracy.
3

---

Under review as a conference paper at ICLR 2021
3 PROPOSED METHODS
3.1 SIGNAL AND NULL CONTENTS OF FEATURE VECTOR
Let = _ be feature vector and W E |m*= be matrix. The operation of fully-connected and convolutionals layers can be expressed matrix-vector and matrix-matrry multiplication. Herein, we let represent the vector in fully-connected layer and a column of the second matrix in convolution layer. Let the singular value decomposition (SVD) of W be W = U S -.V. V. Since the rows of V form an orthonormal basis, we can write the feature vector as
2
T
0. - U
(3)
where U is the i-th row of V and the < > operator denotes inner- product.
Definition 1. The signal content of with respect matrix W, or simply the signal content of 2, denoted by nạ is defined as
28 = arg min |1ll2, s.t,, W. (2 - h) = 0.
(4)
The null content of is also defined as 2N 2 28.
Lemma 1. We have
SS
0
T and AN
a;,y T
(5)
im++1
Proof. We write as the composition orthonormal vectors v, 's as = -i B.u7. We have
n
W(2 h)
Σ aa. W. Σ (a - Vuf
(6)
QER
Since the rows of V are orthonormal, then qi Vel" is one-hot vector with its i-th element equal to 1. By substituting in 6 we obtain
m Σ (a; B.)US
m Σ
W( h) Σ (a; B.UUS
s: B.)UU
where Spai and Up.4] the i-th columns of S and U, respectively. Note that, since S is diagonal matrix, we have S|..1 0. Vi E {m + 1,... n). thus reducing the summation from to components. Also, for i E (1, m), Sk... is column vector with only one non-zero element, 8,. the i-th dimension.
As a result, to obtain W(: h) 0. we must have B, ag Vi E {1, ..). Since v; 's orthonormal, have |/hl|2 VE H. Hence, to minimize |1A]|2, we set 8; 0, Vi e {m + 1.............. .... Therefore, 2s Em a.y. The null content zx can be then computed as 2N zs Ei=m+1 awv""
Definition 2. The normalized signal and null contents of z are defined as Cs(z)
|= and
||
Cw(2)=
respectively. We have Os(z) + Cx(z) Sm | 1
2n Limmt1 E
= 1.
3.2 DEFENSE 1: OBFUSCATING NULL CONTENT OF FEATURE VECTOR
We propose remove all the content in feature vector that is irrelevant to the public attribute. To do so, given a feature vector 2, we find a minimum-nome norm vector z' that generates the same prediction for the public attribute as z, i.e., M²(=') M2(z). Formally,
arg min |//|22, s.t. M2(h) M2(2).
(8)
4

---

Under review as a conference paper at ICLR 2021
Removing null content
Removing part of signal content
Training:
Training:
Server trains M2 o M₁ with (2:, "f ), e [N]
Server trains Md M1 using (f
) ", [N]
Adversary computes 2; (1; [N]
Server augments M₁ - M1
Adversary computes 2; zas. E [N]
Server freezes M, and fine-tunes M2 M
Adversary trains M3 with (2es, u. ), 2 [N]
Adversary computes E; = Mi (2:), E [N]
Inference:
Adversary trains Mg with (2, ". [M]
Client computes M1(x))
Client computes z Zo
Inference:
Client sends to to server
Client computes (z)
Server computes grub M2(Fs)
Client sends 5 to server
Adversary computes
25
Server computes grub M2 Adversary computes gird Mz
Adversary computes jpm = Ms(2s)
(left): Training and inference for defense (client removes null content of =). (right): Training and inference for defense (client removes low-energy content of z). zo the feature vector in which the null content obfuscated. Mi constructed by augmenting Mi with the module that removes the signal content.
Figure 2:
Due to the complex (nonlinear) nature of deep networks, finding such a vector would require doing multiple backpropagations
M2 for each given z. This is, however, not feasible for resource- constrained edge devices. To address this problem, we relax (8) such that the constraint holds for the first layer of M2 (server-side model). i... we modify the constraint to W Wa, where W is the weight matrix of the first layer of M2. As discussed in Section 3.I the solution to this relaxed optimization problem is zs, the signal content of z. Removing or obfuscating the null content of 2 does not change the model prediction on public attribute. It, however, might harm the private accuracy since part of the null content of zn might fall into the signal content of the first linear layer of M3. The method is described in Figure (left).
At inference time, to obfuscate AN; the client constructs 2, using either of the following methods.
Client constructs f -i=++ noy with coefficients, Tii chosen random, and sends 2, z + μ to the server. The adversary can recover 2s = Vi.m Vim 2o but cannot recover 2N- Client computes the signal content of and sends 2, zş Die a.d? the server.
For the first case, since μ is independent of the client can compute it offline, e.g., when the edge device is plugged in and not in use, and store for later use. The second approach does not require storage on the edge device, but extra computation, equal to the complexity of computing the first layer of M2, has to done during inference to extract zş. We next propose a method that reduces the extra cost to only a fraction of the computation cost of the first layer of M2.
3.3 DEFENSE 2: DISCARDING LOW-ENERGY SIGNAL CONTENT OF FEATURE VECTOR
In the first defense method, we proposed to discard the content of the feature vector that will be removed by the first layer of M2. The following layers of M2 will further remove more content from feature vector. Hence, we can potentially discard more content from and still get similar prediction as the original feature vector. For a linear layer, following the same process in Section 3.1 is computed as:
the output
w W 28 = Σ aU S. qi Σ m
au Sk.1 = Σ stayUp
(9)
where is the i-th eigenvalue in S, O; is defined in and Up..) denotes the i-th column of U. From (9) we observe that components with larger 5;0; are contributing more to the layer output since ||U,,,1|2 1 for all columns of U. As such, we approximate z + by only keeping m' < m components of the right-hand- filtering, removing signal content will affect the public accuracy it changes the main network output, but can further reduce the private accuracy. To improve public accuracy when removing signal content of features, the server fine-tunes M2 ž. The method is descrihed in Figure
of 9 that have the largest coefficients. Unlike null content
5

---

Under review as a conference paper at ICLR 2021
Since s; and (...] are fixed at the inference time, the client only needs to send the selected a, values along with their indices to the server; the server knows s; and U, and can reconstruct accordingly, The cnge-computation cost of this process is m' /m times the computation cost of the first layer of the server model. We do experiments in settings where m' /m is about 1%. Hence, the computation cost of our method is only small fraction of computing single layer of the network. Moreover, since m' < m, our method also incurs much smaller communication cost compared to sending the entire feature vector the server.
4 EXPERIMENTS
4.1 SETUP
Datasets. We perform our experiments on four visual datasets listed below.
EMNIST (Cohen et al. (2017) is an extended version of the MNIST dataset where the labels are augmented with writer IDs. We selected 13000 samples from EMNIST written by 100 writers with 130 examples per writer. We then split this dataset into 10000, 1500, and 1500 training. validation, and test sets. We use the digit label and writer IDD the public and private attributes, respectively.
FaceScrub (Ng & Winkler (2014); FaceScrub (2020) is a dataset of celebrity faces labeled with gender and identity. We gender and identity as the public and private attributes, respectively, In experiments, we cropped the face region using the bounding boxes in the image annotations
and resized images 50 x 50.
dataset of face images labeled with gender and race, which we use as the public and private attributes, respectively. We cropped the face region using the bounding boxes the image annotations and resized images to 50 50.
UTKFace (Zhang et al. (017))
CelebA (Liu et al. (2155) a dataset of celebrity images. Each image is labeled with 40 binary attributes. Out of these, we select "Smmllin"" as the public attribute and (Male, Makeupu Wearing Lipstick, Attractive] as private attributes. These attributes have near balanced distribution of positive and negative examples. In experiments, we cropped the face region using the bounding boxes in the image annotations and resized images to 73 60.
Mouth Saattlllllghhyppppp
Model architecture. We present the experimental results on model used in prior work (Song & Shmatikov ).19)). The model architecture and baseline test accuracy results are summarized in and
Table and Table in Appendix.
Adversary capabilities. We use the same architecture for adversary's model M3 as the server model M2. The model M3 is trained using the features extracted by M, and the associated private labels. We also assume that the adversary knows the parameters of M, and M2.
Training settings. We use Adam optimizer with an initial learning rate of 0.001 and drop the learning rate by a factor of 10 after 20 and 40 epochs. All models including the adversary and main models trained for 50 epochs unless stated otherwise.
4.2 EVALUATIONS
We start our analysis by computing the null and signal contents in every layer of M M2 o Mi. Figure 3 (left) shows the content of the input remained at each layer; for the i-th layer, this content is computed as IT; Cs(2g) where 2, denotes the activation vector at the j-th layer and Cs(-) is defined in Section 3.1 As the feature vector propagates through network layers, more content is gradually removed from z until the model outputs the prediction on the public task. We also split the network at different layers and measure the private accuracy of M, trained with the feature vector. As seen in Figure 3 (right), the private accuracy also decreases as we get closer to the output layer, indicating that the discarded content contained information relevant the private attribute.
To reduce the privacy leakage, we proposed to filter out the null content of the feature vector. Figure shows the private accuracy for different split layers. Removing the null content reduces the private

---

Under review as a conference paper at ICLR 2021
JMNN (dkga)
Faceberub (dentey))
LMMM5 (aithor)
racetscuut ticentity)
/Ticaace (aender)
CabibA (smilnng)
UITEace (race)
(webb (gender)
Signal Content
100
Private Accuracy
0.75
75
50
0.25
25
1
2
cowV 3
1
3r3
COWV 1
CONV2 2
FC1 3ON
FC2
Figure 3: (left) Remaining signal content at the output of each layer of the main model, (right) private accuracy for different split layers. The content of the input significantly decreases in deeper layer. Similarly, the private accuracy decreases, indicating that the discarded content contained information relevant the private attribute.
Baseline
Null Removed
EMNIST (author)
UTKFaaee (race)
FaceScrub (identity)
CelebA (gender)
Privete Accurosy (90
254
60
90
20
15
80
10
70
60
comy 1
coNV 2
cONN3 3
FC
2
CONN 1
COW COW 2
PC
PC2
FC3
cowN
CON2
cow
FC1
PC
FC-3
comV1
ceNN
CDWN
FC.1
PC2
PC3
3
1
3
Figure 4: The effect of removing the null content on the private accuracy with different split layers.
accuracy without affecting the public accuracy. Moreover, splitting the network in deeper layers improves the privacy. To further reduce the private accuracy, we discard the low-energy components of the signal content and only keep m' features. Figure 5 illlustrates the effect of m' on the public and private accuracy, when the network is split at the CONV-3 layer. As seen, by setting m' to small value, our method achieves high privacy (low private accuracy) at the cost of a small reduction in public accuracy. In general, the privacy can be controlled using two factors:
The split layer: As we go deeper in the network, i.c., when the edge device performs more computation, better tradcoffs can be achieved. To show this effect, we perform signal-content removal at different layers, with m' set such that the public accuracy is reduced by 1%. The corresponding private accuracy is shown in Figure 6
Number of signal components sent to the server: For the same edge-computation (a given split layer), the number of preserved features (m'' can be tuned as achieve desired tradeoff between utility (higher public accuracy) and privacy (lower private accuracy). Figure] shows the results for the setting that the network is split at the input of the CONV-3 layer.
Comparison Pruning. Similar to our approach, prun- ing network layers can eliminate features that do not con- tribute to the public attributed. In the following, we com- pare our method with pruning in terms of public and pri- vate accuracy. We split the network from the middle layer, ie., the input of the FC-1 layer. For keep the top m' components of = from its signal content and filter out the rest. For pruning, we keep mn' elements in
EMNIST
FaceScruth
UTKFace
ColobA
100
Private Accuracy (%)
60
method, we
20
and set the rest to zero. We adopt the pruning algorithm proposed by (Li al. (2016) which works based on the L1 norm of the columns of the following layer's 's weight matrix. After pruning, we fine-tune M2 to improve the public accuracy. Figure / shows the public and private accuracy for each dataset. As seen, with small m'' both our method and pruning achieve a low private accuracy. Pruning, however, significantly reduces the public accu- racy as well. For example, for the UTKFacee dataset, with m' = 1. both methods result in a private accuracy close to the random guess. However, pruning reduces the public accuracy from 92.25% to 53% (also close to random guess), whereas our method keeps the public accuracy 88.63%.
CONV 1 CONV CONV 3 FC Layer
FC
FC3
Figure 6: Privacy accuracy versus split layer. The number of preserved components of sig- nal content is set such that the public accu- racy is reduced by 1%.
7

---

Under review as a conference paper at ICLR 2021
Public
Private
EMNIST
UTKFace
FaceScrub
CelebA
100
Accuracy (%6
16
16
Preserved Components (ml)
Preserved Components (m')
Preserved Components im')
Preserved Components (m')
Figure 5: The effect of the number of preserved features (ma') on the public and private accuracy when the network is split at the input of CONV-3 layer.
our methed + pruning basellll accuracy -eairaccy----aaai random guess
EMNIST
UTKFace
FaceScrub
CelebA
100
Public Accuracy
60
40
20
100
Private Accuracy
80
60
40
20
veed Components (ml) Preserved Components (ml) Preserved Components (m') Preserved Components im')
Figure 7: Comparison between our method and feature pruning. Both methods reduce the private accuracy.
Pruning, however, significantly reduces the public accuracy.
Comparison with adversarial training. We implement adversarial training framework proposed by (Feutry al. (2018)) and present the utility- privaay tradeoff in Figure g achieve the best tradeoff for adversarial training, train the models in multiple settings with different parameters (Eq, 2} in range of [0.1, 1]. Note that, unlike our method, adversarial training assumes that the private attribute is known at training time. Despite this, Figure shows that our method achieves better utility-accuracy tradeoff than adversarial training.
We also do experiments for the case with multiple (unseen) private labels. Specifically, consider the CelebA model trained to detect "smiling" and evaluate two methods, 1) our method: we keep only m component from the signal content of feature vector and then train adversary model private attribute, and 2) adversarial training: we first adversarially train an M1 model to obfuscate "gender," and then train one model for each private attribute.
For both of the above methods, the network is split at the input of the FC-1 layer. Figure 9 shows the results. Our method outperforms adversarial training method on both public and private accuracy. In our method, the accuracy on all private attributes are significantly lower than the baseline private accuracy. The only exceptions are "high cheekbones" and "mouth open" attributes, which have correlations with public attribute, that is, smiling person likely has high cheekbones and their mouth open. The correlation between public and private attributes causes the signal content of server and adversary's models to have large overlaps and, hence, results in high private accuracy. The adversarially trained model successfully hides the information that it has been trained to obfuscate (the "gender" attribute). Such model, however, fails to remove information of other attributes such as "makeup" or "lipstick". This highlights the applicability of our method practical setting as generic obfuscator compared to specialized techniques such adversarial training.
Ablation study on CONV and FC layers. We compare the performance our method on CONV and FC layers. To do so, train networks the UTKEaace task, (1) network with 10 CONV
8

---

Under review as a conference paper at ICLR 2021
- our method - acversaria training
EMNIST
UTKFace
FaceScrub
CelebA
Public Accuracy (%)
100
95
90
00
75
45
50
55
60
45
50
55
45
50
55
80
Private Accuracy (%))
Private Accuracy (96
Privaze Accuracy (%)
Accuracy (%)
Figure 8: Comparison between our method and adversarial training. For given privacy level, our method
provides higher utility (higher public accuracy) compared to adversarial training method.
baseline
our method
aavvrrsriill traiiinng]
14.8
Accuracy (%)
60
$4.00
00
40
20
Smilling
Figure 9: Accuracy public ("smiling"" and private attributes. Our method obfuscates the feature vector without knowledge of private attribute at training or inference times. Adversarial training method maximizes the accuracy on "smiling." while minimizing the accuracy on "gender." As can be seen, our method reduces accuracy on all private attributes. The adversarially trained model successfully reduces accuracy on "gender" attribute, but fails to remove information of other attributes. This highlights the applicability of our method in practical settings as generic obfuscator compared specialized techniques such adversarial training.
layers each with 16 output channels, and (2) a network with 10 FC layers each with 2304 neurons. Both networks have an extra FC layer at the end for classification. The number of channels/neurons are chosen such that the total number of output features at each layer is the same for the two networks. The public accuracy of the 10-CONV and 10-FC networks 89. 99.26% and 89.07%, respectively. Figure I0 shows the public and private accuracy when we remove low-energy components of the signal space at different layers. The number of preserved features, m'. each layer chosen such that the public accuracy is maintained. As seen, the 10-FC network achieves lower private accuracy compared to the 10-CONV network. The reason that CONV layers are known to be generic feature extractors, while FC layers are more specialized toward the public attribute.
CONCLUSION
10-FC (Pubicc) 10-CONV (Public)
10-FC (Private)
10-CONV (Private)
100
We proposed private inference framework, in which edge devices run several layers locally and obfuscate the intermediate feature vector before sending it
90
80
Accuracy
the server to execute the rest of the model. For ob- fuscation, we proposed remove information that is not relevant to the main task or does not signifi- cantly change the predictions. Specifically, we de- veloped two methods of removing the content of the feature vector the null space of the following linear layer and also removing the low-energy content of the remaining signal. We showed that, unlike existing methods, methods improve privacy without re- quiring the knowledge of private attributes at training or inference times.
70
60
50
40
1
2
3
4
5
6
7
8
9
10
Layer
Figure 10: Comparing performance our method on the 10-FC and 10-CONV networks. The number of preserved features at cach layer is set such that the public accuracy is maintained.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Sattam S Al-Riyami and Kenneth G Paterson. Certificateless public key cryptography. In International conference on the theory and application of cryptology and information security, 2003.
Jianfeng Chi, Emmanuel Owusu, Xuwang Yin, Tong Yu, William Chan, Patrick Tague, and Yuan Tian. Privacy partitioning: Protecting user data during the deep learning inference phase. arXiv preprint 18/2 2018.
Julian Chokkattu. How to make face unlock more secure in the Samsung
Galaxy S10 line, 2019.
https://wwws wwtp:/wwwwiitaaaeenn.dsss www.cooiim/meeeecooooooooo.....e
how to make face un Sanlock secure gwlaxy s10 s10
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. EMNIST: Extending MNIST to handwritten letters. In International Joint Conference on Neural Networks, 2017.
Harrison Edwards and Amos Storkey, Censoring representations with an adversary. International Conference on Leurning Representations, 2016.
FaceScrub. The FaceScrub dataset, 2020.
puthesis (accessed July 3, 2020).
Clément Feutry, Pablo Piantanida, Yoshua Bengio, and Pierre Duhamel. Learning anonymized representations with adversarial neural networks. arXiv preprint urXiv:1802.00386, 2018.
Jihun Hamm. Minimax filter: learning to preserve privacy from inference attacks. The Journal of Machine Learning Research, 2017.
Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. GAZELLE: A low latency framework for secure neural network inference. USENIX Security Symposium, 2018.
Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Computer Architecture News, 2017.
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information.
Physical review E, 2004.
Ang Li, Jiayi Guo, Huanrui Yang, and Yiran Chen. Deepobfuscator: Adversarial training framework for privacy-preserving image classification. Advances Neural Information Processing Systems Workshops, 2019.
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. In International Conference Learning Representations, 2016.
Yitong Li, Timothy Baldwin, and Trevor Cohn. Towards robust and privacy-preserving text represen- tations. In Annual Meeting the Association for Computational Linguistics, 2018.
Sicong Liu, Anshumali Shrivastava, Junzhao Du, and Lin Zhong. Better accuracy with quan- tified privacy: representations learned via reconstructive adversarial network. arXiv preprint 101.70090, 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In International Conference on Computer Vision, 2015.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Conference Computer Vision and Pattern Recognition, 2015.
Fatemehsadat Mireshghallah, Mohammadkazem Taram, Prakash Ramrakhyani, Ali Jalali, Dean Tullsen, and Hadi Esmaeilzadeh. Shredder: Learning noise distributions protect inference privacy. In International Conference on Architectural Support for Progranming Languages and Operating Systems, 2020.
10

---

Under review as a conference paper at ICLR 2021
Daniel Moyer, Shuyang Gao, Rob Brekelmans, Aram Galstyan, and Greg Ver Steeg. Invariant representations without adversarial training. Advances in Neural Information Processing Systems, 2018.
Hong-WWei Ng and Stefan Winkler. A data-driven approach cleaning large face datasets. In IEEE international conference on image processing, 2014.
Seyed Ali Osia, Ali Taheri, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Hamid R Rabiee. Deep private-feature extraction. In JEEE Transactions on Knowledge and Data Engineering, 2018.
M Sadegh Riazi, Mohammad Samragh, Hao Chen, Kim Laine, Kristin Lauter, and Farinaz Koushanfar. XONN: Xnor-based oblivious deep neural network inference. In USENIX Security Symposium, 2019.
Congzheng Song and Vitaly Shmatikov. Overlearning reveals sensitive attributes. In International Conference on Learning Representations, 2019.
Nishant Vishwamitra, Bart Knijnenburg, Hongxin Hu, Yifang P Kelly Caine, et al. Blur vs. block: Investigating the effectiveness of eewacc-enhancing obfuscation for images. In Conference on Computer Vision and Pattern Recognition Workshops, 2017.
Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. Controllable invariance through adversarial feature learning. In Advances in Neural Information Processing Systems, 2017.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progressionnregressoon by conditional adversarial autoencoder. Conference on Computer Vision and Pattern Recognition, 2017.
A
APPENDIX
Table I: Network Architecture. Each row shows split layer, e.g., layer 1, the raw data and, layer 6, the input the last fully-connected layer sent the server.
Split Layer
CONV (3, 10)
ReLU
Max-Pooling 2)
Batch- Maaic--Nommalzzation
CONV(3, 32)
RcLU
Max-Pooling (2 x 2)
CONV (3, 64)
ReLU
Max Pooling (2 x 2)
Batch Batch-Normalize
Normalization
4
FC((18)
ReLU
Butch-Noormaliza
5
FC(64)
ReLU
Batch-Nee Naich-Normalization
FC(n classes)
Softmax
Table 2: Model accuracy for public and private attributes of different datasets.
Dataset
MNITT
UTKFace
FaceScrub
CelobA 2
number of classes
10
2
public attrihute
digit
gender 90.25
gender
smiling 92.25
public accuracy (%)
98 60
97.90
number classes
100
5
530
private attribute
writer
identity 65.52
gender 97.53
makeup 90.00
cheekbones 86,29
mouth-open 92.94
lipstick 93.40
attractive 80.46
private accuracy (%)
26.93
79.18
11


Reviews:
[{"review_id": "Qp4pild5SB4", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes model splitting as a method to perform private inference from an edge device to a cloud provider. The idea being that an edge device owns layers 0 -> i of a model, M, and the cloud device owns layers i+1 -> N. Then, when an edge device wants to perform inference on a private input, x, they send M_i(x) instead of x to the cloud device, who then completes the prediction task. The claim is that if M_i(x) is learned representation that only retains features that are predictive of the public label, there should be no private information leaked about the input to an eavesdropping adversary who can also observe M_i(x). The authors achieve this by finding representations that are approximately equivalent in the following layer (i+1), but have much smaller norms than the original feature vectors. At inference time, the client either directly constructs the signal content part of the feature or randomises over the nuisance components. Experiments on four datasets point to the effectiveness of this idea, and that simple baselines such as adversarial training or pruning perform worse. I thought the core experiments were sound, but I am not totally convinced by the motivation of the approach.\n\n1. I don't understand why cryptographic solutions are not viable for this kind of problem? If I understand correctly, the server is trusted, the only assumed vulnerable step is the transmission of data between the edge device and cloud provider. If this is correct, then would a simple PKE scheme really be prohibitively expensive? Even a homomorphic set-up seems somewhat tolerable (depending on your threat model) given the experiment datasets used in this work. For example, message sizes of CryptoNet for EMNIST size data is ~91KB [1]. \n\n\n2. Following on from (1), I found it hard to reason about trade-offs between public accuracy, private accuracy, computational expense of the defense, bandwidth overheads etc. For example, if I sacrifice 1% of public accuracy for 5% private accuracy, is that a good trade-off? Who makes that decision? The edge device or the cloud provider? The paper would have been much improved if these kinds of questions were captured by a formal threat model.\n\n\n3. Most the experiments concentrated on public / privacy accuracy trade-offs on problems with a balanced distributions over private attributes. How does this scheme cope with more realistic problems where the distributions are highly skewed?\n\n\n4. How weak is the approximation to eq. (8)? I couldn't find any ablation studies to justify that this approximation is reasonable. Additionally, what are the computational overheads of extracting z_S?\n\n\n5. In Figure 4, why is there a large drop at the final Conv layer?\n\n\nMinor: '..do not contribute to the public attributed.\" -> \"..do not contribute to the public accuracy.\"?\n\n\n[1] Gilad-Bachrach, Ran, et al. \"Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy.\" International Conference on Machine Learning. 2016.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "DfVvXj7CZtn", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tackles a timely problem of privacy leakage on the edge devices when applying deep neural networks. Instead of mitigating the leakage of a set of private attributes, the proposed method tries to remove the information irrelevant to the primary task. The proposed method does not need to identify the private attributes. The main contribution of this paper is the two proposed approaches for removing “null content” and “signal content.” The evaluations of the proposed approach are conducted on four image datasets.\n\nPros:\n1. The idea of removing irrelevant information instead of private attributes is an interesting idea.\n2. The paper is well organized and well written.\n3. The experimental evaluation is comprehensive. Feature pruning and adversarial training are included in the evaluation.\n\nCons:\n1. The key concern about the paper is the feasibility of the proposed methods in deep neural networks. Both proposed feature-removing methods are derived from a single linear layer. However, in many cases and even shown in the evaluation, the device side may process more than one layer of neural networks. In addition, the convolution layer is often deployed as the first layer in neural networks. It would be great if the proposed methods can be extended to multiple layers and multiple types of neural networks.\n2. The adversary uses the same architecture in the paper. However, the adversary can choose to use a more complex model to extract the privacy attributes in the evaluation. The different architecture may cause the failure of the proposed methods. It would be nice if more adversarial models can be evaluated in the paper.\n3. In Figure 4, the proposed methods do not perform well in balancing the utilities and privacy achieved. It is hard to tell if the better tradeoffs are due to the deeper layers or fully connected layers. From Figure 4, it seems the proposed methods do not perform well on the convolutional layers.\n4. The experiments only evaluate on a six-layer neural network, which is not a “deep network” claimed in the title. It would be great if the paper can evaluate the performance on other architectures and deeper models.\n5. The algorithms in Figure 2 are hard to understand.\n\n\nMinor comments:\n1. In Equation 2 it should be “M2 * M1” instead of “M1 * M2”\n2. Page 6 Figure 4 shows that the information leakage can be controlled using the following factors “factors”\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "9auAYixxgx2", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is well organized with sufficient background discussion and the related works. The annotation and methodology introduction is clear and mostly without error. This paper lies in an interesting setting of client-server, by sending the shared representation z while decoding the public and private feature at the server side. \n\nThe private information is not necessarily to be orthogonal to the public information. Thus, by the proposed method, without the semantic labels of the private attribute, it is not clear why the orthogonal to public feature would necessarily to be private feature. Yet, public feature would contain private feature, and private feature would contain public feature.\n\nFrom Figure 1, the authors actually claims to utilize adversary training to predict the secrete attribute, which mostly will utilize the private labels. While in the conclusion, the authors mentioned they do not require the knowledge of private attributes, which is a contradictory. Meanwhile, there is no discussion relating to M3 in Section 3 “proposed method”.\n\nThe datasets in the experiment are mostly towards face related data. It would be more convincing with a more general purpose dataset, such as DomainNet, which is aiming at object categories from different resources, sketch, photos, arts, where public information could be the categories, and the private information is the resource type.\n\nFrom the experimental settings, those public information and private information, by the datasets’ setting, are naturally independent, i.e. gender and race in UTKFace, these two are naturally orthogonal. If the public and private information are with some correlation, would the proposed framework still work under this situation?\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: While reviewers believe that the motivation of the paper is strong and the idea is interesting the ultimate execution of the paper is not up to the standards of ICLR. I believe the biggest concern is the precise privacy guarantee of the method. As pointed out, it is an extremely strong assumption that the model structure of the adversary is known (or even approximately known). Standard privacy guarantees are either information theoretic or based in computational hardness. This work does not provide such guarantees. While there has been recent work on using adversarial learning to learn models that are robust to such adversaries, they have been heavily criticized within privacy and security communities due to the lack of such guarantees. I was not convinced by the authors response to such questions: there are plenty of cryptographic/privacy-preserving schemes that work in the honest-but-curious setting, and techniques that use the standard guarantee of differential privacy do not suffer from large slow downs.

Thus, I would urge the authors to modify this work so that it can leverage the guarantees of well-known cryptographic/privacy-preserving schemes. If done so, these arguments about privacy will go away and the paper will have a much better shot at acceptance.
Decision: Reject