Year: 2020

Abstract:
Splitting network computations between the edge device and the cloud server is a promising approach for enabling low edge-compute and private inference of neural networks. Current methods for providing the privacy train the model to minimize information leakage for a given set of private attributes. In practice, however, the test queries might contain private attributes that are not foreseen during training. We propose an alternative solution, in which, instead of obfuscating the information corresponding to a set of attributes, the edge device discards the information irrelevant to the main task. To this end, the edge device runs the model up to a split layer determined based on its computational capacity and then removes the activation content that is in the null space of the next layer of the model before sending it to the server. It can further remove the low-energy components of the remaining signal to improve the privacy at the cost of reducing the accuracy. The experimental results show that our methods provide privacy while maintaining the accuracy and introducing only a small computational overhead.

text: Splitting network computations between the edge device and the cloud server is a promising approach for enabling low edge-compute and private inference of neural networks. Current methods for providing the privacy train the model to minimize information leakage for a given set of private attributes. In practice, however, the test queries might contain private attributes that are not foreseen during training. We propose an alternative solution, in which, instead of obfuscating the information corresponding to a set of attributes, the edge device discards the information irrelevant to the main task. To this end, the edge device runs the model up to a split layer determined based on its computational capacity and then removes the activation content that is in the null space of the next layer of the model before sending it to the server. It can further remove the low-energy components of the remaining signal to improve the privacy at the cost of reducing the accuracy. The experimental results show that our methods provide privacy while maintaining the accuracy and introducing only a small computational overhead.

heading: 1 INTRODUCTION
text: The surge in cloud computing and machine learning in recent years has led to the emergence of Machine Learning as a Service (MLaaS), where the compute capacity of the cloud is used to analyze the data that lives on edge devices. One shortcoming of the MLaaS framework is the leakage of the clients’ private data to the cloud server. To address this problem, several cryptographybased solutions have been proposed which provide provable security at the cost of increasing the communication cost and delay of remote inference by orders of magnitude (Juvekar et al. (2018); Riazi et al. (2019)). The cryptography-based solutions are applicable in use-cases such as healthcare where a few minutes of delay is tolerable, but not in scenarios where millions of clients request fast and low-cost responses such as in Amazon Alexa or Apple Siri applications. A light-weight alternative to cryptographic solutions is to manually hide private information on the edge device; For instance, sensitive information in an image can be blurred on the edge device before sending it to the service provider (Vishwamitra et al. (2017)). This approach, however, is task-specific and may not be viable for generic applications.
The objective of split inference framework, shown in Figure 1, is to provide a generic and computationally efficient data obfuscation scheme (Kang et al. (2017); Chi et al. (2018)). The service provider trains the model and splits it into two sub-models, M1 and M2, where M1 contains the first few layers of the model and M2 contains the rest. The client runs M1 on the edge device and sends the resulting feature vector z =M1(x) to the server, which computes the public label as ypub =M2(z). To preserve the privacy, the client desires z to only contain information related to the underlying task. For instance, when sending facial features for cell-phone authentication, the client does not want to disclose other information such as their mood. As seen in Figure 1, the privacy leakage is quantified by an adversary that trains the model M3 to extract private label ypri from feature vector z.
Current methods of private split inference aim to censor the information corresponding to a list of known private attributes. For example, Feutry et al. (2018) utilize adversarial training to minimize the accuracy of M3 on the private attribute, and Osia et al. (2018) minimize the mutual information between the query z and the private label ypri at training time. The set of private attributes, however, can vary from one query to another. Hence, it is not feasible to foresee all types of attributes that could be considered private for a specific MLaaS application. Moreover, the need to annotate inputs with all possible private attributes significantly increases the cost of model training.
Figure 1: Split inference setup. Client runs M1 locally and sends the features z = M1(x) to the server. The server predicts the intended attribute as ypub = M2(z). An adversary trains a separate model M3 to predict the private attribute as ypri = M3(z).
In this paper, we propose an alternative solution where, instead of censoring the information that is utilized to predict known private attributes, we discard the information that is not used by the main model for predicting the public attribute. Our contributions are summarized in the following.
• We characterize the information that is not relevant to the prediction of the public attribute as part of the content of the feature vector z that will be removed by the server-side model. We then define the null content of the feature vector, zN , as the content in z that is in the null-space of the following linear layer. The remaining content is called signal content and is denoted by zS . We have M2(z) =M2(zS + zN ) =M2(zS).
• We propose to remove zN from features, z, and show that it reduces the accuracy of the adversary (M3), while maintaining the accuracy of the main model (M2). To further discard the private information in z, we propose to remove the low-energy components of zS , through which we achieve higher privacy (lower accuracy of M3) at the cost of a small reduction in utility (lower accuracy of M2).
• We show our methods provide tradeoffs between edge-computation efficiency, privacy, and accuracy. Specifically, with higher edge computation (more layers on the edge device), the client achieves better privacy at the same accuracy. Also, with the same edge computation (a given split layer), removing more components from the signal content provides better privacy at the cost of reduced accuracy.
• We perform extensive experiments on several datasets and show that our methods provide better tradeoffs between accuracy and privacy compared to existing approaches such as adversarial training, despite having no knowledge of the private attribute at training or inference times.

heading: 2 BACKGROUND AND RELATED WORK
text: We consider the supervised learning setting of Figure 1, where the model M2 ◦M1 is trained with a set of examples {xi}Ni=1 and their corresponding public labels {y pub i }Ni=1. At inference phase, the client runs M1 on their data and sends the intermediate feature vector z =M1(x) to the server. The goal of private inference is to ensure that z does not contain information about private attributes.

heading: 2.1 MEASURING PRIVACY
text: Several methods have been proposed to measure the privacy leakage of the feature vector. One approach is computing the mutual information between the query x and the feature vector z (Kraskov et al. (2004)). In practice, measuring the mutual information is not tractable for high-dimensional random variables, unless certain assumptions are made about the probability distribution of the random variables of interest. A more practical approach measures privacy based on the reconstruction error, ||x̃− x||, where x̃ is estimated based on z (Mahendran & Vedaldi (2015)). Finally, attribute privacy is defined based on the accuracy of an adversary model that takes z as input and predicts the private label.
In this paper, we use the attribute privacy notion as it applies to a wide range of applications. Assume each example {xi}Ni=1 has one or multiple private labels {y pri i }Ni=1. The adversary trains a separate model M3 with (zi, y pri i ) where zi = M1(xi), as shown in Figure 1. Note that M3 is used as an aftermath process to evaluate the privacy of the model M2 ◦M1. The split learning framework should achieve high utility, i.e., the server should be able to infer the public attribute from z accurately, while providing privacy, i.e., z should not contain information about ypri. We refer to the accuracy of M2 ◦M1 on ypub as public accuracy and the accuracy of M3 ◦M1 on ypri as private accuracy.

heading: 2.2 THREAT MODEL
text: Honest-but-curious server. The server performs the inference of the public attribute but will potentially try to extract private information from the features, z, as well.
Client capabilities. Upon providing the service, the server also provides a profile of the utility (accuracy on the public attributes), privacy (accuracy on several private attributes), and computation of the edge device. The client then decides on the best tradeoff based on the computational resources and also the desired level of privacy. Such mechanisms are already in use in ML-on-the-edge applications. For example, in the application of unlocking the phone by face recognition, the client can specify the required precision in face recognition, where a lower precision will provide higher utility at the cost of lower security (Chokkattu, 2019).

heading: 2.3 RELATED WORK
text: Prior work has shown that the representations learned by deep neural networks can be used to extract private information (Song & Shmatikov (2019)) or even reconstruct the raw data (Mahendran & Vedaldi (2015)). Current methods for private inference can be categorized as follows.
Cryptography-based solutions. Since the server is not trusted, solutions based on public key encryption (Al-Riyami & Paterson (2003)) are not applicable. We consider scenarios where the server provides service to millions of users (e.g., in cases of Amazon Alexa or Apple Siri), and users expect low communication and fast response. Therefore, classic two-party cryptographic solutions for secure function evaluation (Juvekar et al. (2018); Riazi et al. (2019)) are also not applicable to our scenario.
Noise Injection. A line of work suggests obfuscating private attributes by adding noise to the features, i.e., instead of z, the client sends z + µ to the server, with the noise designed to maintain public accuracy while reducing private accuracy (Mireshghallah et al. (2020)). While noise addition improves privacy, it has been shown to reduce the public accuracy significantly (Liu et al. (2019)).
Information Bottleneck. The notion of mutual information can be used to train private models. Let I(a, b) denote the mutual information between random variables a and b. The idea is to train M1 that maximizes I(z, ypub) while minimizing I(z, ypri) (Osia et al. (2018); Moyer et al. (2018)). The optimization is formulated as follows:
max M1
Ex,ypub,ypri [I(M1(x), ypub)− γI(M1(x), ypri)− βI(M1(x), x)]. (1)
The use of mutual information for privacy, however, has been challenged by practical attacks that extract secret information even when I(z, ypri) is small (Song & Shmatikov (2019)).
Adversarial Training. This defense solves the following min-max optimization problem:
max M1,M2 min M3
Ex,ypub,ypri [γL(ypri,M3 ◦M1(x))− L(ypub,M2 ◦M1(x))], (2)
where L denotes the cross-entropy loss and γ is a scalar. The above objective can be achieved through adversarial training (Edwards & Storkey (2016); Hamm (2017); Xie et al. (2017); Li et al. (2018); Feutry et al. (2018); Li et al. (2019)). At convergence, the trained M1 generates z such that M3(z) is not an accurate estimation of ypri while M2(z) accurately describes ypub.
Existing methods for private split inference have several limitations. First, the underlying assumption in above learning-based defenses is that a set of private attributes along with the public label are provided at training time. In practice, however, it might not be feasible to foresee and identify all possible private attributes and annotate the training data accordingly. It also contradicts deployment at-scale since whenever a new private attribute is identified, the model M1 needs to be retrained and re-distributed to all edge devices that use the service. Second, current approaches for private inference often provide a poor tradeoff between accuracy and privacy. Moreover, the tradeoff of accuracy and privacy with the client-side computation is not well studied in the split learning framework. In this paper, we characterize this tradeoff and propose an alternative approach that, instead of obfuscating the information related to the private attributes, the edge device removes the feature content that is irrelevant to the public task. We empirically show our method successfully reduces the accuracy on private attributes at a small or no cost to public accuracy.

heading: 3 PROPOSED METHODS

heading: 3.1 SIGNAL AND NULL CONTENTS OF FEATURE VECTOR
text: Let z ∈ Rn be a feature vector and W ∈ Rm×n be a matrix. The operation of fully-connected and convolutionals layers can be expressed as matrix-vector and matrix-matrix multiplication. Herein, we let z represent the vector in fully-connected layer and a column of the second matrix in convolution layer. Let the singular value decomposition (SVD) of W be W = U · S · V . Since the rows of V form an orthonormal basis, we can write the feature vector as
z = n∑ i=1 αiv T i , αi =< v T i , z >, (3)
where vi is the i-th row of V and the < ·, · > operator denotes inner-product. Definition 1. The signal content of z with respect to matrix W , or simply the signal content of z, denoted by zS is defined as
zS = argmin h
‖h‖2, s.t., W · (z − h) = 0. (4)
The null content of z is also defined as zN = z − zS . Lemma 1. We have
zS = m∑ i=1 αiv T i and zN = n∑ i=m+1 αiv T i . (5)
Proof. We write h as the composition of orthonormal vectors vi’s as h = ∑n i=1 βiv T i . We have
W (z − h) = n∑ i=1 (αi − βi)WvTi = n∑ i=1 (αi − βi)US V vTi︸︷︷︸ qi∈Rn
(6)
Since the rows of V are orthonormal, then qi = V vTi is a one-hot vector with its i-th element equal to 1. By substituting qi in (6) we obtain
W (z − h) = n∑ i=1 (αi − βi)US[:,i] = m∑ i=1 (αi − βi)US[:,i] = m∑ i=1 si(αi − βi)U[:,i], (7)
where S[:,i] and U[:,i] are the i-th columns of S and U , respectively. Note that, since S is a diagonal matrix, we have S[:,i] = 0,∀i ∈ {m + 1, · · · , n}, thus reducing the summation from n to m components. Also, for i ∈ {1, · · · ,m}, S[:,i] is a column vector with only one non-zero element, si, at the i-th dimension.
As a result, to obtain W (z − h) = 0, we must have βi = αi,∀i ∈ {1, · · · ,m}. Since vi’s are orthonormal, we have ‖h‖2 = √∑n i=1 β 2 i . Hence, to minimize ‖h‖2, we set βi = 0,∀i ∈
{m + 1, · · · , n}. Therefore, zS = ∑m i=1 αiv T i . The null content zN can be then computed as
zN = z − zS = ∑n i=m+1 αiv T i .
Definition 2. The normalized signal and null contents of z are defined as CS(z) = ||zS ||22 ||z||22 and CN (z) = ||zN ||22 ||z||22 , respectively. We have CS(z) + CN (z) = ∑m i=1 α 2 i∑n i=1 α 2 i + ∑n i=m+1 α 2 i∑n i=1 α 2 i = 1.

heading: 3.2 DEFENSE 1: OBFUSCATING NULL CONTENT OF FEATURE VECTOR
text: We propose to remove all the content in feature vector that is irrelevant to the public attribute. To do so, given a feature vector z, we find a minimum-norm vector z′ that generates the same prediction for the public attribute as z, i.e., M2(z′) =M2(z). Formally,
z′ = argmin h ‖h‖2, s.t., M2(h) =M2(z). (8)
Due to the complex (nonlinear) nature of deep networks, finding such a vector would require doing multiple backpropagations on M2 for each given z. This is, however, not feasible for resourceconstrained edge devices. To address this problem, we relax (8) such that the constraint holds for the first layer of M2 (server-side model), i.e., we modify the constraint to Wz′ =Wz, where W is the weight matrix of the first layer of M2. As discussed in Section 3.1, the solution to this relaxed optimization problem is zS , the signal content of z. Removing or obfuscating the null content of z does not change the model prediction on public attribute. It, however, might harm the private accuracy since part of the null content of zN might fall into the signal content of the first linear layer of M3. The method is described in Figure 2 (left).
At inference time, to obfuscate zN , the client constructs zo using either of the following methods. • Client constructs µ = ∑n i=m+1 ηiv T i , with coefficients, ηi, chosen at random, and sends zo =
z + µ to the server. The adversary can recover zS = V T1:m · V1:m · zo but cannot recover zN . • Client computes the signal content of z and sends zo = zS = ∑m i=1 αiv T i to the server.
For the first case, since µ is independent of z, the client can compute it offline, e.g., when the edge device is plugged in and not in use, and store it for later use. The second approach does not require storage on the edge device, but an extra computation, equal to the complexity of computing the first layer of M2, has to be done during inference to extract zS . We next propose a method that reduces the extra cost to only a fraction of the computation cost of the first layer of M2.

heading: 3.3 DEFENSE 2: DISCARDING LOW-ENERGY SIGNAL CONTENT OF FEATURE VECTOR
text: In the first defense method, we proposed to discard the content of the feature vector that will be removed by the first layer of M2. The following layers of M2 will further remove more content from feature vector. Hence, we can potentially discard more content from z and still get similar prediction as the original feature vector. For a linear layer, following the same process in Section 3.1, the output is computed as:
W · z =W · zS = m∑ i=1 αiU · S · qi = m∑ i=1 αiU · S[:,i] = m∑ i=1 siαiU[:,i], (9)
where si is the i-th eigenvalue in S, αi is defined in (3), and U[:,i] denotes the i-th column of U . From (9) we observe that components with larger siαi are contributing more to the layer output since ||U[:,i]||2 = 1 for all columns of U . As such, we approximate z → z̃ by only keeping m′ < m components of the right-hand-side of (9) that have the largest coefficients. Unlike null content filtering, removing signal content will affect the public accuracy as it changes the main network output, but can further reduce the private accuracy. To improve public accuracy when removing signal content of features, the server fine-tunes M2 on z̃. The method is described in Figure 2 (right).
Since si and U[:,i] are fixed at the inference time, the client only needs to send the selected αi values along with their indices to the server; the server knows si and U , and can reconstruct z̃ accordingly. The edge-computation cost of this process is m′/m times the computation cost of the first layer of the server model. We do experiments in settings where m′/m is about 1%. Hence, the computation cost of our method is only a small fraction of computing a single layer of the network. Moreover, since m′ m, our method also incurs a much smaller communication cost compared to sending the entire feature vector to the server.

heading: 4 EXPERIMENTS

heading: 4.1 SETUP
text: Datasets. We perform our experiments on four visual datasets listed below.
• EMNIST (Cohen et al. (2017)) is an extended version of the MNIST dataset where the labels are augmented with writer IDs. We selected 13000 samples from EMNIST written by 100 writers with 130 examples per writer. We then split this dataset into 10000, 1500, and 1500 training, validation, and test sets. We use the digit label and writer ID as the public and private attributes, respectively.
• FaceScrub (Ng & Winkler (2014); FaceScrub (2020)) is a dataset of celebrity faces labeled with gender and identity. We use gender and identity as the public and private attributes, respectively. In experiments, we cropped the face region using the bounding boxes in the image annotations and resized images to 50× 50.
• UTKFace (Zhang et al. (2017)) is a dataset of face images labeled with gender and race, which we use as the public and private attributes, respectively. We cropped the face region using the bounding boxes in the image annotations and resized images to 50× 50.
• CelebA (Liu et al. (2015)) is a dataset of celebrity images. Each image is labeled with 40 binary attributes. Out of these, we select “Smiling” as the public attribute and {Male, Heavy Makeup, High Cheekbones, Mouth Slightly Open, Wearing Lipstick, Attractive} as private attributes. These attributes have near balanced distribution of positive and negative examples. In experiments, we cropped the face region using the bounding boxes in the image annotations and resized images to 73× 60.
Model architecture. We present the experimental results on a model used in prior work (Song & Shmatikov (2019)). The model architecture and baseline test accuracy results are summarized in Table 1 and Table 2 in Appendix.
Adversary capabilities. We use the same architecture for adversary’s model M3 as the server model M2. The model M3 is trained using the features extracted by M1 and the associated private labels. We also assume that the adversary knows the parameters of M1 and M2.
Training settings. We use Adam optimizer with an initial learning rate of 0.001 and drop the learning rate by a factor of 10 after 20 and 40 epochs. All models including the adversary and main models are trained for 50 epochs unless stated otherwise.

heading: 4.2 EVALUATIONS
text: We start our analysis by computing the null and signal contents in every layer of M = M2 ◦M1. Figure 3 (left) shows the content of the input remained at each layer; for the i-th layer, this content is computed as ∏i j=1 CS(zj) where zj denotes the activation vector at the j-th layer and CS(·) is defined in Section 3.1. As the feature vector propagates through network layers, more content is gradually removed from z until the model outputs the prediction on the public task. We also split the network at different layers and measure the private accuracy of M3 trained with the feature vector. As seen in Figure 3 (right), the private accuracy also decreases as we get closer to the output layer, indicating that the discarded content contained information relevant to the private attribute.
To reduce the privacy leakage, we proposed to filter out the null content of the feature vector. Figure 4 shows the private accuracy for different split layers. Removing the null content reduces the private
accuracy without affecting the public accuracy. Moreover, splitting the network in deeper layers improves the privacy. To further reduce the private accuracy, we discard the low-energy components of the signal content and only keep m′ features. Figure 5 illustrates the effect of m′ on the public and private accuracy, when the network is split at the CONV-3 layer. As seen, by setting m′ to a small value, our method achieves high privacy (low private accuracy) at the cost of a small reduction in public accuracy. In general, the privacy can be controlled using two factors:
• The split layer: As we go deeper in the network, i.e., when the edge device performs more computation, better tradeoffs can be achieved. To show this effect, we perform signal-content removal at different layers, with m′ set such that the public accuracy is reduced by 1%. The corresponding private accuracy is shown in Figure 6.
• Number of signal components sent to the server: For the same edge-computation (a given split layer), the number of preserved features (m′) can be tuned so as to achieve a desired tradeoff between utility (higher public accuracy) and privacy (lower private accuracy). Figure 5 shows the results for the setting that the network is split at the input of the CONV-3 layer.
Comparison to Pruning. Similar to our approach, pruning network layers can eliminate features that do not contribute to the public attributed. In the following, we compare our method with pruning in terms of public and private accuracy. We split the network from the middle layer, i.e., at the input of the FC-1 layer. For our method, we keep the top m′ components of z from its signal content and filter out the rest. For pruning, we keepm′ elements in z and set the rest to zero. We adopt the pruning algorithm proposed by (Li et al. (2016)) which works based on the L1 norm of the columns of the following layer’s weight matrix. After pruning, we fine-tune M2 to improve the public accuracy. Figure 7 shows the public and private accuracy for each dataset. As seen, with small m′, both our method and pruning achieve a low private accuracy. Pruning, however, significantly reduces the public accu-
racy as well. For example, for the UTKFace dataset, with m′ = 1, both methods result in a private accuracy close to the random guess. However, pruning reduces the public accuracy from 92.25% to 53% (also close to random guess), whereas our method keeps the public accuracy at 88.63%.
Comparison with adversarial training. We implement adversarial training framework proposed by (Feutry et al. (2018)) and present the utility-privacy tradeoff in Figure 8. To achieve the best tradeoff for adversarial training, we train the models in multiple settings with different γ parameters (Eq. 2) in range of [0.1, 1]. Note that, unlike our method, adversarial training assumes that the private attribute is known at training time. Despite this, Figure 8 shows that our method achieves a better utility-accuracy tradeoff than adversarial training.
We also do experiments for the case with multiple (unseen) private labels. Specifically, we consider the CelebA model trained to detect “smiling” and evaluate two methods, 1) our method: we keep only m′ = 1 component from the signal content of feature vector and then train one adversary model per private attribute, and 2) adversarial training: we first adversarially train an M1 model to obfuscate “gender,” and then train one model for each private attribute.
For both of the above methods, the network is split at the input of the FC-1 layer. Figure 9 shows the results. Our method outperforms adversarial training method on both public and private accuracy. In our method, the accuracy on all private attributes are significantly lower than the baseline private accuracy. The only exceptions are “high cheekbones” and “mouth open” attributes, which have correlations with public attribute, that is, a smiling person likely has high cheekbones and their mouth open. The correlation between public and private attributes causes the signal content of server and adversary’s models to have large overlaps and, hence, results in high private accuracy. The adversarially trained model successfully hides the information that it has been trained to obfuscate (the “gender” attribute). Such a model, however, fails to remove information of other attributes such as “makeup” or “lipstick”. This highlights the applicability of our method in practical setting as a generic obfuscator compared to specialized techniques such as adversarial training.
Ablation study on CONV and FC layers. We compare the performance of our method on CONV and FC layers. To do so, we train two networks on the UTKFace task, (1) a network with 10 CONV
layers each with 16 output channels, and (2) a network with 10 FC layers each with 2304 neurons. Both networks have an extra FC layer at the end for classification. The number of channels/neurons are chosen such that the total number of output features at each layer is the same for the two networks. The public accuracy of the 10-CONV and 10-FC networks is 89.26% and 89.07%, respectively. Figure 10 shows the public and private accuracy when we remove low-energy components of the signal space at different layers. The number of preserved features, m′, at each layer is chosen such that the public accuracy is maintained. As seen, the 10-FC network achieves a lower private accuracy compared to the 10-CONV network. The reason is that CONV layers are known to be generic feature extractors, while FC layers are more specialized toward the public attribute.
5 CONCLUSION
We proposed a private inference framework, in which edge devices run several layers locally and obfuscate the intermediate feature vector before sending it to the server to execute the rest of the model. For obfuscation, we proposed to remove information that is not relevant to the main task or does not significantly change the predictions. Specifically, we developed two methods of removing the content of the feature vector in the null space of the following linear layer and also removing the low-energy content of the remaining signal. We showed that, unlike existing methods, our methods improve privacy without requiring the knowledge of private attributes at training or inference times.

heading: A APPENDIX

Review summaries:
This paper tackles a timely problem of privacy leakage on the edge devices when applying deep neural networks. Instead of mitigating the leakage of a set of private attributes, the proposed method tries to remove the information irrelevant to the primary task. The proposed method does not need to identify the private attributes. The main contribution of this paper is the two proposed approaches for removing “null content” and “signal content.” The evaluations of the proposed approach are conducted on four image datasets.