Interpretable Complex-Valued Neural Networks for Privacy Protection
Authors: Liyao Xiang
Hao Zhang
Haotian Ma
Yifan Zhang
Jie Ren
Quanshi Zhang
Venue:  (2020)
Keywords: Deep Learning
Privacy Protection
Complex-Valued Neural Networks

Abstract: Previous studies have found that an adversary attacker can often infer unintended input information from intermediate-layer features. We study the possibility of preventing such adversarial inference, yet without too much accuracy degradation. We propose a generic method to revise the neural network to boost the challenge of inferring input attributes from features, while maintaining highly accurate outputs. In particular, the method transforms real-valued features into complex-valued ones, in which the input is hidden in a randomized phase of the transformed features. The knowledge of the phase acts like a key, with which any party can easily recover the output from the processing result, but without which the party can neither recover the output nor distinguish the original input. Preliminary experiments on various datasets and network structures have shown that our method significantly diminishes the adversary's ability in inferring about the input while largely preserves the resulting accuracy.

Published as a conference paper at ICLR 2020
INTERPRETABLE COMPLEX- VALUED NEURAL NET- WORKS FOR PRIVACY PROTECTION
Liyao Xiang", Hao Zhang", Haotian Ma', Yifan Zhang", Jie Ren", and Quanshi Zhang Shanghai Jiao Tong University,
'South China University of Technology
ABSTRACT
Previous studies have found that an adversary attacker can often infer unintended input information from intermediate-layee features. We study the possibility of preventing such adversarial inference, yet without too much accuracy degradation. We propose generic method to revise the neural network to boost the challenge of inferring input attributes from features, while maintaining highly accurate outputs. In particular, the method transforms real-valued features into complex-valued ones, in which the input is hidden in a randomized phase of the transformed features. The knowledge of the phase acts like a key, with which any party can easily recover the output from the processing result, but without which the party can neither recover the output nor distinguish the original input. Preliminary experiments on various datasets and network structures have shown that our method significantly diminishes the adversary's 's ability in inferring about the input while largely preserves the resulting accuracy.
INTRODUCTION
Deep neural networks (DNNs) have shown superb capabilities process massive volume of data, and local devices such as mobile phones, medical equipment, Internet of Things (IoT) devices have become major data entry points in recent years. Although on-device machine learning has exhibited various udvantages, it usually burdens thin devices with overwhelming computational overhead. Yet offloading data or processed features to a cloud operator would put the individual privacy risk. For example, if a user has a virtual assistant at home, it should not worry about its private data being collected and processed by an untrusted cloud operator. The operator, on the other hand, should not be able to recover original signals or their interpretation.
However, as shown the previous literature (Dosovitskiy Brox (2016); Zeiler & Fergus 2014);; Ma- hendran & Vedaldi 22055; Shokri et al. (2017); Ganju al. (2018) : Melis et al. (2019). intermediate- layer features face many privacy threats, where the adversary either reconstructs the input or infers unintended properties about the input. Hence, on-device processing encounters dilemma, i.e. while we expect intermediate- layer features to yield high accuracy, we certainly would not want sensitive information to be leaked.
Therefore, we propose a novel method which tweaks a conventional neural network into a complex- valued one, such that intermediate- layer features released without sacrificing input privacy too much. More precisely, we turn the original real-valued features into complex-valued ones, rotate these features by random angle, and feed them the cloud for further processing.
We face two significant challenges in the design. First, the tweaked features have to be correctly handled by the DNN on the cloud. Although network structures vary depending on the input types, it is desired that the piecewise linearity of features to be preserved, so that the resulting accuracy does not degrade too much compared with the original features. Second, an adversary attacker who intercepts the features in the middle should not be able to recover the exact input, nor can it figure out the correct output. Most importantly, the neural network supposed to be trained using the original data without additional human supervision, and is efficient to conduct inference.
"Quanshi Zhang is the corresponding author with the John Hopcroft Center and MoF. Key Lab of Artificial Intelligence AT Institute, Shanghai Jiao Tong University, China. 2gs10222s .tu.edu.ccm

---

Published as a conference paper at ICLR 2020
To overcome the first challenge, tailor operations of DNN to make the succeeding computation at the cloud invariant to feature rotations, L.e., the DNN can correctly handle the feature and preserve the rotated angle in the output. For the second issue, we adopt generative adversarial networks (GAN) to generate synthesized features to hide the original ones. It guarantees k-anonymity such that, when an adversary tries to recover the original input from the complex-valuee features, it would get at least k different inputs (Please see Appendix B for detailed discussion). Each of the k reconstructed inputs has equivalent probabilities of being the original input for the attacker. Apart from the GAN part, the training of the neural network adds little additional computation overhead and requires no extra denotations.
Contributions of this study can be summarized as follows. We propose to transform conventional ones, which hide input information into a randomly chosen phase. Only
DNNs into complex-valee the party with the phase information can retrieve the correct output. Without it, any party can neither recover the input nor the output. Tested on various datasets and DNNs, our method has been shown to yield only moderate computational overhead and little accuracy degradation, while significantly diminishes the adversary's 's ability of inferring about the input.
2 RELATED WORK
Complex-valued neural networks. Using complex- valued features or parameters in neural networks has always been an interesting topic. From the computational perspective, Trabelsi et al. (2018) showed that complex-vail counterparts. By augmenting recurrent neural networks with associative memory based on complex- valued vectors, Danihelka et al. (2016) achieved faster learning on memorization tasks. Complex- valued features can contain information in both the phase and the magnitude. An example is that Reichert & Serre (2013) used the phase to indicate properties of spike timing in cortical information processing. Zhang al.] (2019) used the quaternion-valle valued neural network to learn 3D-rotation- equivariant features for 3D point cloud processing. We take advantage of complex-valuee features such that original features can be hidden in an unknown phase. Also, Yang al. (2019) shared similar transformation-bast than features, and it aimed improve the adversarial robustness of the model.
neural networks have competitive performance with their real-valuee
approach as ours while it performed transformation on inputs rather
preservieeee deep learning. Various privacy-preserving mechanisms have been proposed using different definitions of privacy: Osia et al. (017)) applied the Siamese architecture separate the primary and private information so that the primary information was preserved in the feature. PrivyNet by [i et al. (2017) was proposed decide the local DNN structure under the privacy constraint based on the peak signal-to-noise ratio or the pixel-wise Euclidean distance. Data nullification and random noise addition were introduced by Wang et al. (2018) to protect private information in the features, which guaranteed differential privacy.
Cryptographic tools have been used to learn from sensitive data. Zhang et al. (016)) adopted homomorphic encryption, in particular, BGV encryption, to encrypt the private data and perform the high-order back- proppparpagttion on the encrypted data; Mohassel & Zhang (0177) distributed the non-colluding servers who performed the secure two-party computation to
private data among train models on the joint data. Our scheme achieves k-anonymity privacy guarantee such that any adversary at best reconstruct number of synthetic inputs from the transformed features and thus cannot distinguish the original input from others.
THREAT MODEL AND PRIVACY GUARANTEE
In this section, we briefly introduce the threat model and the privacy guarantee that our scheme aims to achieve.
Threat model. In our model, a user processes its raw data locally and transmits the neural network features to the cloud for further processing. An adversary would intercept the transferred features and perform either feature inversion attacks or property inference attacks on these features to either reconstruct inputs, infer unintended information. Typical examples of the former attacks include feature inversion by Dosovitskiy & Brox (2016). gradient-basee visualization by Zeiler & Fergus (2014); Mahendran & Vedaldi 2015), etc. And the latter ones include membership inference attack
2

---

Published as a conference paper at ICLR 2020
Original
With noises
Adversarial learning
b
Adversary
7"6
With additional
9
exp(1d) (8)
exp(-ii)
d
layers
Rotate
Rotate
Comple-vallad network
Revisedd
Encoder module @ local device
Processing module computation conter
Decoder module @ local device
Adversarial loss
(a) Structure the complex-vaie valued neural network.
(b) Network structures in experiments.
Figure 1:
Design overview and network structures under investigation.
by Shokri et al. 2 passive property inference attack by Ganju et al. (2018) as well as feature
leakage collaborative learning by Melis al. (2019).
k-anonymity. We aim to achieve the following privacy guarantee: when an adversary tries to reconstruct the original input from the transformed features, can sythesize inputs belonging different prototypes with equivalent likelihoods, and only one of them corresponds to the original input.
APPROACH
4.1
OVERVIEW
To fight against the above threats, we propose to revise the conventional DNN. The new structure is shown Fig. Ta where the entire network divided into following three modules.
Encoder is embedded inside local device at the end. The encoder extracts feature from the input, hides true feature by rotating the features by a randomized angle, and sends the encoded result the cloud.
Processing module is located at the public cloud. This module receives and processes the encoded features without knowing the rotation. At the end of processing, the cloud returns the output to the user.
Decoder resides at the user's local device. The decoder receives and decodes results sent from the cloud to obtain the final result.
The three modules are jointly trained. Once trained, the encoder and decoder are placed at the local device, whereas the processing module resides on the cloud for public use. We will illustrate the detail of each module in the following section.
4.2 COMPLEX- VALUED NEURAL NETWORKS
Let (I, y) E D denote an input and its label in the training dataset and be the encoder at the local device. Given the input the intermediate-layer feature computed as
a = g(),
(I)
but we do not directly submit to the cloud. Instead, we introduce fooling counterpart b to construct a complex-valued feature as follows:
== [a bi].
where 9 and are randomly chosen. is the fooling counterpart which does not contain any private information of a, but its magnitude is comparable with to cause obfuscation. The encoded feature is then sent to the processing module 4 which produces the complex-vaie -alued feature h (xx). Upon receiving h, the decoder makes prediction ý on I by inverting the complex-valus feature h back:
i dRR( exp(-io)]),
where d denotes the decoder module, which can be constructed as either a shallow network or just a softmax layer. Ry.) denotes the operation of picking real parts of complex values.
3

---

Published as a conference paper at ICLR 2020
Processing Module Φ. The core design of the processing module is to allow the complex complex-valued feature = (x) to be successfully decoded later by the decoder. Le. we rotate the complex- valued feature bi by an angle 0, all the features of the following layers are supposed to be rotated by the same angle. We represent the processing module as cascaded functions of multiple layers (xx)) &n (I... @1 (x))), where (,(() denotes the function of the j-th layer and f (fz-1) represents the output of the j-th layer. Thus the processing module should have the
following property:
{ = e (f) s.t. f()) a f 4O E 2m).
(4)
In other words, the function of each intermediate layer in the processing module should satisfy
, e (fs--i) s.t. 1 & f2-1, vj E .2, nn,, 40 E [0, 2T).
(5)
to recursively prove Eqn.
Let us consider six most common types of network layers to construct the processing module, i.e. the convolutional layer, the ReLU layer, the batch-normalization layer, the average/max pooling layer, the dropout layer, and the skip-connection operation. For the convolutional layer, remove the bias term and obtain Conv(f) = w & f. which satisfies Eqn. (5) Inspired by Trabelsi et al. (001)). we replace the ReLU with the following non-linear layer:
8(fizk) = max l,,
Sizk
(6)
where Jiyk denotes the neural activation at the location (i in the k-th channel of the feature, and c a positive constant.
Likewise, the batch-normalization operation is replaced by
flge
morm(fijx) = V [Ba() 1
where denotes the complex-valued tensor for the I-th sample in the batch.
For max-pooling layers, we modify the rule such that the feature with the maximum norm in the region selected, which ensures that max-pooling does not change the phase of its input.
For the dropout layer, we randomly drops out both the real and imaginary parts of complex-valuee features. Skip connections can be formulated as f + (ff), where the inner module & (ff) recursively satisfies Eqn. 44)
All above six operations satisfy Eqn. (4). Please see supplementary materials for the proof.
GAN-based Encoder. The objective of the encoder is to hide the real feature a of the input in a certain phase f of the encoded feature x cxp(i0) a + bi]. Let a' R[z exp((i8') Ri(a bi) exp(ii i0') R[(a + bi) exp(i denote feature decoded using a random angle 8' 전 0, where AO = 0'. An ideal encoder requires (i) the decoded feature a' contains sufficient information cause obfuscation with the real feature a: (ii) a' and follow the same distribution so that it is hard to distinguish which one is the real one. Hence, we train the encoder with range of A and b's, and adopt GAN distinguish different values of them. Letting denote the discriminator, we train the encoder g over the WGAN (Arjovsky et al. (2017)) loss:
min max L(g, D) = Eispz D(a)= EA0-0(0.)) D(a'I]
(8)
-Eippz D(g(T)) RA [D bD)
g generates features to fool D. Upon convergence, the distribution a' approximates that of a. Note that the loss is the expectation over uniform-randomly distributed 0 (0, 2T). Ideally, the expectation in Eqn. (8) should also be taken over all possible A + 0, and thus when an adversary tries to recover angle, it will generate infinite number of features following the same distribution as that a. Empirically, we pick & 1 such A&'s uniformly over (0, π) and iterate through these k- values negative samples in the adversarial learning. The GAN-based encoder
by randomly rotating
4

---

Published as a conference paper at ICLR 2020
is the key to our k-anonymity guarantee such that it ensures the real feature cannot be distinguished from at least k - synthetic features. Please find the end of Appendix B to see the visualization effect of our privacy guarantee.
The overall loss of learning is formulated as follows, which contains the adversarial loss in Eqn. and loss for the target task:
min Loss min 9.
max L(g, D) + Lunk (0. (.,y]]
9. .Pdd
where (9, y) represents the loss for the target task.
Note that to simplify the implementation. we compute b = g(()) as the feature of a randomly chosen sample /' # where ideally I' has little correlation with 1.
In sum, the entire neural network [g. 4; d] is trained with randomized variables and to minimize the loss in Eqn. (9). Once trained, the network [g. C d] is fixed and publicly released. At the inference phase, the encoder secretly selects a fooling counterpart b and a rotation angle 0 to encode its real feature The decoder receives the processed result h from the cloud, and make the final prediction based on Eqn. 3
4.3 ATTACKS TO COMPLEX- VALUED DNNs
We will materialize the attacks described in the threat model in 3ed make them specific to proposed complex-valus valued DNNs.
Feature inversion attacks: The adversary performs two types of attacks. In inversion attack the adversary tries to find out the most likely rotated angle 0 to revert the feature x and obtain a". An adversary learns to use a* to reconstruct the input ī == dec(a"), Here we define a* = R[r exp(-16)) as the most probable feature recovered by the adversary. The adversary, for instance, can build new discriminator D' to obtain 0 = max, D' D'(R[rxx((-X learns to directly reconstruct the targeted input from the features, i.e., î dec(x). We show that our adversarial training strategy boosts the difficulty of learning such a decoder. Let g and D" denote the learned encoder and discriminator, respectively. Based on g", let us construct the following discriminator D(a') -lg g" (dec(a'))]. The larger (aa)) indicates the higher similarity with the real input. Because D is learned via D = arg maxp L(g", D), we have T(g", D*) 2 T(gg,, D). The adversarial loss in Eqn. provides an upper bound of T(g", Î), and thus restricts the capability of dec(-).
In inversion attack 2, the adversary
By Ganju al. 2018), 'property' refers the input attribute, which can be any sensitive information to protect or any attribute not expected to be released with features. For example, given facial image dataset and age recognition task, the gender attribute is required to hide features. By labeling features with properties of the corresponding inputs, the adversary is able train classifiers on dataset consisting all the feature proppety pairs. Once trained, the classifiers can use intermediate-layee features to infer if the input possesses certain properties.
Property inference attacks:
We consider four attacking strategies. Inference attack 1: the adversary a classifier to predict hidden properties of the input. At the testing phase, the attacker feeds the reconstructed result dec(a*) from inversion attack
images train
the classifier predict hidden properties. Inference attack 2: the adversary first rotates x's by their respective 0 to estimate the most likely feature a The adversary trains and tests the classifier on a' to predict hidden properties. Inference attack 3: the adversary uses the result of inversion attack dec(a") to train and test a classifier to predict hidden properties. Inference attack 4: different from the aforementioned classification approach, we let the adversary compares a* against features of each training example to find its k-nearest neighbors (k-NNs) in the training set and use them to infer the hidden properties.
a" contains sufficient information to recover some property, aforementioned attackers would learn the relationship between a" (or dec(a"), and the hidden property. Otherwise, the model would fail.
5

---

Published as a conference paper at ICLR 2020
Input
Input
Original AlexNet
Noisy features Y =0.2
AlexNet with additional layers
Noisy features Y =1.0
AlexNet
LeNet
doo Rejrexpí 46j)
dee Refrespí .01))
AlexNet
LeNet
dec(x)
dec(x)
Figure 2: Left: CelebA images reconstructed from different features, From top down: original input, reconstruction from 'encoder' output of the original DNN, dec(a). dec(a*), dec(x:.. Right: CIFAR- 10 images reconstructed from different features. From top down: original input, reconstruction from 'encoder' output of the noisy DNN, -, dec(a*). dec(x). Reconstruction has limited resemblance with the input in the last two rows.
Classifiction
Reconstructon Errors
Danaset
Original DNN
DNN with additional layers
Complex- Valued
Original DNN
DNN with additional layers
Complex- Valued dec(a")
Complex- Valued
DNN
dec(s)
20--
CIFAR-10
ResNet-20-3
CIFAR-10
11.99
9.79
12.28
0.0867
0.1210
0.2124
0.2420
ResNet-32-o
CIFAR-10
11.13
9.67
10.48
0.0930
0.1171
0.2569
0,2412
ResNet-12-8
CIFAR-10
9.40
11.12
0,0959
0.2515
0.2425
ResNet-44-o
CIFAR-10
ResNet-44-3
CIFAR-10
10:50
10.15
10.51
0.1973
0.1210
0.2511
0.2397
ResNet-56-0
CIFAR-10
10.17
9.16
11.53
0.0989
D.1
0,2804
0.2377
RosNei-56-3
CIFAR-10
ResNet-110aa
CIFAR-10
10.19
9.14
11.97
0.0896
0.1079
0.3081
0.2495
ResNet-110-8
CIFAR-10
10.21
9.36
11.85
0.0932
0.1152
0.2582
0,2414
Table 1: Classification error rates and reconstruction errors measured different variants of ResNet and CIFAR-10. Additional layers are introduced due to the GAN structure, but are not trained over adversarial loss. Our results show that the revised DNNs have almost the same utility performance as the original ones, but with greater reconstruction loss.
5 EXPERIMENTS
We revised a variety of classical DNNs to complex-valued DNNs and tested them on different datasets to demonstrate the broad applicability of our method. Without loss of generality, tasks include object classification and face attribute estimation, but do not exclude other tasks on DNNs. We have applied total of two inversion attacks and four inference attacks the complex-valued DNNs for testing the privacy performance.
5.1 IMPLEMENTATION DETAILS
Complex-valued DNNs: We constructed complex-valued DNNs based on 8 classical DNNs in total, which included the ResNet-20032/44456/11 (He et al. (2016)), the LeNet (LeCun et al. (1998)). the V6G-16 (Simonyan & Zisserman (2015) and the AlexNet (Krizhevsky al. (2012)) As shown in Fig. Ta we divided each original DNN into three modules which were referred to as the encoder/processing module/decoder correspondingly. The structure is given by the top row of Fig. [b] The bottom row of Fig. Lb gives the architecture of revised DNNs. Given the original DNNs, the mnooder/procesinngdeeooode modules were divided as follows. For the residual network, we tested two variants - ResNet-a and ResNer-3 - in the a-variant, the output of the layer before the first 16 16 feature map was fed to the aforementioned GAN, and layers following the first 8 feature map constituted the decoder. The B-variant was the same with the o-variant except that the decoder was composed by the last residual block and the layers following it. The processing module of residual network was modified such that
in 6() for all non-lincar layers.
The encoder of the LeNet consisted of the first convolutional layer and the GAN, whereas its decoder only contained the softmax layer. All layers before the last 56 x 56 feature map of VGG-16 comprised the encoder. The decoder consisted of fully-connected layers and the softmax layer. For the AlexNet, the output of the first three convolutional layers was fed into GAN, and the decoder contained fully-connected layers and the softmax layer. In the processing modules of these DNNs, we set < E.yl grlgell for neural activations in the k-th channel.

---

Published as a conference paper at ICLR 2020
Classiiiation Error Rates
Dataset
Original
DNN with additiona hovers
Noisy DNN
Nousy DNN
Noisy DNN
Complex- Valued
DNN
0.2
0.5
DNN
CIFAR-10
19:78
24.15
27.33
17.98
LeNet
CIFAR-100
56.65
78.82
49.76
Ro-Nei-56-
CIPAR-100
53.26
44.3
57.24
61.31
74.17
44.37
ResNet-110-a
CIPAR-100 100
50.64
44.93
55.19
61.12
71.31
50:94
VGG-16
CUB-200
56.78
63.47
69.20
99.48
99.48
78.50 90
AlexNer
CaehA
14.17
15.94
Reconstnacion Frices
Doltet
Original
DNN with additional lnyers
NoNsyDNN DNN
Nony INN
Nuisy DNN
Complex-Valnnl
Valuel DNN
Complex-Vattnl DNN
DNN
0.2
0.5
1.0
deala")
dec(s)
LaNat
CIFAR-TO
017009
0.1208
010548
0.1076
0.1274
0.24015
0.2353
LeNet
CIFAR-100 100
0.7088
0.0950
0.1012
0.2700
ResNet-56-o
CIFAR-100
0.0229
0.1029
0.1461
0.1691
0.2017
0.2593
RosNes-110-o
CIPAR-100
0.1050
0.1092
0.1483
0.1690
0.2116
0.2602
0.2419
VGG+16
CUB-200
0.1255
0.1202
0.1764
0.0972
0.1990
0.280.3
0.2100
AlexNet
ColebA
0.0687
0.1068
0.3272
0.2597
Table 2: Classification error rates and reconstruction errors on variety of DNNs and datasets. We compared the results with noisy DNNs with different noise levels. Complex-Valee Valued DNNs have significantly better accuracy performance and higher reconstruction errors than noisy DNNs over all noise levels.
Dataset
Original DNN
Iadddtiona layers
Noiay DNN
Noisy DNN
Noisy DNN
Complex- Valued dozia")
Complex- Valued dee(s)
7 0.2
Y 0.5
7 1.0
LeNet
CIFAR-TU
0.T6
0.12
0.20
0.200
0.24
0.82
0.92
LeNet
CIFAR-100
0.14
ResNet-56-o
CIPAR-100
8.06
0.06
0:08
0.10
0.36
0.72
0.88
ResNa-110-a
CHFAR-100
0.04
0.12
0.10
0.16
0.36
0,80
0.86
VGG-16
CUR-200
0.06
0.06
0.08
0.02
0.14
0.86
0.84
AlexNet
CelehA
0.04
0.34
1.00
Table 3: Failure rate of reconstructed images identification. We have five human annotators to recognize the object in mixed groups of images. The results show that most fail to recognize the reconstruction from the revised DNNs.
Baselines: Beside the complex-valuee DNN, constructed three baseline networks for comparison.
Original DNNs (baseline 1): The original DNN without any revision was taken as the first baseline network. For case of presentation, we also divided the original DNNs into the encoder, the processing module, and the decoder. The division of modules for original DNNs was the same as that for complex-valued DNNs.
Noisy DNNs (baseline 2): Based on the original DNNs, we injected noise right after the encoder module, which was given as + y e (we set ? 0.5, 1.0, 2.0). The noisy feature was fed to the processing module instead of a, as shown in the second row of Fig. Tb Here e represents a high-dimensional random noise with the same average magnitude as
DNNs with additional layers (baseline 3): Since the encoder was based on GAN, additional layers needed to be added, when we revised the DNN. We chose a GAN which incorporated generator consisting of a convolutional layer with 3 x 3 x K filters, and discriminator composed by a convolutional layer as well as fully-connected layer. We designed this baseline for fair comparison, because additional layers in GAN, although without adversarial loss, would incur model structure changes. Its structure is illustrated by the third row of Fig. [bb]
Attacks: Privacy attacking was implemented as follows. (1) Inversion attacks: we implemented the inversion attacker based on U-net (Ronneberger et al. (205)). The original U-net consisted of blocks, and we revised the U-net to construct the attacker. The input feature of the U-net was first up-sampled to the size of the input, which will be fed into the inversion model. There were four down-sampling blocks and four up-sample blocks. Each block contained six convolutional layers for better reconstruction performance. Each down-sample block reduced the size of the feature by half. Each up-sample block doubled the size of the feature map. Thus, the output of the inversion model was of the same size as the input image. (2) Inference altacks: we selected CelebA and CIFAR- 100 datasets test robustness against property inference attacks. For CelebA, we evaluated the accuracy performance on classifying 30 attributes and the privacy performance on the rest 10 attributes. Likewise, the accuracy of the DNN on CIFAR- 100 was evaluated by the classification error of the major 20 superclasses, and the privacy was gauged by the classification error of the 100 minor classes.
CelebA and CIFAR- 100 adopted the AlexNet and the ResNet-56, respectively, as prototype models. Their attacker nets were implemented as the ResNet-50 and the ResNet-56, respectively.
7

---

Published as a conference paper at ICLR 2020
Datasel
Average
(4] Using CelbM Dataaet
Using CoKBA Dutaset
Res-et-20-a
CIFAR-10
0,7890
0290
Viteerc Artcc 1-ftre Image
ResNet-20-f 8
CIFAR-10
1-00
ResNet-32-a
CIFAR-10
0,7820
ResNet-32-9
CIFAR-10
0.7843
Ielerrss Atauc 3- Comples Nakaed
ResNet-44-o
R4
CIFAR-10
CIFAR-10
0.7853
0,8411
6.90
035
liveree Audii
kusrrc Atack 2: Complen Histadd
ResNet-56-o
CIFAR-10
0,8088
034
ResNet-56.8
CIFAR-10
0,8283
Epech
Epech
ResNet-110-a
CIFAR-I0
0,8048
1et Liing
(r) Living
ResNet-110-3
CIFAR-10
0.7818
LeNet
CIFAR-10
0,7884
LeNet
ResNet-56-a
CIFAR-100
CIFAR-100
0,8046
0,7898
infermes Attack 24 Comples Adi
Friserso Allach SH-ut fo Laya
Res-et--10-o
CIFAR-100
0.7878
VGG-16
CUB-200
Loyer
0.
AlexNet
CelehA
0,8500
0.5
014
100
190
200
53
150 200
Table 4: Average error (absolute value) of the estimated rotation an- gle 0 in radian.
Figure 3: The error rates in inferring hidden properties on CelebA (a)((b) and CIFAR-100 (c)/(d).
Classification Error Rates
Hidden Properties Error Rates
Speed(s/images)
Dataset
DNN structure
I-NN
3-NN
5-NN
CelehA
additiona layers
0.0804
0.2014
0.1726
0.162
0.0026
CelebA
Complex-Valuee Valued DNN
0.1475
0.3169
0.2790
0.2641
0.0027
CIFAR-100
w/ additional layers
0.1873
0.7338
0.6837
0.7116
0.0004
CIFAR-10 100
Complex-Valued Valued DNN
0.2677
0.9444
0.9363
0.925
0.0007
Table 5: The classification error rates, error rates in inferring hidden properties by k-NN, and
processing speed. We found that while introducing little computational overhead, our method diminishes the adversary's s power in inferring hidden properties.
5.2 EVALUATION METRICS
We evaluated three aspects of the revised DNNs' performance: accuracy, privacy, and efficiency. Accuracy concerned the performance of the model on the task. Privacy was measured by the robustness of our DNNs against inversion attacks and inference attacks. More specifically, for inversion attacks, we had the following measures: (1) the average error of the estimation about the rotated angle 8*, i.e. |e* - 0): (2) the pixel-level reconstruction error E|| î Ill] where the value of each pixel was scaled to [o, 1]; (3) failure rate of human identification of the reconstructed images. For inference attacks, we measured the robustness of DNNs by the error rate of each attack model (see Sec. 4.3) As to efficiency, we measured the processing time compared with the baseline.
5.3 EXPERIMENTAL RESULTS AND ANALYSIS
Fig. 2 visualizes images reconstructed using features of different baseline networks. Due to space constraint, this figure only shows partial results. For complete result, please refer to the supple- mentary file. Visual effects were significant as the reconstruction in the bottom two rows greatly shifted away from the original input. Obviously, other misleading input attributes were mixed up with the original ones. To further examine the effect, we compared classification error rates and reconstruction errors on variety of datasets and DNNs in Table [ and 2 Since the revised DNNs introduced additional layers to the original DNNs, we compared the performance of both the original DNNs as well as those with additional layers. Our complex-valuee DNNs achieved similar utility performance, and exhibited significantly higher reconstruction error in almost all groups.
Further, we mimiced the adversary to recover the rotated angle 0* with the help of discriminator model. Most errors fell into (4/4, T/ 22) according to Table 4 which indicates even the discriminator could not tell the original features. We also qualitatively tested the reconstructed result in Table
Fig. 3 and Table show the error rates in inferring hidden properties by attackers. We observed that in Fig. (a). the error rate of the inference attack on the complex-vait significantly higher than on the baseline 1 (raw images) and baseline (dec(a)) and was close to random selection (99%) on CIFAR- 100. We further tested inference attack and 3. In Fig. 3 (d). the error rates of the attacker nets trained a" and dec(a") were both significantly higher than that
DNNs (dec(a") )) was
8

---

Published as a conference paper at ICLR 2020
of baseline 3, i.e. a" indicated little about the hidden properties. k-NN methods yielded similar results. Last but not least, we compared the running time of processing the images with different DNNs. Batch size of 128 and 100 are used respectively for CelebA and CIFAR-100. It is shown that the complex-valued DNNs had comparable performance with the baseline.
6 CONCLUSION AND DISCUSSIONS
We propose novel method to preserve input privacy in the intermediate- features of deep neural networks. The method transforms a traditional DNN into a complex-valuse one. Our method has been tested variety of datasets and models. The experimental results have shown the method effectively boosts the difficulty of inferring inputs for the adversary, while largely preserving accuracy for the user. Please also find in the appendices for supplemental experiments.
Theoretically. not only the encoder output but also all other intermediate-layee features in the processing module preserve privacy. However, only need invert the encoder output to test the privacy performance, since all features in successive layers can be expressed as cascaded functions of the encoder output. Thus we may consider the privacy performance of the encoder output as the worst case for all features in the processing module.
ACKNOWLEDGEMENTS
This work partially supported by National Natural Science Foundation of China (U19B2043, 61906120, and 61902245), and the Science and Technology Innovation Program of Shanghai (Grant 19YF1424500).
REFERENCES
Martin Arjovsky, Soumith Chintala, and Lon Bottou. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference Machine Learning, pp. 214-223, 2017.
Ivo Danihelka, Greg Wayne, Benigno Uria, Nal Kalchbrenner, and Alex Graves, Associative Long Short-Term Memory. 2016.
Alexey Dosovitskiy and Thomas Brox. Inverting Visual Representations with Convolutional Networks. In Proc. of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 4829- 4837, 2016.
Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. Proceedings of the 2018 ACM SIGSAC Conference Computer and Communications Security, pp. 619-633. ACM, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Alex Krizhevsky, llya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep con- volutional neural networks. In Advances in Neural Information Processing Systems (NIPS), pp- 1097-1105, 2012.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basee learning applied to document recognition. In Proceedings of the IEEE, 1998.
Meng Li, Liangzhen Lai, Naveen Suda, Vikas Chandra, and David Z Pan. PrivyNet: A Flexible Frame- work for Preser-Pr Deep Neural Network Training. arXiv preprint 179.6.170.. 2017.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5188-5196, 2015.

---

Published as a conference paper at ICLR 2020
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. IEEE, 2019.
Payman Mohassel and Yupeng Zhang. SecureML: A System for Scalable Learning. 2017 38th IEEE Symposiun on Security and Privacy (SP), pp. 19-38. IEEE, 2017.
Machine
Seyed Ali Osia, Ali Shahin Shamsabadi, Ali Taheri, Kleomenis Katevas, Sina Sajadmanesh, Hamid Rabiee, Nicholas D Lane, and Hamed Haddadi. A Hybrid Deep Learning Architecture for Privacy- Preserving Mobile Analyties. arXiv preprint arXiv: 02952, 2017.
David P Reichert and Thomas Serre. Neuronal Synchrony in Complex- Valued Deep Networks. arXiv preprint orXiv:1312.660 2013.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In CVPR, 2015.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp- 3-18. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
Chiheb Trabelsi, Olexa Bilaniuk, Ying Zhang, Dmitriy Serdyuk, Sandeep Subramanian, João Felipe Santos, Soroush Mchri, Negar Rostamzadeh, Yoshua Bengio, and Christopher Pal. Deep Complex Networks. 2018.
Ji Wang, Jianguo Zhang, Weidong Bao, Xiaomin Zhu, Bokai Cao, and Philip S Yu. Not Just Privacy: Improving Performance Private Deep Learning in Mobile Cloud. In Proc. of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2407-2416. ACM, 2018.
Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation. In Proc. of the 36th International Conference Machine Learning, ICML 2019, 2019,
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pp. 818-833, Springer, 2014.
Binbin Zhang, Wen Shen, Shikun Huang, Zhihua Wei, and Quanshi Zhang. 3d-rotation-equiivriant quaternion neural networks. In 1rXiv:191.099042 2019.
Qingchen Zhang, Laurence T Yang, and Zhikui Chen. Privacy preserving deep computation model on cloud for big data feature learning. IEEE Transactions on Computers, 65(5): 2016.
10

---

Published as a conference paper at ICLR 2020
A PROOF OF x)) = - (x))
In this paper, we design the processing module to ensure features of all layers are rotated by the same angle, in order to enable the later decryption of the feature.
(0)) + bi x() e'0 x00))(1))
}
Vj, 11th = c00 f (())
where f/0 and f/00) represent the feature map computed using the input (0) and that computed using the input æ (0) respectively.
In order to prove the above equation, we revise basic layers/operations in the processing module to ensure
<(e" x) = e (x))
where (-) denotes the function of certain layer/operation. x is given as the input of the specific layer/operation (-). Based on this equation, we can recursively prove f(() ca p(0)
Let us consider the following six most common types of layers/functions to construct the processing module, i.e. the conv-layer, the ReLU layer. the Butch-normalization layer, the average/max pooling layer, the dropout layer, and the skip-connection operation.
We revise the conv-layer by omitting the bias term. Thus, we get
( t0 x) @
[e x]
i0 [w X x|
10 (xx)
2. We replace the ReLU layer with the non-linear activation function of (x(i)))) Thus, we can write the element-wise operation follows.
max
(ax Piik-
dle (l kigh)
I Tigkl 1
[e xaju] e00
Tigk
e (Ixiz)))
max magi c]
Lmmxx
Keihh
3. We replace the nitch-normalization layer with the function of (((t)) write the element-wwn wise operation as follows.
Thus,
v /
e x
6( (()))
50
Engz
20 1i
Begt
b 1ig 112]
4. For the the average/max pooling layer and the dropout layer, we can represent their functions in the form of (xx) = Ar, where x given as a vectorized feature, and denotes a matrix. For the average/max-poolle -pooling operation, A represents the selection of neural activations. For the dropout layer, contains binary values that indicate the dropout state. In this way, we get
( Ø) A(e*)x) e00 [Ax] = (xx)
For the skip-connection operation, we get T) e dh r ())) e [x)) = (x:)
where T-()) denotes the function that is skipped by the connection. We recursively ensure (c x) = 2 (xx).
B ARCHITECTURE OF THE GAN
The architecture of GAN based on the residual network. The generator consists of an input convolutional layer, residual block from the residual network, and convolutional layer for the output. The residual block contains two convolutional layers, and a skip connection. Thus, there four convolutional layers in the generator. The output of the generator of the same size the input. The architecture the discriminator is similar to that of the generator, but the output of the discriminator scalar instead of tensor.
11

---

Published as a conference paper at ICLR 2020
C
EXPERIMENTS: VISUALIZATION
Fig. and Fig. respectively give the visualization effect on CIFAR-10, CUB200-2011, and CelebA, when reconstructing the inputs from features.
Input Original LeNet
LeNet with additional layers
Noisy features 0.2
Noisy features
= Noisy features Y 1.0
LeNet dec(a*)
LeNet dec(x)
Figure 4: CIFAR-10 images reconstructed from different features variety of LeNet-based neural networks.
Input
Original VQG-16
VGG-16 with additional layers
Noisy features 0.2
Noisy features 0.5
Noisy features = 1.0
VGG-16 dec(a*)
VGG-16 dec(x)
Figure 5: CUB200-2011 images reconstructed from different features on a variety of VGG- 16-based neural networks.
Input
Original AlexNet
AlexNet with additional layers
AlexNet
dec(a*)
AlexNet
dec(x)
Figure 6: CelebA images reconstructed from different features variety of AlexNet-based neural networks.
Fig. 7 shows the reconstructed input images when an adversary launches a feature inversion attack: rotates by an angle continuously drawn from (0, 2T) and then reconstruct the input the rotated features. The first image of every two row is the original input image. We found that most meaningful reconstructed pictures mostly drawn from (0 4. 0 37
) ). Most reconstructed images have little
12

---

Published as a conference paper at ICLR 2020
resemblance to the original input, and present meaningful interpolation of the original sample and the randomly chosen sample. The result is an illustration of our k-anonymity privacy guarantee.
o M
Figure 7: CelebA images reconstructed from feature rotated by different angles within 2m. We pick the most meaningful ones. The first image every two row the original input image.
Fig. g shows images reconstructed from correctly decoded features. The result demonstrates that, with the 'key" - 0, one can successfully recover original inputs from features.
Figure 8: CelebA images and CIFAR-10 images reconstructed from feature x correctly decoded with the knowledge of The reconstructed images have high similarity with the original inputs.
13


Reviews:
[{"review_id": "ryxy2OvJjS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel way to outsource a part of the information processing in a deep learning model to an untrusted remote location while revealing only little information about the input or final output of the computation. To this end the result of an on-chip encoder (e.g. the first N layers of a ConvNet) is encoded in a complex number with a random phase, which is then shipped to a remote location and gets processed (the next M layers of a ConvNet) in a way that is phase-equivariant. This result is shipped back to the device which extracts the desired information by inverting the phase randomisation. Additional distractor signals are encoded through a GAN-type approach.\n\nOverall the paper is well written although it experiences a few rough edges (e.g. the conclusions, font size in images, typos). The way the problem or threat scenario is phrased in the introduction, however, should emphasise much more the point that none of the actors, be it an adversary intercepting the communication or the cloud operator itself, have to be trusted. In other words, if I have an Alexa device at home which uses this technique I would not have to trust Amazon to keep my processed data private. Instead, even Amazon itself would not be able to really recover the true speech signals or their meaning. In my opinion, this should be a centrepiece of motivation for this work (unless I have overlooked something).\n\nRight now the manuscript focuses on a large set of experiments to empirically demonstrate the effectiveness of the method. What is missing is a more theoretically founded notion of privacy that can yield trustable guarantees. Nonetheless, the novelty of the ideas would still warrant acceptance.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1gdMyUpYB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Abstract:\nIn this paper, the authors propose to hide information in phase of the input features. They proposed that if each layer of processing layer, sitting outside of the local unit, is phase preserving, then they can recover the phase back. They propose a modification to the most popular layers in DNN to satisfy that property.\n\nI think the general idea of the paper is interesting, but overall, the paper is very poorly written. It appears that it is written in a rush. \n\n*The abstract and introduction are poorly written. There are poorly written sentences in the text. Here are some examples:\n    + \"... We propose a generic method to revise a conventional neural network to boost the challenge of adversarially inferring about the input but still yields useful outputs. ....\"\n    + \"... given a transformed feature, an adversary can at best recover a number of features which contain at least another k − 1 features which are different but cannot be distinguished from the real feature.  ...\" -- This is so central to the whole paper and it is not well-written. Personally, didn't understand this attack.\nPlease proofread your paper.\n\n*Why do you need both theta and b ?  It seems to me if \\| b \\| is comparable to \\| a \\|, it can destroy the information in original the a arbitrary bad, so it makes sense to keep the norm of b small. However, in the paper, it is suggested to use a random sample (I') and set b = g(I'). I don't see any ablation study in the paper. There is no free lunch, if you provide stronger identity preservation, there should be a compromise in the accuracy, and it seems to be the magnitude of the b is that compromise. \n\n* The whole idea of the GAN encoder is not well justified. What does it mean that the fake feature should contain information \"beyond\" a ? This very vague.\n\n* \"Inference attack 1 \" and \"Inference attack 4\" are the same; only the inference models used in each attack are different. I don't know why the author has separated them.   \n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "SJeLYESoKB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "After rebuttal,\n\nI really appreciate the authors' effort during the rebuttal, and most of my concerns are addressed well. \n\n===\n\nSummary:\n\nThis paper proposed a complex-valued neural network to protect the input data from hidden features of DNNs. Specifically, the authors introduce (1) encoder: producing a complex-valued feature, (2) processing module:  extracting useful features for a decoder and (3) decoder: making a final decision from processed features. Using various deep architectures and datasets, the authors showed that the proposed method can hide the input data from hidden features while maintaining the performance of DNNs.\n\nDetailed comments:\n\nThe research topic and main idea of this paper (i.e. introducing a complex-valued neural network for hiding sensitive input data) are interesting and the authors showed that the proposed idea indeed works well using various neural architectures and datasets. It would be more interesting if the authors can consider NLP datasets or other tasks instead of classification. Overall, the paper is well-written and the ideas are novel. ", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers are unanimous in their opinion that this paper offers a novel approach to secure edge learning.  I concur.  Reviewers mention clarity, but I find the latest paper clear enough.
Decision: Accept (Poster)