GINN: Fast GPU-TEE Based Integrity for Neural Network Training
Authors: Aref Asvadishirehjini
Murat Kantarcioglu
Bradley A. Malin
Venue:  (2021)
Keywords: Deep Learning
Trusted Execution Environments
Integrity-Preserving Computation
Intel SGX

Abstract: Machine learning models based on Deep Neural Networks (DNNs) are increasingly being deployed in a wide range of applications ranging from self-driving cars to Covid-19 diagnostics. The computational power necessary to learn a DNN is non-trivial. So, as a result, cloud environments with dedicated hardware support emerged as important infrastructure. However, outsourcing computation to the cloud raises security, privacy, and integrity challenges. To address these challenges, previous works tried to leverage homomorphic encryption, secure multi-party computation, and trusted execution environments (TEE). Yet, none of these approaches can scale up to support realistic DNN model training workloads with deep architectures and millions of training examples without sustaining a significant performance hit. In this work, we focus on the setting where the integrity of the outsourced Deep Learning (DL) model training is ensured by TEE. We choose the TEE based approach because it has been shown to be more efficient compared to the pure cryptographic solutions, and the availability of TEEs on cloud environments. To mitigate the loss in performance, we combine random verification of selected computation steps with careful adjustments of DNN  used for training. Our experimental results show that the proposed approach may achieve 2X to 20X performance improvement compared to the pure TEE based solution while guaranteeing the integrity of the computation with high probability (e.g., 0.999) against the state-of-the-art DNN backdoor attacks.

Under review as a conference paper at ICLR 2021
GINN: Fast GPU-TEE Based Integrity for Neural Network Training
Anonymous authors Paper under double-bling review
ABSTRACT
Machine learning models based on Deep Neural Nerworks (DNNs) are increas- ingly deployed in wide range of applications, ranging from self-driving cars COVID-19 19 treatment discovery. To support the computational power necessary to learn a DNN, cloud environments with dedicated hardware support have emerged as critical infrastructure. However, there are many integrity challenges associated with outsoureing computation. Various approaches have been developed to ad- dress these challenges, building on trusted execution environments (TEE). Yet, no existing approach scales up to support realistie enterrity-rresevve training for heavy workloads (deep architectures and millions of training exam- ples) without sustaining a significant performance hit. To mitigate the time gap between pure TEE (full integrity) and pure GPU (no integrity), we combine ran- dom verification of selected computation steps with systematic adjustments DNN hyperparameters (e.g., a narrow gradient clipping range), hence limiting the attacker's ability to shift the model parameters significantly provided that the step is not selected for verification during its training phase. Experimental results show the new approach achieves 2X to 20X performance improvement over pure TEE based solution while guaranteeing a very high probability of integrity (e.g.. 0.999) with respect to saate-of-the-are DNN backdoor attacks.
DNN model
INTRODUCTION
Every day. Deep Learning (DL) is incorporated into some new aspects of the society. As a result, numerous industries increasingly rely on DL models to make decisions, ranging from computer vi- sion to natural language processing. The training process for these DL models requires a substantial quantity of computational resources (often in distributed fashion) for training, which traditional CPUs are unable fulfill. Hence, special hardware, with massive parallel computing capabilities such GPUs, is often utilized Shi et al. (2016). At the same time, the DL model building process is increasingly outsourced to the cloud. This is natural, as applying cloud services (e.g., Amazon EC2, be more fiscally palatable for companies
Microsoft Azure Google Cloud) for DL training by enabling them to focus on the software aspect of their products. Nevertheless, such outsourcing raises numerous concerns with respect the privacy and integrity of the learned models. In recog- nition of the privacy and integrity concerns around DL (and Machine Learning (ML.) in general). a considerable amount of research has been dedicated applied cryptography, in three general areas: 1) Multi-Party Computation (MPC) (e.g.. Mohassel & Zhang (2017)). 2) Homomorphic Encryption (IEE (e.g.. Galad-Bachrach al. (0166)) and 3) Trusted Execution Environment (TEE) (e.g.. Hunt et al. 2018); Hynes et al. (2018)). However, the majority of these investigations are limited in that: 1) they are only applicable to simple shallow network models, 2) they are evaluated with datasets that have a small number of records (such as MNIST LeCun & Cortes (2010) and CIFAR 10 Krizhevsky al.). and 3) they incur substantial amount of overhead that unacceptable for real-life DL train- ing workloads. In their effort to mitigate some of these problems, and securely move from CPUs to GPUs, Slalom Tramèr & Bonch (2019) mainly focus on the computational integrity at the test phase while depending the application context. can also support enhanced data privacy, however, at a much greater performance cost.
To address these limitations. preserving learning as a service that provides integrity guarantees in outsourced DL model training in TEEs. We assume that only the TEE running in the cloud trusted, and all the other resources
introduce GINN (See Figure D3: framework for integrity-

---

Under review as a conference paper at ICLR 2021
TEE
SGD
SOD/1)
Updats
W
W
W
Batch
V
GPU
Ferwrd
Radand
Figure 1: The main architecture of GINN. The TEE handles mini-batch selection, layer-specific randomness, and parameter initialization. The GPU performs forward and backward passes over the mini-batch (items selected by SGX provided seed) and reports the computed gradients to the TEE. TEE then clips the gradients and performs the weight update. Also, TEE preserves the MAC- authenticated intermediate gradient reports. During verification, TEE performs the forward and backward passes with the batch items along with layer-specific randomness (regenerated) and com- pares the gradients with the GPU's report.
such as GPUs can be controlled by an attacker to launch an attack (e.g., insert a trojan). In this context, our goal is to support the realistic deep learning training workloads while ensuring data and model integrity. To achieve this goal, we focus on the settings where maintaining the learning process's integrity is critical, while the training data may not contain privacy sensitive information. For example, may want to build a traffic sign detection model public traffic sign images and may still like to prevent attacks that can insert trojan during the training phase. Furthermore, we want provide assurances that the model is trained on the specified dataset, with known parameters so that the performance of the model can be replicated and audited for accountability and integrity.
The trivial approach of executing the entire learning process inside a TEE is not scalable since TEES are much slower compared to GPUs. Furthermore, even the existing performance improvement techniques (e.g., random matrix verification provided in Tramer & Boneh (2019)) are not enough to scale up to large DL model learning settings.
To alleviate the TEE bottleneck, we propose incorporating random verification of the computation steps. This strategy is based on the observation that it is unnecessary to verify all of the GPU's computation steps. Rather, we only need to verify occasionally to catch any deviation with a very high likelihood. Given that random verification may itself be insufficient (theoretically, an attacker can launch successful attack by modifying only single unconstrained gradient update), we further show how parts of the DL hyperparameter setting process, such as clipping rale should be modified to prevent single step attacks, and require a larger number of malicious updates by an attacker that model
controls the GPU. Simply. GINN limits the amount of change an adversary can inflict through a single SGD step. As a consequence, the adversary is forced to keep attacking while randomly being verified by the TEE. Using the of-te-of-the-art backdoor attacks, illustrate that random verification technique can detect attacks with a high probability (e.g., 0.99) while enabling 2x-20x performance gains compared to pure TEE based solutions.
The specific contributions of this paper are as follows:
We introduce the first approach to support integrity-preserving DL training by random ver- ification of stochastic gradient (SGD) steps inside TEE ensure the integrity of training pipeline data, parameters, computation function, etc. with a high probability.
We illustrate how gradient clipping can be used as defensive measure against single (or infrequent) step attack in combination with random verification.
We show the effectiveness of our TEE random verification and gradient clipping through extensive experimentation on DNN backdoor attacks.
2

---

Under review as a conference paper at ICLR 2021
2 BACKGROUND AND RELATED WORKS
Our system combines deep learning training on specialized fast hardware such as Graphical Process- ing Units (GPU) with Intel Software Guard Extensions (SGX) based TEE to ensure the produced model's integrity. Details on SGD training and gradient clipping are provided in Appendices B and
2.1 ATTACKS ON DNN MODELS IN TRAINING PHASE
Attacks on DNN models can be realized during both training or test phases. However, GINN is concerned with integrity/accountablitt
issues during the training phase of DNN models, such that attacks related to testing are out of the scope of this paper since test time attacks have been addressed before (e.g.. Slalom Tramer & Boneh (019)) In the literature, particularly in the computer vision domain, targeted trojan attacks on DNN classification models have become a real concern as deep learning has grown in its adoption. These attacks tend to alter the prediction of models if a specific condition in the input is met. These conditions may be feature-based Guu et al. (2017): Liu et al.] (217): Turner et al. (018)) or instance-based Chen al. (2017); (Shafahi et al. (2018). Recently, trojan attacks have been extended to Reinforcement Learning (RL) and text classification models
Panagiota Kiourti (2019) Sun (2020).
In practice, these attacks are implemented by manipulating samples during training through data poisoning. For instance, stamping images with a pattern and modifying its label. Interestingly, these models provide similar competitive classification test accuracy compared to clean models (i.e., mod- els have not been attacked). As consequence, it non-trivial to distinguish these trojaned models from non-trojaned ones based on model accuracy alone. To make matters worse, even if the model owner was aware of examples of the trojan trigger pattern, the owner would need to patch the model through re- re-training to dampen the efficacy of the trojan trigger pattern. Retraining does not always guarantee complete removal of the trojan behavior from the model. To date, various techniques have been proposed to diagnose and mitigate of trojaned models. However, all approaches are either based on unrealistic assumptions or are excessively costly. For instance, the Neural Cleanse Wang jet al. (2019) requires access to sizable sample of clean inputs to everse-enginese the backdoor and has shown to be successful only for trigger patterns with a relatively small size. ABS Liu et al.) (2019) improves upon Neural Cleanse in that requires significantly smaller number of samples; however, it assumes that the responsible trojan neurons can activate trojan behavior independently from each other, which is unlikely be true in practice.
Attacking the training pipeline to inject a trojan(s) in the final model is the cheapest and, thus, is likely the most desirable form of attack for real-world adversaries to launch. As such, throughout this work, we mainly focus on showing our methods' effectiveness in preventing this type of attack from happening. It should be noted that our method is orthogomal to attack type and is sufficiently generic to catch any continuous attack during the training of DNN model. GINN relies upon proactive training as opposed to post-training or deployment-time methods to assess the health of a DNN model. As explain later in section we assume that the initial training dataset is provided by an honest user and is free of manipulation. With this as a basis, GINN limits the amount of change an adversary can inflict on model through single SGD step. As consequence, the adversary is forced to keep attacking while randomly being verified by the TEE.
2.2 INTEGRITY FOR DNN TRAINING
GINN's main goal is enable high- high-interrity training pipeline so that end users are assured that the model is built on the specified dataset, using specified parameters without modification. Thus, the final model users know who built the model, what dataset was used for training, and what algorithms were put in place for building the model. If, at any point during training, GINN GINN detects a deviation from the specified execution, will not sign the final model to ascertain its validity. Tramèr & Boneh (2019) took a first step towards achieving both fast and reliable execution in the test phase but neglected the training phase. The training phase is far more computationally demanding than the test phase, such that verification of all steps in training requires substantially longer time. Since parameters keep changing. we cannot benefit from pre-computation. Second, backward pass involves computing gradients for both the inputs and the parameters and takes longer than forward
3

---

Under review as a conference paper at ICLR 2021
pass. Despite the mentioned hurdles, as investigation shows, it may not be necessary verify every step to achieve integrity guarantees with high probability.
2.3 INTEL SGX
SGX Costan & Devadas (2016) is an example of common TEE that is available in many modern-day computers. As outlined in Table 2 it provides a secluded hardware reserved area, namely, processor reserved memory (PRM), that kept private (i.e., it is not readable in plaintext) from the host, or any privileged processes, and is free from direct undetected tampering. It also supports remote attestation, such that users can attest the platform and the running code within en- clave before provisioning their secrets to remote server. Calls from routines that should transition to/from enclave are handled through predefined entry points that are called Ecall/Ocall that must be defined in advance, before building the enclave image. While it provides security and privacy for numerous applications (c.g., Priebe et al. (2018); Shaon et al. y: Kunkel et al. (2019). due to its limited memory and computational capacity, directly running unmodified applications inside SGX can induce a significant hit on performance. This is especially the case for applications that require large amounts of memory, such as training DNNs.
3 THREAT MODEL
Attacks the integrity of DNNs can be orchestrated different stages of the model learning pipeline (e.g., data collection or training). We assume the TEE node in GINN is trusted, and the bytes stored the PRM are always encrypted and authenticated before they are fetched inside the CPU. We assume that the data sent to GINN comes from honest users via a secure/authenticatee channel and devoid of malicious samples | sary has complete knowledge about the network structure, learning algorithm, and inputs (after TEE performs an initial pre-processing) to the model. In our threat model, the adversary is in complete control of the host system's 's software stack, and hardware (unprotected RAM, GPU), except for the CPU package and its internals. Therefore, the code that runs inside the enclave is free from tamper- ing, and the data that is accessed inside the cache-lines registers are not accessible to the adversary. For the inputs supplied to DNN tasks, the adversary is capable of performing insertion, modification, and deletion influence the final model towards her advantage. As result, an attacker may report wrong gradients as opposed to correctly computed ones.
For the training phase, we assume that the adver-
SYSTEM DESIGN
GINN offers integrity and accountability for the training phase of a DNN model while inducing limited computational overhead. An overview of GINN illustrated in figure D and we refer the reader to table 2 for symbol descriptions and abbreviations.
Before the training phase initiates, the training dataset is decrypted and validated inside the TEE. Besides, for each SGD step, the randomness regarding the mini-batch selection and the network layers (Srivastava et al. (2014) are derived within the TEE and supplied to the GPU. As result, the adversary will face more constrained environment. Moreover, in our design, the GPU always performs a forward and backward pass and reports the computed gradients to the TEE, At this point, the TEE clips the gradients and updates the parameters (low overhead operation). Finally, GINN randomly decides to verify the SGD steps within the TEE or not. GINN optimized to guarantee high level of integrity and the correctness of the model while providing an infrastructure that does not suffer from the substantial computational requirements of pure TEE-based solutions. We assume an honest and authenticated user will send her data encryption key Katiens (after remote- attestation) to the TEE. Next, the TEE decrypts/verifies the initial encrypted dataset using the Katient and supplies the trainer (GPU) the plain-text of the training set. If the TEE fails detect any violations of the protocol during training, it will sign message that certifies the final model.(Pleas)) see Appendix A for more details of the signed message). Then, during testing and deployment, the user can verify the digital signature of the model.
Detecting malicious samples beyond the scope of this work.
4

---

Under review as a conference paper at ICLR 2021
Training with GINN At the beginning of mini-batch iteration i. TEE supplies the untrusted GPU with the randommess for that iteration. After completion of the forward and backward passes over the mini-batch, the computed gradients are sent to the TEE for clipping and updating the parameters. Next, the TEE integrates the clipped gradients with the parameters of the previous step. GINN always clips the reported gradients and ensures that they are within narrow range that evolving the model towards the attacker's 's intended model requires prolonged malicious intervention by the attacker. GINN accepts the reported gradients and only applies the clipped version to the snapshot taken at the specific iteration. If the computation at that step is selected for random verification, then the faulty behavior can be detected. If not, the chance that the model evolved towards the attacker's desired optima will likely require multiple rounds providing ample opportunity for detection. The verification is done randomly to prevent an attacker from guessing which step is verified.
Probabilistic Verification with GINN The TEE randomly decides whether or not to verify the computation over each mini-batch. If the mini-batch is selected for verification, then the interme- diate results are saved, and the verification task is pushed into verification queue. Verification by the TEE can take place asynchronously and it does not halt the computation for future iterations on the GPU. The authenticity of snapshots is always verified with key that is derived from combi- nation of the TEE's session key, S K session and the corresponding iteration. When the TEE verifies SGX step populates the network parameters with the snapshot created for the step i 1. It then regenerates the randomness from step i to obtain the batch indices and correctly sets up the per layer randomness. Given that the TEE's goal is to verify that the reported gradients for step i are correctly computed, GINN does not keep track of the activation results. Rather it only requires the computed gradients, batch mean/std (for BatchNorm layer), and matrix multiplication outcomes (in case random matrix multiplication verification is chosen). These required parameters are saved for verification.
Randomized Matrix Multiplication Verification with GINN Matrix Multiplications (MM) take up the bulk of the resource-heavy computations in DNNs. In modern DNN frameworks, convolu- tional and connected layers computation is implemented in the form of matrix multiplication in both of the forward and backward passes. Table 4 in Appendix G depicts the computations in the forward pass and backward gradient with respect to the weights and previous layers' outputs in the form of a rank 2 tensor multiplication. Fortunately, there exists an efficient verification algorithm for matrix multiplication( [Freivalds (1977)) when the elements of matrices belong to a field. this work, we leverage these random matrix multiplication verification algorithms as well.
5 INTEGRITY ANALYSIS
To achieve the integrity goal, we need to derive the probability Pe (i.e., the verification probability of each step) to achieve our integrity goal ps (i.e., the probability that attacker can modify the result without being detected is less than p.).
Definition 1. Assuming the DNN training requires a total of B steps. for each step report (Rp vb e [1, BJ A Rs E (0, 1}) has probability p. for being corrupted ( i.e., Rg = 1), and the overall integrity requirement probability goal p: (for example p. = 0.999).
Theorem 1. Given total of B steps during SGD training, the required probability of choosing step verify (p.) should be greater than B ( (1og(1 log(1-p.)
1).
[og[I-p-)
Remark. Given each step contains m independent matrix multiplication (MM) operations that is to be repeated & times (independently), and each random entry is chosen from a field of size [S], the probability of error (accepting wrong MM equality) is less than a m Freivalds (1977).
Theorem 2. If random matrix multiplication verification is used, given the configuration of Theorem
7 the required probability of choosing a step to verify (pu) should be greater than Tog p.) 1).
B
We refer the reader to Appendix D.I and D.2 for complete proofs of the theorems. The threshold probability in Theorem 2 yields approximately the same values as Theorem ! when a - 0. How- ever, the randomized matrix multiplication verification requires a O(N2) operations (assuming two N N matrices) compared regular matrix multiplication that requires O(N3) operations.
5

---

Under review as a conference paper at ICLR 2021
Throughput Performance (ImageNet)
Throughput (Images/Sed
4.41
3.93
3.01
0.93
0.42
[0.00 0..99
1.6
1.04
0.63
0.41
0.33
1.63
1.05
0.7
1.05
0.73
0..55
1.15
0.93
00.5
2.24
1.43
1.79
SCX
SGX
SGX
SGX
SGX
SGXXAM
SGX
VGG19
VGG16
ResNet152
ResNet34
Forward
Backward
I
Overall
Figure 2: Throughput of the SGD training step for VGG19, VGG16, ResNer152, and Resnet34 on ImageNet dataset with repect to forward and backward passes. RMM can lead to verification that is twice as fast as full MM verification in case of a VGG architecture.
Table 1: Trained models with restricted clipping and learning rate
Dataset
clip
%clean
#attack
total
GTSRB
10 10-4
10 10-4
%95
%85
124
MNIST
10
5 x 10-6
2 %95
2 %85
49
CIFARIO
x 10
10 10
%90
%885
94
6 EXPERIMENTAL EVALUATION
We executed our experiments server with Linux OS, Intel Xeon CPU E3-1275 v6@3.80GHi 64GB of RAM and an NVIDIA Quadro P5000 GPU with 16GB memory. Our attack code is implemented in python 3.6 using the pytorch Paszke al. (2019) library. We use Intel SGX as our platform for TEE. For SGX proof-of-concept implementation, DarkNet Redmon (0013-0016) library is significantly modified run the experiments. Our SGX code has been tested with SDK 2.9 and the code runs inside a docker container in hardware mode. Our experiments are designed to investigate two aspects. First, we evaluate the impact integrating randomized matrix verification and show that it can potentially increase computational Sffccinncy.Second, we analyze the effective- ness of gradient clipping in foreing the attacker deviate from the honest protocol in significant numbers of mini-batch steps. Various attack hyper-parameters (e.g, poisoning rate) were evaluated in determining their importance towards a successful attack with minimal deviation. This is impor- tant because, if the attacker needs to deviate such deviations with a smaller number of random verification steps (i.e., Pe is higher in equation
mini-batch steps, then the TEE detect
TEE Performance We tested TEE performance on VGG16, VGG19 (Simonyan Zisserman
2014 ) 52, and ResNet34 (He et al. (2016)) architectures with the ImageNet (Deng et al.
2009 ) dataset (see Appendix F22 for results with CIFAR 10 dataset( Krizhevsky et al.)) Figure 2
illustrates the throughput of deep networks. Usually, most of the computation takes place in the volution layers of the network. However, the backward pass on average yields smaller throughput since involves one more MM (weight gradients, and input gradients) than the forward pass which only invokes MM once (i.c., output of convolution or fully-connected layers). The implementation is quite efficient in terms of MM operations which uses both vectorized instructions along with multi- threading. Both VGG networks gives better improvements (= 2.2X) than ResNet (= 1.2X). Consid- ering the TEE randomly decides to verify a SGD step with probability pv, the overall improvement is approximately multiplied by _x. For instance, if we assume the attacker only requires to deviate p- with probability 5.10-5 (i.e., deviating in five steps out hundred thousand of steps), we can detect it with p. x 0.1 (See Figure 5d in Appendix). For VGG networks it means approximately 22X improvements compared to pure T'EE-based solution that verifies every step. Nonetheless, experiments suggest, believe that attacking these deep models that trained massive datasets should require a significantly higher number of deviations. This in return may result a much higher performance gain.

---

Under review as a conference paper at ICLR 2021
Verticatonn Bate (0.00) Fulure)
MNIST haved backdoor trigger
Vericatio Rot: (0.00) Fallnc) )
based trigger
Verication Rat: (0,001 Fallur)
trigger
0.06
0.20
0.04
0.04
0.10
0.02
0.02
0.00
0.00
0.00
2 3
5 6
wigger
backdoon trigger
trigger
(a) MNIST
(b) GTSRB
(c)
CIFARIO
Verlcation m0o0n1 Failar))
MNIST based sizrt spoch of attack
RateeiinnnReteee Rute (0.00) Fvilbro)
GTSRB based on sturt epoch attack
Verfication Fiilure)
CIFARIO based start epoch of attack
0.06
0.20
0.04
0.04
0.10
0.02
0.02
Verfccatin
0.00
0.00
0.00
0.2
0.5
0.9
0.2
0.5
0.9
0.2
0.5
0.9
Attack Start Epocc
Attick Stut Epoch
Ambak Start Fpoch
(d) MNIST
(e)
GTSRB
(f)
CIFARIO
Veriicaion Rute (0.00) Failun))
based poisoning
Verleation Kale (0.001 Falhie)
poisuning mile
Vedicction Rak: (0.00) Fallu))
CIFARIO based on poisoning
0.06
0.20
0.04
0.04
0.02
0.10
$
0.02
0.00
0.00
0.00
0.1
0.5
0.9
0.1
0.5
0.9
0.1
0.5
0.9
Poisoning Rate
Paisoning Rate
Paisoning
(g) MNIST
(h) GTSRB
(i) CIFARIO
Figure 3: A boxplot representation of the verification rates required by TEE for detection failure < 10 Based on the attack hyper-parameters: 1) backdoor trigger type (first row). 2) epoch that
attack starts (second row, 20% x, 5,50% x, and 90% x /eppocss). 3) backdoor poisoning ratio
Combined Impact of Gradient Clipping and Learning Rate Attack Success As shown in Table we selected models that achieved high performance on both clean and poisoned test sam- ples. We conducted the attack in the following manner. First, the trainer follows the correct protocol (e.g. learning with mini-batch SGD) until epochattock. At this epoch, the attacker starts by injecting certain number of poisoned samples (poisrate batch_size) from every class into the training batch and labels it as the target label. The attacker continues to attack until they achieve desired threshold in terms of success rate (correctly classifying backdoored samples the attacker's target label). Once it passes the threshold, the attacker halts the attack and returns to the honest pro- tocol, while observing the decay attack success rate. low-threshold, the attacker transitions back to attack mode and repeats the aforementioned strat- egy. CIFARIO (Krizhevsky et al.),, GTSRB (Stallkamp et al. (2012) and MNIST (LeCun & Cortes (2100), ) were used to analyze the impact of multiple factors imposed by the attacker that also aims to maintain an acceptable accuracy over clean test set. For MNIST, and GTSRB, the Adam Kingma & Ba (014) optimizer (which requires smaller initial learning rate), and for the CIFAR 10 dataset, SGD with momentum are used. Additionally, for MNIST, and GTSRB 10% of the training set was help adjust the learning rate based on the validation loss. Save for CIFARI0 dataset, the learning rate was set to decay (by tenfold) at fixed epochs (40, 70, 100).
success rate falls below the desired
chosen for the validation
Backdoor Trigger Pattern We applied different backdoor triggers of them are shown in Appendix, Figure 9.. The MNIST dataset only has single channel images, we converted it to
We also experimented with the scenario where attacker attack at steps chosen randomly. This type of attack successful. Hence do not report those results here.

---

Under review as a conference paper at ICLR 2021
three channel image apply the triggers 3 to 6. As shown in Figures 3 a [ the trigger pattern can significantly influence the effectiveness of the attack. The red lines show the median, while the green dots correspond to the mean. In all of the datasets, the first trigger pattern (Figure 9a) was the most effective one. It covers a wider range of pixels compared to the second trigger type (Figure (b)). As a consequence, it is more likely for the model to remember the trigger pattern across longer periods of SGD steps. Moreover, because photo filters (c.g. Instagram) are popular these days, we investigate the possibility of conducting attacks using some of the filters(or transformations) as the trigger pattern. However, covering a very wide range of pixels does not lead to a stealthy attack, as illustrated for the last four patterns. These patterns cover the whole imput space and transform it to a new one that they share a lot of spacial similarities while only different in tone scale (e.g. Figure Dd]. Learning to distinguish inputs that are similar, but only different in their tone is demanding in terms of continuity of the attack. In this case, both of the images (w. and w.o. the trigger) are influencing most of the paramcters and filters in a contradictory manner (different classification label). Thus it takes significant number of steps for the network to learn to distinguish them when gradient clipping is applied.
Attack Start Epoch Another major factor influencing the evasiveness of the attack is when at- tacker starts the attack. For instance, early in the training phase the learning rate will be high, such that a savvy attacker might believe they can avoid low clipping values by initiating their attack. However, if the attack begins too early, then it is unlikely that the model has yet converged. There- fore, beginning the attack too early may require an unnecessarily high number of (unnecessarily) poisoned batches, which, in turn, would raise the probability of detection. Yet even the attacker was successful, once they halt the attack, the model will likely evolve the parameters back to a clean state relatively quickly, and may require the attacker to re-initiate their attack. At the same time, given that the system uses a low clipping value, if the attacker waits toward the end of training, the attack is again unlikely to be effective. It would be unlikely that the attack succeeds before the end of the training; particularly due to having a considerably smaller learning rate. As shown in Figures 3 ( the best time for the attack is when the model has relatively low loss on clean training inputs, and the combination of learning rate and clipping value (effective attainable update) could yield the model to move toward attacker's 's desired optima. However, for MNIST. which is an easier learning task, attacking early gives the attacker a better chance to launch stealthier attack. We speculate that this is due to the fast convergence of the model. After few epochs, it quickly reaches a stable low training loss value for clean images. As a result, when the attacker concludes the attack (after reaching to the desired threshold), it is generally preserved far longer than the other two datasets.
Mini-Batch Poisoning Ratio As we stated earlier, the poisrate parameter is the ratio of the num- ber of poisoned samples in the batch to the batch size. It is one of the critical factors amongst those we investigated. Especially when gradient clipping is used, setting poisrate appropriately can help the attacker by moving more parameters toward the desired optima. However going beyond the ratio 0.9 (i.e., poisrate > 0.9 can impact the training negatively for both clean inputs and poisoned inputs. As depicted Figures 3
our experiments suggest that filling more than half the batch with poisoned samples seems to be effective across all datasets. Although for the MNIST dataset, it shows that higher values can slightly perform better, but this not confirmed by experiments on more complex datasets. For example, for the GTSRB dataset, we could not find successful attack model with acceptable clean input accuracy.
Further Analysis On Image Dataset Due to the computational resource limit, we only inves- tigate a few instances of our attack on ResNet34 architecture over the ImageNet dataset. The state of the art reference (pytorch Paszke et al. 2019) repository) achieves an accuracy of 0.73. However, we only reinitialize and train the fully-connected plus the last four convolutional layers (10M out of 21M parameters) with batches of size 256. The initial learning rate is set to 0.05, and poisrate is 0.5. Moreover, the first trigger pattern (Figure (a) is used as the only pattern (resized to 256x256). Finally, our attack starts at the beginning of the third epoch.
Clean Training For 100K Steps In order to see the optimal accuracy that can be achieved for current setting, we selected three varying clipping values, ie.., unbounded, high (2.0), and low (0.0005). Training with unbounded clipping value was futile and the model barely reached 0.01
8

---

Under review as a conference paper at ICLR 2021
Attack Success and Validation Accuracy Rates vs Clipping xlabel
Attack Arocess/Accuraite Rate
1.00
Validation Accuncy, Clip-
Valadation Accuracy.Clg-221
Attack Accunacy Accunay.Clip-..000 Clip=0.0005
Amaak Accumacy.Cii-2.0
0.50
0.00
0
0.5
1
1.5
2
2.5
3.5
4
4.5
-102
(a)
Total Attack Steps xlabel
1,500.00
Siaps.Cli--0.000 Number Sieps, Clip=2.0
Total Attacked
1,000.00
500.00
0.00
1.5
2
2.5
3
3.5
4
4.5
-104
(b)
accuracy. However, training with the high clipping value converged acceptable optima of 0.68 accuracy. Please note that we only trained the model for far less iterations than what is usual for ImageNet (IM-2M steps) and the learning the model converged to an optima of 0.64 accuracy. We want to point out that having a learning decay schedule will make the accuracy approach the 0.73 (similar the reference). However, goal was to analyze the impact of clipping on the attack for a large dataset.
fixed. Additionally, for the low clipping value,
Gradient Clipping Impact Poisoning Backdoor Attack Evasiveness We conducted the tack for a subset of source labels (10 out of 1000 classes) to single target class. This attack requires less interference from the adversary because she does need change all classes recognize the backdoor trigger, influencing a smaller subset of parameters. Figure, Figure.[??ssht shows how smaller clipping rate forees the attacker continue the attack extensively, which will result in a small verification rate.
7 CONCLUSION
This paper introduced the GINN system, which provides integrity in outsourced DNN training using TEEs. As our experimental investigation illustrates, GINN scales up to realistic workloads by ran- domizing both mini-batch verification and matrix multiplication to achieve integrity guarantees with a high probability. We have further shown that random verification in combination with hyperparam- eter adjustment (e.g., setting low clipping rates), can achieve 2X-20X performance improvements in

---

Under review as a conference paper at ICLR 2021
comparison pure TEE-based solutions while catching potential integrity violations with very high probability.
REFERENCES
Xiangyi Chen, Zhiwei Steven Wu, and Mingyi Hong. Understanding gradient clipping in private sgd: A geometric perspective, 2020.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017.
Victor Costan and Srinivas Devadas. Intel sgx explained. IACR Cryptology ePrint Archive, 2016 (086):1-118, 2016.
J. Deng, W. Dong, R. Socher, L.JJ Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPRO9, 2009.
Rusins Freivalds. Probabilistic machines can use less running time. In IFIP congress, volume 839, pp. 842, 1977.
Ran Gilad- Bachhahh Nathan Dowlin, Kim Laine, Kristin E. Lauter, Michael Naehrig, and John Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In Proceedings of the 33nd International Conference on Machine Learning, ICMI. 2016, New York City, NY, USA, June 19-24, 2016, pp. 201-210, 2016. URL http: ixproceedingssee wgw..ml..g.esss......... w.w.:rs/.reevvee.mm. /v8// ad ba ch rach 6.ht.l]
lan Goodfellow, Yoshua Bengio, and Aaron Courville. 6.5 back pock-propagation and other differentia- tion algorithms. Deep Learning, pp. 200-220, 2016a.
Ian Goodfellow. Yoshua Bengio, and Aaron Courville. 10.11 optimization for long-term dependen-
cics. Deep Learning. pp. 408-411, 2016b.
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.0677 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Tyler Hunt, Congzheng Song, Reza Shokri, Vitaly Shmatikov, and Emmett Witchel.
Chiron:
Privacy-preserving machine learning as service. CoRR, abs/1 10t///903. 2018. URL StLhtt::
Nick Hynes, Raymond Cheng, and Dawn Song. Efficient decb learning on multi private data. CoRR, abs/1807 606606689, 2018. hwl.hhttpp//arkki.orr///////////
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14Xiv:1412.6980 2014.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced re- search). URL
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105. 2012.
Roland Kunkel, Do Le Quoc, Franz Gregor, Sergei Arnautov, Pramod Bhatotia, and Christof Fetzer. Tensorscone: A secure tensorflow framework using intel sgx. arXiv preprint arXiv: 1902,0441,, 2019.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. yann. Coann.ecun.com ecun.com httmm//xxbm/mi..////////
URL http:// //
10

---

Under review as a conference paper at ICLR 2021
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. 2017.
Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma. Yousra Aafer and Xiangyu Zhang. Abs: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference Computer and Communications Security, CCS '19, pp. 1265-1282, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450367479. doi: 11145/3319533533622166 URL https: /ttp:::://ai.oor/11011443
$3115555.336311
Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable privacy-preserving machine learning. IACR Cryptology ePrint Archive, 2017:396, 2017. URI. http://
Susmit Jha Wenchao Li Panagiota Kiourti, Kacper Wardega. Trojdrl: Trojan attacks on deep rein- forcement learning agents. CoRR, abs/1903.06638, 2019. URL. htt.t::///a/aii.rrree./..
1903.06638
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché-Buus Buch E. Fox, and
R. Garnett (eds.), ), Advances in Neural Information Processing Systems 32, pp. 8024-8035. Cur- ran Associates, Inc., 2019. http://www////eooeeeeeeeeeeee/ea
an htt.ppppp...ae.eaaaaaaaa.aaaaaaa
Christian Priebe, Kapil Vaswani, and Manuel Costa. Enclavedb: A secure database using sgx. In
2018 IEEE Symposium on Security and Privacy (SP), pp. 264 278. IEEE, 2018.
Joseph Redmon. Darknet: Open source neural networks in c. http://pjrrediie.cmm
darknet/ 2013-2016.
Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathemati- cal statistics, pp. 400-407 1951.
Ali Shafahi, W. Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumi- tras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks neural net- works. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett (eds.), Advances in Newral Information Processing Systems 31, pp. 6103 6113. Curran As- sociates, Inc., 2018 URL http://///.aaape........//////aa
targeted cleeed-clean ceinnlabee label poi son
Fahad Shaon, Murat Kantarcioglu, Zhiqiang Lin, and Latifur Khan. Sgx-bigmatrix: A practical encrypted data analytic framework with trusted processors. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1211-1228, 2017.
Shaohuai Shi, Qiang Wang, Pengfei Xu, and Xiaowen Chu. Benchmarking ofate-of-the-ant deep learning software tools. CoRR, abs/1608.07249, 2016. URL /ttt:://arxivvor///aaa/
1608.07249
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 2014.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1919))992999959 2014.
J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, (0)=,, 2012. ISSN 0893-6080. doi: 10. 02..1...22111111 a

---

Under review as a conference paper at ICLR 2021
Lichao Sun. Natural backdoor attack text data, 2020.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections learning. In Thirty-first AAAI ference on artificial intelligence, 2017.
Florian Tramèr and Dan Boneh. Slalom: Fast, verifiable and private execution of neural networks in trusted hardware. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019, URL IJVorjCcKQ
Alexander Turner, Dimitris Tsipras, and Aleksander Madry. Clean-label backdoor attacks. 2018.
B. Wang. Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao. Neural cleanse: Identi- fying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 707-723, 2019.
Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaic. Why gradient clipping accelerates training: A theoretical justification for adaptivity, 2019.
A MODEL SIGNING BY TEE
We assume, an honest and authenticated user will send her data encryption key Ketient (after remote-attestation) to the TEE. Next, the TEE decrypts and verifies the initial encrypted dataset using the Ketient and supplies the trainer (GPU) the plain-text of the training set. the TEE fails to detect any violations of the protocol during training, it will sign the following message that certifies the final model where W is the model parameters, ds is training dataset, v_numm is the model version number, SH A256 is Sha 256 bit cryptographic hash function.
SH A2 A256(W)) Sww.:/mmHHHH.... (256(ds)) Siggtienn
with signature key Sigsax of the enclave
Table Symbols and Acronyms Description
Category
Symbol K SGX Sigsox
Description
TEE's session key for learning task
SGX signature signing key
Kateent
client's s encryption key
TEE
Sigatent
clients public key
PRM
Processor Reserved Memory
EPC
Enclave Page Cache
RMM
Randomized Matrix Multiplication
Neural Network
FV
Full Verification (No RMM)
DNN
Deep Neural Network
W
model parameters
General
ds
training datuset
num
model version number
B DEEP LEARNING TRAINING
In the recent decade, Deep Neural Networks (DNN) gained an enormous attraction in solving prob- lems related to computer vision and natural language processing Krizhevsky al. (2012); Simonyan & Zisserman 2014); He et al. 2016); Szegedy et al. (2015) 2017). In practice, these networks are stacks of layers that each perform transformation F (-) VI E L where X (Z') and [L] is the number of layers. The training task to learn the correct parameters (point- epsiiatestimates) W that optimizes (commonly minimizes) task-specific (e.g. classification) loss function L.
12

---

Under review as a conference paper at ICLR 2021
The most common way of training the DNN's parameters is through mini- Stochastic Gradient Descent (SGD) Robbins & Monro (195I). A randomly selected mini-batch of dataset is fed to the DNN and the value of objective function L is calculated. This is usually called the forward pass. Next, to derive the partial gradients of L w.r.t W v, backward pass is performed Goodfellow performed(Goodffellow et al. (2016a). Finally, parameters then updated according to the equation I where 0) is called the learning rate. Depending on the complexity of the dataset and the task, it might require hundreds of passes (called epoch) over the input dataset for convergence.
W W
V
WI
C GRADIENT CLIPPING
Gradient Clipping (GC) is method that is mainly known help mitigate the problem exploding gradients during training Goodfellow et al. (2016b). GC forces the gradients to fall within nar- row interval. There have been some efforts to analyze GC with respect to convergence. Authors in Zhang et al. (2019) prove (assuming fixed step size) that training with GC can be arbitrar- ily faster than training without Moreover, the theoretical analysis suggested that small clipping values can damage the training performance. However, in practice that is rarely the case. Chen et al. (2020) has an interesting theoretical analysis coupled with empirical evidence (symmetry of gradients distribution w.r.t SGD trajectory) that answers the gap between previous theoretical and practical observations.
D INTEGRITY PROOFS
D.1 RANDOM MINI- VERIFICATION
Define random variable X = [Bxpu] V, to be the total number of random verifi- cation done. Here V, is the batch chosen verification but failed the verification (due to malicious computation). Please note that we need to catch at least one deviation with probability greater than ps, and invalidate the overall model learning.
Definition 2.
Proof. Proof of theorem
P(X 11211=11 P(X = 0)
>
1 p4
P(X =0)
[ X pol
1 P:
(0 pc) [Axp_]
0
1 p:
IV
Pc) [Bxpo]
log(1
p:)
IV
B x Pol log(1 Pe)
log(1 Pi)
Pc
1)
(2)
log(1 - -pc)
As shown in Figure. 5 is only required to verify much smaller subset of the batch computa- tions inside the TEE to ensure high probability of correct computation. For example, for large datasets such as Imagenet Deng et al. (2009). it is needed to verify less than %1 of computation to have 0.9999 guarantee (when corruption probability %0.5) on the correctness of computation outsourced to the GPU.
D.2 RANDOM MINI-BATCH VERIFICATION WITH RANDOMIZED MATRIX MULTIPLICATION
Definition 3. Define random variable V/g 1 if 1 ^ MM MM.verify(0 0 otherwise 0 and X = [Bxx. Bxx. Vg. We need to detect at least one deviation with probability greater than p; while conducting random matrix multiplication verification.
13

---

Under review as a conference paper at ICLR 2021
Table 3: TEE Architectures Used
Arch
FC1
FC2
FC3
VGG11
(128,10)
(128,64.10)
(256,128,10)
VGGI3
(02
(128,64.10)
(256,128,10)
VGGT6
(128,10)
(128,64,10)
(256,128,10)
Proof. Proof of theorem
P(X 2 1)
N Pi
1 P(X = 0)
M
Pi
B x pol
(pe (1) (11 pe)) + Pea) 1 1Bxpo1
1 Pj
N
0
1 -4 P1 N
Pe) + paa) [Bxp.]
log(1 -p:)
B x Pa log(( (1 - -po) + Pr.a)
log(1 - -p:)
log((1 (a - 1)pe) 1) Pv > B
(3)
E VERIFICATION PROBABILITY GROWTH WITH RESPECT TO DETECTION PROBABILITY
Fig. 5 shows how verification probability changes with respect to the probability that a batch step is maliciously manipulated by the attacker. First row shows the verification probability for a dataset with 60K samples. Second row depicts the required for much bigger dataset (IM samples) over different mini-batch sizes. The smaller the mini-batch size is, there is higher chance for detecting malicious behavior.
F EXPERIMENTAL EVALUATION
F.1 ENCLAVE HEAP SIZE IMPACT ON TEE PERFORMANCE
As shown in Figure 6 which depicts the impact of the heap size on the performance of DNNs for single SGD step. can be seen that increasing the heap size way beyond the available hardware limit (around 92MB) can have a negative impact performance especially in the case of the VGG architecture. This result is mainly due driver level paging, which needs evict enclave pages that require extra level of encryption/decryption and 2) extra bookkeeping for the evicted pages.
F.2
TEE PERFORMANCE ON CIFAR10
Figure shows throughput performance for CIFARIO dataset and 9 different VGG architectures (Table 3) We chose three 1GG architectures adapted for CIFARIO image inputs with custom fully connected layers attached to its end. For CIFAR10, we generally do not benefit from randomized matrix multiplication scheme well ImageNet. Mainly it is because most of the operations and network layers fit well within the hardware memory limit. Therefore, since dimen- sions of MM operations are not too big, it does not improve significantly use randomized matrix multiplications.
F.3
IMPACT OF GRADIENT CLIPPING FOR HONEST TRAINERS
One important question is whether the gradient clipping used to prevent attacker to change param- eters in given batch update would have performance impact during training session where there is no attack. Six experiment configurations were repeated 5 times each with different ran- domness (initialization, batch order, etc.). Initial learning rates set E (0.1, 0..1) and clipping
14

---

Under review as a conference paper at ICLR 2021
epochs= datastt size = 60K, P: 0.99
epochs= 200,datasct size 60K, Ps > 0.999
64
Verification Probability (p.)
128
128
5 256
256
0.86
0.35
5 512
ò 512
0.3
0.3
0.15
0.1
0.1
5-10
Corruption Probability (pa)
10
Corruption Probability (pe)
110
(b)
epochs 200,dataset size
1M, Pi >> 0.99
epochs 200,dttuset size
1MM. pt
0.999
0.0
64
0.6
b 64
0.55
128
0.55
128
Verification Probability (po)
256
256
6 = 512
512
0.4
0.30
0.35
0.3
0.3
0.22
0.2
6.
5-10-2
1.5
2.5
3.5
Corruption Probability (pa) -10-5
Corruption Probability (p.) -10
(c)
(d)
Figure 5: Required verification probability with respect to batch corruption probability and the desired integrity probability for fixed 200 epochs and different SGD batch size.
Thoogghen (IuagseSed)
1.75
200
1.25
VGG
voDls
Sec)
S0X
SUX RXBM
1.0
100
0.5
0.25
0.1
(100.32)
(150,48)
(180,64)
(200,80)
(220.96)
(100.32)
(150.48)
(180,64)
(200,80)
(220.96)
(SGX Max Heup Siz,, Blucking Size) MB
(SGX Max Heup Size. Blocking Size) MB
(a) Available heap with respect throughput
(b) Available heap with respect time spent maarix-matrixxvectee vvecoo)) multiplication
Figure 6: The impact of increasing TEE heap size on (a) overall throughput and (b) the time spent
in matrix multiplication routine... VGG shows significant reduction in performance opposed to ResNet.
thresholds are set E (nil, 0.001, 0.0005). total, there are 30 ResNet56 (complex architecture with state-of-the-an performance) models trained on the CIFAR10 dataset (with attack) for 200 epochs. is shown that the clipping rate have very little impact on the test set accuracy, given an appropriate initial learning rate chosen. Usually, for SGD Robbins & Monro (1951) optimizer
15

---

Under review as a conference paper at ICLR 2021
Throughput Performance (CIFARIO)
69.84
57.
60
47.80
48.31
40.73
Throughput (Images/Se
40,
45.3
40.75
40.18
32. 76.76
31.02
32 00
34 02.02
39.3
30 07
38..1
34.30
24.50
17.07
15.75
17.3
10.76
12.75
9.01
SG05 SGMM SGX5Gx
XGX SGXgax RMN
SGXgGxXOM
SGXSaxxXMM
SGX5Gx
VGGIL-FCI VGGILFFC2 VGGIL-FC3 VGGI3.FCI VGG13.FC2
VGGI3-FC3 VGGI6 VGGI6.FC2 VGI6
Forward I Backward Overall
Figure 7: Throughput of SGD training step for VGG19, VGG16, ResNet1 and Resnet34 CIFAR10 dataset. Randomized Matrix Multiplication can make verification twice faster in case of VGG architecture.
with momentum, value 0.1 is chosen, (and for Adam optimizer a value less than 0.001). For these experiments, we used the configuration with unbounded gradient updates as the main reference point. For learning decay schedule, we used fixed decay by tenfold at epochs (50, 90, 130,160).
In figure 8a describes the mean and standard deviation (dashed lines) of test accuracy taken for repetitions at each epoch for the two learning rate configurations. As can be seen both models start to take giant leaps toward convergence the first two learning decays enforced by the scheduler. Please note that these are reference runs that no gradient clipping is applied during the update step. Toward the end of training, the setting with the higher initial learning slightly performs better in terms of accuracy (the y-axis is not in % scale).
In figure 8b) for each composition of learning rate, and clipping value, the highest difference (aceu- racy rate) with respect to reference run is plotted. The plot shows test accuracy is not influenced so much by the clipping value, rather, it highly dependent on the learning rate value. when (r 0.1, both clipping values can achieve values that are close to the reference runs that has no gradient clip- ping, however, slightly smaller (most epochs is negative, except jumps in the start). In figure &c the opposite of the previous measure is plotted. Again, by the end of the training, the gaps are signif- icantly tightened the case where better learning chosen. Therefore, having a smaller clipping value is not really impacting the performance in any considerable way.
Overall, figure shows that clipping does not really impact the learning task negatively, once good learning rate is chosen. One can observe that if the trainer choose acceptable learning rate for the task, small clipping values such as 0.001 or 0.0005) does not impede the learning task. Once the model passes the first learning rate decay schedule, all the configuration behave the same in terms of their test performance compared to their reference model (no gradient clipping limit).
F.4 ALL BACKDOOR TRIGGER EXAMPLES
G MATRIX MULTIPLICATION OPS OF COMMON DNN LAYERS
Table, shows common MM operations in DNNs. Connected and convolutional layers use MM routines to compute feed forward output, parameter gradients, and input gradients.
16

---

Under review as a conference paper at ICLR 2021
Table 4: Matrix Multiplication Operations
Layer Type
Computation
USubiaacched Precomp.
T
(W opr I Rojn
Oato Tajn (Wou))
Zogp Toui
Fully
Ousio Ropl
Backwar
Gradiont
Zuri
(Via)
Ppnn Vi Rignl
Backward Inputs Gradioni
2'enn) - Viero Ryni
Dile Rooii x Plypec
Vibonal Wyybecia x op
2ul Tpqu- Aqueog
Convolutional
2 21
Rockward
Mil Vilon (Aga mall
Zrm Pinoa.. T Tu.AD
Tpor Rpja+ ca x (Mogge og)
Rackoaed Inputs Gradient
V Vanno (Myneesaa (io Aa)
mun. Tuun Braad
NO/YES
Mionn Rppen of VisccpaA
H GINN BLOCKING OF BIG MATRICES
By default, GINN allocates/releases resources on a per layer basis. the event that even for single sample, it is not possible to satisfy the memory requirements of network (either large network or large inputs), GINN breaks each layer even further.
For convolutional layers, the main memory bottleneck is im2co which converts the layer's 's input (for each sample) of size 63 Wt; 14 to 1.2.c]] x [w, ho] (k kernel window size) matrix for more efficient matrix multiplication. GINN divides the inputs across the channel dimension and processes the imoc on maximum possible channels that can be processed at once.
For fully-connected layers, the main memory bottleneck is the parameters matrix Woo that does not depend on the batch size. GINN divides the matrix across the first dimension (rows), and pro- cesses the outputs the maximum possible size of rows that fits inside the TEE for the correspond- ing layer.
"extracts redundant patches from the input image and lays in columnar format
17

---

Under review as a conference paper at ICLR 2021
Mean and STD of reference accuracy across 5 repeats for each epoch
0.90
0.06
0.80
Mean Accuracy
STD Accuracy
0.70
Ir=10-1 1
Ir=10 -2
0.04
0.60
mean
std
0.50
0.02
0.40
0.30
0.00
20
40
60
80
100
120
140
160
180
200
Epoch
(a)
Highest test accuracy gap across 5 repeats
0.05
о Ir=10
1r-10
clip=5.10-4
clip=10-*
Best Difference
0.00
-0.05
0.100
60
80
100
120
140
160
180
200
Epoch
(b)
Lowest test accuracy gap across 5 repeats
0.05
Worst Difference
0.00
-0.05
-0.10
-0.15
60
80
100
120
140
160
180
200
Epoch
(c)
Figure 8: Reference Models (no gradient clipping) mean/std on accuracy repeats for two different learning rates. Each configuration had 5 repeats and reference model (no attack and unbounded updates). For each run configuration test accuracy difference di Sir,tip is defined as mar (aecir.eiip
aecree)
Vrep E [1,5]. Semin (acc
accçep)
rep
Vrep [1,5] 5]
18

---

Under review as a conference paper at ICLR 2021
(3) Large with multiple colorvrriations
(b) Small with low color variations two
(c) Gray scale
Instagram Kelvin Filter
(e) Coler Rotation Filter
(f) Mis of Rotation and Instagram Nashville Filters
Figure 9: All examples of triggers on CIFARIO images
19


Reviews:
[{"review_id": "wkTlp0FQB4N", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper discuss how to detect erroneous steps in gradient descent on a non-trusted GPU using a separate slower trusted execution environment,\nby randomly deciding in each step whether to check the values returned by the GPU, as well as using small learning rates and clipping the gradients to ensure all updates are small.\n\nThe reason for checking this is based on the assumption that since GPU calculations are out sourced there may be trust issues and attackers with control of the\nvalues returned by the GPU can alter the final network in subtle ways. \n\nThe paper includes experiments and shows that this approach is faster than just running everything on the trusted execution environment.\nThe experiments test an attack approach where the attacker tries to inject some bad samples to get some success of making the network being trained output some particular class on a particular image kind.\n\n\nI have questions about the model. It is explained that the attacker is full control of CPU and GPU etc. but it is also assumed the attacker runs the code on GPU as expected. I would prefer a complete black box model, where the system can input model, batches, parameters etc. to the GPU and get whatever it wants back, i.e. gradients, activations, whatever you desire and given an input the attacker can decide to return whatever you want.\n\nIt seems the attacker is seemingly completely oblivious to the actual network, and parameters being used, and only measures his current success rate, and seems very naive. \n\nFinally, the time is measure compared to a pure based TEE solution which is stated in the paper as completely unsatisfactory, and should instead be compared with training time without using any form of verification as this is the target.\n\nIn my opinion this paper is an implementation of a straight forward idea and the theorems for setting the probability parameters  are basic probably computations.\nThere is no theoretical contribution in the paper and all the arguments are heuristically based on some intuition about training dynamics.\n\nIn short, in my opinion the paper is simply not relevant or strong enough to warrant acceptance at ICLR.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "tpjTJ4yKgR1", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper Summary:\n\nThe paper presents new techniques to enable secure neural training in the TEE+GPU paradigm. This is a natural extension of previous work like Slalom which only handles the inference case. The authors propose a two-step approach. First, they clip the gradients during training to force the attacker to insert multiple deviations to influence the model. They then randomly verify the integrity a subset of the gradient updates to check for tampering. Combining these two ideas the authors demonstrate a 2-20x improvement over a TEE only benchmark.\n\nScore Rationale:\n\n- The paper presents two novel ideas\n  - The use of probabilistic checks on a random subset of the gradient update steps\n  - The use of gradient clipping to force the attacker to tamper with multiple update steps to affect the final model\n- While the core idea of verifying the linear layers is a direct extension of previous work, the probabilistic checks offer a significant improvement in performance\n\nStrengths:\n\n- The paper presents a detailed empirical evaluation of the required verification rate as function of multiple hyper-parameters such as the style of the injected error, aggressiveness, and timing of the poisoning\n- The paper demonstrates that gradient clipping has minimal impact on the network accuracy if a suitable learning rate is chosen\n\nWeaknesses:\n\n- The core contribution of this work relies on the effectiveness of clipping procedure in forcing the attacker to introduce a large number of faults for a successful attack. However, this conclusion is not directly supported by empirical evidence, i.e. there is no ablation study of the verification rate required in the absence of clipping.\n\nMinor Comments/Questions:\n\n- There is a typo in the caption of Fig. 6 which should specify the CIFAR-10 dataset rather than ImageNet\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "EM-3XJqchqY", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "It seems that the paper is focusing on the privacy preserving training of deep neural networks by developing a Learning-as-a-Service framework. It assumes that all resources may be penetrated by adversaries except the TEE. In this paper, the authors leverage random verification to detect the attacks and shows how gradient clipping can defend against attacks.\n\nAs an avid user of cloud training, I can agree upon the motivation of the paper that the integrity of training models in the cloud. However, I want to write about some issues that I have about the paper.\n\nFirst, it seems that the main contribution of the paper is two folds (1) using randomized verification over full verification and (2) verifying gradient clipping to defend against the attacks. However, mere randomized runs of verifications speak for a rather humble contribution. It would be interesting to see whether there can be other more intelligent methods that can identify them. It would add to the novelty and merit if there were some evaluations too for this.\n\nFurthermore, to verify the real effectiveness of the gradient clipping as the defense mechanism, it would be nice to see some theoretical analysis on this front.\n\nFurthermore, I have some issues with the evaluation. First, the paper compares to a full verification in Figure 2. However, this information is given without any breakdown of \"actual training\" vs. \"verification\". This makes it hard for the readers to see exactly where the speedups come from and where.\n\nThe paper does not seem to demonstrate any evaluation n terms of the potential accuracy loss for large datasets. While F.3 seems to demonstrate this on CIFAR10, this cannot fully verify that such gradient clipping wont hurt the accuracy of the models.\n\nI am open to reevaluating the paper given the authors' feedback.\n\n============================================\n\nI thank the authors' for their response.\nI am satisfied with the answer regarding gradient clipping. As such, I raised my score to 5.\n\nHowever, I still cannot get away from the thought that random verification seems to speak for a rather humble a contribution. Combined with the lack of breakdown of \"actual training\" vs. \"verification\", I am rather hesitant to give a score higher than this. Regarding the experimentation on ImageNets, I understand that there was lack of time to add more experiments on this front. However, it would provide a more concrete message if the paper includes these additional experiments.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BmaRCq7TxrS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper targets security challenges of deep neural networks. While solutions can hardly scale up to support realistic DNN model training workloads, the authors propose GINN to support integrity-preserving DL training by random verification of stochastic gradient steps inside trusted execution environments (TEE). GINN combines random verification and gradient clipping to achieve good performance. The experimental results show that GINN achieves 2x-20x performance improvement compared with the pure TEE based solution while guaranteeing the integrity with high probability.\n\n#######################\nPros\n1.\tThe security of deep neural networks is a very interesting and challenging topic.\n2.\tThe paper is well structured, and the assumptions are clear.\n3.\tThe authors provide extensive evaluation results to demonstrate their work. \n\n#######################\nCons\n\n1.\tThe authors illustrated how to do probabilistic verification: randomly decide to verify the computation over each batch. But how is a batch verified inside the TEE? It is not clearly stated in the paper. \n2.\tWhen discussing the limitation of existing solutions, the authors claim that they are evaluated with small datasets (MNIST, CIFAR10). But there is not much evaluation on large datasets for GINN either. Maybe adding an experiment on a large dataset in Figure 3 will be helpful? \n3.\tThe random verification strategy is based on the observation that it is unnecessary to verify all of the computation steps. Where does the observation come from? More details, data or citations will help.\n\n#######################\nMinor comment\n1.\tAs far as I understand, the use of gradient clipping is intended to reduce the bias introduced by the adversary (which might be missed by random verification ) so that the adversary tends to launch multiple attacks? It would be better to explain that clearly in the paper.\n2.\tIt would be good to see some results about Verification Rate v.s. clipping rate.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: While all reviewers agree the problem of TEEs for model training is well motivated, the reviewers remain divided on whether the concept of randomly selecting computations to verify has sufficient novelty, and whether the proposed gradient clipping method is well motivated.

Decision: Reject