How to 0wn the NAS in Your Spare Time
Authors: Sanghyun Hong
Michael Davinroy
Yiǧitcan Kaya
Dana Dachman-Soled
Tudor Dumitraş
Venue:  (2020)
Keywords: Reconstructing Novel Deep Learning Systems

Abstract: New data processing pipelines and novel network architectures increasingly drive the success of deep learning. In consequence, the industry considers top-performing architectures as intellectual property and devotes considerable computational resources to discovering such architectures through neural architecture search (NAS). This provides an incentive for adversaries to steal these novel architectures; when used in the cloud, to provide Machine Learning as a Service (MLaaS), the adversaries also have an opportunity to reconstruct the architectures by exploiting a range of hardware side-channels. However, it is challenging to reconstruct novel architectures and pipelines without knowing the computational graph (e.g., the layers, branches or skip connections), the architectural parameters (e.g., the number of filters in a convolutional layer) or the specific pre-processing steps (e.g. embeddings). In this paper, we design an algorithm that reconstructs the key components of a novel deep learning system by exploiting a small amount of information leakage from a cache side-channel attack, Flush+Reload. We use Flush+Reload to infer the trace of computations and the timing for each computation. Our algorithm then generates candidate computational graphs from the trace and eliminates incompatible candidates through a parameter estimation process. We implement our algorithm in PyTorch and Tensorflow. We demonstrate experimentally that we can reconstruct MalConv, a novel data pre-processing pipeline for malware detection, and ProxylessNAS-CPU, a novel network architecture for the ImageNet classification optimized to run on CPUs, without knowing the architecture family. In both cases, we achieve 0% error. These results suggest hardware side channels are a practical attack vector against MLaaS, and more efforts should be devoted to understanding their impact on the security of deep learning systems.

Published as a conference paper at ICLR 2020
How TO OWN NAS IN YOUR SPARE TIME
Sanghyun Hong, Michael Davinroy', Yigitcan Kaya, Dana Dachman-Soled, Tudor Dumitras
University of Maryland, College Park
shhengees. umd. edu,
mi michael... davinnoyygmaaall...... com,
yigitcanécs. und.edy
danadachéece, umd.edu
tdumi tummittreumaaas umd.
ABSTRACT
New data processing pipelines and novel network architectures increasingly drive the success of deep learning. In consequence, the industry considers top- performing architectures as intellectual property and devotes considerable com- putational resources to discovering such architectures through neural architecture search (NAS). This provides an incentive for adversaries to steal these novel ar- chitectures; when used the cloud, to provide Machine Learning as a Service (MLaaS), the adversaries also have an opportunity to reconstruct the architectures by exploiting a range of hardware side channels. However, it is challenging to reconstruct novel architectures and pipelines without knowing the computational graph (c.g., the layers, branches or skip connections), the architectural parameters (e.g., the number of filters in convolutional layer) the specific pre- proocesssnng steps (e.g. embeddings). In this paper, we design an algorithm that reconstructs the key components of novel deep learning system by exploiting a small amount of information leakage from cache side-channel attack, Flush+Reload. We use Flush+Reload to infer the trace of computations and the timing for each computa- tion. Our algorithm then generates candidate computational graphs from the trace and eliminates incompatible candidates through a parameter estimation process. We implement our algorithm in PyTorch and Tensorflow. We demonstrate exper- imentally that we can reconstruct MalConv, novel data pre-processing pipeline novel network architecture for
for malware detection, and ProxylessNAS-CPU the ImageNet classification optimized to run on CPUs, without knowing the archi- tecture family. In both cases, achieve 0% error. These results suggest hardware side channels are a practical attack vector against MLaaS, and more efforts should be devoted to understanding their impact the security of deep learning systems.
INTRODUCTION
To continue outperforming state-of ofttt--artttttt results, research in deep learning (DL) has shifted from manually engineering features to engineering DL systems, including novel data pre-processing pipelines (Raff al. 20188] Wang al., 2019) and novel neural architectures (Cai al.) 2019 Zoph et al., 2018). For example. a recent malware detection system MalConv, with manually designed pipeline that combines embeddings and convolutions, achieves 6% better detection rate over previous state-of-the-ar technique without pre-processing (Raff et al.) 2018). In addition to designing data vre-processing pipelines, other research efforts focus on neural architecture search (NAS)-a a method to automatically generate novel architectures that are faster, more accurate and more compact. For instance, the recent work of ProxylessNAS (Cai et al., 2019) can generate a novel architecture with 10% less error rate and 5x fewer parameters than previous state-of-the-arr generic architecture. As a result, in the industry such novel DL systems are kept as trade secrets or intellectual property they give their owners competitive edge (Christian & Vanhoucke (017).
These novel DL systems are usually costly to obtain: generating the NASNet architectures (Zoph et al. 2018) takes almost 40K GPU hours and the MalConv authors had to test large number of failed designs in the process of finding a successful architecture. As a result, an adversary who wishes have the benefits of such DL systems without incurring the costs has an incentive steal them. Compared to stealing a trained model (including all the weights), stealing the architectural
This work done when Michael Davinroy was research intern the Maryland Cyhersecurity Center.

---

Published as a conference paper at ICLR 2020
details that make the victim DL system novel provides the benefit that the architectures and pipelines are usually applicable to multiple tasks. Training new DL systems based on these stolen details still provides the benefits, details, an attacker can train a functioning model, even on a different data set, and still benefit from the stolen DL system (So al.] 2019 Wang et al. (209)). Further, against novel system, stealing its architectural details increases the reliability of blxck-box poisoning and evasion attacks (Demontis)
when the training data is different. After obtaining these
al., 2019). Moreover, stealing leads to threats such Camouflage attacks (Xiao al.) 2019)
that trigeer misclassifications by exploiting the image scaling algorithms that are common in DNN
pre-processing pipelines.
The emerging Bacchin-Learringg a--evvice (MLaaS) model that offers DL computation tools in the cloud makes remote hardware side-channel attacks practical vector for stealing DL sys- tems (Liu et al., 2015). Unlike prior stealing attacks, these attacks do not require physical proximity to the hardare that runs the system (Batina et al. 2019 Hua et al.) 2018) or direct query access to train an approximate model (Tramèr al.) 2016). Cache side-channel attacks have especially been shown as practical in cloud computing for stealing sensitive information, such as cryptographic keys (Liu et al. 2015). Cache side-channel attacks are ubiquitous and difficult to defeat as they are inherent to the micro architectural design of modern CPUs (Werner et al. 2019).
In this paper, considering the incentives to steal novel DL system and applicability of cache side- channel attacks in modern DL settings, we design a practical attack to steal novel DL. systems by leveraging only the cache side-channel leakage. Simulating common cloud computing scenario, our attacker has a co-located VM on the same host machine as the victim DL system, and shares the last-level cache with the victim (Liu et al.) 2015). As result, even though the VMs running on separate processor cores, the attacker can monitor the cache accesses a DL Tramework-Pylie or TensorFlow- makes while the victim system running (Liu et al. 2015).
The first step of our attack is launching a cache side-channel attack, Flush+Reload (Yarom & Falkner 2014), to extract a single trace of victim's 's function calls (Section B}. This trace corre- sponds to the execution of specific network operations DL framework performs, e.g., convolutions or batch- normalizations, while processing an input sample. However, the trace has little information about the computational graph, e.g., the layers, branches skip connections, or the architectural pa- rameters, e.g., the number of filters a convolutional layer. The limited prior work on side-channel attacks against DL systems assumed knowledge of the architecture family of the victim DNN (Yan) ctt al. 2018; Duddu et al.) 2018); therefore, these attacks are only able to extract variants of generic architectures, such as VGG (Simonyan & Zisserman 2015) ResNet (He al. 2016). To over- come this challenge we also extract the approximate time each DI. operation takes, in addition to the trace, and leverage this information estimate the architectural parameters. This enables us to develop a reconstruction algorithum that generates a set of candidate graphs given the trace and eliminates the incompatible candidates given the parameters (Section 4. We apply our technique to two exemplar DL systems: the MalConv data pre- pre-prccessing pipeline and novel neural architecture produced by ProxylessNAS.
Contributions. We design an algorithm that reconstructs novel DL systems only by extracting cache side-channel information, that leaks DL. computations, using Flush+Reload attack. We show that Rlush+Reload reliably extracts the trace of computations and exposes the time each computa- tional step takes in a practical cloud scenario. Using the extracted information, our reconstruction algorithm estimates the computational graph and the architectural parameters.
We demonstrate that our attacker can reconstruct a novel network architecture found from NAS process (ProxylesNNAS) and a novel manually designed data pre-processing pipeline (MalConv) with no reconstruction error.
We demonstrate the threat of practical stealing attacks against DL by exposing that the vulnerability is shared across common DL frameworks, Py Torch and TensorFlow.
BACKGROUND
Here, we discuss prior efforts in both crafting and stealing network architectures. There is a growing interest in crafting novel DI. systems as they significantly outperform their generic counterparts. The
2

---

Published as a conference paper at ICLR 2020
immense effort and computational costs of crafting them, however, motivates the adversaries steal them.
Effort to Design Deep Learning Systems. Creating deep learning systems traditionally takes the form of human design through expert knowledge and experience. Some problems require novel designs to manipulate the input domain -specfcc way that DNNs can process more effectively. For example, MalConv malware detection system (Raff et al.] 2018) uses manually designed pre- processing pipeline that can digest raw executable files as a whole. Pseudo LIDAR (Wang et al.) 2019), by pre-processing the output of a simple camera sensor into a LIDAR-like representation, achieves four times better object detection accuracy than previous state-of- of-t-of-theearrr technique. More- over, recent work also focuses automatically generating optimal architectures via neural archi- tecture search (NAS). For example, reinforcement learning (Zoph & Le, 2016) or gradient- approaches (Cai et al.) 2019) have been proposed for learning generate optimal architectures. Even though NAS procedures have been shown to produce more accurate, more compact and faster neural networks, the computational cost of the search can be an order of magnitude higher than training a generic architecture (Zoph et al. 2018).
Effort to Steal Deep Learning Systems. Prior work on stealing DNN systems focus on two main threat models based on whether the attacker has physical access to the victim's 's hardware. Physical access attacks have been proposed against hardware accelerators and they rely on precise timing measurements (Hua et al. 2018) electromagnetic emanations (Batina et al. 2019). These attacks are not applicable in the cloud setting we consider. The remote attacks that are applicable in the cloud setting, on the other hand, have limitation of requiring precise measurements that are impractical in the cloud (Duddu et al. 2018). Further, the attack without this limitation (Hong et al. 2018) requires the attacker know the family the target architecture comes from; thus, it cannot steal novel architectures. In our work, we design an attack to reconstruct novel DL systems by utilizing a practical cache side-channel attack in the cloud setting.
EXTRACTING THE SEQUENCE OF COMPUTATIONS VIA RELOAD
3.1 THREAT MODEL
We consider an attacker who aims to steal the key components in novel DL system, i.c., a novel pre-processing pipeline or novel network architecture. We first launch Flush+Reload (Yarom & Falkner 2014) attack to extract cache side-channel information leaked by DL computation. Our target setting a cloud environment, where the victim's DL system is deployed inside VM-or or a container-to serve the requests of external users. Rluood in this setting, is known to be a weak, and practical, side-channel attack (Liu al. 2015). Further, as in MLaaS products in the cloud, the victim uses popular open-source DL frameworks, such as PyTorch (Benoit Steiner 2019) or TensorFlow (Abadi et 2016).
Capabilities. We consider attacker that owns co-located VM- container in the same physical host machine as the victim's 's system. Prior work has shown that spinning-up the co-located VM in the third-party cloud computing services does not require sophisticated techniques (Risten- part et al. 2009 Zhang et al. 2011 Bates et al.) 2012 Kohno et al. 20055 Varadarajan et al. 2015). Due to the co-location, the last- cache (L3 cache) in the physical host is shared between multi- ple cores where the attacker's s and victim's 's processes are; thus, our attacker can monitor the victim's 's computations leaked at the L3 cache. We also note that, even if the victim uses GPUs, our attacker can still observe the same computations used for CPUs via cache side-channels (see Appendix A
Knowledge. We consider our attacker and the vietim use the same version of the same open-source DL framework. This is realistic, in MLaaS scenarios such AWS SageMaker Google Cloud's AutoML, as cloud providers recommend practitioners to use the common frameworks to construct their systems] These common practices also allow our attacker to reverse-engineer the frameworks offline and identify the lines of code to monitor with the Flush+Reload technique.
For example, AWS provides convenient deployment options for both PyTorch and TensorFlow:
httpss/
5x5. wws..ems.com www.xxxoccooom cwo/ www.seeeeeeeeeeeeeee /laeeee////eeteeeeeeeeeee ateett httpyessttt//tyttttoottooo and stt //oos. avs amezon, con aagerrkeer attett /óg/t h.hmm)
3

---

Published as a conference paper at ICLR 2020
Online (Co-located)
Offiin (Separate)
Identify the lines code to monitor
2 Monitor the lines of code via -lusodd
De-noise the Flush+Reload observations
Profile the computations with of parameters
Perform the reconstruction process with the data
Figure I: Overview of our attack procedure. Our attacker only requires to be online (co-located) while the attacker is monitoring the computations of a victim DL system.
3.2 FLUSHI+RELOAD MECHAANISM
Rlush+Reload allows an adversary to continually monitor victim's instruction access patterns by observing the time taken to load them from memory. This technique is effective to extract the computation flow of the victim's program when the attacker and victim share memory (i.e., shared library or page deduplication (Bosman et al. 20166). The attacker flushes specific lines of code in a shared DL framework from the co-located machine's 's cache-hierarchy and then measure the amount of time it takes to reload the lines of code. If the victim invokes the monitored line of code, the instruction will be reloaded into the shared cache, and when the attacker reloads the instruction, the access to it will be noticably faster. On the other hand, if the vietim does not call the monitored line of code, the access it will be slower because the instruction needs be loaded from main memory (DRAM). By repeating this process, our attacker can tell when a victim has accessed a line of code.
3.3 OVERVIEW OF OUR ATTACK PROCEDURE
In Figure I. we illustrate our attack procedure. We split the steps into two phases: the online phase and the offline phase. In the online phase (step 2), the attacker needs co-location to monitor the computations from the victim's 's system. In the offline phase (steps D, 3, ④ and (5) attacker does not require the co-location with the victim.
First, our attacker analyzes the open-source DL framework to identify the lines of code to mon- itor. The attacker monitors the first line of each function that corresponds to the start of DL computation.
Next, the attacker spins up a co-located VM and launches the Flush+Reload attack to extract the trace of victim system's function calls. As the trace does not depend on the input sample, we only require to extract single trace from one full invocation of the victim system. Since the raw observations with Flush=Reload are noisy, the attacker applies filtering to highlight the regularities of DL. computations reflected in the trace.
To estimate the architectural parameters, e.g., the input/output channels, kernel size, or strides, our attacker creates lookup tables of timings and performed number of matrix multiplications by collecting traces from various parameter combinations.
5 Finally, using the victim's 's computational trace lookup tables for estimating architectural param- eters, the attacker starts the reconstruction process steal the victim's DL system (Sec 4)
3.4 MONITORING THE TOY NETWORK COMPUTATIONS VIA FLUSH+RELOAD
Experimental Setup. We implement our attack on Ubuntu 18.04 rumning on host machine equipped with the Intel E3-1245v6 3.7GHz processors (8 cores, 32GB memory and 8MB cache shared between cores). For the step ( we analyze two popular open-source DI. frameworks, Py- Torch and TensorFlow, and identify the list of functions to monitor (see Appendix for the full list of functions). We leverage the Mastik toolkit (Yarom) 2016) to launch the MlushRReload attack. and while a victim DL system is running on VM, our attacker monitors the list of functions- step ②. For the reconstruction process conducted in offline after the extraction, we use Python v3.6 implement the procedure.
httpps/ www...gytt...................... hon..og
4

---

Published as a conference paper at ICLR 2020
ToyNet (PyTorch)
Raw Trace
Processed Trace
class ToyNet(
Module )
52230076,Comv2d
[0] Conv2D.52230076...2
init [self. implace=True
[1] (.tthhNoommDD54230700000
super(ToyNet. self). init. iiit).
[2] Conv2D,55600076,11000
self el =an. mn..Con2d(3310.111 1.1
self.bl mn.BatchNorm2d
53440820.GEMMM00nce
[3] BatchNNorm2D.See 0tt.i//0omm222222777700000
self.c2 = an mm.Con2dr10,11 10.10,100.1110111 .100
[4] RcLJU6.5988M07660000
self.h2 = an BatchNorm2d (10)
542420776.Buchhorrm22
[5] TensorAddd,602200766000
self.r2
ReLU6(inplace)
def forward x):
$5600076.Conv/
sx)
56040820. (
x self.bl(x).
56048200,EEMM(occcpp)
skip = # skip connection
x sxxl.b2(x)
== se)).
59880076,ReLLU
skip skip connection
return
60270076,add
Figure 2: Extracted traces while ToyNet is use. the left, we place our network definition in PyTorch. We show the raw trace from Flush+Reload (middle) and the de-noised trace (right).
we demonstrate the extracted trace via Flush+Reload while ToyNet is processing an input. ToyNet is composed of one convolution followed by a batch-norm and one depthwise convolution followed by a batch-norm and ReLU activation. The 1st convolution has the parameters (in, out, kernel, stride) as (3, 10, 3. 1), and the depthwise convolution's 's parameters are (10, 10, 1, 1). The network has a skip connection that adds the intermediate output (from the 1st convolution) to the final output. During inference, we feed in an input with dimensions 3x32x32.
ToyNet Results. Figure 2
In the middle panel of Figure we also show the -oiiyo-tt-treee from the Rlush+Reload output. The trace only includes cache-hits where the attacker's accesses to the lines of code are faster, ie., when the victim invokes the function. Each element of the trace includes a timestamp and a function name. The name corresponds to the ToyNet layers, such as Conv2d and BatchNorm22, and it also contains additional information such as the tensor (add) and the BLAS operations, e.g., GEMM(oncopy).
Our attacker filters the raw trace according to the regular patterns in the DL computation. For example, a long function call, e.g., Conv2d in the ToyNet trace, can appear multiple times in the trace, as the cache can hit multiple times during Flush+Reload. In this case, we condense the multiple occurrences into single invocation using heuristic based how close the timestamps are. We also observe the matrix multiplications such as GEMM(conv) and GEMM(oncopy)) while DL computation is being processed. We count the individual occurrences and sum them up them based on the timestamp. After obtaining the processed trace (in the right panel). the attacker starts the reconstruction procedure.
RECONSTRUCTING NOVEL DEEP LEARNING SYSTEMS
After processing the Flush+Reload trace, our attacker reconstructs the key components of the vic- tim's DL system. In this process, the attacker aims generate the candidate computational graphs of the victim system and to eliminate the incompatible candidates by estimating the correct param- eter set for each computation. For instance, our ToyNet example, the attacker wants to identify the computational orders and the location of the start and end of a branch connection (computa- tional graph). Also, the same attacker wants to estimate the parameters for each computation; for example, the input/output channels and the kernel size in the 1st Conv2d. In this small network that has one branch, there are only 10 candidate computational graphs; however, considering all possible combinations of parameters, this will result in untractable number of candidates. Prior work, in reconstruction, only considered generic architectures such as VGGs or ResNets with the unrealistic assumption that attacker knows the architecture family (backbone); however, steal novel DL systems, we do not make this assumption. To overcome this problem, we design a reconstruction procedure, which describe next.
aim is to
5

---

Published as a conference paper at ICLR 2020
Knowledge of Our Attacker in Reconstruction. Here, consider attacker knows what ten- sor operations and functions to monitor in the victim's 's open-source DL framework. These functions moodel-indeeendent they correspond to architectural attributes designated by the deep learning framework (see Appendix [C. We show that this knowledge is sufficient to reconstruct novel data- preprocessing pipelines, such as MalCony, that are usually shallower than the network architectures.
To reconstruct the deeper network architecture automatically designed by NAS algorithms, we assume our attacker has some knowledge about the NAS search space-e.g. NASNet search space (Zoh et al.) 2018) layers used and the fact that set of layers (known as blocks) are repeatedly used such as Normal and Reduction Blocks in NASNet. We make this assumption because, from the sequence of compu- tations observed via Flush+Reload, our attacker can easily identify a set of layers and the repetitions of the layers. However, we do mot assume how each block is composed by using the layer obser- vations directly; instead, we identify candidate blocks by using a sequence mining algorithm. We demonstrate that, under these assumptions, attack reconstructs the ProxylessNAS- in 12 CPU hours rather than running a NAS algorithm from scratch that takes 40k GPU hours]
victim's 's search process relies on. This knowledge includes the list of
4.1 OVERVIEW OF OUR RECONSTRUCTION PROCEDURE.
We first focus on the invariant rules in the computations used for DL. computations. For instance, there are unary operations and binary operations. The tensor addition used to implement skip connection is binary operation; thus, our attacker can supplement the reconstruction process by pruning the incompatible candidates. We also exploit the fact that computation time is proportional to the number of element-wise multiplications in a computation. In the ToyNet example, the time the 1st convolution (2 million cycles) takes shorter than the 2nd depthwise convolution (3.668 million cycles); thus, our attacker further eliminates the candidates by comparing the possible parameters for a computation with her offline profiling data the lookup table.
Our reconstruction procedures consist of two steps:
! Generation: The attacker can generate the candidate computational graphs from the Flush+Reload trace based on the invariant rules in DL computations. Using the rules, our at- tacker reduces the number of candidates significantly. Elimination: Our attacker compares the time for each computation takes with profiling data and prunes the incompatible candidates. We estimate the parameters sequentially starting from the imput. When the output dimension from a candidate does not match with the observation, we eliminate.
Error Metries. To quantifyy the error of our reconstruction result, we use two similarity metrics. First, we use the graph edit distance (GED) (Abu-Aisheh et al. 2015) compare the reconstructed computational graph with that of the victim. Second, we use the di-distance to compute the error between the estimated architectural parameters and those in the victim system.
Victims. We first reconstruct the MalConv (Raff al. 2018). a novel data pre- pre-processing pipeline that converts the binary file into a specific format so that a neural network can digest easily. Also, we show that our attacker can reconstruct the novel ProxylessNAS (Cai al.) 2019) architecture that shows the improved accuracy on the ImageNet classification with the less computational cost on a CPU.
4.2 RECONSTRUCTING NOVEL PRE-PROCESSING PIPELINES
Here, we elaborate the reconstruction process of the MalCony (Raff et al., 2018) data pre-processing pipeline. MalConv receives the raw bytes of exe files and determines whether the file is malicious or not. The uniqueness of MalConv comes from the way that treats the sequence of bytes: 1) Code instructions in binary file are correlated spatially, but the correlation has discontinuities from function calls and jump commands that are difficult to capture by the sequence models, e.g., RNNs.
Note that ProxylessNAS starts its searching process from backbone architecture such as NASNet; thus, even if the paper reported search took 200 GPU hours, this number does not include the time spent searching backbone architecture, i.e., the 40k GPU hours find NASNet.

---

Published as a conference paper at ICLR 2020
Reconstruction Errors
Vietim
Type
# Candidates
# Compatibles
Topology (GED) Parameters (é)
MalConv
Pre-processing Pipeline
20
0
ProxylesNAS-CCUU
Deep Neural Network
180,224
Table 1: Reconstruction results. We report the number of candidates and compatible architectures.
Also, we report the reconstruction errors in the resulting topology and parameters, and they 0%.
2) Also, each sequence has the order of two million steps which far exceedes the length of an input to any previous neural network classifier. MalConv tackles this problem by pre-processing the sequence of bytes (Figure (3). first splits the upper four bits and the lower four bits (narrow operations) of byte information; this helps the network capture the locality of closer bytes and distant bytes. Next, the pipeline uses one dimensional convolution to extract such localities and performs the element-wise multiplications of two outputs. Before feeding this information to the neural network, the pipeline uses max- max-pooling to reduce the training time caused by processing inputs with large dimensions, All these heuristics are examined manually (see Section of the original paper); thus, our attacker can save time and effort by stealing the pipeline.
Generate Computational Graphs. The first step of our attacker is reconstruct the computa- tional graph candidates for the victim pipeline from the Flush+Reload trace. As we can see in the trace in Figure B] the attacker cannot simply connect the components in the traces sequentially be- cause the branch connection, e.g., [7] (mu 1 t tipyy) ) Also, from this component, our attacker knows which is the end of branch but cannot know when the branch has started. We solve this problem by populating all possible candidates and pruning them later with the parameter estimation.
Our algorithm populates the candidate computational graphs and the sample candidates found (see Appendix E) Our solution uses a recursive algorithm. Given a trace from Flush+Reload (7), we pop each computation t from the back and construct the list of candidates 1. At a high-level, the algorithm first traverses all the possible connections starting from the last computation to the first by using recursion. Then, when the base condition met (i.e., the algorithm arrives the first com- putation, Embeddings), we backtrack the recursions to construct the list of candidate computational graphs. We focus on the computation type this backtracking process; there are unary and binary computations. For the unary operations, we simply connect the current and preceding computations. However, for the binary operations, we split all the preceding computations into a set of two lists. Each set of two lists corresponds to branch, and continue backtracking for each branch and include all of the construction into our results. At the end, we found 20 candidates.
Eliminate Candidates with Computational Parameters. Next, our attacker further prunes the candidates based on the computational parameter estimation process. Our attacker, most impor- tantly, focuses on the fact that computation time is dependent on the size of the matrix multi- plication. This enables our attacker to profile the computational time taken for a set of param-
MalCony Novel Pre-processing Pipeline
Processed Trace
2000000)
2000000)
1:2g 6820)
[228, 4003|
Embedding.4.0
transpose.2106 1001100,
narrow
Convid
Max Pool
narrow,210754.0
[3] Conv Id,941699,8871999
[128. 1)
narrow,94736200
(0000000,
2000000)
Linear
[S] Conv Id, Id, 10144145..885888
66
Embedding
Transpose
(128, 1)
[7] (multiply),187755 18775738.8
Linear
[8] Mdd.Poolld..883355
[9] transpose, 189.49244,0
20000000
(128, 4000)
1I
[10] L 18949546,96
narrow
Convid
[12] 1994983119
[11] Lincar,
Figure 3: MalCony and the extracted trace. While the MalConv (left) is processing a sample, we extract trace via Flush+Reload and process the trace (right) for the pipeline reconstruction.
7

---

Published as a conference paper at ICLR 2020
Block Size
Counts
Identified Block
AvgPool2d. Linear
2
38
Conv2d BatchNorm2d
19
Conv2d BatchNorm2d ReLU
5
Conv2d BatchNorm2d- Conv2d BatchNormdd
5
17
DepthiConvdd BatchNorm2d ReLU6 Conv2d BatchNorm2d
DepthComv2d- BatchNorm2d Rel.U6 Conv2d BatchNorm2d
0
15
Conv2d - BatchNorm2d- ReLU6- DepthCons2d BatchNorm2d- ReLU6 - Conv2d BatchNorm2d
Conv2d BatchNorm2d ReL.U6 DepthCom22d BatchNorm2d ReLUU Conv2d BatchNorm2d add
Table The 9 candidate blocks identified from the Flush+Reload trace.
eter combinations in advance. The attacker is able to perform this offline by taking advantage of cloud infrastructure: that the hardware and software stacks composing the cloud are consis- profile the timing of the convolution and linear op-
tenf In the MalConv reconstruction, erations. For the convolutions, we consider input/output channels {1,2, 4, 8, 16, 32, 128, 256), kernels {1, 3, 5, 7, 11, 100, 200, 500, 1k, 10k), and strides {1, 2, 5, 10, 100, 200, 500, 1k; 10k). For the linear layers, we use input {4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048} and output dimensions {1, 10, 16, 20, 32, 40, 100, 128, 256, 512, 1k, 1024, 2048).
Once our attacker has the timing profiles with these parameter combinations, the attacker defines the potential parameter sets for the convolutions and linear layers. Then, the attacker checks, in each candidate, if the computational graph returns the correct output dimension (1.) for the input (8, 2000000.. this pruning process, there are the other operations such as Sigmoid, * (multipy), transponse, narrow, or pooling. We applied the universal rules for each case: 1) the Sigmoid and multiply do not change the input/output dimensions, 2) the transpose only swaps two dimensions in input, 3) the narrow slice one chosen dimension, e.g., (8,2000000) to (4, (4,1000000); thus we consider all the possible slice in checking, and 4) the pooling only requires us to estimate its window size, we match this value to the stride of preceding convolution. At the end of this parameter estimation, we can narrow down to only one architecture with the correct set of computational parameters, i.e. 0% error.
4.3 RECONSTRUCTING NOVEL NETWORK ARCHITECTURES
Here, we show our attacker is able to steal novel network architecture by describing the recon- struction process of the ProxylessNAS-CEE architecture, MobileNetV2 (Sandler et al. 2018), , and also reduces the computation time. Indeed, the NAS search procedure warm-start from an over- weer-paaaeterizeed MobileNetV2 as a backbone; however, in attack, we hypothesize our attacker is not aware of the backbone. Instead, sume our attacker only knows the search space of MNasNet (Tan et al.] 2019) (see Appendix D where the authors come up with the MobileNetV2, opposed the recent attacks in Sec
(Cai et al. 2019) that improves the accuracy of existing
Knowing the search space does not, however, reduce the amount of efforts by our attacker in recon- struction. The network architectures found from the NAS procedure commonly are wide and deep, and they include multiple branch connections; thus, our attacker requires to consider exponential number of candidate computational graphs and the computation parameters, which makes the attack infeasible. To tackle this issue, we focus on the NAS procedure-aee -ereeduure-hisss process factorizes the entire architecture into blocks by their functions. For instance, NASNet (Zoph et al. 2018) is composed of normal cells (blocks) and reduction cells. Within each block, the process considers the architecture combinations that provides the optimal performance. Thus, we first identify the potential blocks before we initiate the process for reconstructing candidate computational graphs.
Identifying Candidate Blocks. We utilize the frequent subsequence mining (FSM) method to identify the blocks composing the ProxylessNAS-CPU architecture. Our FSM method is simple: we iterate over the Flush+Reload trace with the fixed windows and count the occurrences of each subsequence. Since the attacker knows that in the search space that the victim uses, maximum of nine computations are used to compose a block, we consider the window size from one to nine. Once we count the number of occurrences for each subsequence (candidate blocks), and we prune
httpst// /ewss www.ss.naaeo anazon .con/ we? cesttanee-ttt pataanne-ttttee typen/
8

---

Published as a conference paper at ICLR 2020
them based the rules in the search space: 1) Conv2d operation is followed by BatchNorm, 2) a block with DepthConv2d must end with Conv2d and BatchNorm (for depthwise separable convolution), 3) branch connection cannot merge (add) in the middle of the block, and 4) we take the most frequent block in each window. In T we describe the 9 identified blocks. We then run the generation process of reconstructing candidate computational graphs with the blocks instead of using each computation in the trace. At the end, we have 1 180,224 candidate computational graphs.
Processed Trace
Identified Block
Estimated Parameters
[8] Comv20d,00569998411.200
[B-8] 1omv2d,1,20
C₁ :24 in, 144 out channels 144 in/out channels
[9] BatchNoorm22132224559660000
[B-8] BatchNorm22.000
[10] ReLU6.37611
[B-8] ReLU6,0,0
[11] DgpphhcCmm2224222222994444413333
[B-8] Depthhonn22,143.
143,200
144 in/out channels
[12] BaichNoomm225555559954.000
[B-8] Bat.hNorm22.000
144 in/out channels
[13] ReLUU6.59233218840000
[B-8] ReLU6,0,0
[14] 1omv26.6095579844..77
[B-8] Conv2d,1,7
144 in, C2:32 out channels C2:32 in/out channels
[15] Bacchoom2246143319660000
[B-8] BatchNorm2d,000
Figure 4: The reconstruction of the ProxylessNAS-CPU architecture. From the Flush+Reload trace (left), we find the candidate block (middle) and estimate the computation parameters (right).
Eliminate Candidate with Computational Parameters. For each candidate composed of known blocks, our attacker estimates the computation parameters. However, the number of parameter com- binations are also exponential; for example, within the search space, a Conv2d can have any number of input/output channels, kernel size {1,3.5 5), and strides (1, 2}. Thus, we focus on the computation only possible have the same input/output
rules in block. 1) We first found that DepthConv2d channels. Also, the channel size can be identified by the number of GEMM(conv) operations. For instance, in Figure 4 the DepthConv2d has 143 GEMM(conV) invocations, which is close to the channel size. Since commonly the operation has reduce the candidates to 142 or 144. 2) We also know that the number of GEMM GEMM(oncopy)) invoca- tions is proportional to the matrix multiplication size in Conv2d; thus, the attacker can compare the offlin profiling results with the processed traces and estimate the parameters. For instance, the 1st Conv2d has 20 GEMM(oncopy), and we approximately have of input dimensions, e.g., (20- 30, 12) from the previous block estimation. Thus, our attacker only profiles the variations of input channels {20 120-30] 30). kernels 31,3,55, 5), and strides (1,2)-tota 60 cases and check if there is match. Moreover, 3) the Conv2d after DepthConv2d is the pointwise linear operation whose kernel and stride one, which will further reduce the attacker's efforts. Our attacker runs this elimina- tion process and finally narrows down to only one architecture with the correct set of computational parameters, i.e., 0% error.
number of channels, the attacker can easily
DISCUSSION
In this section, we discuss defense mechanisms that prevent our attacker from reconstructing the victim's 's DL system with an exact match. Prior work on defenses against cache side-channel attacks proposed system-level solutions (Kim al. 20122] Liu et al. 2016 Werner al.) 2019). However, applying them requires infrastructure-wiie wide changes from cloud providers. Also, even if the infras- tructure is resilient to cache side-channel attacks, an attacker can leverage other attack vectors to leak similar information. Thus, we focus on the defenses that can be implemented in DL frameworks.
We design our defense mechanisms to obfuscate what the attacker observes via cache side-channels by increasing the noise in computations supported by DI. frameworks. We discuss four approaches that blend noise into components of a DI. framework; however, these mechanisms introduce a com- putational overhead by performing additional operations. This highlights that defending against our attack is not trivial and efficient countermeasures require further research.
Padding Zeros to the Matrix Multiplication Operands. Our reconstruction algorithm estimates the computational parameters such as kernel sizes or strides based on the time taken for matrix multiplication. Hence, we consider increasing the size of operands randomly by padding zeros to

---

Published as a conference paper at ICLR 2020
them. We keep the original sizes of the operands and, after the multiplication of augmented tensors, we convert the resulting tensor into that of the correct dimensions by removing the extra elements. With the augmentation, our attacker finds it difficult to reconstruct the victim's 's DL system exactly by monitoring a single query. However, if our attacker can observe computations with multiple queries, the attacker cancel-out the noise and estimate the parameters correctly.
Adding Null/Useless Network Operations. This reconstruction attack assumes all the computa- tions observed in the Flush+RReooad trace are used to compute the output of DL system. Thus, a defender can modify the victim's architecture that it includes the identity layers or the branches whose outputs are not used, We hypothesize a small number of null/useless operations will not in- crease the attacker's computational burden significantly; this addition only increases the time needed to reconstruct the victim's 's architecture by a few hours. If the defender includes an excessive amount of null/useless layers or branches, this can significantly increase the reconstruction time. However, this defense suffers from two issues: 1) the defense may still not make the reconstruction impossi- ble, and 2) the victim also requires perform the additional operations, which increases network evaluation time significantly.
Shuffling the Computation Order: We have seen in popular DL frameworks that, once network architecture is defined, the computational order of performing operations also invariant. We are able to shuffle the computation order of the victim's 's DL system each time when the system processes an input. In particular, we can identify the dependency of operations in victim's 's DI. system and compute the independent operations in different order each time. This approach will make the observations from cache side-channels inconsistent, which results the exponential number of candidate architectures that attacker needs to consider. However, to compute the independent operations separately, the defender needs to store intermediate results in memory while processing an input; thus, this approach increases the space overhead of the DL computations.
Running Decoy Operations in Parallel. Lastly, we can make DL framework run separate net- works (decoy operations) in parallel on the same physical host. These networks obfuscate what our attacker will observe via Rlush+Reload. Here, the attacker cannot reconstruct the victim architec- ture by monitoring a single query because the computational order does not reflect how the victim's 's architecture is defined. However, if our attacker can observe the computations the attacker can use the frequent sequence mining (FSM) that we used in the block identification to identify a repeated set of operations and can reconstruct the victim architecture. This defense also increases network evaluation time by running extra operations on the same machine.
multiple queries,
6 CONCLUSIONS AND FUTURE WORK
This work presents an attack that reconstructs a victim's novel DL system through the information leakage from cache side-channel, Flush+Reload. We steal key components of the victim's 's system: a novel pre-processing pipeline and a novel network architecture. Observing the DL computations and the time complete each computation enables the attacker to populate all candidate computa- tional graphs and prune them with our parameter estimation process. In experiments, we demon- strate the feasibility of this reconstruction attack by reconstructing MalConv, a novel pre-processing pipeline for malicious file detection, and ProxylesSNAS MARCCPs novel architecture for the ImageNet classification optimized to run on CPUs. We do this with 0% error. As novel DI. systems become trade secrets, our results highlight the demands for future work on countermeasures against model theft.
ACKNOWLEDGMENTS
We thank the anonymous reviewers for their valuable feedback. This research was partially sup- ported by the Department of Defense, by NSF grants #CNS-1933033, CENS-1840899 #CNS- 1453045 (CAREER), by research partnership award from Cisco and by financial assistance award 7ONANB 15H328 from the U.S. Department of Commerce, National Institute of Standards and Tech- nology. We would like thank the NSF REU-CAAR program (NSF grant #CCF-1560199
10

---

Published as a conference paper at ICLR 2020
REFERENCES
Martán Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, San- jay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu. and Xiaoqiang Zheng. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposiun on Operating Systems Design and Implementation (OSDI 16), pp. 265 283, Savannah, GA, November 2016, USENIX Association. ISBN 978 9-1-19931977-3-111 URLhttps: /www. usenix. org/ abadi
Zeina Abu-Aisheh, Romain Raveaux, Jven-Yves Ramel, and Patrick Martineau. An exact graph edit distance algorithm for solving pattern recognition problems. 2015.
Adam Bates, Benjamin Mood, Joe Pletcher, Hannah Pruse, Masoud Valafar, and Kevin Butler. Detecting co-residency with active traffic analysis techmiques. In Proceedings of the 2012 ACM Workshop on Cloud Computing Security Workshop, CCSW '2, pp. 1-12. New York, NY, USA, 2012. ACM. ISBN 978-1- 1503-1665-1. doi: 00.1144/228119112231991 URL http:://pio..... acm. /10 111.1113111211131 2381915
Lejla Batina, Shivam Bhasin, Dirmanto Jap, and Stjepan Picek. CSI NN: Reverse engineering of neural network architectures through electromagnetic side channel. In 28th USENIX Security Symposiun (USENIX Security 19). pp. 515 532, Santa Clara, CA. August 2019. USENIX Association. ISBN 978 -9999133-9--- URL wtt.///enneeeeeeeeeeeeeeeeeeeeee
https: www. usenix. /ttp:///teerrreee/eeeeeaaaaattte
Soumith Chintala Sam Gross Adam Paszke Francisco Massa Adam Lerer Gregory Chanan Zeming Lin Edward Yang Alban Desmaison Alykhan Tejani Andreas Kopf James Bradbury Luca Antiga Martin Raison Natalia Gimelshein Sasank Chilamkurthy Trevor Killeen Lu Fang Junjie Bai Benoit Steiner, Zachary DeVito. Py- torch: An imperative style, high-performance deep learning library. In Advances Neural Information Processing Systems 32, 2019.
E. Bosman, K. Razavi, H. Bos, and C. Giuffrida. Dedup est machina: Memory deduplication as an advanced exploitation vector. In 2016 IEEE Symposiun on Security and Privacy (SP), pp. 987-1004, May 2016. doi: 10.1 109/SP.2016.63.
Han Cai, Ligeng Zhu, and Song Han. ProxylesNNAS: Direct neural architecture search on target task and hard- ware. In International Conference on Learning Representations, 2019. URL http::ttt////t /ttpp:://ppenrevie..
net net/forune Set/EoruunIdd-HyLBBB3
Szegedy Christian and Vincent O. Vanhoucke. Processing images using deep neural networks, July 2017. URL Patent 9,715,642.
https::/pattets: wwwpgle..ommppatet////eetee
Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita- Rotaru, and Fabio Roli. Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. In 28th USENIX Security Symposium (USENIX Security /9), pp. 321-338, Santa Clara, CA. August 2019. USENIX Association. ISBN 978-1-939133-06-9 URL Intpps: : www.usenix. org htt///onfeeencc
htt.eesssttattiooo/mmmttiiii demontis
Vasisht Duddu, Debasis Samanta, D. Vijay Rao, and Valentina E. Balas. Stealing neural networks via timing side channels. CoRR, abs/1812.11120., 2018. / waapgg//bb/181/ 11720
Kazushige Goto and Robert A. van de Geijn. Anatomy high-performance matrix multiplication. ACM Trans. Math, Softw. 34(3):12:1 12:25, May 2008. ISSN 3500, doi: 10. 1411111350022.15500055 URL http:/ //tp:///cc..io acn. 11tp//0001115511512222222 1366053
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In The IEEE Conference on Computer Vision and Puttern Recognition (CVPR), June 2016.
Sanghyun Hong, Michael Davinroy. Yigitcan Kaya, Stuart Nevans Locke, Ian Rackow, Kevin Kulda, Dana Bechmman-Soled and Tudor Dumitras. Security analysis of deep neural networks operating in the presence of cache side-channel attacks. CoRR, abs/1810.,3387, 2018. URL
03487
Weizhe Hua, Zhiru Zhang, and G. Edward Suh. Reverse engineering convolutional neural networks through side-channel information leaks. In Proceedings of the 55th Annual Automatttooooionn Automation Conference, DAC pp. 4:1-4:6, New York, NY, USA, 2018 ACM. ISBN 978- 1 -500-570-5. doi: 10.1145/3195977.31966 URL U hott::///oo acm org/ 10. 95 95970.997770

---

Published as a conference paper at ICLR 2020
Taesoo Kim, Marcus Peinado, and Gloria Mainar-Ruiz STEALTHMMEM: System-level protec- tion against cache-based side channel attacks cloud. In Presented as part of the 21st
USENIX Security Symposium (USENIX Security 12), pp. 189 204, Bellevue, WA, 2012. USENIX.
ISBN 978-931177--95- URLptttp:://wwt usenix http://ooo.oooooo//////oooooooo. technical sess sessi ons
T. Kohno, A. Broido, and K. C. Claffy. Remote physical device fingerprinting. IEEE Transactions on Depend- able and Secure Computing, 2(2):93-108, April 2005, doi: 10.1 00.1109/DDC.2205.22
E. Liu, Y. Yarom, Q. Ge, G. Heiser, and R. B. Lee. Last-level cache side-channel attacks are practical. In 2015 IEEE Symposium on Security and Privacy, pp. 605-622, May 2015, doi: 10.1 1.91S....2011
F. Liu, Q. Ge, Y. Yarom, E. Mckeen, C. Rozas, G. Heiser, and R. B. Lee. Catalyst: Defeating last-level cache side channel attacks in cloud computing. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 406-415 March 2016. doi: 10.1 200////A..211...44666660
Edward Raff, Jon Barker, Jared Sylvester, Robert Brandon, Bryan Catanzaro, and Charles K. Nicholas. Mal- detection by eating whole In Workshops the Seiony- AAAI Conference Artificial Intelligence, 2018
Thomas Ristenpart, Eran Tromer, Hovav Shacham, and Stefan Savage. Hey, you, get off of my cloud: Ex- ploring information leakage in third-party compute clouds. In Proceedings the 16th ACM Conference on Computer and Communications Security, CCS '09, pp. 199-212, New York, NY, USA, 2009. ACM. URL http://tpt/o acm. org. 10////10.100 10.1111111 11457
ISBN 978- -60558-894-0. doi: 10. 10.1145/163362216652220 1653687.
1653662. 653687
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chich Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In The IEEE Conference Computer Vision and Pattern Recog- nition (CVPR), June 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. In International Conference Learning Reprexentations, 2015. URL http:://axii....tt abs 1409. 1556
David So. Quoc Le, and Chen Liang. The evolved transformer. In Kamalika Chaudhuri and Ruslan Salakhut- dinov (cds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Pro- ceedings of Machine Learning Research, pp. 5877-5886 Long Beach, California, USA, 09-15 Jun 2019. PMLR. URL htt.tttp:
ml r wwwppspweeeeee.p st///sllaaa html
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quac L.c. Mnasnet: Platform-aware neural architecture search for mobile. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Florian Tramèr, Fan Zhang, Ari Juels, Michael Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In 25th USENIX Security Symposium (USENIX Security 16), Austin, TX, August 2016. USENIX Association. URL URL/ttts:://///
hattp:://ww/ wsw.ix wwg. coneeeenneeeece//// usealxsecurityy/teemee www.eaaaaasssssssspeeeeeeee htt.a///wws.oeeeeeeeeaaaaaaaaaaa
Venkatanathan Varadarajan, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. A place- ment vulnerability study in multi-tenant public clouds. In 24th USENIX Security Symposium (USENIX Security 15), pp. 913-928, Washington, D.C., Aug 2015. USENIX Association. ISBN
978-19931977-233. URL huttps://wwe uswnix.cccc......o iereenix.oo.... wwt.eorr/ccooo http://awo....oooooooooooo////ea htt.s//esssssssseeeeeeeaaattoooa htt.s//aenaaattooooaaaraarraaaaa
Yan Wang, Wci-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Camphell, and Kilian Q. Weinberger. Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Mario Werner, Thomas Unterluggauer, Lukas Giner, Michael Schwarz, Daniel Gruss, and Stefan Mangard. Scattereache: Thwarting cache attacks via cache randomization. In 28th USENIX Security Sympo- siam (USENIX Security 19), pp. 675-692. Santa Clara, CA. August 2019. USENIX Association. ISBN 978- 19881-99913--- URL
presentaticn/wee wwr.errwweeeerreee
Qixue Xiao, Yufei Chen, Chao Shen, Yu Chen, and Kang Li. Seeing is not believing: Camouflage attacks on image scaling algorithms. In 28th USENIX Security Symposium (USENIX Security 19), pp. 443 460, Santa Clara, CA, August 2019. USENIX Association. ISBN 978-1-929133-06-9 URL https: //www. usenix htwpni/..rgg/eeeeeee/ee/eeeeeeee www.eeeeeeeeennnneeeeeeee aarane//seemmxoorrrrtyoy//eeeeee
cen
12

---

Published as a conference paper at ICLR 2020
Mengjia Yan, Christopher W. Fletcher, and Josep Torrlllss. Cache telepathy: Leveraging shared resource attacks learn DNN architectures. CoRR, abs/1808.04761, 2018. URL http://arx...og//aabb.
1808.04761
Yuval Yarom. Mastik: A micro- aecro-aachitettualll side-channel toolkit. Reirieved.from School of Computer Science Adelaide: http://cs. adelaide. edu. auf" yval/Mastik, 16, 2016.
Yuval Yarom and Katrina Falkner. Flush+reload: high resolution, low noise, cache side-channel attack. In 23rd USENIX Security Symposium (USENIX Security 14), pp. 719 732, San Diego, CA. August 2014. USENIX Association, ISBN 978-1-931971-1577 URL https: www. usenix x org/ conference/
usen wwweeurity14/teeeee
ca sess on / in
Y. Zhang, A. Juels, A. Oprea, and M. K. Reiter. Homealone: Co-residency detection in the cloud via side- channel analysis. In 2011 IEEE Symposiun on Security and Privacy, pp. 313-328, May 2011. doi: 10.1109/ SP.201
Barret Zoph and Quoc V L... Neural architecture search with reinforcement learning. arXiv preprint 161.v//61..0157. 2016.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. Learnnng transferable architectures for scalable image recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
13

---

Published as a conference paper at ICLR 2020
A APPLICABILITY TO GPUs
Our attack is not fundamentally different for GPUs. In most deep learning frameworks, when a network performs a computation, it invokes the same function implemented in C++ and the func- tion decides whether the back-end computation can use GPUs or not. This practice] maximizzs the hardware compatibility of a framework; however, this also makes the framework vulnerable to our attacker who can still observe the common functions listed in Table by by monitoring the shared cache. On GPUs the timings would be different, so we would have to profile the computational times, e.g., the time taken for the matrix multiplication with various sizes of tensor operands. How- ever, on both CPUs and GPUs, the computation time is proportional to the size of tensor operands, which enables our attacker to estimate the architecture parameters with timing observations.
B DL COMPUTATIONS MONITORED IN PYTORCH AND TENSORFI.OW
Input
Conv
Processed Trace (PyTorch)
Processed Trace (TensorFlow)
Batch
[0] Conv2d, 52230076, (1 and 2)
[0] Conv2d, 4717917, (20 and 15)
Depii wice Cony
[1] BatchNorm2d, 54230076,
[1] Narmhd, 4725816. 0
Reish Nom
[3] Conv2d, 55600076, (10 and 10)
[2] DepthwiseConv, 4725912, 0
[4] BatchNorm2d, 59268076, 0
[3] BatchNorm22, 4726228, O
Rotu5
[5] ReLUU6, 59880076, 0
[4] ReIJU6, 4727308, 0
[6] add, 60270076, 0
[S] TensorAdd, 4727644,
Ouiput
Figure 5: The traces extracted from PyTorch and TensorFlow via Flush+Reload. We illustrate the architecture of a small network (ToyNet) on the left. The sequence of computations observed from the our attack are listed in the middle (Py and the right (TensorFlow).
descriees the reconstruction process of a small network in both the PyTorch and TensorFlow frameworks. On the left, we have the ground truth of the ToyNet architecture, which represents an example of possible residual block in a victim network. In the middle and right, we show the observations of an adversary monitoring both PyTorch and TensorFlow code. The first entry indicates the monitored function corresponding to the desired architectural attribute. The second entry indicates the timestamp at which the adversary observes these functions, and the last entry is the number of general matrix multiplication (GEMM) function calls for the given layer observation.
Naming conventions vary slightly between the two frameworks, but the information inferred is the same. The adversary attacking both networks sees functions calls that correspond to architectural at- tributes in the same order: Conv2d, BatchNorm22,
BatchNorm2d, ReLU6, and TensorAdd. PyTorch does not distinguish between Conv2d and DepthwiseConv, but as stated in we can differentiate the layers by timing data. Additionally, Py PyTorch and TensorFlow use dif- ferent linear algebra libraries to perform matrix computation, so the implementations differ slightly. However, they both use variations on matrix multiplication algorithms that take into account system level optimizations, such as cache size (e.g. Goto's algorithm). In both cases, we observe operations in nested iterations of these implementations and are able to monitor instructions that correspond to the size of the matrices being multiplied, giving an adversary the ability to estimate the parameters of the convolution layers.
4.1
To perform the estimations of these layer parameters, the adversary can profile candidates offline on similar hardware. They can then create data set of candidate parameters for given observation ranges. For instance, the number of observed GEMM calls in the PyTorch example for the depthwise convolution layer gives the attacker the information that there 10 output channels, and therefore also ten output channels in the 1st convolution. Additionally, the observed GEMM calls for the 1st convolution layer give the candidate kernel sizes of and 5. Likewise in TensorFlow, the observed instructions fits the candidate kernel sizes of 3 or 5, and 0-24 output channels. Therefore, these
Customization of operations in TensorFlow:
het //woe, www.wwttwttott..ooooo... El oow..ogg///... /ggi /wwee/etetee... .pp
14

---

Published as a conference paper at ICLR 2020
Framework
Computations
Line of Code
Conv2d
Convld
BatchNorm
FC (Begin)
FC (End)
ReLU6
Sigmoid
Py Poroch
Core (C++)
AvgPool MaxPool Embeddings
aeerfscc/Ateee/nniiii//ooooogoop
add
(multiply)
transpose
narrow
GEMM(conv)
driver/lovel3//ovel3.e
OpenBLAS
GEMM(oncopy)
kerne//x86..44/ssemmm ncopy 4.kylake..c.
Conv2d
corc/Kemels/comv 2d.h:81
BatchNorm
core/kemelw/fused. hawpluueedd_ahhhhooooooo 2awptcchnommoo.cc22222..
DepthConv2d
core/kernels/eeptiiisee http:///ddwpwii..ooeoooooooee www.e/oovvooopccoe _oo.voop.cc:661......
Core (C++)
FC
core/kemees//matnuu.opp .pp.c:542
ReLU6
core/Kereels/relu. op functor.h:68
TensorFlow
AvgPool
conc//cmels/awgpooling op.cc:0000
TensorAdd
Eigen
#FCNodes
GEMM
stc/epu/gemm//gemm ddr.emr.mriiieepp2222222222
(loop)M-outer
sedcpu/ggmm//gemm http:////emmmm.dv..pp...... driver.epp:349
MKL-DNN
(loop)
stc/cpu/gemm/gentt d.tprmm..rvvieep:33555
(loop)N
src/cpu/gemm/geee driver.epp:372
(loop)M-innee
crc/cpu/gcmm/gecme driver.cpp:3887
Table 3: Monitored lines of code in PyToorch and TensorFlow.
exploitable vulnerabilities exist independent of the specific deep learning framework a victim is using.
C LIST OF FUNCTIONS MONITORED VIA FLUSH+RELOAD
Table 3 shows the exact lines of code monitor in the PyTorch and TensorFlow frameworks. We use Py Torch v v1.2.qJand Tensorflow v v1.14.( In both the frameworks, we are able to monitor a similar of DL computations the C++ native implementations. However, the back-end libraries sup- porting the matrix multiplications are different, i.c., PyTorch is compiled with OpenBLAS whereas TensorFlow uses Eigen and MKL-DNN. Even if the libraries are different, the multiplications are implemented using GOTO's algorithm (Goto & Geijn 2008). Therefore, we monitor the number of iterations of for-loops estimate the overall size of matrix multiplication.
D MNASNET SEARCH SPACE
Tan et al. (2019) utilize a hierarchical search space over six parameters: ConvOp, KernelSize, SER- atio, SkipOp. FilterSize, and #Layers. They choose to partition a CNN into known, finite set of blocks and then further divide these blocks into possibly repeated layers. The number of repeats per layer in given block i is searchable parameter Nr. which is bounded at + the number of layers in MobileNetV2 on which block is based. These layers are further divided into three possible
hL de Li coand pow.occhhheoo.....o......ooo.cce oruch (pylured Aurmmi /855441488 1o 201:22339
hteps:/ won beneorf www./teneeeeeeeeeeee cww/tnnart www/comnit / (8w19/9999999776666666666
Sceee 2
15

---

Published as a conference paper at ICLR 2020
network layers (ConvOp): regular convolution, depthwise convolution, or mobile inverted bottle- neck convolution. Additionally, the network layer parameters can vary. These parameters include the convolution kernel size (KemnelSize), the squeeze-and-exeee sudecze-add-xxcitation ratio (SERatio), possible skip op (SkipOp), and the output filter size (FilterSize). The squeeze-and-excitatio ration (SERatio) of given layer varies between and 0.025; the convolution kernel size varies between and 5; the skip op is either pooling, identity residual, or no skip; and the filter size varies between 0.75, 1.0, and 1.25 the filter size of the corresponding block in MobileNetV2. Overall, this gives an claimed typical search space size of 10 possibilities with 5 blocks, 3 average layers per block, and 432 options for the sub search space of each block. This size compares to the per-layer approach with the same parameters that has a search space size 10
E SEARCHING CANDIDATE COMPUTATIONAL GRAPHS
Generated Candidates
Algorithm Populate computational graphs
1: procedure POPULATEGRRAPHS(T)
2:
t.= T-pppp)
3:
if t empty then
4:
g CreateAGraphiC
Embedding
Transpose
5:
[ = Candidate
6:
1= 1 9
7:
return
narrow
Corvid
8:
else
9:
if / is unary operator then 1 PopmlateGraphos( for g in do Create Andget, end for return
11:
12:
13:
14:
Embedding
Transpose
else if is binary operator then
16:
for each preceding element e in T do
TI, Tr = Splil ()
nartow
Convid
18:
11 PopulateGraapss(T) PopulateGiraphs(T1) for each gl. gr of I1 X Ir do
20:
g Compose(el, er)
22:
1 - uolug g
narrow
Convid
23:
end for end for return end if
24:
25:
Embedding
Transpose
26:
27:
end
28:
end procedure
narrow
Convid
Figure 6: The algorithm for searching candidate computational graphs. We describe our algo- rithm to populate the candidate graphs of MalConv (left) and the sample candidates (right).
16


Reviews:
[{"review_id": "rkxi9slCKr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a way to attack and reconstruct a victim's neural architecture that is co-located on the same host. They do it through cache side-channel leakage and use Flush+Reload to extract the trace of victim's function call, which tells specific network operations. To recover the computational graph, they use the approximate time each operation takes to prune out any incompatible candidate computation graph. They show that they can reconstruct exactly the MalConv and ProxylessNAS. \n\nThe paper looks very interesting but also alarming -- more research should be done to countermeasure this attack. I have the following questions: \n\n1. To reconstruct the network, you need to generate potentially exponentially number of candidates and do some pruning based on the estimated parameters. This also looks very expensive. I am wondering compared to just doing NAS yourself, how much gain in terms of resources and time this attack can give? \n\n2. What is the limitation of the proposed approach, i.e., does it work on any network structures, e.g., sequence networks, graph convolutional networks, etc. \n\n3. In the experiments shown, you can reconstruct MalConv and ProxylessNAS with zero error, does the proposed approach alway find the exact match? Under what circumstances can you find the exact match?", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "rJgGcD4RtS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposed a method to reconstruct machine learning pipelines and network architectures using cache side-channel attack. It is based on a previous proposed method Flush+Reload that generates the raw trace of function calls. Then the authors applied several techniques to rebuild the computational graph from the raw traces. The proposed method is used to reconstruct MalConv which is a data pre-processing pipeline for malware detection and ProxyLessNas which is a network architecture obtained by NAS. \n\nOverall, the paper is well-written and easy to read. The problem of stealing machine learning pipelines/architectures is interesting and important, since it enables an attacker to actually know the private networks that are being used for prediction. Therefore, I think this is a promising direction for future work.\n\nI hope the authors can address my concerns as follows:\n\nQ1: What is the knowledge of the attacker? The authors should be explicit in summarizing the detailed search space of the attacker. Currently i found it very hard to understand the capability of attacker. This is important in evaluating this work.\n\nQ2: Can the authors add some discussion on how to defend against the proposed attack? For example, one can add some null/useless operation during execution to make the reconstruction process harder? \n\nQ3: I am curious why the authors choose ProxylessNAS-CPU for evaluation. There is a bunch of other architectures found by NAS, e.g. MNas, ENas?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SkgYKSmAtS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n---\nThis paper proposes to use a computer security method, \"Flush+Reload\" to infer the DNN architecture of a victim in the setting where both the attacker and the victim share the same machine in a cloud computing scenario. This does not require any physical access to the machine, however it does require that a CPU is shared, and the inference of the architectural details is based on the time it takes to reload computations from cache. \nThe paper is overall clear and well written.\n\nMotivations of the paper\n---\nHowever, concerning the motivations of the paper, I'd like some clarifications. As far as I know, in the deep learning community, the most effective architectures are published and public (VGG, Inception, ResNet, Transformer...).\nI am a bit confused by the sentence \"As a result, in the industry such novel DL systems are kept as trade secrets or\nintellectual property as they give their owners a competitive edge (Christian & Vanhoucke, 2017).\" which justifies that architectures are kept secret and thus may be prone being stolen.\nThis US patent is public and explains the method. As far as I know, it has never been enforced. Furthermore, this patent is associated with the paper \"Going deep with convolutions\", Szegedy et al. which introduced the Inception architecture, is public, very well-known, and thus I do not believe anyone would have any commercial interest in stealing it. \nFurthermore, I do have the impression that the edge many companies have over their competitors is the private datasets they own much more than the architectural details.\n\nMethod and applicability\n---\nWhile the method Flush+Reload itself is not novel, its application to the DNNs case and the way to reconstruct the architecture (generating the candidates, pruning) is.\nHowever, I do have some practical concerns about the applicability of the method.\n\nAs far as I understand, it can only work on one CPU. Most DNNs, even for inference, are run on (one or multiple) GPUs. Can the method be extended to work on GPUs?\n\nAlso, while the assumption that both the attacker and the victim use the same framework is realistic to me, I believe, they should also both use the same version of the said library, no? Otherwise some operations might be faster in some versions and slower in others, this is thus an additional and much stronger assumption to make.\n\nAt last, this would require the victim to use a public cloud service. However, as far as I know, many of the companies who could potentially design new architectures have their own private cloud. I am not certain that someone disposing of a new, private, and powerful architecture would use it on a public cloud service. \n\nExperiments\n---\nThe experimental section seems very limited to me. The authors show that they are able to reconstruct perfectly 2 architectures. While this is encouraging, I would like to see the limits of the proposed method.\nWhy not generate N random (or not so random) architectures and try to reconstruct them? Where does the method fail, where does it succeed?\nWhat if the victim used a custom layer that the method could not recover? Does it still recover a similar architecture?\n\n\nConclusion\n---\nWhile the paper, proposed to use Flush+Reload for recovering DNNs architectures and succeeds for at least 2 non trivial architectures, I do not recommend acceptance.\nFirst I am concerned by the problem this paper is tackling. Can this realistically happen in a real-life scenario?\nSecond, I am worried that the method suffers from very strong limitations in practice (eg the usage of a CPU for both victim and attacker).\nFinally, and importantly, while the experiments show some interesting first results, they are limited, I am not able to judge the strengths and weaknesses of the method, and thus I cannot assess the usefulness of the proposed method.\n\nNote: I have to say that this paper is definitely out of my area of expertise, even though I am confident in my understanding of the paper, it may be that some of my concerns are unfounded. If this is the case I will adjust my score accordingly.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes using Flush+Reload to infer the deep network architecture of another program, when the two programs are running on the same machine (as in cloud computing or similar). 

There is some disagreement about this paper; the approach is thoughtful and well executed, but one reviewer had concerns about its applicability and realism. Upon reading the author's rebuttal I believe these to be largely addressed, or at least as realistically as one can in a single paper. Therefore I recommend acceptance.
Decision: Accept (Poster)