Can one hear the shape of a neural network?: Snooping the GPU via Magnetic Side Channel
Authors: Henrique Teles Maia
Chang Xiao
Dingzeyu Li
Eitan Grinspun
Changxi Zheng
Venue:  (2021)
Keywords: side channel
model extraction
GPU
magnetic induction
sensors

Abstract: We  examine  the  magnetic  flux  emanating  from  a  graphics  processing  unit’s (GPU’s) power cable, as acquired by a cheap $3 induction sensor, and find that this signal betrays the detailed topology and hyperparameters of a black-box neural network model.  The attack acquires the magnetic signal for one query with unknown input values, but known input dimension and batch size. The reconstruction is possible due to the modular layer sequence in which deep neural networks are evaluated. We find that each layer component’s evaluation produces an identifiable magnetic signal signature, from which layer topology, width, function type, and sequence order can be inferred using a suitably trained classifier and an optimization based on integer programming.  We study the extent to which network specifications can be recovered, and consider metrics for comparing network similarity. We demonstrate the potential accuracy of this side channel attack in recovering the details for a broad range of network architectures including also random designs.  We consider applications that may exploit this novel side channel exposure, such as adversarial transfer attacks. In response, we discuss countermeasures to protect against our method and other similar snooping techniques.

Under review as a conference paper at ICLR 2021
CAN ONE HEAR THE SHAPE OF A NEURAL NETWORK?: SNOOPING THE GPU VIA MAGNETIC SIDE CHANNEL
Anonymous authors Paper under double-blind review
ABSTRACT
We examine the magnetic flux emanating from a graphics processing unit's (GPU) power cable, as acquired by a cheap $3 induction sensor, and find that this signal betrays the detailed topology and hyperparameters of a black-box neural network model. The attack acquires the magnetic signal for one query with unknown input values, but known input dimension and batch size. The network reconstruction is possible due to the modular layer sequence in which deep neural networks are evaluated. We find that each layer component's evaluation produces an identifiable magnetic signal signature, from which layer topology. width, function type, and sequence order can be inferred using a suitably trained classifier and an optimiza- tion based integer programming. We study the extent which network specifi- cations can be recovered, and consider metrics for comparing network similarity. We demonstrate the potential accuracy of this side channel attack in recovering the details for broad range of network architectures, including random designs, We consider applications that may exploit this novel side channel exposure, such as adversarial transfer attacks. In response, we discuss countermeasures to protect against our method and other similar snooping techniques.
INTRODUCTION
The Graphics Processing Unit (GPU) is favored vehicle for executing neural network. As it com- putes, it also hums-electree elecrromaegaeet What can this hum tell us? Could listening to the GPU's electromagnetic (EM) radiation reveal details about the neural network? We study this question and find that magnetic induction sensing reveals detailed network structure, including both topology and hyperparameter values, from inferences of otherwise unknown networks running on GPUs.
Reverse engineering a network structure has attracted increasing research effort, motivated by sev- eral concerns. First, it has been well known that the performance of a network model hinges on its judiciously designed structure-bur finding an effective design is no easy task. Significant time and energy is expended in searching and fine-tuning network structures (Zoph al. 2018). Moreover, in industry, optimized network structures are often considered confidential intellectual property. It therefore important to understand the extent to which this valuable, privileged information can be compromised.
Worse yet, reverse engineered "surrogate" model also makes the black-box "victim" model more susceptible to adversarial transfer attacks (Papernot et al.) 2017; Liu et al., 2016), in which vul- nerability identified in the surrogate is exploited on the victim. Success in the exploit is contingent on the ability of the surrogate to successfully model the vulnerabilities of the victim. Recovering accurate, detailed network topology and hyperparameters informs the modeling of a good surrogate.
We examine the fluctuation of magnetic flux from the GPU's power cable, and ask whether a passive observer can glean the information needed to reconstruct neural network structure, Remarkably, we show that, through magnetic induction sensing, passive observer can reconstruct the complete network structure even for large and deep networks.
Threat model. We consider an adversary that (i) is able to place a magnetic induction sensor in close proximity to the GPU's power cable, (ii) knows the dimension of the input feature vector, and (iii) is able to launch query of known batch size. We also consider that our attacker uses the same deep learning framework (e.g.. Py Torch, TensorFlow) as the black-box model. The adversary is otherwise weak, lacking access to the model source, binaries, training data, and underlying training data distribution; without ability to execute code on the host CPU and GPU; and without knowledge of the input values and output results of the launched queries. Not only that-it also lacks direct

---

Under review as a conference paper at ICLR 2021
55.
4.5
4.0
3.5
3.0
2.5
5
10 15 20 25 ms
D0FY retu
coIV BAg GO en
Figure I: Leaked magnetic signal. (left) Our induction sensor captures a magnetic signal when a CNN is running on the GPU. We observe strong correlation between the signal and the network steps. Across two steps, the GPU has to synchronize, resulting sharp drop of the signal level (highlighted by selected red circles). (right) We can accurately classify the network steps and re- construct the topology, as indicated by the labels under the æ-axis. Here we highlight the signal regions associated with convolutions (conv), batch-norm (BN), Relu non-linear activations (relu), max-pooling (MP), and adding steps together (add).
access to the GPU hardware, beyond the proximity to the power cable. The adversary only requires access to their own GPU hardware, matching the brand/version the victim, c.g., as purchased on the open market.
Physical principle. The GPU consumes energy variable rate that depends on operations per- formed. Every microprocessor instruction is driven by transistor electron flows, and different in- structions require different power levels (Grochowski cores of GPU amplify the fluctuation in energy consumption, and so too the current drawn from the power cable. Current induces magnetic flux governed by the Biot-Savart law (Griffiths) 2005), and current fluctuations induce EM ripples whose propagation through the environment governed by the Ampère-Maxe Maxwell law. Even a cheap, $3 magnetic induction sensor (see Fig. placed within few millimeters of the power cable suffices to record these EM ripples.
Annavaram, 2006). The many compute
Technique and results. To reconstruct the black-box network's structure, we propose two-step approach. First, we estimate the network topology, such as the number and types of layers, and types activation functions, using a suitably trained neural network classifier. Then, for each layer, estimate its hyperparameters using another set of deep neural network (DNN) models. The in- dividually estimated hyperparameters are then jointly optimized by solving an integer programming problem enforce consistency between the layers. We demonstrate the potential accuracy of this side-channel attack in recovering the details for wide range of networks, including large, deep networks such ResNet101. We further apply this recovery approach to demonstrate black-box adversarial transfer attacks.
1.1 RELATED WORK: MODEL EXTRACTION QUERIES AND SIDE-CHANNER ANALYSIS
Our work falls under the umbrella of black-box model extraction. Absent access to the model's internals, one might infer structure from observed input-output pairs. For instance, Tramèr et al. (2016) demonstrated that, for simple models such as decision trees and support vector machines hosted a cloud, certain internal information can be extracted via multitude of queries. This approach, which was extended to infer details deep neural networks (Oh et al. 2019 Liu et al,, 2016 Duddu & Rao 2019), is typically able to recover certain information, such optimization learning rate and network structure type, but has not demonstrated recovery of full structural details.
A contrasting approach, side-channel analysis (SCA), extracts information gained from the physical implementation of model, rather than the mathematical model itself. Analysis sis of timing (Kocher 1996), power (Kocher et al. 1999 Luo et al. 2015). cache flushes (Yarom & Falkner 2014). and audio (Genkin et 20144) have been prominently demonstrated extract secret keys from crypto- graphic procedures such as the Digital Signature and Advanced Encryption Standards.
SCA was recently used to infer machine learning models by observing power consumption pro- files (Xiang et al. 2020; Wei et al., [2088] Dubey et al.] 2019), timing information (Duddu et al, 2018) and memory/cache access (Hu et al. 20200 Hong et al. 2019 Hua al. 2018 Yan et al. 2020 These methods placed a malware process on the machine hosting the black- box model. Our
threat model does not involve introducing processes the host.
2

---

Under review as a conference paper at ICLR 2021
Recently, |Batina et al. (2019) exploited EM radiation for network model extraction. They focused on EM radiation from embedded processors. In contrast to GPUs, embedded processors emit a relatively weak EM signal, necessitating delicate measurement devices and even mechanical opening of the chip package.
Our advance. All these previous works are demonstrated on shallow networks (c.g., fewer than 20 layers). It remains unclear whether these methods can also extract deep network models, ones that are structurally more complex and more prevalent in practice. We demonstrate successful recovery of the full structure of deep networks, such as RetNet101 (He et al.] (016)). With that, we hope to raise awareness of the GPU's EM radiation as an information-rich, easily-probed side channel.
MAGNETIC SIGNALS FROM GPUS
Before getting into the weeds, let us provide our intuition: we think of the magnetic signal as the GPU's "speech." The GPU speaks series of "words,"" demarcated by silence. Each word names the computational step that was executed. Let us now refine this explanation further, and ground in physical principles.
We use step to refer performing specific kind of network operation, such linear operation, batch normalization, pooling, activation function, etc. A layer is a sequences of steps, e.g., (i) lin- car operation, then (ii) pooling, then (iii) activation. While there may be data dependencies between steps, there are no such dependencies within a step.
The parallel nature of GPU computation lends itself to natural implementation of networks, wherein each step is executed in parallel, i... single instruction multiple data (SIMD) parallelism. Transitions between steps, however, are synchronized (Buck 2007): in our example above, activa- tion begins only after pooling completes. This cross-step synchronization allows for implementa- tions structured into modules, or GPU kernels. This modular approach is employed in widely-used deep learning frameworks such as Py PyTorch and TensorFlow (Paszke et al.) 2019 Abadi et al. 2016).
Signal. Kernel execution demands transistor flips, which place electric load on the GPU processor, in turn emitting magnetic flux from its power cable. An induction sensor measures this flux and produces proportional voltage. The time-varying voltage is our acquired signal (see Fig. D
Different steps correspond to different GPU kernels, transistor flips, electric loads, and signal char- acteristics, which are distinguished even by the naked eye (see Fig. []. Cross-step synchronization involves idling, dramatically reducing electric load and signal level (see Fig. [T. These rapid sharp drops demarcate steps.
We observe that signal strongly correlates to the kind of GPU operations, rather than the specific values of computed floating point numbers. We verify this by examining signals using both PyTorch and TensorFlow and on multiple kinds of GPUs (see Scc.
The signal is also affected by the input to the network. Although the specific input data values do not influence the signal, the input data size does. When the GPU launches network, the size of its single input (e.g., image resolution) is fixed. But the network may be provided with a batch of input data (e.g., multiple images). As the batch size increases, more GPU cores will be utilized in each step. The GPU consequently draws more power, which in turn strengthens the signal. Once all GPU cores are involved, further increase of input batch size will not increase the signal strength, but elongate the execution time until the GPU runs out of memory.
Therefore, in launching query to the black-box network model, the adversary should choose batch size sufficiently large to activate a sufficient number of GPU cores to produce a sufficient signal-to-noise ratio. We find that the range of the proper batch sizes is often relatively large (c.g., 64 ~ 96 for ImageNet networks). loosely depending on the size of the single input's 's features and the parallel computing ability of the GPU. In practice, the adversary can choose the batch size by experimenting with their own GPUs under various image resolutions.
Notably however, we do not require knowledge of batch size to robustly recover network topology (as opposed to hyperparameters), only that the batch size is sufficiently large enough to provide clear signal. While used consumer friendly sensor with limited sampling rate (see 4.1 and corresponding signal-to-nois to-ise ratio (SNR), a sensor with high sampling rate and SNR would correspondingly require smaller minimum batch size.
3

---

Under review as a conference paper at ICLR 2021
3 SIGNAL ANALYSIS AND NETWORK RECONSTRUCTION
We prepare for the attack by training several recovery DNN models: we refer to training before the attack as pretraining. After the attacker launches a batch query (whose input and output values are irrelevant) we recover structure from the acquired signal in two stages: (i) topology and (ii) hyperparameters. To recover topology, pretrained DNN model associates a step to every signal sample. This per-sample classification partitions the signal into segments corresponding to steps. We estimate hyperparameters for each individual segment in isolation, using a step-specific pre- trained DNN model, and resolve inconsistencies between consecutive segments using an integer program. The pretraining of our recovery DNN models is bardware,specific, and good recovery requires pretraining on like hardware.
3.1 TOPOLOGY RECOVERY
Bidirectional Long Short Term Memory (biLSTM) networks are well-suited for processing time- series signals (Graves et al. 2005). We train biLSTM network to classify each signal sample 8; predicting the step ((s)) that generated s; (see Fig. b)). The training dataset consists of annotated signals constructed automatically (see Sec. 4.2p We train the biLSTM by minimizing the standard cross-entropy loss between the predicted per-sample labels and the ground-truth labels (see Appx. Appx.[A] for details). By identifying the sequence of steps, we recovered the layers of the network, including their type (e.g., fully connected, convolution, recurrent, etc.), activation function, and any subsequent forms of pooling or batch normalization. What remains to recover layer hyperparameters.
3.2 HYPERPARAMETER ESTIMATION
Hyperparameter consistency. The number of hyperparameters that describe a layer type depends on its linear step. For instance, a CNN layer type's linear step is described by size, padding, kernel size, number of channels, and stride hyperparameters. Hyperparameters within a layer must be intra-consistent. Of the six CNN hyperparameters (stride, padding, dilation, input, output, and kernel size), any one is determined by the other five. Hyperparameters must also be inter-consistent across consecutive layers: the output of one layer must fit the input of the next. A brute-force search of consistent hyperparameters easily becomes intractable for deeper networks; we therefore first estimate hyperparameters for each layer in isolation, and then jointly optimize to obtain consistency.
Initial estimation. We estimate specific hyperparameter of specific layer type, by pretraining a DNN. We pretrain a suite of such DNNs, one for each (layer type, hyperparameter) pairing. Once the layers (and their types) are recovered, estimate each hyperparameter using these pretrained (layer type, hyperparameter) recovery DNNs.
Each DNN is comprised of two 1024-node fully connected layers with dropout. The DNN accepts two (concatenated) feature vectors describing two signal segments: the linear step and immedi- ately subsequent step. That subsequent step (e.g., activation, pooling, batch normalization) tends to require effort proportional to the linear step's ss output dimensions, thus its inclusion informs the esti- mated output dimension. Each segment's feature vector is assembled by (i) partitioning the segment uniformly into N windows, and computing the average value of each window, (ii) concatenating the time duration of the segment. The concatenated feature vector has a length of 2N + 2.
The DNN is trained with our automatically generated dataset (see Sec. 4.2). The choice of loss function depends on the hyperparameter type: For a hyperparameter drawn from a wide range, such as a size, we minimize mean squared eитоr between the predicted size and the ground truth (i.c., regression). For hyperparameter drawn from a small discrete distribution, such as stride, we mini- mize the cross-entropy loss between the predicted value and the ground truth (i.e., classification). In particular, we used regression for sizes, and classification for all other parameters.
Joint optimization. The initial estimates of the hyperparameters are generally not fully accurate, nor consistent. To enforce consistency, we jointly optimize all hyperparameters, seeking values that best fit their initial estimates, subject consistency constraints. Our optimization minimizes the convex quadratic form
Σ 15-21 -x,-2))
subject to consistency constraints,
(1)
2
where X is the set of all hyperparameters across all layers; xं and 2; are the initial estimate and optimal value of the i-th hyperparameter, respectively. The imposed consistency constraints are:
4

---

Under review as a conference paper at ICLR 2021
placedion cabie
Ang Digital Converter
END
GPU
GPU Bower cable
USB outpui
top view
side view
Figure 2: Placement of the magnetic induction sensor on the power cord works regardless of the GPU model, providing a common weak-spot to enable current-based magnetic side-channel attacks.
The output size of a layer agrees with the input size of the next next layer.
(ii) The input size of the first layer agrees with the input feature size.
(iii) The output size of CNN layer does not exceed its input size (due to convolution).
(iv)
The hyperparameters of a CNN layer satisfy
Sin + 2B 7
Sout
+
where a, B, ", and & denote the layer's stride, padding, dilation, and kernel size, respectively.
(V) Heuristic constraint: the kernel size must be odd.
Among these constraints, (i-iii) are linear constraints, which preserves the convexity of the problem. The heuristic (v) can be expressed
linear constraint: for every kernel size parameter ky, we introduce a dummy variable Ty, and require k; 2T and Ty Z Constraint (iv), however, is troublesome, because the appearance of kernel size a and dilation y, both of which are optimization variables, make the constraint nonlinear.
Since all hyperparameters are non-negative integers, the objective must be optimized via integer programming: IP in general case is NP-complete (Papadimitriou & Steiglitz, 1998), and the non- linear constraint (iv) does not make life easier. Fortunately, both a and y have very narrow ranges in practice: is often set to be 1 or 2, and is usually I, and they rarely change across all CNN layers in a network. As a result, they can be accurately predicted by our DNN models; we therefore retain the initial estimates and do not optimize for and 7, rendering (2) linear. Even if DNN mod- els could not reliably recover a and 7, one could exhaustively enumerate the few possible a and ? combinations, and solve the IP problem (T) for each combination, and select the best recovery.
The IP problem with quadratic objective function and linear constraints even when the number of hyperparameters is large (e.g., > 1000). In practice, we use IBM CPLEX (Cplex 2009), widely used IP solver. Optimized hyperparameters remain close to the initial DNN estimates, and are guaranteed to define a valid network structure.
be easily solved,
4
EXPERIMENTAL SETUP
4.1 HARDWARE SENSORS
We use the DRV425 fluxgate magnetic sensor from Texas Instruments for reliable high-frequency sensing of magnetic signals (Instruments) 2020 Petrucha Novotny 2018). This sensor, though
costing only $3 USD, outputs robust analog signals with a 47kHz sampling rate and #2mT sensing range. For analog to digital conversion (ADC), we use the USB-204 Digital Acquisition card, 5-Volt ADC from Measurement Computing (Computing. 2020). This allows 12-bit conversion of the signal, mapping sensor readings from -2mT~>mm to OV~5V
4.2 DATASET CONSTRUCTION
Sensor placement. To avoid interference from other electric components, we place the sensor near the GPU's magnetic induction source, anywhere along the power cable. Because magnetic flux decays inversely proportional to the squared distance from the source, according to the Biot-Savart law (Griftiths 2005), position the sensor within millimeters of the cable casing (see Fig. 2).
Data capture. Pretraining the recovery DNN models (recall Sec. B) requires annotated dataset with pairwise corresopndence between signal and step types (see Fig. 2 We can automat- ically generate annotated signal for given network and specific GPU hardware, simply
5

---

Under review as a conference paper at ICLR 2021
by executing query (with arbitrary imput values) the GPU to acquire the signal. Times- tamped ground-truth GPU operations are made available by most deep learning libraries (e.g., torch. autorrad profiler in PyTorch and tf. tf.proiiler in TensorFlow). A difficulty in this process lies in the fact that the captured (47kHz) raw signals and the ground truth GPU traces run on different clocks. Similar to the of clapperboard to synchronize picture and sound in filmmaking, we precede the inference query with a short intensive GPU operation to induce a sharp spike in the signal, yielding a synchronization landmark (see Fig. S3 We implemented this "clapperboard" by filling vector with random floating point numbers.
Training Set. The set of networks to be annotated could in principle consist solely of randomly generated networks, the basis that data values and "functionality" are irrelevant us, and the training serves to recover the substeps of a layer; or of curated networks or those found in the wild, on the basis that such networks are more indicative of what lies within the black-box. We chose to construct our training set as mixture of both of these approaches. All in all we consider 500 networks for training leading total of 5708 network steps we aim to identify. We will release the complete training and test datasets, along with source code and hardware schematics for full reproducibility.
RESULTS
This section presents the major empirical evaluations of our method. We refer the reader Appx. for complete results, additional experiments, and more thorough discussion.
5.1 ACCURACY OF NETWORK RECONSTRUCTION
Test dataset. We construct test dataset fully separate from the training dataset. Our test dataset consists of 64 randomly generated networks in way similar to Sec. 4.2 The number of layers ranges from 30 to 50 layers. To diversify our zoology of test models, we also include smaller networks that under 10 layers, LSTM networks, as well as ResNets (18, 34, 50, and 101). Altogether, each test network has up to 514 steps. In total, the test dataset includes 5708 network steps, broken down into 1808 activation functions, 1975 additional batch normalization and pooling, and 1925 convolutional, fully connected, and recurrent layers. When we construct these networks, their input image resolutions randomly chosen from x224 224, 96x96, 64 64, 48 X48, 32x32]: the highest resolution is used in ImageNet, and lower ones are used in datasets such as CIFAR.
Topology reconstruction. As discussed in Sec. we use a biLSTM model to pre- dict the network step for each single sample. Table I reports its accuracy, measured on Nvidia TITAN V GPU. There, we also break the accuracy down into measures of individ- ual types of network steps, with an overall accuracy of 96.8%. An interesting observa- tion is that the training and test datasets are terms of signal samples
Table I: Classification accuracy of network steps on TITAN V.
Layer Type
Prec.
Rec.
F]
samples
LSTM Conv
.997
.992
.955
8704
.993
.996
.944
447968
Fully-connected
.901
.796
.866
10783
Add
.984
.994
.989
22714
BatchNorm
.953
.955
.954
47440
MaxPool
.957
.697
.806
4045
both unbalanced (see last column of Table [.. This is because in practice convolutional layers are computa- tionally the most expensive, while activation functions and pooling are lightweight. Also, certain steps like average pooling much less frequently used. While such data imbal- them
AvgPool
.371
.760
.499
675
ReL.U
.861
.967
.911
28512
ELU
.464
.825
.594
2834
LeakyReL.U
.732
.578
.646
9410
Sigmoid
.694
.511
.588
8744
Tanh
.773
.557
.648
4832
Weighted Avg.
.968
.967
.966
ance does reflect reality, when train and test, most of the misclassifications occur at those rarely used, lightweight network steps, whereas the majority of network steps are classified correctly.
We evaluate the quality of topology reconstruction using normalized Levenshtein distance (i.e., one of the edit distance metrics) that has been used to evaluate network structure similarity (Graves et al.) 2006 Hu et al. 2020).. Here, Levenshtein distance measures the minimum number of operations- including adding/removing network steps and altering step type- nepe-deeded to fully rectify recovered topology. This distance is then normalized by the total number of steps of the target network.
We report the detailed results in Fig. S2 in the appendix. Among the 64 tested networks, 40 of the reconstructed networks match precisely their targets, resulting in Levenshtein distance. The

---

Under review as a conference paper at ICLR 2021
Table 2: Model extraction accuracy CIFAR-10 across different GPUs.
Model
Target
Titan-V
Titan-X
GTX-1080
GTX-960
VGG-11
89.03
89.61
89.63
88.46
88.3
VGG-16
90.95
91.08
91.03
89.33
90.78
AlexNet
81.68
85.26
85.11
85.27
85.03
ResNet-18 18
92.77
92.61
92.82
92.79
92.04
ResNet-34
92.21
92.28
92.95
90.81
92.71
ResNet-50
90.89
91.8
91.97
91.2
91.29
ResNet-101
91.58
91.91
91.85
91.37
91.72
average normalized Levenshtein distance of all tested networks is 0.118. To provide sense of how the Levenshtein distance is related to the network's ' ultimate performance (i.c., its classification accuracy), conduct additional experiment and discuss it in Appx. B.
DNN hyperparameter estimation. Next, report the test accuracies of our DNN models (dis- cussed in Sec. 3.2 for estimating hyperparameters of convolutional layers. Our test data here con- sists of 1804 convolutional layers. On average, our DNN models have 96~-97% accuracy. The break-down accuracies for individual hyperparameters are shown Table [f the appendix.
Reconstruction quality measured as classification accuracy. Ultimately, the reconstruction quality must be evaluated by how well the reconstructed network performs in the task that the original network aims for. To this end, we test seven networks, including VGGs, AlexNet, and ResNets, that have been used for CIFAR-10 classification (shown in Table 2p. We treat those net- works as black-box models and reconstruct them from their magnetic signals. We then train those reconstructed networks and compare their test accuracies with the original networks' performance. Both the reconstructed and original networks are trained with the same training dataset for the same number epochs. The results Table 2 show that for all seven networks, including large networks (e.g., ResNet101), the reconstructed networks perform almost as well as their original versions. We also conduct similar experiments on ImageNet and report the results in Table Appx. B.2
GPU transferability. Our proposed attack requires the adversary have the same brand/version of GPU as the victim, but not necessarily the same physical copy (see Fig. [S3). Here, we obtain two copies an Nvidia GTX- -000 GPU running on two different machines, using one to generate training data and another one for black-box reconstruction. We demonstrate that in this setting the models can still be well reconstructed. The experiment details and results are described in Appx. B.3
5.2 TRANSFER ATTACK
To demonstrate a potential exploit of this side-channel exposure, we use reconstructed networks to launch adversarial transfer attack. Transfer attack relies surrogate model, one that approximates the target model, to craft adversarial examples of the target model. black-box setting, is known to be hard in general case to find an effective surrogate model (Demontis et al.) (2019). But under our threat model, the adversary can recover the network structure of a black-box model from the leaked magnetic signals, and use it as the surrogate model.
Here we on six networks found in the wild, ranging from VGGs AlexNet ResNets (listed in Table 3). For each network (and each column in Table 3. we treat it as a black-box model and reconstruct its structure by running on four different GPUs (listed in Table 3 obtaining four reconstructed versions. Afterwards, we train each reconstructed network and use it to craft adversarial examples for transfer attacking the original network. The transfer attack success rates are reported in the top four rows of Table which are compared against several baselines shown the bottom half of the table. Using our side-channel-ban -baed surrogate model, the transfer attack success rate is comparable the white-box transfer attack baseline, that is, using the target model's network structure (but trained separately) to attack the target model itself. In other words, our side- channel-based reconstruction effectively turn black-box attack into white-box attack.
We also conducted additional experiments for transfer attacks MNIST dataset. We reconstruct network models downloaded online and then launch attacks. The results are reported in Appx. B.4
5.3 DISCUSSION: DEFENSES AGAINST MAGNETIC SIDE CHANNEL LEAKAGE
At this point, we have shown the robustness and accuracy of the magnetic side channel exploits. Traditionally countermeasures fall under the category of either prevention, detection, jamming.

---

Under review as a conference paper at ICLR 2021
Table 3: Transfer attack results on CIFAR-10.
Target Model
ResNet-18
ResNet-34
ResNet-101
VGG-11
VGG-16
AlexNet
GTX-960
98.56
92.51
91.20
63,41
72.57
58.90
(0X--188)
97.88
90.86
86.24
64.69
55.19
56.83
TITAN-X
98.32
93.45
84.47
61.89
77.36
68.41
Source Model
TITAN-V
98,48
93.65
91.27
64.39
72.77
60.17
VGG-IT
65.86
57.82
57.52
60.24
65.50
39,95
VGG-16
74.00
61.54
54.23
41.60
74.29
29.57
R8sNet-18
97.70
90.72
80.27
47.98
86.64
30.56
ResNet-34
97.21
92.46
82.30
51.42
85.60
32.34
ResNet-101
92.53
86.98
92.95
53.98
83.04
30.55
AlexNet 10.11
9.59
10.19
11.60
10.42
62.70
Since our approach is passive in that does not alter any code or hardware operation GPUs, detection methods which consist of somehow discovering someone is listening to the GPU applicable to magnetic leakage. Here we focus on prevention and jamming.
Prevention shown in Figure T each rise and drop of the magnetic signals correspond to the boundary between GPU operations. This is only possible when the input batch is large enough to keep every GPU operation sustained and stabilized keep the input sufficiently small (e.g. single image) such that the magnetic signals never reach any stable state and suffer from a low signal-to-noise ratio, rendering our sensing setup fu- non-standard framework for
high-load state. To prevent this behavior,
tile. Another way prevent magnetic side channel leakage is inference which the adversary does not have any training data to start with.
Jamming While running on tiny input batch size might be infeasible for large input dataset, we find jamming also effective defense mechanism. Specifically, during the inference of large input batch, we ran a third-party CUDA GPU stress test in the background (Timonen) 2020p. We found that the magnetic signals completely distorted because of the constantly high utilization of GPU. Moreover, we observe little speed degradation for the foreground inference. The main caveat with this approach higher power consumption and the possible effects on the lifetime of GPU.
Another possible defense mechanism results from the fact that dataflow in the GPU. For example, we can correctly identify two GPU operations, convolution and batch norm, in a long sequence. But there no evidence proving the dataflow follows the same pattern- the output from convolution could be deadend and batch norm takes input from previ- ous GPU operation. This mismatch between the dataflow and the underlying network model makes it hard decipher robustly. While this defense can handle arbitrary input and networks in theory, are unsure about the implementation hurdles this defense and how modern deep learning libraries optimize for such unconventional graphs.
not tracking the actual
CONCLUDING REMARKS
We set out to study what can learned from passively listening magnetic side channel in the proximity of running GPU. Our prototype shows it is possible to extract both the high-level net- work topology and detailed hyperparameters. To better understand the robustness and accuracy, we collected dataset of magnetic signals by inferencing through thousands of layers four different GPUs. We also investigated how one might use this side channel information to turn a black-box box attack into white-box transfer attack.
Limitations. In our formulation, we assume networks progress in linear fashion and do not handle complex graph network with intricate branching topologies. We cannot tell a network is trained with dropout since dropout layers do not appear at inference time. Indeed, any operation that only appears during training beyond the capability magnetic side channel snooping.
Our reconstruction DNNs require knowledge of the victim's 's GPU model and version. When these unknown, the adversary may still exhaustively pretrain multiple sets of reconstruction DNNs for all GPU models and at runtime sean through all reconstruction options. Software upgrades, which can lead significant performance boost and therefore alter the emitted magnetic signals, may be
8

---

Under review as a conference paper at ICLR 2021
viewed as further increasing the set of possible GPU models. In our experiments, keep all the software versions constant, including OS verion, CUDA version, and ByTorch/Tensoriin version.
Ethical Considerations. Our main intention is to study the extent to which a magnetic side chan- nel can leak information. We hope help people understand that running deep learning inferences on GPUs can leak critical information, and in particular model architecture and hyperparameters, nearby induction sensors. Sharing these findings creates the potential for malicious use. We introduced several viable defense mechanisms in 5.3
REFERENCES
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: A system for large- scale machine learning. In 12th [USENIX] symposium on operating systems design and imple- mentation ((OSDI) 16). pp. 265-283, 2016.
Lejla Batina, Dirmanto Jap, Shivam Bhasin, and S Picek. Csi nn: Reverse engineering of neural network architectures through electromagnetic side channel. In Proceedings of the 28th USENIX Security Symposium. USENIX Association, 2019.
Ian Buck. Gpu computing:
Programming massively parallel processor. In International Sympo- sium on Code Generation and Optimization (CGO (7), pp. 17-17. IEEE, 2007.
Measurement Computing.
USB-200 Series Single Gain Multifunction USB Devices (accessed
Oct I, 2020), 2020. URL https: /twp:://wwww... wwwdaq.cccooooooooo com. www.commsssaae usb data data-aaquisiiion.
USB-200-Seriess aspx
IBM ILOG Cplex. V12. 1: User's manual for cplex. International Business Machines Corporation, 46(53):157, 2009.
Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, and Fabio Roli. Why do adversarial attacks transfer? explaining transfer- ability of evasion and poisoning attacks. In 28th USENIX Security Symposium Security 19), pp- 321-338, 2019.
Anuj Dubey, Rosario Cammarota, and Aydin Aysu. Maskednet: The first hardware inference engine aiming power side-channel protection. arXiv: Cryptography and Security, 2019.
Vasisht Duddu and D Vijay Rao. Quantifying (hyper) parameter leakage in machine learning. arXiv preprint arXiv:1910. 1et:::990..444/4 14409, 2019.
Vasisht Duddu, Debasis Samanta, D Vijay Rao, and Valentina E Balas. Stealing neural networks via timing side channels. arXiv preprint arXiv: 188.. 2018.
Daniel Genkin, Adi Shamir, and Eran Tromer. Rsa key extraction via www-bandwidth acoustic crypt- analysis. In Annual Cryptology Conference, pp. 444-461. Springer, 2014.
Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. Bidirectional Istm networks for im- proved phoneme classification and recognition. In International Conference Artificial Neural Networks, pp. 704. Springer, 2005.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 369-376, 2006.
David Griffiths. Introduction to electrodynamics, 2005.
Ed Grochowski and Murali Annavaram. Energy per instruction trends in intel microprocessors. Technology @ Intel Magazine, 4(3):1-8, 18, 2006.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.

---

Under review as a conference paper at ICLR 2021
Sanghyun Hong, Michael Davinroy. Yigitcan Kaya, Dana Dachman- Soled, and Tudor Dumitraş. How to Own the nas in your spare time. International Conference on Learning Representations, 2019.
Xing Hu, Ling Liang, Shuangchen Li. Lei Deng, Pengfei Zuo. Yu Ji. Xinfeng Xie, Yufei Ding, Chang Liu, Timothy Sherwood, et al. Deepsniffer: A dnn model extraction framework based on learning architectural hints. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 385-399, 2020.
Weizhe Hua, Zhiru Zhang, and G Edward Suh. Reverse engineering convolutional neural networks through side-channel information leaks. In 2018 55th ACM/ESDA//EFE Design Automation Con- ference (DAC), pp. 1-6. IEEE, 2018.
Texas Instruments. DRV425: Fully-integrated fluxgate magnetic sensor for open-loop applications (accessed Oct 1, 2020), 2020. URL/https:/ www. ti. cwtoo//poouc/////
Paul Kocher, Joshua Jaffe, and Benjamin Jun. Differential power analysis. In Annual international cryptology conference, pp. 388-397. Springer, 1999.
Paul C Kocher. Timing attacks on implementations of diffic-hellman, rsa, dss, and other systems. In Annual International Cryptology Conference, pp. 104-113. Springer, 1996.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks, arXiv preprint arXiv:1611.0277, 2016.
Chao Luo, Yunsi Fei, Pei Luo, Saoni Mukherjee, and David Kaeli. Side-channel power analysis of a gpu aes implementation. In 2015 33rd IEEE International Conference on Computer Design (ICCD), pp. 281-288. IEEE, 2015.
Seong Joon Oh, Bernt Schiele, and Mario Fritz. Towards reverse-engineering black-box neural networks. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pp. 121- 144. Springer, 2019.
Christos H Papadimitriou and Kenneth Steiglitz. Combinatorial optimization: algorithms and com- plexity. Courier Corporation, 1998.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp- 506-519. 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In Advances in neural information processing systems, pp. 8026-8037, 2019.
Vojtech Petrucha and David Novotny. Testing and application of integrated fluxgate sensor drv425. Journal of Electrical Engineering, 69(6):418-42) 2018.
Ville Timonen. Multi-GPU CUDA stress test (accessed Oct 1, 2020), 2020. URL https://
github. www.cc//wwwlccgggccc -uubburrnpppuu
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. 25th {USENIX} Security Symposium ((USENIX) Security 16), pp. 601-618, 2016.
Lingxiao Wei, Bo Luo, Yu Li, Yannan Liu, and Qiang Xu. know what you see: Power side-channel attack convolutional neural network accelerators. In Proceedings of the 34th Annual Computer Security Applications Conference, pp. 393-406, 2018.
Yun Xiang, Zhuangzhi Chen, Zuohui Chen, Zebin Fang, Haiyang Hao, Jinyin Chen, Yi Liu, Zhefu Wu, Qi Xuan, and Xiaoniu Yang. Open dnn box by power side-channel attack. IEEE Transactions Circuits and Systems II: Express Briefs, 2020.
10

---

Under review as a conference paper at ICLR 2021
Mengjia Yan, Christopher W Fletcher, and Josep Torrellas. Cache telepathy: Leveraging shared source attacks to learn (DNN) architectures. In 29th {USENIX) Security Symposium ({USENIX) Security 20), pp. 2003-2020 2020.
Yuval Yarom and Katrina Falkner. Flush+ reload: a high resolution, low noise, 13 cache side-channel attack. In 23rd {USENIX] Security Symposium ( Security 14), pp. 719-732, 2014.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 8697-8710, 2018.

---

Under review as a conference paper at ICLR 2021
Supplementary Document
Can one hear the shape of a neural network?: Snooping the GPU via Magnetic Side Channel
A BILSTM NETWORK STRUCTURE
Classifying steps in network model requires taking in time-series signal and converting it to labeled operations. The EM signal responds only to the GPU's instantaneous performance, but because the GPU executes neural network sequence, there is rich context in both the window before and after any one segment of the signal. Some steps are often followed by others, such as pooling operations after series of convolutions. We take advantage of this bidirectional context in our sequence to sequence classification problem by utilizing a BiLSTM network to classify the observed signal, To retrive network's topology, pass normalized EM values into two-layer BLSTM network, with dropout of 0.2 between. From there we compute categorical cross- entropy loss on time-distributed output that we achieve by sliding an input window across our EM signal. This approach proves robust, and is the method used by all of our experiments, and on all GPU's tested.
The segmented output of our BiLSTM network on our extracted signal is for the most part unambigu- ous. Operations that follow one another (i.e. convolution, non-linear activation function, pooling) are distinct in their signatures and easily captured from the context enabled by sliding window input the BiLSTM classifier. Issues arise for very small-sized steps, closer to sensor's sampling limit. In such regions a non-linear activation may be over-segmented and split into two (possibly different) activation steps. To ensure consistency results to merge identical steps that are output in sequence, cull out temporal inconsistencies such pooling before non-linear activation, and remove activation functions that larger than the convolutions that precede them.
signal
postprocess the segmented
B ADDITIONAL EXPERIMENTS
B. USING LEVENSITEIN DISTANCE TO MEASURE NETWORK RECONSTRUCTION QUALITY
To provide a sense of how the Levenshtein edit distance is related to the network's ultimate perfor- mance, consider AlexNet (referred model A) and its five variants (refered as model B, C, D, and E, respectively). The variants are constructed by randomly altering some of the network steps model A. The Levenshtein distances between model A and its variants are 1, 2, 2, 5, respectively (see Fig. SS]. and the normalized Levenshtein distances are shown in the brackets of Fig. [S] We then measure the performance (i.e., standard test accuracy) of these models on CIFAR-10 As the edit distance increases, the model's performance drops.
Classifccation test accuracy
0.83
0.822
0.798
0.811
0.806
0.808
0.765
0.771
0.732
0.7
A
B
Edit Distance:
0
100.55)
(0.11))
(0.11)
5 (0.28)
Figure S1: The model's classification accuracy drops as its Levenshtein distance from the original model (model A: AlexNet) increases.
B.2 RECONSTRUCTION QUALITY ON IMAGENET
We treat ResNet18 and ResNet50 for ImageNet classification black-box models, and recon- struct them from their magnetic signals. We then train those reconstructed networks and compare their test accuracies with the original networks' performance. Both the reconstructed and original
12

---

Under review as a conference paper at ICLR 2021
Table S1: Model reconstruction evaluated ImageNet classification.
ResNet18
ResNet50
Model
Original
Extracted
Original
Extracted
Top-1 Acc.
64.130
64.608
62.550
61 61.842
Top-5 Acc.
86.136
86.195
85.482
84.738
KL Div.
2.39
4.85
40%
40%
10%
10%
10%
0.1
0.15
0.7
0:45
0.5
nonmalized
cdit
Figure S2: Distribution of normalized Levenshtein distance. (left) We plot the distribution of the normalized Levenshtein distances between the reconstructed and target networks. This results, corresponding to Table in the main text, use signals collected Nvidia TITAN V. (right) We also conduct similar experiments on two Nvidia GTX-1080 GPUs. One is used for collecting training signals, and the other is used for testing our tite-channel-basee reconstruction algorithm.
networks are trained with the same training dataset for the same number of epochs. The results are shown in Table [ST] where we report both top-1 and top-5 classification accuracies. In addition, also report a KL-divergence measuring the difference between the 1000-class image label distri- bution (over the entire ImageNet test dataset) predicted by the original network and that predicted by the reconstructed network. Not only are those KL-divergence values small, we also observe that for the reconstructed network that has smaller KL-divergence from the original network (i.e., ResNet18), its performance approaches more closely to the original network.
B.3 GPU TRANSFERABILITY
Here we verify that (i) the leaked magnetic signals are largely related GPU brand/version but not the other factors such CPUs and (ii) the signal characteristics from two physical copies of the same GPU type stay consistent.
We obtain two copies of an Nvidia GTX-1080 GPU running on two different machines. When the same network structure both GPUs, the resulting magnetic signals similar to each other, as shown in Fig. S3 This suggests that the GPU cards are indeed the primary sources contributing the captured magnetic signals.
Next, we use one GPU to generate training data and another one to collect signals and test our black-box reconstruction. The topology reconstruction results are shown in Table arranged in the way similar Table [] and the distribution normalized Levenshtein edit distance over the tested networks are shown Fig. -22-right. These accuracies are very close to the case wherein a single GPU used. The later part of the reconstruction pipeline (i.e., the hyperparameter recovery) directly depends the topology reconstruction. Therefore, it is expected that the final reconstruction is also very similar to the single-GPU results.
B.4 TRANSFER ATTACKS ON MNIST
We also conduct transfer attack experiments on MNIST dataset. We download four networks online, which are not commonly used. Two them are convolutional networks (referred as CNNI and CNN2), and the other two are fully connected networks (referred as DNNI and DNN2). None of these networks appeared in the training dataset. We treat these networks as black- box models, and reconstruct network for each of them. We then the four reconstructed models to transfer attack the four original models, and the results are shown Table [S4] As baselines, we also use the four original models transfer attack each other including themselves.
13

---

Under review as a conference paper at ICLR 2021
Table S2: Classification accuracy of network steps 108X-
Prec.
Rec.
# samples
LSTM
.97
.999
998
12186
Conv
985
989
.987
141164
Fully-connected
.818
.699
.887
9301
Add
.622
.941
.951
30214
BatchNorm
.56
.94
.950
48433
MaxPool
.809
.701
.751
1190
AvgPool
.97
.874
.900
294
ReLU
.868
.859
.863
11425
ELU
.861
.945
.901
8311
LeakyReLU
.962
.801
.874
3338
Sigmoid
.462
.801
.585
5106
Tanh
.988
.884
.53
8050
Weigted Avg.
.955
.955
.945
Table S3: DNN estimation accuracies. Using the 1804 convolutional layers our test dataset, we measure the accuracies of our DNN models for estimating the convolutional layers' hyperparame- ters. Here, we break the accuracies down into the accuracies for individual hyperparameters.
Kernel
Stride
Padding
Image-in
Image-out
Precision Recall
0.971
0.976
0.965
0.968
0.965
0.969
0.975
0.964
0.969
0.968
Fl Score
0.969
0.975
0.962
0.967
0.965
Table S4, every row shows transfer attack success rates when we use different source (surro- gate) models attack specific original model (CNN1, CNN2, DNNI, or DNN2). Each column labeled as "extr." corresponds to the extracted (reconstructed) model whose target model is given in the previous column right before it. In addition, also show all the models' test accuracies on MNIST in the last row the table. The results show that all the reconstructed models approx- imate their targets closely, both in terms of their abilities for launching transfer attacks and their classification performance.
C SENSOR SETUP
The magnetic induction signal we utilize comes from digitally converting analog readings of Texas Instruments DRV425 fluxgate sensor with Measurement Computing's USB-204 Digital Acquisition Card. The sensor samples at a frequency 47Khz and the converter operates at 50Khz to map the originally 22mT~2mT readings across 0 to 5 Volts using 12-bit conversion. Calibrating the sensor requires (a) that the sensor within range of the electromagnetic signal and that (b) the sensor orientation is consistent. The magnetic induction signal falls off at a rate inversely proportional to distance squared, and so the sensor must be placed within 7mm of the GPU power cable for reliable
2.5
5
2.5 0
12
18 24
30
36
42
48
54
60
ms
Figure S3: Here we plot the resulting signals from the same network model deployed on two differ- instances of a NVIDIA-GTX 1080 (running on two different computers). In the green boxes on the left are the spikes that we inject purpose (discussed in Sec. 4.2 to synchronize the measured signal with runtime trace GPU operations.
14

---

Under review as a conference paper at ICLR 2021
Table S4: MNIST results.
Source Model
CNNI
extr.
CNN2
extr.
DNN1
extr.
DNN2
extr.
CNNI
.858
.802
.226
.202
.785
.795
.476
527
Target
CNN2
.395
.319
.884
.878
.354
.351
.354
.211
DNNI
.768
.812
.239
.223
.999
.999
.803
.885
DNN2
.703
.768
.219
.194
.955
.979
.860
.874
Accuracy
.989
.987
.993
.991
.981
.981
.980
.983
measurement. Flipping the flat sensor over will result in sign change of the magnetic induction signal, thus a uniform orientation should be maintained to avoid preprocessing readings across the dataset align.
15


Reviews:
[{"review_id": "x0SBDu3Vmz", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \n- This paper studies the effectiveness of inferring a neural network’s layers and hyperparameters using the magnetic fields emitted from a GPU’s power cable. The results show that (under certain assumptions) one can reconstruct a neural network’s layers and hyperparameters accurately, and use the inferred model to launch adversarial transfer attacks.\n\nStrong points: \n- The idea of using magnetic side channels to infer network structure is interesting.\n- The paper is well-written with ideas and limitations explained clearly.\n- The experiment results are thorough and explained clearly\n\nWeak points: \n- The threat model seems impractical. Attacker assumptions include:\n  - have physical access to the GPU\n  - know the exact input feature dimensions and batch size.\n  - know the deep learning framework, GPU brand and hardware/software versions.\n- The main innovation is demonstrating magnetic side channels from GPU cables reveal information about network structures. However, I’m not sure if ICLR is the best venue for this type of contribution. This paper could be a much stronger submission to other security and system conferences.\n\nRecommendation: \n- I’m inclined to recommend a reject. The main reason is that the results are based on multiple impractical assumptions, limiting the impact of this paper in reality. \n\nComments & questions:\n- How do the authors imagine launching this attack in reality? Specifically, how would one know the input dimensions and batch size of a black-box model? A clear explanation of this will help readers understand the value of this work.\n- Using consistency constraints to optimize for hyperparameter estimation is interesting. How effective is this additional optimization compared with only using the initial estimation?\n\nMinor comments\n- “But there is ‘not’ evidence proving” -> ‘no’\n\n==== Updates after the response ====\n\nI thank the authors for answering the questions in detail. Providing an example application does help readers understand scenarios where the threat model could apply. However, I still think such scenarios are not common but agree that the findings in this paper could be helpful for future security research. I adjusted my rating based on this better understanding. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "fAkPLJj4IaF", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary:\nThe paper presents a method for capturing the shape (type of layers) and their respective parameters of a neural network through the magnetic field induced as the GPU drains power. In particular, the GPU is snooped using an off-the-shelf magnetic induction sensor which is placed along the power cable of the GPU. It turns out that under some assumptions (knowledge of GPU model and deep learning framework, knowledge of input size and ability to launch query of specific batch size), based on the correlation of the power consumption pattern of the GPU with the operations being performed it is possible to recognize the type of operation being performed as well as the respective hyper-parameters with very few errors.\n\n\n# Strengths:\nThe paper presents a very interesting and novel method for \"spying\" on the Neural Network being executed on a GPU by capturing the magnetic signal induced by the GPU's power consumption. A BiLSTM is employed to classify captured samplers of the magnetic signal to segments corresponding to specific steps/operations of the neural network. Then DNNs specific for each type of step are employed for estimating the hyper-parameters of each step. Consistency between the hyper-parameters is enforced by solving an integer-programming problem.\n\nThe proposed method is able to recover the entire structure of various types of neural networks (including randomly generated ones) and the parameters of each step with very few errors. The similarity of the reconstructed networks is evaluated under various metrics, including the performance of target and reconstructed networks on a classification task. An interesting application is the use of the method to build surrogate models for performing black-box adversarial attacks with very high success rates. Possible counter-measures are proposed and the limitations of the method are also discussed.\n\nThe paper is very well-written and easy to read. The evaluation is quite comprehensive showing the ability of the method to fully recover the architecture of various networks while further results and clarifications are provided in the supplemental material.\n\n\n# Weaknesses and Questions:\nSome interesting aspects are not completely covered. For example, although transferability to different hardware is discussed, still the sensitivity of the method with respect to some factors is not discussed. In particular, are there any calibration issues that should be taken into account? How does the method perform if a sensor of different make is used with possible different sampling rate? In what range of sampling rates/placement distances the method works? Regarding GPU transferability, are the GPU pairs of the same make, frequency settings, etc.?\n\nAdditionally, regarding the segmentation of the samples in steps, one would expect that some over-segmentation issues would be present. Are there any measures for enforcing temporal consistency and for avoiding over-segmentation, or it is not an issue because the samples are so unambiguous?\n\nRegarding the applicability of the proposed method, in real-world conditions it may not always be possible to define the input of the batch size. On this topic, more details can be provided on the strategy to be used to find a suitable batch size for the method to work. Additionally, the paper considers only networks acting as encoders. Would the method naturally extend to decoders or are there any difficulties/ambiguities introduced? Also the paper focuses on the inference task, can the method be applied also for training tasks and with what adaptations?\n\nRegarding the defenses, what happens if inference optimizers (e.g. Tensor RT) are used? Is the performance of the method affected? Also, does multi-GPU setups introduce interference?\n\n## Minor comments:\n* Figure 1 right: the abbreviations (BN, MP), although common, should be explained in the text/caption.\n* Joint optimization paragraph: \"will generally not fully\"\n* page 5 before Section 4: \"Were it that DNN\"\n \n# Rating Justification:\nOverall, I think that the paper is highly novel and introduces an interesting way to recover the architecture of a NN by using a cheap sensor. There are many questions that rise by this work mainly though due to its novel and intriguing nature. I am not 100% sure that ICLR is the most suitable venue for this work, I am not considering though this aspect in my rating.\n\n# Rating and comments after the rebuttal\nI share the concerns of fellow reviewers regarding the practicality of the assumptions needed for launching the attack presented in the paper, however, based also on the discussion with the authors, I think that the paper is interesting regardless as it can lead to better insights regarding the development of suitable defence mechanisms for securing the architecture and the information carried by a Deep Learning model.\n\nI share also the concerns raised in the other reviews that the paper might be better appreciated by an audience focusing on cyber-security. However, I think that the subject can also be of relevance to ICLR as the paper made an effort to highlight the aspects more relevant to the ML community.\n\nHence, leaving the aspect of relevance to the ACs' discretion, I think that overall this is a clearly written paper based on well executed research that presents some interesting results that are potentially impactful in the aspects concerning security of systems employing Deep Learning models.", "extra_scores": {}, "extra_text": {}}, {"review_id": "LcwmFL9WSxv", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper demonstrates that magnetic side channel information from a GPU (that is processing a deep neural net) can be snooped to recover the architecture and hyperparameters of the neural network. While the concept of side channel information snooping to recover codes/software (including ML models) is widely studied, the novelty claim is that recovering detailed structures of deep models is new. The paper also demonstrates that black-box attacks mounted using a recovered model is quite powerful compared to traditional black-box attacks. \n\nThe paper is well-written and quite useful for the safety-critical applications community that use ML. However, there is no core ML contribution made in this paper. The authors use standard ML models to map the side channel signal to deep learning model architecture. The use of model architecture consistencies as constraints is clever, but nothing significant in terms of contributions to the ML community. So, in my opinion, this paper is better suited for other venues such as cyber security conferences.  ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: 
The paper presents a side-channel attack in a scenario where the attacker is able to place a induction sensor near the power cable of the victim's GPU. The authors train a neural network to analyse the magnetic flux measured by the sensor to recover the structure (layer type and layer parameters) of the target neural network. The authors also show that for a wide range of target network structure, by training a network with the inferred structure, they produce adversarial examples as effective as a white box attack.

The points raised by the reviewers were the following: 1) the result that this type of side-channel attack works is interesting, 2) the practicality of the attack is unclear because the attacker needs hardware access to the victim's GPU, 3) the ML contribution is not really clear and a venue on cyber-security might be more appropriate. 

Side-channel attacks on deep neural networks can be of relevance to ICLR (as pointed to by the authors by the ICLR papers/submissions on system side-channel attacks). Nonetheless, I tend to agree with R1 and R2 that the ML contribution is limited (either in terms of application of ML or methodology), and the concerns of practicality of the approach make me lean towards rejection.
Decision: Reject