S ECURITY A NALYSIS OF D EEP N EURAL N ETWORKS O PERATING IN THE P RESENCE OF C ACHE S IDEC HANNEL
A TTACKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent work has introduced attacks that extract the architecture information of deep neural networks (DNN), as this knowledge enhances an adversary’s capability to conduct attacks on black-box networks.
This paper presents the ﬁrst in-depth security analysis of DNN ﬁngerprinting attacks that exploit cache side-channels.
First, we deﬁne the threat model for these attacks: our adversary does not need the ability to query the victim model; instead, she runs a co-located process on the host machine where the victim’s deep learning (DL) system is running and passively monitors the accesses of the target functions in the shared framework.
Second, we introduce DeepRecon, an attack that reconstructs the architecture of the victim network using the internal information extracted via Flush+Reload, a cache side-channel technique.
Once the attacker observes function invocations that map directly to architecture attributes of the victim network, the attacker can reconstruct the victim’s entire network architecture.
In our evaluation, we demonstrate that an attacker can accurately reconstruct two complex networks (VGG and ResNet50) having observed only one forward propagation.
Based on the extracted architecture attributes, we also demonstrate that an attacker can build a meta-model that accurately ﬁngerprints the architecture and family of the pretrained model in a transfer learning setting.
From this meta-model, we evaluate the importance of the observed attributes in the ﬁngerprinting process.
Third, we propose and evaluate new framework-level defense techniques that obfuscate our attacker’s observations.
Our empirical security analysis represents a step toward understanding DNNs’ vulnerability to cache side-channel attacks.
I NTRODUCTION
Deep neural networks (DNNs) have become an essential tool in various applications, such as face recognition, speech recognition, malware detection, and autonomous driving or aviation (Parkhi et al., 2015;
Amodei et al., 2016;
Arp et al., 2014; Chen et al., 2015; Smolyanskiy et al., 2017).
A
DNN’s performance depends widely on the network architecture—the number and types of layers, how the layers are connected, and the activation functions—and, unfortunately, there is no universal architecture that performs well on all tasks.
Consequently, researchers and practitioners have devoted substantial efforts to design various DNN architectures to provide high performance for different learning tasks.
Owing to their critical role, DNN architectures represent attractive targets for adversaries who aim to mount DNN ﬁngerprinting attacks .
In such an attack, the adversary probes a DNN model, considered conﬁdential, until she infers enough attributes of the network to distinguish it among other candidate architectures.
In addition to revealing valuable and secret information to the adversary, DNN ﬁngerprinting can enable further attacks on black-box models .
While the prior work on adversarial machine learning often assumes a white-box setting, where the adversary knows the DNN model under attack, these attacks are usually unrealistic in practice (Suciu et al., 2018).
In consequence, researchers have started focusing on a black-box setting, where model architecture is unknown to the adversary.
However, in this setting, the adversary must make some assumptions about the victim model in order to craft successful adversarial examples (Papernot et al., 2017).
Instead of guessing, the adversary can start by conducting a DNN ﬁngerprinting attack to infer the information required about the model, then use this information to craft adversarial examples that can evade the model.
This can also enable model extraction attacks (Tram`er et al., 2016; Kurakin et al., 2016; Wang & Gong, 2018) and membership inference or model inversion attacks (Shokri et al., 2017; Long et al., 2018).
Because of the large number and types of architectural attributes, and the subtle effect that each attribute has on the model’s inferences, DNN ﬁngerprinting is challenging when using the typical methods employed in the adversarial machine learning literature.
For example, Wang & Gong (2018) propose a hyperparameter stealing attack that requires knowledge of the training dataset, the
ML algorithm, and the learned model parameters, yet is unable to extract the model architecture.
Wang et al.
(2018) demonstrate a ﬁngerprinting attack against transfer learning; however, they rely on the assumption that the teacher model and learning parameters are known to the attacker.
To overcome these challenges, recent work has started to investigate attacks that utilize information leaked by architectural side-channels on the hardware where the DNN model runs.
Hua et al.
(2018) extract the network architecture of a model running on a hardware accelerator by monitoring off-chip memory addresses.
Yan et al.
(2018) reduce the search space from 10 35 to 16 candidates within a given architecture framework by exploiting cache side-channels.
In this paper, we ask the question: how vulnerable are DNNs to side-channel attacks, and what information do adversaries need for architecture ﬁngerprinting?
We perform, to the best of our knowledge, the ﬁrst security analysis of DNNs operating in the presence of cache side-channel attacks.
Speciﬁcally, we deﬁne the threat model for these attacks, including the adversary’s capabilities and limitations.
We then introduce DeepRecon, an efﬁcient attack that reconstructs a black-box DNN architecture by exploiting the Flush+Reload (Yarom & Falkner, 2014) technique, and we further evaluate the importance of speciﬁc architectural attributes in the success of ﬁngerprinting.
Finally, we propose and evaluate new framework-level defenses against these attacks.
Our attack works by targeting lines of code corresponding to the execution of speciﬁc network architecture attributes of a deep learning (DL) framework.
Speciﬁcally, these lines of code correspond to instructions to execute functions that are mapped into the instruction cache when the functions are invoked.
Once these lines of code are identiﬁed, our attack ﬂushes them from the instruction cache shared by the attacker and the victim.
The attacker waits for the victim’s process to run and then measures the time it takes to re-access those same lines of code.
If the victim’s DNN model has accessed any of these particular functions, the corresponding lines of code will be present in the instruction cache when the attacker tries to re-access them.
Therefore, the access time to call these functions will be measurably faster than if the victim had not loaded them back into the shared instruction cache.
On the other hand, if the victim DNN model did not access these particular functions, the corresponding lines will not be present in the cache when accessed by the attacker, and thus the access time will be measurably slower.
We show that from this seemingly small amount of information that is leaked to the attacker, much of the victim’s DNN architecture can be extracted with no query access required .
To launch this attack, we only assume that: (1) an attacker and a victim are co-located in the same machine, and (2) they use the same shared DL framework.
In evaluations, we demonstrate that, by learning whether or not speciﬁc functions were invoked during inference, we can extract 8 architecture attributes across 13 neural network architectures with high accuracy.
Based on the extracted attributes, we demonstrate how an attacker can reconstruct the architectures of two common networks, VGG16 (Simonyan & Zisserman, 2014) and
ResNet50 (He et al., 2016)
as proof of concept.
We also demonstrate a useful example of DeepRecon through model ﬁngerprinting in a transfer learning attack.
Finally, we propose countermeasures to obfuscate an attacker from extracting the correct attributes and sequences using observations attacks like DeepRecon and show that these defenses signiﬁcantly increase the errors in the extracted attributes and can be implemented in various DL frameworks without hardware or operating system support.
B ACKGROUND
As opposed to attacks that exploit vulnerabilities in software or algorithm implementations, sidechannel attacks utilize information leaks from vulnerabilities in the implementation of computer systems.
Due to modern micro-processor architecture that shares the last-level cache (L3 cache) between CPU cores, cache side-channel attacks have become more readily available to implement.
Since the cache is involved in almost all the memory access activities on a machine, it can be a medium that includes abundant information about programs running on the host.
The fundamental idea of the attack is to monitor the access time to the shared contents, e.g., shared libraries or credentials, between a victim and an attacker while the attacker ﬁlls the cache set with the addresses known to her (Prime+Probe (Liu et al., 2015)) or keeps ﬂushing the shared data from the cache (Flush+Reload (Yarom & Falkner, 2014)).
In both the cases, once the victim accesses memory or shared data, the attacker can identify which memory addresses or shared data is accessed.
Prior work has demonstrated that, with cache side-channels, an attacker can construct covert channels between processes, stealing cryptographic keys, or breaking the isolation between virtual machines (Zhang et al., 2014; Liu et al., 2015).
F LUSH +R ELOAD
Our attack leverages the Flush+Reload technique, which monitors accesses to memory addresses in shared contents.
The technique assumes that an attacker can run a spy process on the same host machine.
This enables the attacker to monitor the shared data or libraries between her and the victim.
During monitoring, the attacker repeatedly calls the clflush assembly instruction to evict the L cache lines storing shared content and continually measures the time to reload the content.
A fast reload time indicates the data was loaded into the cache by the victim whereas a slow reload time means the data is not used.
From this information, the attacker determines what data is currently in use and identiﬁes the control ﬂow (order of function calls) of the victim’s process.
We chose Flush+Reload over Prime+Probe since the results from Flush+Reload includes less noises.
A TTACKS ON B LACK -B OX D EEP N EURAL N ETWORKS
Prior work has proposed various methods to attack black-box DNNs.
Tram ` er et al.
(2016) and
Papernot et al.
(2017) demonstrated model extraction attacks to black-box DNNs that aim to learn a substitute model by using the data available to the attacker and labeled by observing the query results.
Fredrikson et al.
(2015) and Shokri et al.
(2017) demonstrated model inversion attacks on black-box DNNs that reveal a user’s private information in the training data leveraging model predictions.
Wang & Gong (2018) proposed a hyper-parameter stealing attack that aims to estimate the hyper-parameter values used to train a victim model.
However, these attacks require unrealistic conditions, e.g., the architecture of the victim network needs to be known to attackers, or the victim uses a network with simple structures, such as multi-layer perceptrons.
Thus, the capability of
DeepRecon attack that reconstructs black-box DNN architectures can bridge the gap between the realistic black-box and their conditions.
R ECONSTRUCTING B LACK -B OX DNN S VIA S IDE -C HANNELS
Recent studies have discovered various methods to extract the architecture of a black-box DNN.
Memory and Timing Side-Channels: Hua et al.
(2018) monitored off-chip memory accesses to extract the network architecture of a victim model running on a hardware accelerator.
They estimated the possible architecture conﬁgurations and extracted model parameters.
However, the attack requires physical accesses to the hardware, whereas our attack does not.
Power Side-Channel: Wei et al.
(2018) demonstrated that an attacker can recover an input image from collected power traces without knowing the detailed parameters in the victim network.
However, this approach also assumed an attacker who knows the architecture of a victim network, so our attack could help meet the assumptions of this attack as well.
Cache Side-Channel: Concurrent work by Yan et al.
(2018) demonstrates that an attacker can reveal the architecture details by reverse engineering and attacking generalized matrix multiply (GeMM) libraries.
However, GeMM-based reverse engineering can only reveal the number of parameters of convolutional or fully connected layers because others such as activations and poolings are difﬁcult to characterize by matrix multiplications.
Also, in order for the monitored functions in GeMM libraries to be in a shared instruction cache of an attacker and a victim, the multiplications must occur on the CPU.
However, DeepRecon can be performed independent of the hardware on which the computations occur, generalizing better common hardware on which DNNs run (e
Using Known Student Models: Wang et al.
(2018) proposed a transfer learning technique in which an attacker identiﬁes teacher models by using known student models available from the Internet.
This approach assumed that the victim selects the teacher from a set of known architectures.
We, however, take this a step further and ﬁngerprint families of architectures as well as many commonly known teacher models.
Additionally, we are able to reconstruct arbitrary teacher model architectures with high accuracy.
Meta-Models: Oh et al.
(2018) demonstrated that an attacker can estimate the victim’s architecture by using a brute-force approach and meta-models.
They ﬁrst trained all the possible architectures of a given set and pruned the models with inferior performance.
Then, they trained a meta-model that identiﬁed the network architecture using mutated samples and labels.
However, the pruning process is time intensive (i.e., 40 GPU days for 10k candidates of LeNet (LeCun, 1998)), and the candidates were selected from limited architectural choices, whereas we again go a step further in identifying families of architectures and can generalize to previously unknown teacher models.
D EEP R ECON A TTACK 3. T HREAT M ODEL
Our threat model requires an attacker who can launch a co-located user-level process on the same host machine as the victim.
This ensures the attacker and the victim’s process share the same instruction cache.
The co-location also allows our attacker to observe the victim DNN’s behaviors without actively querying the model, which is commonly assumed in the black-box attack literature.
Consider the example of any computer that an attacker has access to at a user-level, the attacker can log into this machine and attack other users with DeepRecon.
Another way for an attacker to achieve co-location is to disguise her process as a benign program such as an extension for a browser.
Once some victims install the extension in their browser, the attacker can easily launch a monitoring process.
We also assume that the attacker and victim use the same open-source DL frameworks shared across users.
Importantly, this assumption can be easily met because many popular DL frameworks such as Tensorﬂow (Abadi et al., 2016) or PyTorch 1 are provided as open-source libraries, and this practice of sharing libraries across users is default on major operating systems, e.g., Windows, MacOS, and Ubuntu.
Thus, our attacker can identify the addresses of functions to monitor in the instruction cache by reverse-engineering the shared framework’s source code.
Motivating Attack Example: We provide a practical example where our threat model is applicable. Suppose an attacker aims to install malware on a victim’s machine where an anti-virus system, based on a DNN model, is running. To evade malware detection in common black-box attacks, an attacker ﬁrst actively drops malicious programs to monitor the model’s decisions and then synthesizes evasive samples based on the collected data. However, when the attacker drops multiple ﬁles, her behavior can be detected by the victim.
On the other hand, our attacker induces the victim to install a chrome add-on (which runs at a user-level) that passively monitors cache behaviors of the model and extracts the architecture. Then, the attacker trains a surrogate model with public datasets (including malware and benign software). With the surrogate model, the attacker crafts her malware to evade the detection by the model. As opposed to common black box attacks models, our attacker lowers the possibility of being caught because she only monitors the behaviors of the victim model passively and does not need to actively query the victim model.
3. A TTACK O VERVIEW
The overview of DeepRecon attack is described in Fig.
The victim’s behaviors are depicted with the dotted lines (black), and the attacker’s actions are described with the solid lines (red).
While preparing the attack ofﬂine, the attacker ﬁrst analyzes the deep learning framework that the victim uses and collects the target functions corresponding to the architecture attributes that the attacker wants (Table 1).
Then later, the attacker launches a co-located process at the user-level that runs along with the victim’s process on the same host machine.
When the victim’s process runs training or predictions with its model, the target functions are invoked and are loaded into the shared instruction cache.
The attacker periodically ﬂushes the cache lines and measures the access time to the target functions.
If the victim invokes any of the target functions after ﬂushing, the following access time measured by the attacker will be measurably faster than if the victim does not invoke them.
The attacker collects the number and sequence of invocations and then extracts the victim model’s architecture attributes.
The attacker reconstructs the victim model’s architecture.
3. R EVERSE E NGINEERING
In Table 1, we analyze the TensorFlow v1.9.0-rc0 framework 2 and list the target functions corresponding to the architecture attributes.
We chose TensorFlow due to its popularity as an open source machine learning (ML) framework, but believe that the methods we describe will be applicable to most, if not all, other popular frameworks.
In addition to having found some corresponding functions in another popular framework, PyTorch/Caffe2, our attack leverages the inherent structure of a scalable and widely deployable ML framework, namely library layer abstraction.
All the of the functions we monitor in TensorFlow are in the core of the library, both below the API interface and above the system dependent code.
Because of this, our attack not only does not depend on the speciﬁc TensorFlow API a victim uses, but also is agnostic to the type of processing hardware the victim is using, from a single CPU to a cluster of GPUs.
The speciﬁc functions we monitor in Table 1 represent two subgroups: those corresponding to control ﬂow and those corresponding to architecture attributes.
The control ﬂow functions allow us to observe the number of queries the victim makes to the model and the number of layers that are updated by gradient decent if we observe the model when it is being trained The function that monitors the number of queries is especially important, as it allows us to separate individual observations.
The architecture attribute functions are called once per instance of an architecture attribute being present in the neural network, allowing us to see the number of each attribute and the sequence in which they occur in the victim’s architecture.
Combined, these functions allow us to observe the architecture attributes of a neural network from start to ﬁnish on a given observation.
Additionally, the bias operator gradient function, given in the table by #grads, can allow an attacker to ﬁgure out the total number of layers that are updated during training time if the attacker observes the training of the model.
Using this information, the attacker, already knowing the total number of layers in the architecture, can ﬁnd the point at which the victim is freezing the backpropagation.
This allows the attacker to know which layers are directly inherited from the training model and [Table] Arch. [Table]
which layers are speciﬁcally trained by the victim.
The relevance of this will be discussed in our application of DeepRecon to model ﬁngerprinting (Sec. 4). Limitations.
Similar to concurrent work (Yan et al., 2018), we are also able to extract additional information, such as the number of parameters in convolutional and fully connected layers by monitoring the matrix multiplications in the Eigen library 3 on which TensorFlow is built. This attack provides more ﬁne-grained information, but it does not generalize to computations on hardware other than a CPU. We also examine whether our attack can recover the inputs to the model and its parameters. By varying inputs and parameters while monitoring the functions used to compute these parameters using a code coverage tool, GCOV 4 , we ﬁnd that the framework implements matrix multiplications of parameters in a data-independent way.
Thus, we are unable to estimate the inputs and parameters of a victim model.
We hypothesize that this is a general limit of cache based side-channel attacks on DNNs that target instructions
[Table]
and that obtaining the parameters is reducible to the problem of reading arbitrary victim memory
[Table]
3. E XTRACTING A RCHITECTURE A TTRIBUTES Arch.
[Table]
VGG19 and ResNet50.
We consider the 8 attributes extracted from short and long attacks.
In the short attacks, We report the average values observed in ten random queries whereas the long attacks include the averages of ten continuous queries.
We run our attack on Ubuntu 16.04 running on a host machine equipped with the i7-4600M processor (8 cores and 4MB L3 cache).
Our victim and attacker processes are running at the user-level on the same operating system (OS).
Both the processes utilize the TensorFlow v1.9.0-rc0 framework.
The victim runs predictions with the DNN model, and the attacker launches the Flush+Reload attack using the Mastik toolkit (Yarom, 2016) to monitor the target functions.
A total of 13 convolutional neural network (CNN) architectures are considered in our experiment:
DenseNet121, DenseNet169, DenseNet201 (Huang et al., 2017), VGG16, VGG19 (Simonyan &
Zisserman, 2014), ResNet50, ResNet101, ResNet152 (He et al., 2016), InceptionV3, InceptionResNet, Xception (Szegedy et al., 2015), MobileNetV1, and MobileNetV2 5 (Howard et al., 2017).
Table 2 includes the extraction results from monitoring VGG16 and Resnet50.
The full extraction results from the 13 networks are in Appendix D.
We ﬁrst show the results from a S hort attack, where an attacker can only run her process on a short interval of time, observing only a single query of the network.
We randomly choose ten individual queries and average the attributes.
We report errors as the sum of absolute deviations from ground truths.
In VGG19, our attacker has 2.6 errors on average and 3.1 in ResNet50.
We also show the extraction results from 10 continuous observations, in which the attacker runs her process for a more extended period of time.
The error rates in both the networks are similar.
These results demonstrate that DeepRecon achieves better accuracy by only observing a running network than prior work that assumes query access (Oh et al., 2018).
3. R ECONSTRUCTING THE A RCHITECTURE OF B LACK -B OX D EEP N EURAL N ETWORKS
[Figure]
[Table]
[Figure] [Table] [Figure] Based on the extracted architecture information, DeepRecon reconstructs the entire DNN architecture of the victim model.
In these examples, we focus on the fact that most CNN architectures consist of the feature extractor and classiﬁer layers.
The feature extractor is located in the earlier layers and is the combination of basic building blocks.
The classiﬁer is a set of fully connected layers at the end of the network.
In VGGs and ResNets, there are standard blocks used in each CNN architecture as we depict in Fig. 2.
Each block includes activation layers with preceding convolutional layers.
In the classiﬁer layers, each fully connected layer is followed by an activation layer.
We describe the reconstruction process of ResNet50 in Table 3. (Note that we also reconstructed VGG16 and show the result in Appendix A.) In the table above, we compare the computation sequences observed by the attacker with the actual computations in the ResNet50 network.
We can see the sequences are accurately captured with a few errors.
The three steps at the bottom describe the reconstruction processes of our attacker.
Our attacker ﬁrst identiﬁes (1) the number of blocks by counting the (max-)pooling layers.
Once the attacker separates blocks with the pooling layer locations, she counts (2) the number of convolutional layers in each block.
In ResNets, we know that
the
Residual block has four convolutional layers, and the Identity block has three convolutional layers in each.
Thus, the attacker can identify the type of each block.
After that, the attacker estimates (3) the number of fully connected layers attached to the last.
Finally, with this block-level information, our attacker successfully estimates the victim architecture is the ResNet50 with high accuracy.
Discussion about the Reconstruction Errors.
We also examine whether the errors in our experiments have speciﬁc patterns, allowing our attacker to cancel them out.
However, we could not ﬁnd any pattern: the types and locations of error attributes are different in each run (over 10 runs).
Thus, we attribute these errors to two primary causes.
First, there can be background noise from other
[Table]
ResNet
G
C R P M
C R C R C C M R C R C R C M R C R C R C M R C R C R C C M R C R C R C M R C R C R C M R C R C R C M R C R C R C C M R C R C R C M R C R C R C M R
C R C R C M R
C R C R C M R C R C R C M R C R C R C C M R C R C R C M R C R C R C M R
P A F
So
S
C R P M
C C R C R C M R C R C R C M R C R C R C M R C C R C R C M R C R C R ...
M R C R C R C M R
C R C R C M R C C R C R C M R C R C R C M R C R C R C M R
C R C R C M R
C R C R C M R
R C R C M R C C R C R C M R C R C R C M R C R C R C M R
P A F
So
[Table] [Table]
[Table] [Table]
(Note
C , P , F , M indicate the C onvolutional, P
, F ully
, M layers, and the mean
( R : ReLU and
So : processes that our Flush+Reload attack picks up, e.g., a process can pull data into the L3 cache and evict the target function between when the victim calls the function and we reload it.
In this case, our attacker cannot observe the victim calling the function.
Second, our attack can experience common errors associated with Flush+Reload attack (Yarom & Falkner, 2014), e.g., a victim invokes the target function when we reload, causing our attacker to see a cache miss instead of correctly observing a cache hit.
F INGERPRINTING B LACK -B OX N EURAL N ETWORKS
Our attacker identiﬁes victim neural network architectures using statistical models trained on the attributes as features and the architectures as labels.
This is a powerful capability to have if our attacker aims to ﬁngerprint the architecture of the pre-trained models used in transfer learning.
Transfer learning is typically done in a ﬁne-tuned manner: a user creates a student model that uses the architecture and parameters of a teacher model and trains only a few fully connected layers at the end of the network by freezing the backpropagation of the preceding layers.
We also found that our attacker can learn the layers in the student model not updated during training when they observe the training process (Sec. 3.3).
Thus, our attacker can extract both the network architecture and frozen layers of the student (or victim) model.
Once our attacker identiﬁes the victim network with this information, the attacker can utilize several pre-trained models available from the Internet to perform further attacks. For instance, an attacker can increase the success rate of black-box attacks by crafting adversarial samples with the internal representation from the teacher models (Wang et al., 2018). Additionally, since adversarial samples transfer across different models for the same task (Tram`er et al., 2017), the attacker is not required to use the exact same teacher model—i.e., she can use any pre-trained model of an architecture family that achieves the similar accuracy on a task. An attacker can also perform model extractions easily since the model parameters from the pre-trained models are used in a victim model in this setting, and they can also be found readily available on the Internet.
Finally, if the attacker has a partial knowledge of the victim’s training data and can gain the knowlege of which layers were frozen during training (see Sec. 3.3), she can fully estimate the parameters of the entire network by independently training the last few layers that were not frozen.
To evaluate, we train decision tree classiﬁers on the 13 networks used in Sec. 3.4 that identify the network architectures using the extracted attributes and labels.
We extract the attributes overqueries in each network (650 in total) and utilize 5-fold cross-validations.
We measure the classiﬁcation accuracy and analyze the four most essential attributes based on the mutual information (MI)
[Table] scores. Since the attributes are not affected by the host machines or operating systems, the attacker can train the models ofﬂine for use in attacks.
Table 4 shows the results of ﬁngerprinting the neural networks.
We conduct three types of classiﬁcation tasks with the aim of identifying 1) the entire 13 networks, 2) 5 network families, and 3) architecture variants in each network 6 .
We report the accuracy of best decision trees and the average accuracy over the cross-validations.
In all the tasks, our decision trees achieve 100% accuracy, which demonstrates, once trained, these statistical models can be perfect predictors.
(Note that we also visualize our data in an attribute space via PCA analysis in Appendix C).
We also identiﬁed the four essential attributes across all the classiﬁcations: 1) #relus , 2) #merges , 3) #convs , and 4) #apools .
Identifying these inﬂuential attributes can guide a potential obfuscation-based defensive strategy against such side-channel attacks.
D EFENSES TO D EEP R ECON A TTACK
Previous studies on defenses against cache side-channel attacks (Kong et al., 2013; Zhou et al., 2016) require speciﬁc hardware (page-locked cache) or kernel-level features (page coloring).
These solutions have not been widely deployed or have remained as optional features because of their impact on computational performance.
Hence, we propose framework-level defenses that do not require specialized hardware or kernel-level updates.
Our ﬁndings in Sec.
4 regarding the essential architecture attributes for an attack, e.g. #relus , #convs , and #merges , guide our search for defenses.
As a result, we propose obfuscating the attacker’s observations of these attributes.
We show that these defenses signiﬁcantly reduce the success of our DeepRecon attack, and they can be extended against a more general class cache side-channel attacks against DL frameworks.
5. R UNNING D ECOY P ROCESSES WITH T INY M ODELS
DeepRecon and other potential cache side-channel attacks on deep learning frameworks can only observe that a library function is called, but not by whom.
By running an extra process (i.e. a decoy process ) simultaneously with the actual process, we develop a simple but effective defensive strategy.
The decoy process also invokes the target functions in the shared framework, which obscures the architecture attributes and computation sequences.
To minimize the computational overhead we utilize networks that only have a few layers, referred to as TinyNets .
To evaluate our defense, we train these TinyNets at the same time as the victim’s process is running ResNet50, and we measure the number of errors in the extracted attributes.
The results are listed in Table 5.
We experiment with three TinyNets: 1) only with a Conv. layer (C:1), 2) Conv.
with a ReLU layer (C:1, R:1), and 3) two Conv.
and ReLU layers with a Merge layer.
We found the existence of a decoy process signiﬁcantly hinders the attacker’s ability to extract correct attributes, causing 1283-2211 errors in the attribute extractions.
Contrasted with up to 2.
errors on average from DeepRecon’s previous extractions (Section 3.4), we ﬁnd that this defense is exceedingly effective at curbing this type of reconstruction attack.
In Table 5, we can increase 6 In the task 3), we consider MobileNet and MobileNetV2 as the same family.
[Table]
the errors associated with the attributes that we aim to obfuscate.
For instance, when we run the
TinyNet with only one convolutional layer, we observe the #convs is signiﬁcantly increased. This is important because with our defenses, a defender can choose the attributes to obfuscate. Since the defender can control what noise gets introduced, they can also dynamically and adaptively change what noise is added into the attackers observations, thereby increasing our defenses effectiveness and generalizability. To quantify the overhead required of this defense, we measure the average network inference time with and without a decoy process
[Table]
and we observe that the defense increases the inference time by only 5.91-8.38 seconds per inference.
Thus, this defense is a reasonable measure to combat cache side-channel attacks that reconstruct DNNs.
5. O BLIVIOUS M ODEL C OMPUTATIONS
One defense against DeepRecon is to obfuscate the computations (i.e., function invocations) observed by our attacker.
We propose two approaches.
First, we can update the victim’s architecture by adding extra layers.
To minimize the side-effects such as performance loss, these layers returns the output with the same dimensions as the input.
For instance, the convolutional layer with the kernel 3, padding 1, and strides 1 can preserve the input dimensions, and the identity block of the
ResNets preserves as well.
Thus, we augment the original architecture by adding such layers at the random location to make the same architecture look different in the attacker’s point of view.
Prior work (Targ et al., 2016) has shown the unraveled view of the ResNet architecture.
Under this view, the skip-connections of a network can be expressed as the ensemble of multiple computational paths that can be computed independently.
Hence, we secondly split a computational path with skip-connections into multiple paths without skip-connections.
In forward propagation, the multiple paths are randomly chosen and computed so that our attacker is hard to capture the exact architecture.
To evaluate our intuition, we construct the obfuscated architecture of ResNet50 (see Appendix B) and extract the attributes by using our attack.
Our results are shown in Table 6.
[Table]
Using this defense, the errors detected by DeepRecon increased from 2-3 to 28 for ResNet50.
During this test, the ﬁrst 3 blocks over the entire 16 blocks of ResNet50 are obfuscated.
While less effective than our previous defense, we conclude that this defense still can marginally obfuscate the observations of our attacker our attacker.
Additionally, the gain on computational time is also small: it increased from 17.88 to 24.03 seconds.
C ONCLUSION
This paper conducts the ﬁrst in-depth security analysis of DNN ﬁngerprinting attacks that exploit cache side-channels.
We ﬁrst deﬁne the realistic threat model for these attacks: our attacker does not require the ability to query the victim model; she runs a co-located process on the host machine where the victims DL system is running and passively monitors the accesses of target functions in a shared framework.
We also present DeepRecon, an attack that reconstructs the architecture of the victim network using the architecture attributes extracted via the Flush+Reload technique.
Based on the extracted attributes, we also demonstrate that an attacker can build a meta-model that perfectly ﬁngerprints the architecture and family of the pre-trained model in a transfer learning setting.
With the meta-model, we identiﬁed the essential attributes for these attacks.
Finally, we propose and evaluate new framework-level defense techniques that obfuscate our attackers observations.
Our empirical security analysis represents a step toward understanding how DNNs are vulnerable to side-channel attacks
A PPENDIX A R ECONSTRUCTION OF VGG16 A RCHITECTURE
Data
[Table] Sequences (Layers in the Ground Truth)
VGG
G
C R C R P
C R C R P C R C R C R P
C R C R C R P C R C R C R P
F R F R F So S
C R C R P
C R C R P C R C R C R P
C R C R C R P C R C R C R P
F R F R F
So
[Table]
We describe the reconstruction process of VGG16 in Table 7.
The upper table indicates the sequences that our attacker captured, and the bottom shows the actual reconstruction steps.
Our attacker identiﬁes the basic building blocks by splitting the sequence with pooling layers.
Then, the attacker counts the number of convolutional layers in each block.
In VGG16, we found the ﬁrst two ConvNet blocks have two convolutional layers, and the next three ConvNet blocks have three convolutional layers in each.
Additionally, the attacker estimates the number of fully connected layers attached at the end.
Once the attacker recovers all the blocks, our attacker can identify the victim architecture as being VGG16 with the ConvNet conﬁguration ‘C’ or ‘D’
[Figure]
In Sec. 5.2, we construct the obfuscated ResNet50 architecture by using the unraveled view of the ﬁrst three blocks in ResNet50 as shown in Fig.
The upper architecture depicts the block connections in the original ResNet50.
In this network, the blocks are computed sequentially, e.g., Residual Block → Identity Block → Identity Block.
However, in our unraveled architecture at the bottom, there are individual 8 paths that can be computed independently.
We use this architecture to compute the blocks as follows: Residual Block 1, 2 → Identity Block 1 → Residual Block 3,→ Identity Block 2 → Identity Block 3.
This makes our attacker have difﬁculty in estimating the architecture attributes and computation sequences of ResNet50.
C V ISUALIZING C LUSTERS OF N EURAL N ETWORKS IN A TTRIBUTES S PACE
In the Fig.
4, we clearly see that each network forms a distinct cluster in the attribute space.
Also, the networks of the same family, e.g., VGGs, DenseNets, MobileNets, has the clusters close to
[Table]
D A TTRIBUTES E XTRACTION R ESULTS FROM O THER N ETWORKS
We show the attributes extraction results with the other 11 networks in Table
Arch.
Data
Attributes
[Table]
VGG
G
S 13.
L 13.
ResNet
G
S 106.
L 107.
ResNet
G
S 164.
L 164.
DenseNet
G
S 125.
L 125.
DenseNet
G
S 174.
L 175.
DenseNet
G
S 203.
L 203.
InceptionV
G
S 86.
L 86.
InceptionResNet
G
S 234.
L 234.
Xception
GS 41.
L 41.
Mobile (1.0)
GS 15.
L 15.
MobileV2 (1.0)
GS 35. L 35.