Model Obfuscation for Securing Deployed Neural Networks
Authors: Mingyi Zhou
Xiang Gao
Jing Wu
John C. Grundy
Xiao Chen
Chunyang Chen
Li Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: model obfuscation
AI safety
AI system

Abstract: More and more edge devices and mobile apps are leveraging deep learning (DL) capabilities. Deploying such models on devices -- referred to as on-device models -- rather than as remote cloud-hosted services, has gained popularity as it avoids transmitting user's data off of the device and for high response time. However, on-device models can be easily attacked, as they can be accessed by unpacking corresponding apps and the model is fully exposed to attackers. Recent studies show that adversaries can easily generate white-box-like attacks for an on-device model or even inverse its training data. To protect on-device models from white-box attacks, we propose a novel technique called model obfuscation. Specifically, model obfuscation hides and obfuscates the key information -- structure, parameters and attributes -- of models by renaming, parameter encapsulation, neural structure obfuscation, shortcut injection, and extra layer injection. We have developed a prototype tool ModelObfuscator to automatically obfuscate on-device TFLite models. Our experiments show that this proposed approach can dramatically improve model security by significantly increasing the overhead of extracting models' inner information, without increasing the latency of DL models. Our proposed on-device model obfuscation has the potential to be a fundamental technique for on-device model deployment. Our prototype tool is publicly available at https://github.com/AnonymousAuthor000/Code2536.

Under review as a conference paper at ICLR 2023
MODEL OBFUSCATION FOR SECURING DEPLOYED NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
More and more edge devices and mobile apps are leveraging deep learning (DL) capabilities. Deploying such models devices referred on-device models rrather than as remote cloud-hosted services, has gained popularity as it avoids transmitting user's data off of the device and for high response time. However, on-device models can be easily attacked, as they can be accessed by unpacking corresponding apps and the model fully exposed attackers. Recent studies show that adversaries can easily generate white-box-liee model or even inverse its training data. protect on-device models from white- box attacks, we propose novel technique called model obfuscation. Specifically, model obfuscation hides and obfuscates the key information - structure, param- eters and attributes of models by renaming, parameter encapsulation, neural structure obfuscation, shortcut injection, and extra layer injection. We have de- veloped a prototype tool ModelObfuscator to automatically obfuscate on-device TFLite models. Our experiments show that this proposed approach can dramati- cally improve model security by significantly increasing the difficulty of extract- ing models' inner information, without increasing the latency of DL. models. Our proposed on-device model obfuscation has the potential be a fundamental tech- nique for on-device model deployment. Our prototype tool is publicly available at com httt:s//tfthttoo.cocoooooooooooo
attacks for an on-device
INTRODUCTION
Numerous edge and mobile devices are leveraging deep learning (DL) capabilities. Though DL mod- els can be deployed on a cloud platform, data transmission between mobile devices and the cloud may compromise user privacy and suffer from severe latency and throughput issues. To achieve high-level security, users' personal data should not be sent outside the device. To achieve high throughput and short response time, especially for large number of devices, on-device DL models are needed. The capabilities of newer mobile devices and some edge devices keep increasing, with more powerful systems on chip (SoCs) and large amount of memory, making them suitable for running on-device models. Indeed, many intelligent applications have already been deployed on devices (Xu et al. 2019) and benefited millions of users.
Unfortunately, it has been shown that on-device DL models can be easily extracted. Then, the extracted model can be used to produce many kinds of attacks, such adversarial attacks, member- ship inference attacks, model inversion attacks, etc. (Szegedy et al.
2013 Chen et al., 2017b] Shokri al. 2017 Fang et al. 2020). The deployed DL model can be extracted by three kinds of attacks: (1) extracting the model's 's weights through queries (Tramèr et al.
2016 (2) extracting the entire
model from devices using software analysis (Vallée-Rai et al. 2010) or reverse engineering (Li al.)
2021bb. (3) extracting the model's architecture by side-channel attacks et al. 2021a).
According to our observation, existing defense methods can be categorized into two different levels: (1) algorithm level, and (2) side-channel level. For securing the Al model at the algorithm level,
some studies (Orekondy et al. 2019b; Kariyappa & Qureshi 2020: Mazeika et al.) 2022) propose methods degenerate the effectiveness of query- query-based model extraction. While other studies (Xul cet al.] 2018; Szentannai et al. 2019; 2020) propose methods to train a simulating model, which has similar performance to the original model but is more resilient extraction attacks. For securing

---

Under review as a conference paper at ICLR 2023
the Al model at the side-channel level, a recent work modifies the CPU and Memory costs resist the model extraction attacks (Li et al. 2021a).
Although many attacks have been proposed to extract DL models, it is hard for adversaries to pre- cisely reconstruct DL models that are identical to the original oncs using queries or side-channel information. These attack cannot access the inner information of the model, which means they are black-box attacks. In contrast, since on-device models are delivered in mobile apps and hosted on mobile devices, adversaries can easily unpack the mobile apps to extract the original models for exploitation. It will enable serious intellectual property leakage and adversaries can further gener- ate white-box attacks, which are much more effective than black-box attacks (Zhang et al., 2022). Despite that model extraction using software analysis may lead to severe consequences, to the best of our knowledge, the community has not yet been aware of this attack, and no effective defense method has been proposed against it.
In this paper, we propose a novel model protection approach based on model obfuscation, which focuses on improving AI safety for resisting model extraction using software analysis. Given a trained model and its underlying DL library (e.g., PyTorch, TensorFlow, TFLite and etc.), an end2end prototype tool, ModelObfussator,, is developed to generate the obfuscated on-device model and the corresponding DL library. Specifically. ModelObfuscatto first extracts the information of the target model and locates the source code in the library used by its layers. Then, it obfuscates the information of the models and builds a customized DL library that is compatible with the obfuscated model. To achieve this, we design five obfuscation methods, including: (1) renaming, (2) parame- ter encapsulation. (3) neural structure obfuscation, (4) random shortcut injection, and (5) random extra layer injection. These obfuscation methods significantly increase the difficulty of parsing the information of the model. The model obfuscation can prevent adversaries from reconstructing the model. In addition, adversaries also hard to transfer the trained weights and structure of models to steal intellectual property using model conversion because the connection between the obfuscated information and the original one randomly generated. Experiments on 10 different models show that ModelObfuxcator can against state-of-the-art model parsing and attack tools with a negligible time overhead and 20% storage overhead. Our contributions in this work include:
We propose the model obfuscation framework to hide the key information of deployed DL models at the software level. It can prevent adversaries from generating white-box attacks and stealing the knowledge of on-device models.
We design five obfuscation strategies for protecting on-device models and provide an end2end prototype tool, ModelOOfuscator. This tool automatically obfuscates the model and builds a com- patible DL software library. The tool is open-source available.
We provide a taxonomy and comparison of different obfuscation methods in terms of effectiveness and overhead, to guide model owners in choosing appropriate defense strategies.
2 RELATED WORK
Model Extraction Attacks and Defenses For model extraction attacks, adversaries can effec- tively extract the model in black-box setting. They can use collected samples to query the target model to reconstruct the target model (Tramer et al., 2016 Papernot et al. [017 Orekondy et al, 2019a He et al.] [021 Rakin et al.] (022), or use the synthetic sample to steal the information of target models (Zhou et al. 2020 K Kariyappa et al. [2021] Yuan et al. [202
| [Sanyal et al. 2022). For defending against the model extraction attacks, various methods (Orekondy et al. [019b]] Kariyappa & Qureshi 2020; Mazeika et al. 2022 have been proposed to degenerate the performance of model extraction attacks. Some methods (Szentannai et al. 2019: 2020) have been proposed to train simulating model, which has similar performance to the original model, but can reduce the effee- tiveness of attacks. In addition, watermarking is also promising method to defend against the model extraction (Yang et al. 2019 Fan et al. 2019 Lukas et al. 2019).
Ad Adversarial Machine Learning: Currently, adversaries can use many kinds of attacks to chal- lenge the reliability of DL models, such as adversarial attacks, membership inference attacks, model stealing attacks, and model inversion attacks. For the adversarial attack, depending on the knowl- edge required by the adversary, adversarial attacks can be categorized into white-box attacks such as gradient- attacks (Croce & Hein 2020 Goodfellow et al. 2013 Kurakin et al. 2017 Papernot et al. 2016, Moosavi-Deztood et al.)
2016 Madry al. 2018 Moosavi-Deztooli et al. 2017). and
2

---

Under review as a conference paper at ICLR 2023
Producing L'te model
compatibilty chock
program
generate
compilo
inference
Python
TEL library
user
TemsodFlow library
TF model
Tiii model
b. ModetObfuscattor
Renaming
Paratteter encupaulation
assembling
inference
modified iF library
Neural mode obitiscation
obiluscasod TFI ite model
(1) model passing
(3) package compillttoo
Shortcst injectioon
Shortcut injection
TFLite source code
modified source cod2
(2) model obliscation
Figure 1: The working process of our ModelObfuscator on-device DL model obfuscation tool.
black-bor attacks such as query-based attacks (Chen et al. 201 Ilyas et al.
2018a Guo et al,
2019 Brendel et al. 2017 Cheng et al. 20188] Chen al. [020 Mopuri al.
2018). For member- ship inference attacks, several studies (Shokri et al. 2017) Truex et al. 2019 Choquette-Chune et al.)
2021 Carlini et al.] (202) challenge the
ability of model by predicting whether
a sample is in the collected training set.
Code obfuscation: Code obfuscation methods are initially developed for hiding the functionality of the malware. Then, the software industry also uses it against reverse engineering (Schrittwieser
itt al.
2016). They provide complex obfuscating algorithms for programs like JAVA code (Collberg
1997 1998), including robust methods for high-level languages (Wangl 2001) and machine code level (Wroblewski, 2002) obfuscation. Code obfuscation is a well-developed technique to secure the source code. However, traditional code obfuscation approaches are not capable of pro- tecting on-device models, especially for protect the structure of the models and their parameters. In this work, inspired by traditional code obfuscation, we propose a novel model obfuscation approach to obfuscate the model files and then produce corresponding DI. library for them.
al.
METHODOLOGY
Threat Model The on-device model is usually saved as separate file (e.g., .tflite file) and packed into the app package. The attackers can either download the target app from the app markets (e.g.. Google Play and iOS App store) or extract the app package file (e.g., APK file for Android, and IPA file for iOS) from the hosting devices. These app package files can then be decompiled by off-the-shelf enggree-ngiineerin tools (e.g., Apktool I file. Although many on-device DL platforms do not support some advanced functions like back- propagation, attackers can assemble the model architecture and weights into a differentiable model format, or they can use software analysis methods to generate attacks for the target model file (Li al., 202Tb] [Huang & Chen 2022). In this study, we will obfuscate the information of the model to disable the software analysis sis and model conversion tools, and generate compatible DL library for the obfuscated model that only supports the forward inference function.
and IDA Pro i to get the original DL model
We chose the TensorFlow Lite (TFLite) platform to demonstrate our model obfuscation approach. TFLite is currently the most commonly used on-device DL platform. The steps to produce TFLite models are shown in the top half of Figure [I] Usually, DL developers use TensorFlow application programming interfaces (APIs) to define and train the TensorFlow (TF) model. The trained TF model is then compiled to TFLite model. Note that TFLite has different implementations with TF, their operators (i.e., layers) may not be compatible. Therefore, the TFLite library will check the compatibility during compilation. Once the compatibility check passed, the compiled TFLite model can run on devices using the TFLite library. Hence, the problem of obfuscating TFLite models is to design obfuscation strategies and make the obfuscated model compatible with the TFLite library.
We analyzed the current mainstream DL platforms (i.e., TensorFlow and Py Torch) and identified the following two main findings. First, these platforms are open-source and provide a set of tools to build
ic Apktoo1/ watpb/ii//pkktoooooo
https://hex patp
3

---

Under review as a conference paper at ICLR 2023
the library (e.g., TFLite library) from the source code. Second, they officially support customized operators (e.g.. neural layers). Specifically, on top of these DL platforms, users could implement cus- tomized layers in C/C++ and compile customized layers to executing files (. so file in TensorFlow). Then, users can use these customized layers via high-level Python interfaces. Those features enable us design obfuscation techniques to obfuscate model and DL. library code together. The bottom half of Figure [ shows the overview of our model obfuscation framework. Specifically, ModelOb- fuscator has three main steps: (I) model parsing, (2) model obfuscation, and (3) model assembling & library recompilation. In the following subsections, we detail our proposed ModelObfuscatoo.
3.1 MODEL PARSING
The first step of ModelObfuseator model obfuscation is to parse the deployed model to extract its key information. ModelOb- fuscator first extracts the structure information of each layer, in- cluding the name of layers (e.g., Conv2D) and model structures (including model's input, output, layer ID and etc.). The ex- tracted structure information are depicted using a data structure as shown in the right example. Then, ModelOhfuscator will ex- tract the parameter of each layer. Moreover, ModelOhfusccator will identify the source code used by each layer by referring to the underlying libraries. The identi- fied source code includes relevant packages or functions of the TFLite layers.
Minimal model structure
"Layer name": "Conv2D",
"Input": (0].
"OOtpul": [1]. "ID":
3.2 MODEL OBFUSCATION
After getting the model information and corresponding source codes, ModelObfuscator will obfuscate the model as well as the source codes. ModelObfuscator uses five obfuscation strategies: renaming, parameter encapsulation, neural struc- ture obfuscation, shortcut injection, and extra layer injection.
Obfuscated minimal model structure
"Layer name": "'buuzzp" "Input": [0].
"Ourpat": [I]. "ID": 0.
The most straightforward obfuscation strategy is the renaming of layer. Usually, the layer's name contains important information, which is the function of this layer. For instance, "Conv2D" indicates a 2D convolution layer. Such information is useful for adversaries to reconstruct the model to generate white-box attacks or to obtain a similar surrogate model to conduct effective black-box attacks. To hide such important information, we randomly change each model layer's 's name. On the right an example of an obfuscated Conv2D layer. Mod- elObfuscator automatically replaces the real name with the random meaningless string Tbuszp. Meanwhile, ModelObfuscator creates copy of Conv2D's 's source code and replaces the layer name (i.e., Conv2D) in the source code with Tbuszp. Note that we modify the duplicate of Conv2D in case the modification affects other parts of the TFLite library. After adding modified source codes into the TFLite project, the recompiled TFLite library will recognize obfuscated layers custom layers and correctly execute them at runtime.
Renaming
Source code registration (C/C++ code)
Til Rteeegittrtionn " Register Conv2D0)
TILiicccgisttation Register.Tbusap()
Parameter encapsulation Existing TFLite models have two main assets: model structure and pa- rameters. The parameter can be obtained in the training period. Given an input, a TFLite model will compute the results using the input tensor and parameters that are stored in the model file. As we discussed above, parameter exposure is very dangerous. An adversary could use the parameter in- formation to perform many kinds of white-box box attacks, e.g., adversarial attacks and model inversion attacks. Besides, an adversary can guess the function of this layer according to the shape of the pa- rameter, because different layers have different numbers of parameters (e.g., two in the convolution layer) and different shapes of parameters (e.g.. (3, 3, 64) in the convolution layer).
To hide key model parameter information, we instead encapsulate parameters into their correspond- ing generated custom source codes of the obfuscated layer. For example, for simple one-layer feed-forward neural network, the output can be computed by Y = 8(W"X x + b), where X, W, and b is input tensor, parameter of the layer, and bias, respectively. 0 is the activation for neural nodes. For ModelObfusscato obfuscation, the network layer can be disguised as Y = g(X), where g is an unknown function. We then implement the correct computation (ie., g) in the generated
4

---

Under review as a conference paper at ICLR 2023
Ippel Spore
lrpelt byer
Inpull Lawer
Heden
Hedden Lapar
Layer
Caya
Outbut
(a) Orginal neural nodes
(b) Random neural nodes
(c) Unchunged neural nodes
Figure 2: Neural structure obfuscation for a simple feed- heed-forward neural network.
Coro2D
DepohaaleCorr20
Cowwd
Wer Cfficta
wigght Clatessrtt
1x112-10011
1-50x
Aelus
Aelii
Ref
Aelus
(a)
neural structure obfuscation
renaming
parameter encapsulation
Wizwh
Vinbee
Cinsy
1x236x22843
1x2341224r5
Yiuumu
extra layer injection
22200-2003
1x220-224-3
shortcut injection
(b)
Figure 3: Example of shortcut injection and extra layer injection for the TFLite model extract from real-world app. (a) part of the original model. (b) the corresponding obfuscated model. This
visualization is generated by Netron (Roeder 2017).
custom TFLite source code, which then obfuscate. At runtime, function will be invoked to achieve the computation from X to Y. Now an adversary is unable to extract the key parameter information from obfuscated model. Furthermore, the implementation of be obfuscated us- ing transitional and well- well-proven code obfuscation strategies (Collberg & Thomborson (2002). After compiling the modified TFLite, adversaries will find very hard identify key model parameters by reverse engineering the compiled library.
Neural structure obfuscation Just obfuscating layer names and parameters is not enough, since an adversary may still infer the function of each layer according to the model structure. For instance, Figure a() presents the structure of the neural network, where the input, hidden and output layers include four, two, and one node, respectively. Attacker could search for a surrogate model according neural structure obfuscation
to the neural architecture. To solve this problem, ModelOhfuscator to obfuscated neural architecture with the goal of confusing the adversary. We propose two strategies for network structure obfuscation: random and align-to-largest Given model with output shapes (So. s,), s, where Sn refers to the number of dimensions for the n-th channel, the random strategy generate random shape r (Io...- .In) of the output for each layer. Figure Figure2(bb shows an obfuscated model of Figure a) using random strategy. Second, the align-to-largen strategy finds the largest output shape and then fill the output shapes of other layers to the size of s' Figure (c) shows such an obfuscated model, where the output shapes of each layer are filled up to (4). Note that this will not affect the performance models because the modified TFLite library will not compute the output using the provided neural structure information.
Shortcut injection & extra layer injection Neural structure obfuscation changes the network struc- ture by inserting new nodes, but the spatial relationships of original layers remain the same. There- fore, even with the above three obfuscation strategies, the adversary can still infer node informa- tion by analyzing the spatial relationships of runtime data (e.g., actual input-output values of each node). To further obfuscate the model structure, hence, ModelObfuscator applies two more strate- gies: shortcut and extra layer injection. The injected shorteut and extra layers would destroy the original spatial relationships of TFLite models.
To automatically inject random shorteuts, ModelObfuscator first randomly select shortcut pair (r1, 12). The outputs r -th layer are then added to the input list r2-th layer. For example, the blue solid line in Figure (b) is a shorteut inserted into the model extracted from fruit recognition app, which connects the 2-nd convolution layer and the 5-th convolution layer. Second, to inject extra layers, just like the shorteut injection, ModelOhfuscator randomly picks layer pair (()
5

---

Under review as a conference paper at ICLR 2023
The obfuscation error of the proposed model obfuscation method. 'Error': the output difference between the original model and the obfuscated model.
Table 1:
03
Average
Error 0.0
0.0
0.0
0.0
0.0
0.0
0.0
The input of the extra layer is the output node list of r'-thh layer, and its output is added to the input list of r2-th layer. For instance, an extra layer Ypxumu (as shown in the red dotted line in Figure B)) is injected between the 2-nd and 3-rd convolution layers.
Note that the injected shortcut and layers will not affect the prediction results of the deployed model because the modified TFLite library will ignore the obfuscation part (e-g., Ypxumu layers) in model inference. Specifically, the extra layer Y f(X) just needs to create the output with a specific shape to confuse the adversary. In addition, extra layer injection will not significantly increase the latency of on-device models because the extra layer does not need much computational resources.
Combine five obfuscation strategies The proposed five obfuscation strategies are applied to the target model sequentially. Figure show an example, where Figure Baa) is the original model, while Figure 3bbb shows the final obfuscated model. The structure and parameters of the obfuscated model quite different from that of the original model, which will make the obfuscated model hard to attack. To prevent an adversary from parsing the obfuscated model through reverse enginecring the modified TFLite library and customized layers, use code obfuscation strategies, well-developed technology to make the code unreadable and unable to be reverse engineered (Appendix [B] The combination of model and code obfuscation would hide the key information of models.
3.3 MODEL ASSEMBLING AND LIBRARY RECOMPILATION
After obtaining the obfuscated model and modified TFLite source codes, ModelObfuscator assem- bles the new obfuscated model using the obfuscated model structure and then recompiles the modi- fied TFLite library support the newly generated obfuscated model.
EVALUATION
4.1
EXPERIMENTAL SETTING
Dataset To evaluate ModelObfuscator'ss performance on models with various structures for mul- tiple tasks, collected 10 TFLite models including fruit recognition model, skin cancer diag- nosis model, MobileNet (Howard et al. 2017), MNASNet (Tan et al., 2019), SqueezeNet (landola
2016), EfficientNet (Tan & Le 2019 MiDaS (Ranftl al. 2020). Dnception-ResNetty Pose et (Kendall et al., 2013). and SSD (Liu et al. 2016), which are referred to as model 1 1 re- spectively. The fruit recognition and skin cancer diagnosis model were collected from Android apps (see the provided code repository). The other models were collected from the TensorFlow Hub
Experimental Environment ModelObfuscator is evaluated on a workstation with Intel(R) Xeon(R) W-2175 2.50GHz CPU, 32GB RAM, and Ubuntu 20.04.1 operating system.
4.2 EFFECTIVENESS OF ModelObfuscator
We first evaluate the effectiveness of ModelObfascctoor. Ideally, ModelObfasscutor should not affect the prediction accuracy of the original models, while providing sufficient defense against model attacks. To this end, apply all five proposed obfuscation strategies to each model and compare the prediction results based on 1,000 randomly generated inputs. The obfuscation error is calculated as Ilv-y'lla where y and y' is the output of original models and obfuscated models, respectively. Note that the number of extra layers and shortcuts is set to 30 in shortcut injection & extra layer injection. model obfuscation strategies have no impact the
Table [ demonstrates that ModelObfuscate prediction results of the original models.
To show the effectiveness of ModelObfuscator in hiding the model's key information, we try to extract the model information of obfuscated models using five software analysis tools. The tools
https::/tt dev/

---

Under review as a conference paper at ICLR 2023
Table 2: The success number of existing software analysis tools extract mobile models with each obfuscation strategy. 'Basic obfuscation': renaming + parameter encapsulation.
TF-ONNX
TFLite2TF TFLite2ONNX
FlatBuffer
App Attack
Original
10
9
9
10
8
Renaming
0
8
Parameter encapsulation
0
0
0
2
Neural structure
0
0
0
0
8
Shortcut injection
0
0
8
Extra layer injection
0
8
Basic obfuscation
0
0
0
0
include TF-ONNX (Deeloperr) 2022) TFLite2ONNX (Wang) 2021 TFLite2TF (Hyodo 2022)
FlatBuffer (Li et al. 2021b). and Smart App Attack (Huang & Chen 2022), which is proposed to attack the on-device models. In this experiment, we apply the basic obfuscation strategies renam- ing and parameter encapsulation on the original models. For model conversion tools, they can successfully convert the model format, we consider it successful case for extracting the model information. If ModelOhfuscator method is effective, these tools cannot work on the models. For the app attacking method, if can correctly identify the obfuscated model that has the same model structure as the original one on TensorFlow Hub, we consider it a success case. If ModelObfusca-- tor is effective, the FlatBuffer extractor cannot parse the information of the obfuscated model and reverse it to the original one. As shown in Table 2 two basic obfuscation strategies can prevent all existing model extraction tools from parsing the deployed TFLite model. Besides, parameter encapsulation could prevent App Attack from finding surrogate models on 6 models. When com- bining renaming and Parameter encapsulation, the App Attack on all the models can be prevented. However, except for parameter encapsulation, applying other strategies separately does not confuse the App Attack, because App Attack can identify the same model through the parameter comparison.
(8, 138, 125)
I8 124, 128)
88. 124, 128)
64)
483
(24
(74, 48j
(24,
(24, 16, 169
(12, 24
(24, 16, 30
(22 24, 24)
(1,250)
(1,25
2509
(1.)
(1.128)
(1.138)
(3,128)
Figure 4: Visualization of structure obfuscation for LeNet. (a) original data flow of LeNet (b) obfuscated model with shortcut and extra layer (c) obfuscated model with three shorteuts and three extra layers. Red dotted line and red dotted block represent injected shortcut and extra layer.
As we discussed above, adversaries may infer the functionality of each layer by analyzing the data flows of the model. To demonstrate the capability of ModelObfusccaoo in obfuscating the data flows, we show a visualization of LeNet (LeCun et al.) 1998) before and after applying Shortcur injection & extra layer injection. The visualization of model structure and data flows before and after obfuscation are shown in Figure + As can be seen, difficult to recognize the shortcut or extra layer without prior knowledge of the model structure. When large number (e.g., more than 30) of shortcuts and extra layers is used to obfuscate the TFLite, the structure of the obfuscated model will become extremely confused (Figure 7 Therefore, the shortcut and extra layer injection are effective in making it extremely difficult to understand the model structure.
4.3 EFFICIENCY OF ModelObfusscator
We evaluate the efficiency of ModelObfusscator obfuscation strategies by demonstrating the run- time overhead of each model in our dataset. Specifically, we report both the time overhead and memory overhead of the proposed obfuscation method under various settings based 1,000 ran- domly generated instances. The results of time and memory overhead are shown in Table 3 and Table respectively. "(n1,"2)" in the table indicates the the number of shortcuts (n1) and num-
7

---

Under review as a conference paper at ICLR 2023
Table Time overhead (seconds per 1000 inputs) of the original model and obfuscated model. We use five obfuscation strategies. n?)); obfuscated models with ni shortcuts and n2 extra layers.
(
x
Average
Original
30.3
98.5
62.1
72.5
33.4
81.2
346.6
247.4
130.4
231.2
133.4
(0, 0)
30.4
98.7
61.1
70.4
33.5
82.3
346.4
247.0
130.7
232.1
133.3
(30, 0)
30.7
97.9
62.3
72.7
33.1
82.0
346.9
247.1
130.2
231.0
133.4
(0, 10)
30.2
98.6
61.1
74.1
34.9
82.3
346.7
251.5
130.9
231.6
133422
(0, 20)
30.6
98.3
63.3
74.1
36.8
81.3
348.2
247.1
130.5
231.3
134.2
(0, 30)
30.4
98.5
63.2
74.6
34.5
82.8
349.0
251.9
130.4
236.7
135.2
Table 4: Overhead of the model obfuscation on random access memory (RAM) cost (Mb per model).
We use five obfuscation strategies. To eliminate the influence of other processes the test machine, we show the increment of RAM usage for the model inference.
4
5
x
Average
Original
18.8
49.5
33.1
46.5
41.3
51.9
241.1
373.8
39.4
97.8
99.3
(0, 0)
18.9
53.3
33.1
49.9
45.5
56.8
244.8
381.2
40.6
99.8
102.4
(30, 0)
18.7
51.4
32.6
49.5
46.0
55.8
244.6
381.6
40.3
100.2
102.0
(0, 10)
25.9
58.8
42.5
53.6
49.2
69.5
248.1
386.7
55.0
107.5
109.7
(0, 20)
30.0
67.3
50.4
56.7
56.0
66.2
253.1
387.7
66.9
112.0
114.6
(0, 30)
35.9
81.9
55.2
57.3
71.6
73.0
267.7
397.3
73.4
123.8
123.71
ber of extra layers (n2) applied, where (0, 0) indicates only the basic obfuscations (i.e., renaming, parameter encapsulation) and neural structure are involved. We also include the time and memory consumption of the original models as the baseline. As shown Table even though extra layers injected into the obfuscated model, ModelObfuscator obfuscated models incur negligible time overhead (i.e., approximately 1% on average for the most time-consuming obfuscation). The differences between using various obfuscation settings are also not significant. Because the param- eter encapsulation will remove some data processing steps in the source code of APIs, the basic obfuscation ("00,)) in Table may reduce the latency of TFLite models.
The memory overhead for ModelObfiuscator obfuscated models shown in Table To eliminate the impact of different memory optimization methods, preserves all intermediate tensors. It can be seen that the other obfuscation strategies do not affect memory usage except for the extra layer injection. Memory usage increases when the number of extra layers increases. Therefore, a trade-off between the complexity of the obfuscation and the memory overhead is worth considering when choosing the obfuscation strategy. Nevertheless, we argue that even with the most complex settings in our experiment, which provide sufficient protection the original model, the memory overhead is acceptable (approximately 20%).
peak RAM usage where the model
Table 5: Size change (Mb) of the TFLite library and models after the obfuscation. The size of the obfuscated model is reduced few Kb. The original library size is 191.5 Mb. '+' and '-' refer to the increase and decrease, respectively.
6
@
@
library-all model
+8.7
+31.9
+19.6
+38.7
+8.8
+40.6
+126.1
+231.8
+9.7
+52.1
-5.5
-16,9
-10.3
-17.5
-5.0
-1866
-66.3
-121.1
-5.0
-2..5
total
+3.2
+15.0
+9.3
+21.2
+22.0
+59.8
+110.7
+4.7
+24.4
library-renaming
+8.7
+31.9
+19.6
+38.7
+8.8
+40.6
+126.1
+231.7
+9.7
+52.1
Considering that the models are deployed on mobile devices that have limited storage space, we also present the size differences of the modified TFLite software library and the obfuscated models. Note that the size change caused by creating additional so files to support the inference of the obfuscated layer. Hence, the size difference will be the same with different implementations (e.g., Python, Java). Table5] the size change to the TFLite software library and the TFLite models after applying ModelObfuscator obfuscation strategies. We use all obfuscation strategies in this experiment and the number injecting shortcuts and extra layers is 30. Our results show that the library size change is mainly caused by the renaming method, because it will create for the renamed layer in TFLite library. addition, the extra layer injection will also increase the size of library because it will create the new API support the extra layer. As the extra layer just
API
8

---

Under review as a conference paper at ICLR 2023
Canoerlage Oeftacatio
obfineaation
Neural mode obfuscation
injection
Stutter de oiimruttoon
Parameter encapeulation
Renaming
layer injection
obfuseaiion
expand ubfuscation
Figure 5: Taxonomy of model obfuscation methods.
has simple function, the affect of the extra layer injection is negligible. Our obfuscation strategies significantly reduce the size of the TFLite model file because it only keeps the obfuscated minimal structure information. However, the size of the TFLite library is significantly increased, and increases the size of the application deployed on the mobile device.
5 DISCUSSION
Taxonomyy of model obfuscation methods In this paper, proposed five different model ob- fuscation strategies. Figure [ shows a preliminary taxonomy of the different model obfuscation methods and the best practice for model deployment. First, developers can use the renaming and parameter encapsulation to prevent most model parsing or reverse engineering tools from extract- ing the information of the deployed model. In the scenarios that computational costs are critical, developers can use the neural structure obfuscation and shorteut injection, as they do not introduce additional overhead. For structure obfuscation, developers can use neural structure obfuscation, shoricul injection, and extra layer injection. These three methods can significantly increase the difficulty of understanding the data flow of the deployed models. In addition, developers can use renaming and neural structure obfuscation to disguise the deployed model, which mislead the adversary into choosing the wrong architecture to produce a surrogate model. If the size of the app package is critical (c.g.. deploying the model on devices with limited storage), developers need to carefully consider the trade-off between the number of obfuscated layers leveraging extra layer in- jection with renaming. The reason is that if renaming is used to create different obfuscated layer for every layer, ModeiObfusccttor will need to create the corresponding APIs in the TFLite library support the obfuscated layers, hence increasing the size of the library.
Limitations Although the proposed model obfuscation does not introduce significant computa- tional overhead, it will increase the size of the modified TFLite library. This is because need to provide support for the new obfuscations made. For a huge network like a 1000-layer network deployed model (although it is unlikely to find such deployed model in real world), the size of the modified TFLite library will significantly increase if we rename every layer. As a result, the app package also increases the modified TFLite library will also need to be deployed on the device.
Extracting obfuscated model When ModelObfuscator obfuscates the model, will create a cache file guide the tool to generate compatible DL. library. Attackers cannot automatically extract the obfuscated model unless they obtain the cache file from the developer's 's computer (which is unlikely to happen). Generally speaking, attackers must reverse engineering get source codes from the compiled library file and cost manual effort to understand them for extracting the obfuscated model.
CONCLUSION
In this work, we analyzed the risk of deep learning models deployed on mobile devices. Adversaries can extract information from the deployed model perform white-box attacks and steal its intellec- tual property. To this end, we proposed a model obfuscation framework to secure the deployment of DL. models. We utilized five obfuscation methods obfuscate the information of the deployed model, ie.,, renaming, parameter encapsulation, neural node obfuscation, shortcut injection, and extra layer injection. We developed a prototype tool ModelObfussatoo to automatically obfuscate a TFLite model and produce compatible library with the model. Experiments show that our method is effective resisting the model parsing tools without performance sacrifice. Considering the neg- ligible extra resources required, and the extra security it achieves, we believe that model obfuscation has the potential to be a fundamental step for on- on-device model deployment in the future. In future works, optimizing model obfuscation reduce overhead is worthwhile to be explored.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-bars adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv:1712.04245 2017.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Mem- bership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897-1914. IEEE, 2022.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjumpattack: A query-efficient decision- based attack. In 2020 ieee symposium security and privacy (sp), pp. 1277-1294. IEEE, 2020.
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15-26. ACM, 2017a.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 2017b.
Minhao Cheng, Thong Le, Pin-Yu Chen, Jinfeng Yi, Huan Zhang, and Cho-Jui Hsieh. Query- efficient hard-label black-box attack: An optimization-based approach. arXiv preprint 1807.0800 2018.
Christopher Choquette- Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In International conference on machine learning, pp. 1964-1974. PMLR, 2021.
Christian Collberg, Clark Thomborson, and Douglas Low. A taxonomy of obfuscating transforma- tions, 1997.
Christian Collberg, Clark Thomborson, and Douglas Low. Manufacturing cheap, resilient, and stealthy opaque constructs. In Proceedings of the 25th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 184-196, 1998.
Christian S. Collberg and Clark Thomborson. Watermarking, tamper-proofing, and obfuscation- tools for software protection. IEEE Transactions on software engineering, 28(8):735-766, 2002.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206- 2216. PMLR. 2020.
Developers. tf2onnx Convert TensorFlow, Keras, Tensorflow.js and Tilite models ONN, August 2022. URL
Lixin Fan, Kam Woh Ng, and Chee Seng Chan. Rethinking deep neural network ownership ver- ification: Embedding passports to defeat ambiguity attacks. Advances in neural information processing systems, 32, 2019.
Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. Local model poisoning attacks to (Byzantine-Reee Robust)) federated learning. In 29th USENIX Security Symposium (USENIX Secu- rity 20). pp. 1605-1622. 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. Simple black-box adversarial attacks. In International Conference on Machine Learning, pp. 2484-2493, 2019.
Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang. Stealing links from graph neural networks. 30th USENIX Security Symposium (USENIX Security 21), pp- 2669-2686, 2021.
10

---

Under review as a conference paper at ICLR 2023
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 2017.
Yujin Huang and Chunyang Chen. Smart app attack:
Hacking deep learning models in android apps. IEEE Transactions on Information Forensics and Security, 17:1827-1840, 2022.
Katsuya Hyodo. uffite2tensorflow, 2022. t Fflitettenssrfoow
URL
Forrest N landola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters andj 0.5 mb model size. arXiv preprint urXiv: 1602.07360 2016.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with 2018a.
limited queries and information. In ICML, pp.
Andrew Hyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversarial attacks with bandits and priors. arXiv preprint 1107:01777 2018b.
Sanjay Kariyappa and Moinuddin K Qureshi. Defending against model stealing attacks with adap- tive misinformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pp. 770-778, 2020.
Sanjay Kariyappa, Atul Prakash, and Moinuddin K Qureshi. Maze: Data-free model stealing attack using zeroth- gradient estimation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 13814-13823, 2021.
Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real- time 6-dof camera relocalization. In Proceedings of the IEEE international conference com- puter vision, pp. 2938-292999 2946, 2015.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. International Conference Learning Representations (ICLR), 2017.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2288-2324, 1998.
Jingtao Li, Zhezhi He, Adnan Siraj Rakin, Deliang Fan, and Chaitali Chakrabarti. Neurobfuscator:
A full-stack obfuscation tool to mitigate neural architecture stealing. In 202/ IEEE International Symposium on Hardware Oriented Security and Trust (HOST), pp. 148-258. IEEE, 2021a.
Yuanchun Li, Jiayi Hua, Haoyu Wang, Chunyang Chen, and Yunxin Liu. Deeppayload: Black-box backdoor attack on deep learning models through neural payload injection. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 263-274. IEEE, 2021b.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Kann Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference computer vision, pp. 21-37. Springer, 2016.
Nils Lukas, Yuxuan Zhang, and Florian Kerschbaum. Deep neural network fingerprinting by con- ferrable adversarial examples. arXiv preprint 1rXiv:19/2.00888 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations/CCRR, 2018. URL
http://pppneeiie.......ee
FJZIBfZAD
Mantas Mazeika, Bo Li, and David Forsyth. How to steer your adversary: Targeted and efficient. model stealing defenses with gradient redirection. In International Conference on Machine Learn- ing, pp. 15241-15254 PMLR, 2022.
Seyed-Mohsen Moosavi- Dezaool., Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on com- pufer vision and pattern recognition, pp. 2574-2582, 2016.

---

Under review as a conference paper at ICLR 2023
Seyed-Mohsen Moosavi- -eziool,,, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1765-1773, 2017.
Konda Reddy Mopuri, Phani Krishna Uppala, and R Venkatesh Babu. Ask, acquire, and attack: Data-free uap generation using class impressions. In Proceedings of the European Conference Computer Vision (ECCV), pp. 19-34, 2018.
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 4964-4963, 2019a.
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Prediction poisoning: Towards defenses against dnn model stealing attacks. arXiv preprint arXiv: 10908, 2019b.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, 7. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEF European Sympo- sium on Security and Privacy (EuroS&P), pp. 372-387. IEEE, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506-519, 2017.
Adnan Siraj Rakin, Md Hafizul Islam Chowdhuryy, Fan Yao, and Deliang Fan. Deepsteal: Advanced model extractions leveraging efficient weight stealing memories. 2022 IEEE Symposium on Security and Privacy (SP), pp. 1157-1174. IEEE, 2022.
René Ranfil, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transac- tions on pattern analysis and machine intelligence, 2020.
Lutz Roeder. Netron, Visualizer for neural network, deep learning, and machine learning mod- els, December 2017. URL htthtts::: //to///oooo/ org. /10, 5281/ 7109451] If you use Netron in your research, please cite it using these metadata.
Sunandini Sanyal, Sravanti Addepalli, and Venkatesh Babu. Towards data-free model stealing in hard label setting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15284-15293, 2022.
Sebastian Schritwieser,, Stefan Katzenbeisser, Johannes Kinder, Georg Merzdovnik, and Edgar Weippl. Protecting software through obfuscation: Can it keep pace with progress in code analy- sis? ACM Computing Surveys (CSUR), 49(1):1-37, 2016.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE symposium security and privacy (SP), pp. 3-18. IEEE, 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6 2013.
Kálmán Szentannai. Jalal Al-Afandi, and András Horváth. Mimosanet: An unrobust neural network preventing model stealing. arXiv preprint arXiv: 10077.00000600 2019.
Kálmán Szentannai, Jalal Al-Afandi, and András Horváth. Preventing neural network weight steal- ing via network obfuscation. In Science and Information Conference, pp. 1-11. Springer, 2020.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International conference on machine learning, pp. 6115- PMLR, 2019.
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoe V Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 2820-2823 2019.
12

---

Under review as a conference paper at ICLR 2023
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction (APIs). 25th USENIX security symposium (USENIX Security 16), pp. 601-618, 2016.
Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. Demystifying membership inference attacks in machine learning as a service. IEEE Transactions on Services Computing, 2019.
Raja Vallée-Rai Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. Soot: A java bytecode optimization framework. In CASCON First Decade High Impact Papers, pp. 214-224. 2010.
Chenxi Wang. A security architecture for survivability mechanisms. University of Virginia, 2001.
Zhenhua Wang. tflite2onnx Convert TensorFlow Lite models to ONNX, August 2021. URL
Gregory Wroblewski. General method program code obfuscation. 2002.
Hui Xu, Yuxin Su, Zirui Zhao, Yangfan Zhou, Michael Lyu, and Irwin King. Deepobfuscation: Securing the structure of convolutional neural networks via knowledge distillation. arXiv preprint arXiv:1806. 10313, 2018.
Mengwei Xu, Jiawei Liu, Yuanqiang Liu, Felix Xiaozhu Lin, Yunxin Liu, and Xuanzhe Liu. A first look deep learning apps on smartphones, In The World Wide Web Conference, pp. 2125-2136, 2019.
Ziqi Yang, Hung Dang, and Fe-Chien Chang. Effectiveness of distillation attack and countermeasure on neural network watermarking. arXiv preprint arXiv: 060466, 2019.
Xiaoyong Yuan, Leah Ding, Lan Zhang, Xiaolin Li, and Dapeng Oliver Wu. Es attack: Model stealing against deep neural networks without data hurdles. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022.
Chaoning Zhang, Philipp Benz, Adil Karjauv, Jae Won Cho, Kang Zhang, and In So Kweon. In- vestigating top-k white-box and transferable black-box attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15085-15094, 2022.
Mingyi Zhou, Jing Wu, Yipeng Liu, Shuaicheng Liu, and Ce Zhu. Dast: Data-fre free substitute training for adversarial attacks. In Proceedings of the IEEE/VV Conference on Computer Vision and Pattern Recognition, pp. 234-243 2020.
13

---

Under review as a conference paper at ICLR 2023
A APPENDIX THE VISUALIZATION OF THE MODEL OBFUSCATION
Figure 6: The visualization of the original SqueezeNet model. It is plotted by Netron Roeder 2017).
14

---

Under review as a conference paper at ICLR 2023
Figure 7: The visualization the obfuscated SqueezeNet model with all obfuscation strategies. This figure is plotted by Netron (Roeder 2017). The number of injected shorteuts and extra layers 30. The obfuscated model can run on the modified TFLite library. The computational time and memory overhead of model obfuscation for this model can be found in Figure and 4 The size change can be found in Figure 5
B APPENDIX EXAMPLE OF CODE OBFUSCATION
The following code the 'Eval' function the modified source code to support the Concatenation layer. It will finally call the 'Evallmpl' function, which is the detailed im- plementation for the forward inference.
template <KernelType kernel type>
Staiue Eval (fLLiteeCooeet context, TíLiteNode node)
auto params 11 reinterprel.case
C P buwliodddd--uiiiiddddd )
11 int axis = params->axis;
TfLiteTensor* output;
ENSUREEEENUEE..... (context,
comteett, GetOutputSafe (context, node, 0, &output)) ));
if (IsConstantOrPeesitteet Tensor (output)) {
// Output is computed in Prepare. return kTfLiteOk:
if (axis 0) axis += output->dims>>szze
return Exallmpl<kkerel.uugg (contexxt, node, axis, output);
After obfuscating the simple code by some simple obfuscation method (e.g., comments removal, functions and variables renaming, whitespaces removal. We use open-source tool implement [. the code will become the following form:
tovpoipkaqun>TTTiitssaauu ojmfjdgxlkrj ( (TFLiteContext zsflmtabissg { (gggyxvwaxa output: DEFFLLIEEEESSREE 2sflmtabfslq, G node ,,,& output)); ));
kemplate-<lpuraacgelt
LLCoosstnOOPeccsisoooooooooonnon tsciszklzszy ;) ifff( if
tigeeettagkccorrpoiooaaonnnnnnnn node. dwzzpibyhyhk. output):]
15

---

Under review as a conference paper at ICLR 2023
Note that this is just simple example. For the case when the original model is 30-layer neural network, will create more than thousand lines of code for supporting the obfuscated model, and then add the code to the TFLite project to compile modified TFLite library. If adversaries use the reverse engineering method to extract the source code of the obfuscated layer, ideally they can get the above obfuscated code. But the code will be extremely difficult read. This is the reason why we think developers can use code obfuscation to resist further reverse engineering for the source code of the modified TFLite library.
16


Reviews:
[{"review_id": "qRi4Gud7xl", "reviewer": "Reviewer_Lpda", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper raises a bar against an adversary who aims to reverse-engineer efficient models deployed to edge devices. A naive deployment practice allows the attacker to just extract the model's architecture and parameters which work as a prior for other attacks (such as adversarial attacks). To address this issue, the paper proposes ModelObfuscator that converts a victim model's architecture and parameters. It also obfuscates the ML frameworks by obfuscating the function names (corresponding to layers) and the ways they perform the layerwise computations. In evaluation, the paper shows that the resulting models return the same outputs on a set of input samples even after applying the obfuscation, and the obfuscation method does not incur large extra computations.", "summary_of_the_review": "This paper studies a research problem that can be important in the future (as we deploy more and more models to edge devices). To address this issue, the paper proposes a method that obfuscates the model internals (such as architectures, parameters, and computation details).\n\nHowever, this defense has the following major problems:\n\n(1) The defense heavily relies on \"security by obscurity.\"\n(2) The defense was not evaluated against any reconstruction attacks.\n(3) The defense was not evaluated against an adaptive adversary.\n\nThus, I believe this paper (as-is) has more flaws than strengths. But, I am happy to update my review score if the authors address all the requested changes.", "strength_and_weaknesses": "Strengths:\n\n1. The paper studies a research problem that may carry some importance in the future.\n2. The paper proposes ModelObfuscator that employs six different ways to obfuscate a model.\n3. The paper shows the proposed mechanism does not change a model's behaviors nor increase its computations.\n\n\nWeaknesses:\n\n1. The approach relies on \"security by obscurity.\"\n2. The evaluation against existing reconstruction attacks and adaptive attacks should be done.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ocCcGFmkzx", "reviewer": "Reviewer_exnF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work introduces ModelObfuscator, an obfuscation strategy for deep neural networks. ModelObfuscator introduces 5 model obfuscation strategies, namely renaming, parameter encapsulation, neural structure obfuscation, shortcut injection, and extra layer injection. The authors demonstrate that ModelObfuscator (a) prevents model extraction tools from parsing the deployed model, (b) obfuscated models incur a negligible time overhead with (c) a model overhead of around 20%.", "summary_of_the_review": "I believe that this work holds some novelty. However, given the insufficient related work and my concern if this work will be of interest to the ICLR community I am currently leaning toward rejection. \n", "strength_and_weaknesses": "## Strengths\n(+) The topic of deep neural network obfuscation has not been studied extensively in the literature. This work holds novelty in the proposed model obfuscation techniques.  \n(+) ModelObfuscator achieves convincing results, supporting the claims made by the paper.  \n(+) The overall presentation of the paper is well-structured and easy to follow.  \n\n## Weaknesses\n(-) While the authors claim that their proactive defense could disallow a transformation from a non-differentiable model to a differentiable one, in a realistic attack scenario an attacker could simply train a differentiable substitute model by querying the deployed model to create his own dataset. It seems ModelObfusctator cannot prevent this simple attack. It would also be beneficial if the authors could elaborate on their claim why ModelObfuscator could disallow a transformation from a non-differentiable model to a differentiable one, and in which scenarios it is possible and in which it is not.  \n(-) In essence, the shortcut injection and extra layer injection are decoys to confuse the attacker. However, under the hood, the model architecture is not changed since the library will ignore these parts. It would be more interesting if such an obfuscation could be achieved by really changing the model architecture with the proposed methods.  \n(-) The authors did not include related work on model obfuscation [A-F]. Consequently, the authors failed to differentiate themselves from the existing literature and to compare their approach against existing methods.  \n(-) Given, that this work introduces obfuscation tricks, which are not really changing the underlying model architecture to make it more secure, I am not sure if this work is in the scope of the topics for ICLR. This work might be more suitable for a conference on the topic of security & privacy or software engineering.  \n\n[A] Anonymizing Machine Learning Models; ESORICS 2021  \n[B] DeepObfuscation: Securing the Structure of Convolutional Neural Networks via Knowledge Distillation; ArXiv 2018  \n[C] Mimosanet: An unrobust neural network preventing model stealing; CVPR-W 2019  \n[D] Preventing Neural Network Weight Stealing via Network Obfuscation; Intelligent Computing. SAI 2020   \n[E] Hardware-Assisted Intellectual Property Protection of Deep Learning Models; ACM/IEEE Design Automation Conference (DAC), 2020  \n[F] Preventing DNN Model IP Theft via Hardware Obfuscation; EEE Journal on Emerging and Selected Topics in Circuits and Systems 2021  \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "AvxJ0j_Nrdi", "reviewer": "Reviewer_pBz1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Motivated by the security of machine learning models running on device, this paper proposes ModelObfuscator, an obfuscation technique designed to protect models from multiple types of attacks, including adversarial examples, model extraction and membership inference. The proposed obfuscation includes five types of model changes, ranging from layer renaming, introduction of new layers and connections, and operation size masking. The correct predictions of the neural network are maintained after the obfuscation process. ModelObfuscator is implemented for TFLite models. Experiments are performed on ten TFLite architectures.", "summary_of_the_review": "Paper of limited scientific contribution, with weak motivation and poor evaluation.", "strength_and_weaknesses": "Strengths:\n- The paper proposes a method and implementation for obfuscating TFLite models.\n- The experiments show that models that use ModelObfuscator cannot be parsed by model parsers.\n- The implementation of ModelObfuscator is open-source.\n\nWeaknesses:\n- The motivation of the paper related to model security is loosely covered, inexact, and not followed throughout the paper.\n- The experiments do not evaluate the proposed obfuscation w.r.t. the main motivation of the paper, security goals.\n- More generally, the paper fails to motivate the need for neural network obfuscation, as neural network architectures are not particularly interpretable and sometimes even generated automatically via neural architecture search.\n- The paper seems to lack scientific novelty and can be viewed, at best, as an engineering feat (trivial obfuscation strategies).\n\nPlease see details below.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9aE-zK5a6dC", "reviewer": "Reviewer_17Yg", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose ModelObfuscator, a methodology that applies novel obfuscation techniques customised to hide relevant information regarding architecture and operations of the input neural network.\nAmong the obfuscation, ModelObfuscator can hide the real shape of the layers of the input neural network, re-arrange the data flow inside layers (but preserving the original one), and rename classes and functions.\nThe effectiveness and preciseness of the original model is preserved, with a negligible overhead.", "summary_of_the_review": "+ Obfuscation techniques customised for hiding meaningful information of deployed model, such as architecture and layers' shape\n+ Original performance is assured with negligible overhead", "strength_and_weaknesses": "**Strenghts**\n\n+ novel manipulations of neural networks that preserve the original performance, but rearranging and complicating the structure\n+ the core contribution of the paper can help bringing the gap between software obfuscation techniques and AI technologies\n+ converters can not deal with this new format of models\n+ open-source code is released (but maybe the authors could have submitted a Dockerfile that would have make the testing easier)\n\n**Weaknesses**\n\n+ the authors do not show how their manipulations are more effective than regular Android obfuscation techniques. In particular, the code-related manipulations (such as the class renaming) are not novel. An example of previous implementations can be found in ObfuskAPK [a]. The authors might compare their techniques, showing that regular tools can not obfuscate part of the machine learning model.\n+ converters do not work because this obfuscation is not known yet. Hence the authors should add this limitations to their work, or at least discuss what can be done once this obfuscation technique is known and how converters could be patched to take into account this technique.\n\n[a] Aonzo, S., Georgiu, G. C., Verderame, L., & Merlo, A. (2020). Obfuscapk: An open-source black-box obfuscation tool for Android apps. SoftwareX, 11, 100403.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject