# The Limits of SEMA on Distinguishing Similar Activation Functions of Embedded Deep Neural Networks
## 6 Related Works  
As DNNs are intellectual property, there are many attacks targeting these network models and parameters. DNNs on edge devices especially are quickly becoming a target. There are many model extraction attacks being researched using side-channel attacks. Some of the previous works were focused on an NN embedded on an FPGA accelerator. [PERSON] et al. used power analysis to extract weights and bias from the adder-tree function of the binarized neural network (BNN) hardware by focusing on the switching activity of registers [27]. [PERSON] et al. used EM side-channel information leakages to accurately recover the large-scale BNN accelerators embedded on an FPGA [11]. This study is based on a black box setting, but can only collect EM side-channel information under inference operations and observe the networks' outputs. They recovered the architecture of a targeted BNN through the analysis of EM emanations from the data bus and built substitute models with both a classification performance and layout architecture similar to the victim BNNs on the same hardware platforms using such information. [PERSON] et al. targeted a practical BNN hardware and showed that the overall structure of the BNN can be identified by the corresponding EM emanations [28]. In addition, they targeted the convolution layers. With actual experiments, the weight values convolved in the layers were revealed by EM side-channel attacks. There are also works around GPUs. [PERSON] et al. used EM emanations to recover the architecture in a black box setting, but requiring physical access to the system [29]. Their objective was to learn the relation between extracted architectural hints such as the volumes of memory reads/writes obtained by side-channel and model internal architectures, and the results showed that a high accuracy of model extraction could be achieved. There are also works on VGG architectures. The black box NN extraction attack was presented by recovering the layer's depth through exploiting timing side-channel information [30]. [PERSON] et al. [30] utilized a reinforcement-learning-based neural architecture search to search for the optimal substitute model with functionality and test accuracy similar to the victim networks. [PERSON] et al. demonstrated that even simple power and timing analysis enable recovering an embedded neural network model with different precision on three microcontroller platforms [31]. They used the same microcontroller platform Atmel ATmega328P; however, they did not distinguish activation functions like in our work. [PERSON] et al. demonstrated that one can utilize the cache-based SCA of Flush+Reload to spy on DNN models' computations and recover the label information [32]. There also exists side-channel analysis, which targets the parameters of the NN model, one of the parameters being activation functions. [PERSON] et al. showed that NNs implemented on CPUs can be fully reverse-engineered using merely side-channel leakages in a black box setting [15]. In [15], the activation function was discerned by the timing attack from its distinctive computation time with multiple executions and inputs. [PERSON] et al. reverse-engineered neural network designs, such as the number of layers and neurons, as well as the types of activation functions on GPUs through SEMA [26]. In [26], the activation function was discerned by the timing observed from a singular execution of SEMA. [PERSON] et al. identified activation functions using SEMA [16]. In [16], the activation function was discerned by observing the timing and unique patterns from a singular execution of SEMA. In [15; 16; 26], the activation functions distinguished were the Sigmoidfunction, Tanh function, softmax function, and ReLU function. In this work, we additionally distinguished similar activation functions, the Leaky ReLU function, SELU function, Swish function, and Fast Sigmoid function. These additional activation functions have not yet been distinguished to the best of our knowledge.