# The Limits of SEMA on Distinguishing Similar Activation Functions of Embedded Deep Neural Networks
## 3 Identification Using Timing
### Timing Attack on Activation Functions  
The Tanh function is distinguished from the Sigmoid function since it is more symmetric in the timing pattern for positive and negative inputs compared to Sigmoid, as shown in Figure 2a,c. We also recreated the attack on the Swish function, drawn in Figure 2b. By observing the timing signatures, it is difficult to tell the Sigmoid function apart from the Swish function. The two functions are computed in a very similar way, and the pattern is indistinguishable with the timing attack. However, the minimum computation time for the Sigmoid function and Swish function is slightly different, being the only element to distinguish.  
The three functions, ReLU, Leaky ReLU, and SELU, have a similar computation time; however, they have different equations for positive and negative inputs, causing distinct patterns. The timing patterns of these functions are depicted in Figure 2d-f. The ReLU function and Leaky ReLU function are very similar and are a mirror image of each other. The Leaky ReLU function has a independent computation time at input 0; however, this is due to the implementation method. The Leaky ReLU function, i.e., Equation (6), can also be implemented like Equation (10) to blend the computation time of input 0 with the other positive inputs.  
\[h(a)=\begin{cases}\beta a,&\text{for }a<0\\ a,&\text{for }a\geq 0\end{cases} \tag{10}\]  
By comparing Figure 2a,b,d-f, the SELU function appears to be a mix of a pattern similar to the ReLU function for positive inputs and a pattern similar to the Sigmoid function for negative inputs. The unique pattern for the negative input is due to the exponential operation in the function.  
The Fast Sigmoid function is an approximation of the Sigmoid function. By observing Figure 2g,h, the patterns are very unique to the other functions. The timing patterns appear to be symmetrical with some individual timing for some inputs. The computation times also differ quite significantly from the other functions, making for easy classification.  
Figure 2: Timing Behavior of Similar Activation Functions. (**a**) Timing Behavior for Sigmoid. (**b**) Timing Behavior for Swish. (**c**) Timing Behavior for Tanh. (**d**) Timing Behavior for ReLU. (**e**) Timing Behavior for Leaky ReLU. (**f**) Timing Behavior for SELU. (**g**) Timing Behavior for Fast Sigmoid when \(n=1\). (**h**) Timing Behavior for Fast Sigmoid When \(n=9\).  
The features of the timing behavior can be summarized as follows:  
* The Sigmoid and Swish functions are the same pattern, with a slightly different minimum computation time.
* The Tanh function is not symmetrical compared to the Sigmoid function, which has a similar computation time.
* The ReLU and Leaky ReLU functions are mirror images of each other with a similar computation time.
* The SELU function is similar to the Sigmoid function for negative inputs and similar to the ReLU function for positive inputs.
* The Fast Sigmoid function is symmetrical, yet significantly different from the other functions in the computation time.