# Physical Side-Channel Attacks on Embedded Neural Networks: A Survey
## 2 Background  
### Some Background on Neural Networks (NN)  
The existing attacks on the literature have mainly targeted two classes of NNs (in the remainder of this manuscript, we use the acronym NN to refer generically to any type of neural network, irrespective of whether the network is actually deep or not. We use the terms DNN/CNN whenever required to be more specific) mostly used for classification problems: Multi Layer Perceptron (MLP) and Convolutional Neural Networks CNN. This section provides some background on NNs as related to the attacks surveyed in this paper. The reader is referred to [9] for more detailed information with respect to DNNs. An MLP is a feed-forward network which is composed of several layers and each layer comprises a number of neurons. Each neuron of each layer is connected with a certain strength connection called weight to every node of the next layer. Data circulates in one manner, from the input layer to the output layer (feed-forward). MLPs include at least three different layers: an input layer, an output layer and one or more hidden layers (see Figure 1). Each neuron receives as inputs (\(x\)) the results of its previous layer connection(s), calculates a weighted summation added with a certain _bias_ (see Equation (1)) and then applies a nonlinear transformation to finally compute its activation output (\(y\)). This latter output is passed onto the next connected layer neurons as their input. \[y=Activation\big{(}\sum(weight\cdot x)+bias\big{)} \tag{1}\]  
Different nonlinear _activation_ functions exist, including logistic (sigmoid), Tanh, softmax and Rectified Linear Unit (ReLU) functions. ReLU is one of the most widely used ones as it has the particularity of activating only a certain number of neurons at the same time, rather than activating them all at a given moment. The network is then sparse and easier to compute as only a subset of neurons is activated at any time [10]. CNNs are a variant of the standard MLP NNs mainly used for the classification of multi-dimensional data, especially images. CNNs are distinguished by the convolution layer. The latter is responsible for the feature extraction through a convolution operation between the high-dimensional input data and kernels, which are small matrices of parameters, in order to generate the layer output. Finally, each output layer is passed as input to the next layer. In addition to the convolution layer, other layers might be included in CNNs such as non-linear functions, pooling and fully-connected layers. Further details on CNNs can be found in [9; 11]. BNNs (Binarized Neural Networks) [12] are a subfamily of CNNs that binarize input and output activations and weights values. This converts floating point multiplication operations into single-bit XNOR operations, which significantly reduce storage requirements and computational complexity. However, binarization causes a loss of information and deviation from full-precision results; therefore, optimized training strategies are still being studied [13]. BNNs are a promising solution for IoT/edge resource-constrained and power-constrained devices where trading some accuracy loss for efficiency can be accepted. All NNs have two stages: training and inference. In the training stage, the network is fed with pre-classified training inputs along with their classification results, eventually converging into a set of parameters values (weights and bias) able to compute the correct output results for its specific classification problem. During the inference stage, the pre-trained network can perform its task by processing the input data and computing the output by using the weights determined during the training stage. When looking at a standard NN model, we can refer to its architecture consisted of _macro-parameters_, such as number of layers, neurons per layer and activation function, and to its _micro-parameters_ (or simply _parameters_) such as weights and biases. Figure 1: Multi Layer Perceptron, where \(x\) is the input and \(y\) is the output of a neuron.