# Smart Embedded System for Skin Cancer Classification
## 2 Background and Related Work  
### Convolutional Neural Networks  
A convolutional neural network (CNN) is a type of deep neural network used to analyze images. It has been applied successfully for image classification, image segmentation and object detection. The main layer of a CNN is the convolutional layer. Given a set of input maps, the convolutional layer runs the convolution of several kernels of weights of size \((n_{out},\leavevmode\
obreak\ in_{z},\leavevmode\
obreak\ k_{xy},\leavevmode \
obreak\ k_{xy})\) and a bias vector of size \((1,n_{out})\), where \(n_{out}\) is the number of different filters, \(in_{z}\) is the depth of each filter, equal to the depth of the input image, and \(k_{xy}\) is the size of the filter. Each convolution between one kernel and the input maps generates an output map. After running the convolution with multiple kernels, the result is a 3D map of size \((n_{out},\leavevmode\
obreak\ n_{y_{in}}-k_{xy}+1,\leavevmode\
obreak\ n_{x_{in}}- k_{xy}+1)\). The behavior of each convolutional layer can be modified by two parameters: the stride and the padding. The stride determines the sliding size of the convolution; that is, the number of input pixels that each kernel slides over the map. In a normal convolution, where the filter slides through all input pixels, the stride is one. The stride is used to reduce the size of output maps relative to the size of the input map. Padding is used to preserve the size of the output map. One form of padding adds zeros at the border of the image. Convolutional layers are used to extract features of an image. These features are then forwarded, for example, to a classifier to determine the class that the image belongs to in image classification problems. This classifier is usually implemented with a dense layer that determines the matrix multiplication between a matrix of weights, \(w\), of size \((n_{in},\leavevmode\
obreak\ n_{out})\), and the input map flattened to a vector of size \(1\times n_{in}\) followed by an addition with a bias vector, \(b\), with size \((1,\leavevmode\
obreak\ n_{out})\). Its output function is \(A(x.w+b)\), where \(A(\cdot)\) is an activation function. Equation (1) illustrates the multiply and accumulate operation required to calculate the \(n_{out}\) outputs. \[out_{j}=\sum_{i=1}^{n_{in}}x_{i}\cdot w_{ij}+b_{j},\leavevmode\
obreak\ \leavevmode\
obreak\ out\in\mathbf{R}^{n_{out}} \tag{1}\]  
An important operation of deep neural networks is batch normalization [12], a method used to accelerate the training of the neural network model. It normalizes the layer inputs to a mean of 0 and variance of 1, and then scales and shifts the normalized inputs with learnable variables, \(\gamma\) and \(\beta\). During training, the training dataset is partitioned into batches. Batch normalization estimates a vector of means, \(\mu\in\mathbf{R}^{N}\), and a vector of variances, \(\sigma^{2}\in\mathbf{R}^{N}\), for each channel of the input image for each batch. The input shape will be \((M,C,Y,X)\), where \(M\) is the batch size and \(C\) is the number of \(X\times Y\) planes. First, as seen in Equation (2), where \(I\) represents the inputs, \(X\times Y\) values are averaged per channel, resulting in an \(\overline{I}\), shaped as \((M,C)\). Then, the mean and variance values are calculated using Equations (3). \[\overline{I}(m,c) =\frac{1}{X.Y}\sum_{i=1}^{Y}\sum_{j=1}^{X}I(m,c,i,j) \tag{2}\] \[\mu(c) =\frac{1}{M}\sum_{i=1}^{M}\overline{I}(i,c)\] (3) \[\sigma^{2}(c) =\frac{1}{M}\sum_{i=1}^{M}(\overline{I}(i,c)-\mu(c))^{2}\]  
The mean and variance determined during training are used to normalize, scale and shift the inputs according to Equation (4), where \(\gamma\) and \(\beta\) are the variables learned during the training process and \(\epsilon\) is a small constant value, commonly \(10^{-5}\), used for numerical stability [12].