## 2 Theoretical Background  
### Convolutional Neural Networks (CNNs)  
Convolutional neural networks (CNNs) consist of deep layered architectures, in which multidimensional algebra operations are carried out sequentially. These architectures match especially well with multidimensional data since the spatial information of the latter is preserved while being treated. For instance, the algebraic representation of RGB images needs two dimensions for populating pixel matrices, and a third dimension for RGB channels. The capabilities of CNNs were originally exploited for image classification tasks [22; 50], but nowadays the potentials of these deep architectures are acknowledged in numerous fields concerning multidimensional data [25; 26; 27; 42; 51]. Generally, CNNs include three types of layers, corresponding to specific operations: convolution, pooling, and fully connected layers (Figure 1). Figure 1: General architecture of a CNN. Convolutional layers convolve input tensors \(x^{l-1}\) of the layer \((l-1)\) by using filter kernels \(k^{l}\)[5; 51; 52] as reported in Equation (1):  
\[x^{l}=\sigma\Big{(}x^{l-1}*k^{l}+b^{l}\Big{)} \tag{1}\]  
where \(*\) is the convolution operator, \(b^{l}\) the bias term of the layer \(l\), and \(\sigma\) an activation function, which introduces nonlinear effects. Rectified Linear Unit (ReLU) is one of the most common activation functions for convolutional layers: \(ReLU(x)=\max(0,x)\). The output of the convolutional layer is the feature map \(x^{l}\). A training process essentially attempts at optimizing the elements of the kernel \(k^{l}\). Such an optimization produces output feature maps that emphasize the most significant attributes for categorizing input data. This aspect is factually the core idea underlying automated feature extraction in CNNs, which apprehend dominant patterns in training data. The feature extraction occurring in stacked convolutional layers is a hierarchical process: first convolutional layers typically learn general and low-level features, whereas deeper layers learn more complex, specific, and high-level features. Figure 2 shows an example of hierarchical feature learning for the well-known CNN AlexNet [22] trained on the ImageNet dataset. Such representations can be produced by passing a random noise image through the network for several iterations. Namely, the input noise is gradually modified by updating the network gradient in order to maximize the activation of a chosen layer. Then, the convolutional structure outputs images that mostly activate the channels of the specific layer. These maps highlight the features learned by the network in such a layer. Pooling layers usually follow convolutional layers in CNNs architectures (Figure 1). The output of pooling layers is a low-resolution feature map, which is obtained through downsampling the input map. Pooling operations reduce the number of network parameters in order to mitigate data overfitting. Furthermore, the low-dimensional outputs of pooling layers extract local features, while preserving the multidimensionality of data. An example of downsampling is provided by the Max Pooling function, which only saves the maximum value of the input map in the regions passed by the filter. Instead, Global Average Pooling performs a global mean over the input map (Figure 3). The features learned over stacked convolutional and pooling layers merge in the fully connected layers which receive flattened input vectors. The number of neurons of the last fully connected layer coincides with the number of output classes. For typical classification tasks, the activation function of output layers is a saturating function such as Softmax, which can account for class probability. When new data are provided to the network, the learned features activate differently and culminate in the last fully connected layers for the assessment of the output class. Figure 2: Examples of features extracted from AlexNet trained with ImageNet data: (**a**) low-level features; (**b**) medium-level features; (**c**) high-level features.