## 3 Data and Methods
### CNNs
#### 3.1.1 Architecture of CNNs  
The typical CNN structure consists of three types of layers, which are the convolutional layer, the pooling layer, and the fully connected layer. In addition to the above main layers of a CNN, it may include layers such as a non-linear activation layer, a classification layer, a dropout layer, and a pre-processing layer. The convolutional and non-linear layers are very commonly combined into one. Brief information about these layers is presented below. The convolutional layer, as its name suggests, is the most significant component in CNN architecture and plays an important role in its operation. The main purpose of these layers is feature extraction: they learn feature representations of their input images. The convolutional layer consists of a set of convolutional filters called kernels. The kernel is a grid of discrete numbers or values called kernel weights. At the beginning of the CNN training process, random numbers are assigned as the weights of the kernel. The input image is convolved with filters to generate the feature map of the output. The kernel slides over the whole image horizontally and vertically. A dot product is determined between the input image and the kernel, where their corresponding values are simultaneously multiplied and then summed up to calculate a single scalar value. The whole process is repeated until no further sliding is possible. The calculated dot product values create the output feature map. By optimizing the output of convolutional layers, they are able to significantly reduce model complexity. The output is optimized through three hyperparameters: the depth, the stride, and setting zero-padding. Non-linear activation functions allow for the extraction of non-linear features. The main purpose of all types of activation functions is to map the input to the output. The neuron's input value is determined by computing the summation of weighted outputs from all neurons in the previous layer. In the first layer of the CNN, the neuron's input is the numeric representation of the image. Thus, the activation function decides which neuron to fire with reference to a particular input by creating the corresponding output. These layers allow the CNN to learn more complex things. Non-linear activation layers are applied after all convolutional and fully connected layers, i.e., after layers with weights. The following types of activation functions are commonly used in CNNs: sigmoid, tanh, ReLU (rectified linear unit), and leaky and noisy ReLUs [20]. In this study, an ReLU is chosen as the activation function [21]. The choice of this activation function is explained by the fact that it has simple computation, good sparsity, a fast convergence speed, and solves the gradient dispersion problem caused by sigmoid and tanh functions [22]. The purpose of the pooling layer is to reduce the spatial resolution of the feature maps, due to which spatial invariance to input distortions and shifts is achieved. At this layer, part of the data is discarded, decreasing the complexity of the model. This, in turn, reduces the chance of it overfitting. There are several types of pooling methods. These methods include average pooling, min pooling, max pooling, tree pooling, gated pooling, and global average pooling [23]. This study uses max pooling because it provides unique performance features. Max pooling calculates the maximum value in each patch of the feature map covered by the kernel, thereby highlighting the most present feature in the patch. The fully connected layers are located at the end of each CNN architecture. These layers acquired their name because of the approach of the same name, which is that each neuron in the layer is connected to all neurons of the preceding layer. The input of these layers is in the form of a vector, which is created from the feature maps after flattening. The fully connected layer is used as a CNN classifier: it takes input from feature extraction stages and globally analyzes the output of all previous layers.