## 2 Deep Learning Algorithms
### Fundamentals of CNNs  
In this study, we delve into the convolutional neural network (CNN) architecture, a widely acknowledged model. CNNs have gained popularity for their enhanced capabilities in image recognition and classification, as previously mentioned. Visually, CNNs resemble feedforward artificial neural networks (ANNs), as depicted in Figure 1. They are distinguished by their hierarchical arrangement of layers and incorporate filters, kernels, or neurons endowed with adaptable weights and biases. Every filter accepts input, executes convolution operations, and may perform nonlinear transformations. A standard CNN configuration encompasses the following elements:  
* The input data are processed by the convolutional layer (CONV). * Information compression is achieved by the pooling layer (POOL), typically through subsampling, reducing the size of the intermediate image. * Acting as a perceptron-type layer, the fully connected layer (FCL) performs its function. * Predictions regarding the class of the input image are made by the classification layer (Softmax). The convolutional network's cornerstone lies in its convolutional layer, extracting features from input data, especially images. Through convolution, it preserves spatial relationships between pixels, producing feature maps crucial for subsequent processing. Following convolution, a nonlinear activation function is used (typically ReLU), further processing the results and enhancing the network's ability to capture complex patterns. The convolutional layers are critical in enabling the network to learn hierarchical representations of input data, which is essential for tasks such as image recognition and classification [29; 30]. Pooling layers play a pivotal role in reducing the spatial size of representations, thereby decreasing the number of parameters and computational load. Additionally, they help prevent overfitting by capturing essential information while discarding irrelevant details. It is important to note that pooling layers do not engage in any learning process but employ functions like max-pooling or average pooling to achieve their objectives. The fully connected layer (FCL) serves as the final pooling layer in the network architecture. It receives the extracted features from the preceding layers and forwards them to a classifier, typically employing the Softmax activation function. Softmax ensures that the output probabilities from the fully connected layer sum up to 1, facilitating interpretation and decision making. Lastly, the choice of a suitable loss function is crucial for training the network effectively. In multiclass classification tasks, the cross-entropy loss function is commonly used, while regression tasks involving continuous values often utilize the mean squared error. Selecting the appropriate loss function is a hyperparameter choice that heavily depends on the specific requirements and characteristics of the task at hand. For a visual reference, see Figure 1 for a representation of the convolutional and activation layers, as well as an illustration of the pooling operation process. For more information, please refer to [31]. Hyperparameter Optimization  
The performance of a convolutional neural network (CNN) depends heavily on the hyperparameters--parameters specified before training and not learned during the process. These hyperparameters are crucial for optimizing the model's accuracy and efficiency. Key examples include the following:  
* Network Depth [32]: A CNN typically comprises multiple layers, such as convolutional, activation (e.g., ReLU), pooling, and fully connected layers. * Filter Size (Kernel Size) [33]: This parameter defines the dimensions of the filters used in the convolutional layers to extract features. Common filter sizes are \(3\times 3\), \(5\times 5\), and \(7\times 7\). * Number of Filters [34]: The number of filters in each convolutional layer determines the depth of the feature maps generated. More filters offer greater expressive power but increase computational demands.