## 2 Deep Convolutional Neural Networks in Image Classification  
The first neural network model with convolutional layers, named Neocognitron by [PERSON], its author, dates from 1979 and was inspired by the hierarchical biological model of the visual nervous system proposed by [PERSON] and [PERSON] in the 1960s. In this model, neurons from deeper layers tend to respond selectively to more complicated features of the stimulus pattern, have a larger receptive field, and are more insensitive to variations in position compared to neurons from earlier layers [22; 23]. In 1989, [PERSON] presented the first convolutional network with learning through the backpropagation method by applying the gradient descent rule [24]. However, in the following years, there was a consensus among researchers about the difficulty of applying backpropagation for deep networks, mostly due to the problem of gradient vanishing and exploding [8; 23]. In 2006, [PERSON] et al. [7] found a combination of unsupervised and supervised learning, which again encouraged research into learning in multilayer neural networks. This work meant the renaissance of deep learning [8; 10; 23]. Subsequently, various contributions enabled fully supervised learning in DCNNs. We refer to regularization and normalization mechanisms, features of network architecture, nonlinear activation functions, supervision components, optimization techniques, and strategies to reduce the computational cost [8; 10; 23; 25]. In 2012, for the first time, a DCNN model outperformed other machine learning techniques in image classification tasks [26; 27]. Since then, this type of problem has been solved better with DCNNs than with any other category of machine learning, surpassing even the precision of human perception [8; 9; 10]. The power of a DCNN lies fundamentally in its convolutional layers. This kind of layer works as a set of local receptive fields extracting information at a certain level of abstraction according to the location of the layer in the depth of the network [9; 23]. In convolutional layers, the adjustable weights are organized into two-dimensional sub-sets, a filter bank, which are convolved with the input to the layer, which also occurs in two dimensions per channel. Due to the nature of the convolution operation, the synaptic weights turn out to be shared, with the benefit of a lower count of them compared with an equivalent layer of a multilayer perceptron network [8; 10; 23]. A detailed explanation of the operation of convolutional layers can be found in [10; 23]. A convolutional layer is typically followed by rectified linear units (ReLU) instead of traditional sigmoidal functions like logistic or hyperbolic tangent [27]. Unlike these, which tend to saturate in certain regions, ReLU maintains a non-saturating behavior in the positive region. This characteristic plays a crucial role in mitigating the vanishing gradient problem, a common obstacle in the training of DCNNs. Remarkably simple ReLU has a profound impact on the training process [27]. Its simplicity translates into computational efficiency and ease of differentiation, contributing to faster training compared to more intricate activation functions. Some researchers have explored certain varieties, and sometimes outcomes have slight improvements, but they have not deviated significantly from the widespread adoption of ReLU [10; 23]. A comprehensive overview of these alternatives can be found in [10]. The three activation functions mentioned above are shown in Figure 1.  
Towards the end of a DCNN, fully connected layers are usually found; these operate as a conventional layer of multilayer perceptron networks, omitting the activation function, that is, performing the powered sum. The fully connected layers that follow these layers interpret these feature representations and perform the function of high-level reasoning [10].