## 2 Materials and Methods
### Deep Learning Approach
#### 2.4.2 Convolutional Neural Networks CNN  
CNN represent a leading category for image recognition and classification [38]. CNN can strongly model nonlinear functions. Unlike SVM and KNN, this one does not converge toward the global optimum. It can also be used directly on raw data without the need for handcrafted features. The artificial neurons in a CNN are inspired by the structure of biological neurons, consisting of a summation term, connection weights, and a nonlinear activation function. The number of hidden layers and neurons in a CNN is often determined through trial and error. Each neuron X is characterized by certain of its bias, weights, and activation function. Images are introduced into the deep learning model through the input layer, where the neuron carries out a transformation as depicted in Equation (31). \[\text{X}=\text{(weights * input) + bias} \tag{27}\]  
An artificial neural network without an activation function is weaker and less suitable for tasks involving complex patterns. Without the activation function, it is equivalent to a linear regression model. So, the proper choice of activation function enables the DL to effortlessly learn very challenging patterns and the following are among the prominent purposes of the activation function:* To hold the output restricted to the desired range. * To encompass a non-linear functionality in the data. Different Kinds of Activation Functions  
Several activation functions are available in the literature [39; 40; 41] and some of them are outlined in detail below:  
**-Sigmoid:** It stands for a non-linear function producing output values ranging from 0 to 1, and the resulting output will be non-linear with an identical sign. The function has no symmetry around zero:  
\[\mathrm{f(x)}=\frac{1}{1-\mathrm{e}^{-\mathrm{x}}} \tag{28}\]  
**-Tanh:** It is akin to a sigmoid function but is symmetrical around 0 and produces output values in the range [\(-1\), \(1\)], and the output sign can vary. \[\mathrm{tanh(x)}=2*\mathit{sigmoid}(2x)-1 \tag{29}\]  
\[\mathrm{g(x)}=1-2\,\mathrm{tanh}^{2}\mathrm{x} \tag{30}\]  
The generated output for the sigmoid and tanh functions features upper and lower bounds. **-ReLu:** It denotes a rectified linear unit and is a non-linear function. The ReLu function possesses an edge over other functions because it rarely activates all neurons at once. If the output value is inferior to 0, the neuron is disabled. It is substantially more efficient computationally than the tanh and sigmoid functions:  
\[\mathrm{f(x)}=\mathrm{max}\;(0,\mathrm{x}) \tag{31}\]  
**-Softmax:** It is built up of several sigmoids. It yields values ranging from 0 to 1 and processes data probabilities related to the class employed in the multiclass task:  
\[\mathrm{\pi(z)j}=\frac{\mathrm{e}^{zj}}{\sum_{k=1}^{K}\mathrm{e}^{zj}},\; \mathrm{for}\;\mathrm{j}=1,2,3,4,\ldots\mathrm{k} \tag{32}\]  
**-Softplus:** It is a sort of conventional function released in 2001. It is differentiable and simple to prove thanks to its derivative. This is an optional replacement for the dead ReLu. The outcome ranges from [0 to \(\infty\)]:  
\[\mathrm{y}=\log\;(1+\;\mathrm{e}^{x}) \tag{33}\]  
**-Softsign:** It resembles the tanh much more; except that tanh is exponentially converging while Softsign is polynomially converging. The resultant outcome lies between [\(-1\), \(1\)]:  
\[y=\frac{1}{1-\frac{1}{x}} \tag{34}\]  
**-Swish function:** It is one of the earliest hybrid combination AFs proposed, combining the input function and the sigmoid AF to produce a hybrid AF. It was introduced by [PERSON] et al. [42] and uses an autonomous search technique based on reinforcement learning to compute the function. The Swish function has properties of non-monotonicity, and smoothness being bounded below and unbounded above. Its smoothness makes it effective for training deep learning (DL) architectures and improving generalization and optimization outcomes.