## 2 Materials and Methods
### Deep Learning Approach
#### 2.4.2 Convolutional Neural Networks CNN
\begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Activation Function** & **Advantages** & **Disadvantages** & **When Might It Be Used?** \\ \hline Softmax [39] & -Proficient in managing different classes, single class among other functions & -Does not consider the rejection of null values & when predicting a probability for a multi-class task, the -This is unlikely to work if data are non-linearly separable & when predicting a probability for a multi-class task, the -Softmax function must be implemented in the last layer \\ \hline Softplus [41] & -Soft derivative employed in backpropagation and it is equivalent to a sigmoid function & -Operation not as affordable as ReLU & Rather never \\ \hline Softsign [42] & -Better and faster learning due to the absence of difficulties related to the vanishing gradient & -Frequently, the gradient yields either an extremely high or low value & Rather never \\ \hline Softplus [41] & -Botter and faster learning due to the absence of difficulties related to the vanishing gradient & -Frequently, the gradient yields either an extremely high or low value & It is used in very deep networks, when the number of hidden layers is high (nearly 30) \\ \hline \multirow{2}{*}{LeakyReLU [42]} & -Seeks to overcome the “dead neuron”-Straightforward implementation and low-cost operation & -Large gradients can change the weights so that neural units are permanently disabled (never activated). & To be used only if you expect a “dying ReLU” problem, so it should be applied in hidden layers. \\ \hline ELU [44] & -It can deliver negative outputs -Resolves both the vanishing gradient and the dying ReLU problem & -Computationally demanding -For x \(>\)0, it can jump the activation along with the output range of [0, \(\infty\)] & When the risk of overfitting is high, and it should be used in hidden layers. \\ \hline \hline \end{tabular}
\end{table} 
Table 4: _Cont._should be activated based on the weighted summation and bias. Activation functions that can be used in this layer include ReLU, Leaky ReLU, ELU, and Softmax. The purpose of determining the activation function is to illustrate non-linearity in the neuron's output. Spatial pooling also termed down-sampling or subsampling, decreases each feature map's dimensionality while retaining the relevant information. It also helps to prevent overfitting by reducing the number of computations and parameters. There are various types of pooling, such as average, sum, and maximum pooling. The final layer in a convolutional neural network (CNN) is a dense or fully connected layer, where every neuron receives input from all the neurons in the previous layer. This layer produces the model's output with probabilities between 0 and 1 [46]. The model depicted in Figure 9 represents a complete CNN process for input image classification according to assigned values. The key configuration settings for a convolutional neural network (CNN) include: The number of concealed layers, the activation function used in each layer, the method of regularization, the optimization technique, the technique of regularization, and method of optimization. In this work, we not only compared several activation functions that exist in the DL literature and applied them to leaf disease detection tasks, but also implemented several DL optimizers and compared them to our target problem of classifying leaf diseases.