# Plant Disease Detection Using Deep Convolutional Neural Network
## 3 Materials and Methods
### Model Design
The \(ReLu\) activation function was performed by using Equation (3). \[ReLu(x)=Max(0,\ x) \tag{3}\]  
Moreover, the flatten layer was introduced after the fifth convolutional and pooling layer. It reduces the three-dimensional data to one-dimensional data for a traditional neural network approach. The flatten layer converts the output of the fifth max-pooling layer from 2, 2, 128 values to 512 values. The first dense layer was introduced after the flatten layer in the 14-DCNN. The first dense layer increases input value from 512 to 2048 values. Equation (4) represents the individual neuron output (\(z_{j}\)) of the first dense layer. The \(i\) represents the number of inputs of the first dense layer, and it ranges from 1 to 512. Additionally, the \(j\) denotes the number of outputs of the layer its range from 1 to 2048 values. \[z_{j}=ReLu\left(0,\ \sum_{i}^{512}b_{j}+x_{i}w_{i}\right) \tag{4}\]  
The \(x_{i}\) and \(w_{i}\) represent the value and weight of the \(i\)th input of the \(j\)th output. Additionally, the \(b_{j}\) denotes the bias value of the \(j\)th node. The dropout layer was used between the first dense and second dense layer in the 14-DCNN to avoid the overfitting issue. The second dense layer was initiated after the first dense layer and dropout layer. Similarly, the number of inputs of the second dense layer is 2048 and the output values of the neural network is 59 values. Additionally, this layer uses the softmax activation function to classifythe plant leaves. The softmax function (\(\sigma\)) value of \(i\)th neuron of the dense layer can be calculated using Equation (5). \[softmax(\sigma(z_{i}))=\frac{e^{z_{i}}}{\sum_{j=1}^{59}e^{z_{j}}} \tag{5}\]  
The output class of the input image can be discovered using Equation (6). \[Output\ Class\ (z_{out})=max(z_{1},\ z_{2},\ldots\ z_{59}) \tag{6}\]  
This output value from \(z_{1}\) to \(z_{59}\) represents the number of diseases and healthy plant leaf and non-leaves classes in the PlantDisease59 dataset. The total number of training parameters is 5,424,583 in the 14-DCNN model. The layered structure of the 14-DCNN model is shown in Figure 2.  
Figure 2: Layered structure of 14-DCNN model. After designing the 14-DCNN model, the most suitable hyperparameter values were identified using hyperparameter tuning techniques. The random search and coarse-to-fine techniques were used to discover the suitable values of the optimizer function, mini-batch size and dropout probability of the 14-DCNN. The most common optimizers considered for the hyperparameter searching are adaptive moment estimation (Adam), stochastic gradient descent (SGD) and root mean square propagation (RMSprob). The range of the mini-batch size is between 8 and 256, incremented by 8 per value. Additionally, the dropout range varies between 0.0 and 0.5, incremented by 0.1 per value. The random searching technique chooses the random combination of hyperparameter values from the search space. The selected combination of hyperparameter values is applied to the 14-DCNN and trained with 100 epochs in parallel. From the training result, the coarse-to-fine process helps to identify the most possible hyperparameter values from the search space. Finally, the hyperparameter tuning technique discovers the SGD optimizer with a mini-batch size of 32 and the dropout probability of 0.2 gives a better performance than other values. To identify the learning rate (Lr) and momentum of the SGD optimizer, a similar random search approach was used with a possible combination of the values. Table 4 shows the most common hyperparameters of the 14-DCNN model for the plant leaf disease detection model with their values. The random search with the coarse-to-fine technique offers significantly improved searching performance than the grid search and simple random search optimization techniques. The optimized hyperparameter values and the PlantDisease59 were used to train the proposed 14-DCNN model for diagnosing the diseases from the plant leaf images.