## 3 Materials and Methods
### Development of a Deep Learning Model by Fine-Tuning a VGG-16 Network  
#### 3.1.1 Theoretical Background of the VGG-16 Network  
The VGG-16 network was proposed by [PERSON] and [PERSON] of the Visual Geometry Group Lab of Oxford University in 2014 [37]. This model took the first and second prize in the ImageNet Large Scale Visual Recognition competition on categories of object localization and image classification, respectively. There were 200 classes in the object localization category, and 1000 classes in the image classification category. The architecture of the VGG-16 is illustrated in Figure 7.  
It can be seen in Figure 7 that the VGG-16 network has an input tensor of (224,224,3). This model processes the input image and outputs a vector of 1000 prediction values (probability) as shown in Equation (9). \[\hat{y}=\left[ \begin{array}{c}\hat{y}_{0}\\ \hat{y}_{1}\\.\\ \hat{y}_{999}\end{array} \right] \tag{9}\]  
The classification probability for a corresponding class is determined by a Softmax function as shown in Equation (10). Equation (10) shows the prediction probability for the \(j\)th class given a sample vector \(X\) and the weighting vector \(W\) using a Softmax function. \[P(y=j|X)=\frac{e^{X_{W_{j}}^{T}}}{\sum_{k=1}^{K}e^{X_{W_{k}}^{T}}} \tag{10}\]  
Figure 6: Final arrangement of the training, validation, and test data sets for the prediction of the maize common rust disease severities by a fine-tuned VGG-16 network. 3.1.2. Fine-tuning and Training a VGG-16 Network for Maize Leaf Disease Severity Prediction  
There are four scenarios in which a pre-trained model can be fine-tuned. These scenarios are summarized as follows:  
Scenario 1: The target dataset is small and quite similar to the source dataset. Scenario 2: The target dataset is large and quite similar to the source dataset. Scenario 3: The target dataset is small and very different from the source dataset  
Scenario 4: The target dataset is large and very different from the source dataset. The guidelines for the appropriate fine-tuning level to use in each of the scenarios are summarised in Table 4.  
Our model for predicting the severities of the maize common rust disease was developed by fine-tuning a VGG-16 network as guided by scenario 3 of Table 4. This was achieved by freezing all the layers of a VGG-16 network, except for the top four layers. The fine-tuned model that resulted is summarised in Table 5. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Scenario**} & **Size of the Target Data** &
\begin{tabular}{c} **Similarity of the** \\ **New and Original** \\ **Data Sets** \\ \end{tabular} & \multirow{2}{*}{**The Approach Used**} \\ \hline \(1\) & Small & Similar & The pre-trained network is used as feature extractor. \\ \hline \(2\) & Large & Similar & Fine-tuning is conducted through the full network. \\ \hline \(3\) & Small & Very different & Fine-tuning is conducted from activations earlier in the network. \\ \hline \(4\) & Large & Very different & Fine-tuning is conducted through the entire network. \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Summary for the fine-tuning of pretrained models. Figure 7: The VGG-16 architecture. Our model had a total of 15,517,668 parameters, of which 802,980 of them were trainable, while 14,714,688 were not. It can be seen in Table 5 that our FC (fully connected) layer consisted of a dense layer with 32 nodes and an activation function of \"relu\". The dropout layer was also used to randomly switch off 20% of the nodes in the first dense layer of our FC, during each training epoch. The output layer consisted of a dense layer with four nodes and \"Softmax\" as an activation function. Each class of the four classes was trained with 400 images and validated with 50 images. This means that the training data consisted of 1600 images while a total 200 images were used for validation. With a batch size of 32, the training steps per epoch were determined by a number of training samples divided by a batch size, which in this case was 1600/32.