## 3 Results and Discussion
### Artificial Neural Network Model Results
#### 3.3.2 Performance of the Best ANN Model in the Test Dataset  
The out-of-sample test dataset has 96 data points across a 2-year period. The out-of-sample data was neither used in the training nor the validation processes of the ANN and even of the GAM process. It represents the model's performance in the real world. We describe the performance of the best ANN models both as a regressor and as a classifier.  
Performance of ANN in Regression: the ANN prediction was formulated as a regression problem. The performance of the best ANN model in regression is indicated in the plot of the actuals versus the predicted real values for all the counties ordered by county and time period as shown in Figure 10.  
The plot of the actual versus the predicted values represents quite a good agreement. In the test data, the best model posted an \(R^{2}\) of 0.78 and RMSE of 7.03 on the actual data values. The above performance over the 96 data points for testing is an acceptable performance in the prediction of drought events.  
Figure 10: Plot of the actual values of VCI3M versus the best ANN model’s predicted values in test data over 24 months for (**a**) Mandera (\(R^{2}\) = 0.71); (**b**) Marsabit (\(R^{2}\) = 0.77); (**c**) Turkana (\(R^{2}\) = 0.83) and (**d**) Wajir (\(R^{2}\) = 0.71). Predictions were done 1 month ahead.  
Performance of ANN in classification. Operational drought monitoring involves the definition of thresholds on indices used for drought monitoring so as to realise a phase approach to drought monitoring. We use the approach documented in [PERSON] (2012), [PERSON] et al. (2010) and [PERSON] et al. (2012) and earlier presented in Table 3 to classify drought in five phases.  
The best model had an overall accuracy of 67% over all the counties, with the highest accuracy of 71% for Wajir and Marsabit counties as indicated in the matrix provided in Figure 11. Additional analysis of the classification performance of the model in the prediction of moderate to severe drought is provide in Appendix C with an accuracy of 54% (Mandera), 71% (Marsabit), 74% (Wajir) and 58% (Turkana).  
When formulated as a multi-class classification problem and multiple receiver operating characteristic (ROC) curves plotted for each of the pairwise comparisons of the classes following on the approach in [PERSON] and [PERSON] (2007), we obtained the ROC plot in Figure 12. The multi-class area under the curve (AUROC) is the average of the 10 areas under all the ROCs. The ROC for the five classes provides a reasonable trade-off between sensitivity and specificity at an overall AUROC of 89.99%. The AUROC indicates quite a good trade-off between sensitivity and specificity and is ranked within the good performance category, as it is way above the 50%, which represents a worthless test (in gray).  
Figure 11: Performance of the classifier for the each of the counties showing months of difference in grey and those of agreement in blue. Predictions are done 1 month ahead. The classification accuracies are: (**a**) 63% for Mandera county; (**b**) 71% for Marsabit county; (**c**) 63% for Turkana county and; (**d**) 71% for Wajir county.