# Groundwater Prediction Using Machine-Learning Tools
## 5 Performance Results and Discussion
### Performance Comparisons  
The data in Tables 1-4 are summarized in Figures 6 and 7. For MAE, XGB with all features (including same-pixel, spatial-temporal, and square root rescaling) gives the overall best performance. However, SVR is clearly the best performer for most feature sets. SVR tends to work best with fewest same-pixel features (i.e., configuration **a**, which is the previous month + 12 months prior). SVR with configuration **a + (i, j)** reduces the MAE by about 7% over the best result without spatial features. Adding time stamp and square root rescaling gives little additional improvement. In general, SVR gave MAE values that were between 7 and 20 % better than the worst-performing algorithms, which were  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline \hline
**Features** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** \\  & **XGB** & **XGB** & **LR** & **LR** & **RF** & **RF** & **MLP** & **MLP** & **SVR** & **SVR** & **Mean** & **Mean** \\ \hline a & 2.887 & 5.790 & 2.915 & 5.649 & 2.911 & 5.878 & 2.843 & 5.639 & 2.700 & 5.720 & 2.851 & 5.735 \\ b & 2.890 & 6.064 & 2.840 & 5.642 & 3.047 & 6.402 & 3.008 & 5.952 & 2.677 & 5.895 & 2.892 & 5.991 \\ c & 2.912 & 6.078 & 2.909 & 5.630 & 3.048 & 6.255 & 2.782 & 5.844 & 2.640 & 5.861 & 2.858 & 5.933 \\ d & 2.928 & 6.145 & 2.900 & 5.625 & 3.074 & 6.407 & 2.844 & 5.657 & 2.626 & 5.857 & 2.874 & 5.938 \\ e & 2.890 & 6.060 & 2.913 & 5.621 & 3.034 & 6.351 & 2.829 & 5.723 & 2.617 & 5.751 & 2.856 &random forest (for same-pixel and same-pixel + spatial location feature sets) or linear regression (for other feature sets). It is noteworthy that adding a time stamp brought large error reductions to random forest and XGB, while having little effect on linear regression, MLP, and SVR. The RMSE results resemble the MAE results in that SVR consistently gives the lowest error. This time however, XGB with **a** + **i, j** + **t** + **s** does not outperform the SVR results. Once again, same-pixel configuration **a** tends to give the best results for SVR, and **a** + **i, j** has about 4% lower error than **a** only. In general, SVR gave MAE values that were 2.5-15.5% better than the worst-performing algorithms. Figure 8, shows the percentage performance improvement (i.e., percentage error reduction) of the overall best-performing model from Tables 1-4 (SVR) compared to the untrained model based on the previous month. The MAE and RMSE values for the untrained model were 2.988 and 6.771,  
Figure 6: MAE Graph for the different set configurations. Figure 7: RMSE Graph for the different set configurations. respectively. When square root rescaling is used, all predictions reduced MAE and RMSE by over 15% and 20%, respectively. The overall best predictor (**a + i + j + t**) reduced both MAE and RMSE by more than 20%. This result is consistent with [39], which found that SVM outperformed RF when predicting rainfall images up to 3 h ahead on a per-pixel basis. When GMM was added to XGB with **a + i, j + t + s**, the values of MAE and RMSE obtained were 2.258 and 4.838, respectively. These values were better than the corresponding best results without GMM by 3.6% and 9.4% respectively. Compared to the untrained predictor, this XGB+GMM model gave 25% improvement in MAE and 29% improvement in RMSE. Figure 9, shows an example of an actual image and its prediction using XGB+GMM. All of the above results were obtained using the default parameters in sklearn for their respective methods. Parameters were not optimized because of the large number of different methods involved. In particular, optimizing GMM is very expensive since it used 100 trained models which would all have to be optimized separately. We did conduct a preliminary investigation into parameter optimization by tuning parameters used in RF and XGB for the models in Tables 1-3. For this purpose, the scikit-optimize package was used, which employs Bayesian optimization.