## 5 Experimental Design
### Optimization of Classifiers: Forward Feature Selection and Hyper-Parameter Tuning  
The classifiers implemented within the scikit-learn library come within a set of default hyper-parameters that should perform well for a wide range of datasets. Nevertheless, this flexibility comes at the price of suboptimal accuracy, which can be further improved by the tuning of hyper-parameters. In this respect, the scikit-learn library version 0.24.2 offers a suite of useful routines, amongst which the _HalvingGridSearchCV_ routine from the sklearn.model_selection module was used within this study. This routine starts by evaluating all combinations of hyper-parameters with a small amount of resources and iteratively selects the best ones, using more and more resources. The term \"resources\" typically denotes a number of training samples, but it can also be an arbitrary numeric parameter such as a number of trees in a random forest. The optimal combination of hyper-parameters maximizes some classification quality metric ([PERSON]'s kappa score in this study), which is derived internally by the _HalvingGridSearchCV_ routine, which divides the input samples into the training and held-out samples. It has to be emphasized that the input samples used for the forward feature selection and hyper-parameter tuning consisted of training datasets generated within the experiments with the 2D datasets and MODIS data. The optimal set of hyper-parameters varies for different combinations of input features. Therefore, feature selection was performed, prior to the hyper-parameter tuning, by means of the _SequentialFeatureSelector_ routine from the sklearn.feature_selection module. This routine performs forward feature selection using 5-fold cross-validation, which maximizes some classification quality metric ([PERSON]'s kappa score in this study). The feature selection procedure was performed only for the real case datasets (summary of the selected features is given in Appendix A). Within the experiments with synthetic 7D \begin{table}
\begin{tabular}{l l} \hline \hline
**Abbreviation** & **Full Name** \\ \hline B28\_7325 nm & Band 28 brightness temperature at 7.325 \(\upmu\)m \\ \hline B29\_8550 nm & Band 29 brightness temperature at 8.550 \(\upmu\)m \\ \hline B30\_9730 nm & Band 30 brightness temperature at 9.730 \(\upmu\)m \\ \hline B31\_11030 nm & Band 31 brightness temperature at 11.030 \(\upmu\)m \\ \hline B32\_12020 nm & Band 32 brightness temperature at 12.020 \(\upmu\)m \\ \hline B33\_13335 nm & Band 33 brightness temperature at 13.335 \(\upmu\)m \\ \hline B34\_13635 nm & Band 34 brightness temperature at 13.635 \(\upmu\)m \\ \hline B35\_13935 nm & Band 35 brightness temperature at 13.935 \(\upmu\)m \\ \hline B36\_14235 nm & Band 36 brightness temperature at 14.235 \(\upmu\)m \\ \hline B1\_645 nm\_B2\_858 nm\_ratio & Reflectance ratio between Bands 1 and 2 \\ \hline B31\_11030 nm\_B20\_3750 nm\_diff & Brightness temperature difference between Bands 31 and 20 \\ \hline B32\_12020 nm\_B20\_3750 nm\_diff & Brightness temperature difference between Bands 32 and 20 \\ \hline B31\_11030 nm\_B32\_12020 nm\_diff & Brightness temperature difference between Bands 31 and 32 \\ \hline B28\_7325 nm\_B31\_11030 nm\_diff & Brightness temperature difference between Bands 28 and 31 \\ \hline B29\_8550 nm\_B31\_11030 nm\_diff & Brightness temperature difference between Bands 29 and 31 \\ \hline B29\_8550 nm\_B28\_7325 nm\_diff & Brightness temperature difference between Bands 29 and 28 \\ \hline sunz & Sun zenith angle \\ \hline satz & Satellite zenith angle \\ \hline suna & Sun azimuth angle \\ \hline sata & Satellite azimuth angle \\ \hline razi & Relative azimuth angle \\ \hline scat & Scattering angle \\ \hline NDVI & Normalized Difference Vegetation Index \\ \hline NDVI & NDVI = B350\_850 nm\_81 6450 nm \\ \hline B2\_850 nm\_81,450 nm & \\ \hline \hline \end{tabular}
\end{table}