# An Investigation on Radiomics Feature Handling for HNSCC Staging Classification
## 3 Results  
Machine learning algorithms have been parameterized by means of a grid search optimization strategy. Selected parameters for each tool, including the number of artificial samples generated by SMOTE, are summarized in the Supplementary Materials (Table S1). Both classification and feature selection algorithms have been implemented in Python, using the Scikit-learn framework [53]; as for SMOTE, we considered the implementation provided by the Imbalanced-learn library [54]. Several experiments using selected features coming from both CT and PET images have been carried out. In detail, we define with \(M_{ALL}\) the set that includes all features from original, wavelet-filtered and LoG-filtered images. The best results in terms of AUC for the selected number of features in \(M_{ALL}\), i.e., \([5,10,20,50,100]\), obtained from the LOO-CV on \(DS_{1}\) and validated on \(DS_{2}\), are reported in Figure 2. Both in this and in the following figures, for visualization reasons, we chose to insert only the values of F1 score and the AUC. As shown, the results were not very encouraging. This was most likely due to the well-known over-fitting problem on the training data [41]. To avoid this problem, [PERSON] et al. suggested that the number of selected features should be approximately 10% of the number of patients in the training set. Thus, in our case, we selected the best models among those trained with a number of features equal to 5, 10 or 20, because the total number of patients in \(DS_{1}\) is 182. Among the experiments, we added clinical information, i.e., age and gender, to try to enhance the classification performance, and moreover, we combined the features as follows:  
* selected features from CT + selected features from PET;
* selected features from CT + age + gender;
* selected features from PET + age + gender;
* selected features from CT + selected features from PET + age + gender. All results are shown in Figure 3, while the selected features are reported in the Supplementary Materials (Tables S2-S5). The red arrows indicate the results obtained by using the best model in terms of AUC for the LOO-CV strategy for each outcome. For pT-Stage, the best result was obtained by using features coming from PET images, with 9 selected features. The classifier with the best results was NB. For pN-Stage and Overall Stage, the best results were obtained by using features from both CT and PET images, using RF and MLP classifiers, respectively. In detail, for pN-Stage, 8 features were selected from CT images and 10 from PET images, while for Overall Stage, 39 features in total were used, 19 from CT and 20 from PET images. Figure 2: Best results on \(DS_{2}\) for number of features in \([5,10,20,50,100]\) in \(M_{ALL}\) set. For the sake of completeness, experiments using features selected only from original images or from LoG- or wavelet-filtered images have been performed. In order to compare the results of the models which use the \(M_{ALL}\) set and the models which use only features coming from original or LoG-filtered or wavelet-filtered images, namely \(M_{O}\), \(M_{L}\) and \(M_{W}\), for each outcome, we selected the source images that, for \(M_{ALL}\), have given the best AUC for each model. Next, the feature selection step was applied for \(M_{O}\), \(M_{L}\) and \(M_{W}\), by using the selected source images. Finally, the feature classification step was applied on the selected features. Results are reported in Figures 4-6, with the use of \(M_{O}\), \(M_{L}\) and \(M_{W}\), respectively. For the experiments in the above figures, the only case worth dwelling on is the use of \(M_{O}\) for pN-Stage classification. Indeed, the AUC outperformed the one calculated by using \(M_{ALL}\) and only 17 features were used, 7 from CT and 10 from PET. Finally, in the Supplementary Materials, we report the selected features for the best models for all sets of features: \(M_{ALL}\) (Table S2), \(M_{O}\) (Table S3), \(M_{L}\) (Table S4) and \(M_{W}\) (Table S5).