# A Machine Learning and Radiomics Approach in Lung Cancer for Predicting Histological Subtype
## 4 Results and Discussion  
The results of each experiment are reported in Figure 7. Each matrix in the figure shows the mean AUC per model per number of features of the input pattern. **Experiment 1** aimed to assess if the radiomic signature obtained from Dataset1, thanks to the implemented approach for features reduction, could be discriminative for classifying LUAD and other histological types also on the other datasets. To do this, we trained the classification models on the selected features from D1, D2 and D3, respectively. The performances of these models are reported in Figure 7(I-a, I-b and I-c), respectively. The classifiers reached the highest robustness in classifying samples from D1, regardless of the number of features and the classifier itself. Specifically, the mean (\(\pm\)standard deviation) AUC for all the considered models and features was 0.64 (\(\pm\)0.04), 0.54 (\(\pm\)0.06), and 0.56 (\(\pm\)0.09), considering data from D1, D2 and D3, respectively. Considering input data from D1, the best model was LogReg, which allowed us to obtain an AUC of 0.70 when trained with the input pattern size of six features. Internal cross-validation with input data from D2, instead, showed an increased variability among models' performances, regardless of the number of features, as can be noticed in the heatmap reported in Figure 7(I-b). In this case, the best model was the SVM trained on only one feature, which allowed us to achieve an AUC of 0.65. Considering the models trained on input data from D3, LogReg and MLP allowed us to achieve the best results in the univariate version; also in this case, with a mean AUC of 0.75. **Experiment 2** aimed to assess if models trained on the histotype signature from D1 could generalise to make decisions also on D2 and D3. The results obtained show that GB, trained on two features, and AdaBo, trained on four features, allowed us to achieve an AUC of 0.62 considering data from D2 (Figure 7(II-a)). However, better performances were obtained, validating models on D3, with an AUC of 0.70 for SVM trained with one, five, and six features (Figure 7(II-b)). In **Experiment 3**, instead, we investigated if increasing the training sample would allow us to obtain a higher performance on D3 as a validation set with respect to **Experiment 2**. As shown in Figure 7 III, the MLP trained with an input pattern of seven features allowed us to obtain an AUC of 0.69 on data from D3 considering a joint train set composed of D1 and D2. Comparing this experiment to the previous one, a small improvement in terms of mean performances were obtained; in fact, this training configuration allowed us to achieve a mean (\(\pm\)standard deviation) AUC of 0.58 (\(\pm\)0.07), instead of 0.59 (\(\pm\)0.09), on the external validation set D3. Concerning **Experiment 4**, it has been designed to evaluate how the models would have performed if D1, D2 and D3 would be merged. The internal cross-validation revealed that merging D1, D2 and D3 led to more robust performance of the classifiers, except for the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Hyperparameters** & **Value** \\ \hline \multirow{8}{*}{MLP} & neurons\_per\_layer & 100 \\ \cline{2-3}  & hidden\_layers & 1 \\ \cline{2-3}  & b1 & 0.9 \\ \cline{2-3}  & b2 & 0.999 \\ \cline{2-3}  & solver & adam \\ \cline{2-3}  & learning\_rate\_init & 0.001 \\ \cline{2-3}  & max\_iter & 200 \\ \cline{2-3}  & early\_stop & None \\ \cline{2-3}  & penalty & 12 \\ \cline{2-3}  & alpha & 0.0001 \\ \cline{2-3}  & activation & relu \\ \hline \multirow{2}{*}{GB} & learning\_rate & 0.1 \\ \cline{2-3}  & n\_estimators & 100 \\ \cline{2-3}  & criterion & friedman\_mse \\ \hline \hline \end{tabular}
\end{table} 
Table 3: _Cont._comparison with the case of internal cross-validation performed on D1 (in **Experiment 1**). LogReg, RF, and Ensemble-5 models allowed us to achieve an AUC of 0.65 in their best case, i.e., trained considering two, six and seven features, respectively (as shown in Figure 7 IV).