## 3 Results
### Machine Learning Modelling  
The pipeline of the data analysis is shown in Figure 1.  
Before feature selection, more than two hundred features of both PT and LN were significantly different between HPV-negative and HPV-positive patients, based on the Mann-Whitney test (Supplementary Table S3). These statistical analyses were conducted to infer the relationships between radiomic features and HPV status and facilitate the interpretability of the following ML-driven models. The results of feature selection on the training set are illustrated in Figure 2, which shows the ranks of the top ten predictors. Four features were dominant over all the others: the 50 th percentile of \(\mathrm{D_{PT}}\) (i.e., the median value), the inverse difference moment of \(\mathrm{ADC_{LN}}\) (which is a measure of homogeneity from the GLCM family), smoking habits, and tumor subsite (tonsil versus base of the tongue). Box-and-whisker plots of the 50 th percentile of \(\mathrm{D_{PT}}\) and Inverse Difference Moment of \(\mathrm{ADC_{LN}}\), sorted by HPV status, are illustrated in Figure 3; the median (inter-quartile range) of the 50 th percentile of \(\mathrm{D_{PT}}\) was \(1.15\times 10^{-3}\) mm\({}^{2}\)/s (0.25) and \(0.91\times 10^{-3}\) mm\({}^{2}\)/s (0.24) for HPV-negative and HPV-positive patients, respectively (\(p<0.001\), Mann-Whitney test); analogously, the median (inter-quartile range) of Inverse Difference Moment of \(\mathrm{ADC_{LN}}\) was \(0.30\) (\(0.07\)) and \(0.41\) (\(0.12\)) for HPV-negative and HPV-positive patients, respectively (\(p<0.001\)). \begin{table}
\begin{tabular}{l l l l} \hline \hline \multirow{2}{*}{**Patient and Tumor Characteristics**} & \multicolumn{2}{c}{**HPV-Negative (n = 16)**} & \multicolumn{1}{l}{**HPV-Positive (n = 33)**} \\ \hline  & & **n** & **n** \\ \hline
**Gender** & Male & 14 (88\%) & 29 (88\%) \\  & Female & 2 (12\%) & 4 (12\%) \\
**Age (mean \(\pm\) SD)** & & 66.3 \(\pm\) 8.8 & 61.8 \(\pm\) 11.5 \\ \hline
**Tumor subsite** & Tonsil & 7 (44\%) & 20 (61\%) \\  & Base of the tongue & 9 (56\%) & 13 (39\%) \\ \hline
**T-stage** & T1 & 1 (6\%) & 4 (12\%) \\  & T2 & 3 (19\%) & 14 (43\%) \\  & T3 & 1 (6\%) & 2 (6\%) \\  & T4 & - & 13 (39\%) \\  & T4a & 11 (69\%) & - \\ \hline
**N-stage** & N0 & 2 (12.5\%) & 4 (12.1\%) \\  & N1 & 2 (12.5\%) & 16 (48.5\%) \\  & N2 & 4 (25\%) & 12 (36.4\%) \\  & N3 & 8 (50\%) & 1 (3\%) \\ \hline
**Smoke** & No & 2 (12.5\%) & 24 (72.7\%) \\  & Moderate & 3 (18.8\%) & 3 (9.1\%) \\  & Heavy & 9 (56.3\%) & 5 (15.2\%) \\  & Unknown & 2 (12.5\%) & 1 (3\%) \\ \hline
**Alcohol intake** & No & 3 (18.8\%) & 26 (78.8\%) \\  & Moderate & 6 (37.5\%) & 5 (15.1\%) \\  & Heavy & 5 (31.3\%) & - \\  & Unknown & 2 (12.5\%) & 2 (6.1\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Characteristics of patients and tumors in the validation set. Figure 1: Data analysis pipeline. Figure 2: Selection of the most relevant features and the corresponding predictor ranks obtained from the Minimum Redundancy Maximum Relevance (MRMR) algorithm. Among the different families of ML algorithms, which were trained on the first ten predictors, the Naive Bayes classifier provided the best results in terms of accuracy, as depicted in Supplementary Figure S2. Therefore, we explored the model performance by various combinations of the most relevant predictors using the Naive Bayes algorithm. We applied the SMOTE-NC technique during training after the feature selection, in order to reduce the number of variables and thus the impact of missing values and sparse data on the random oversampling. In fact, the SMOTE-NC algorithm does not work with NA (not available) values; NA values were due to both the intrinsic nature of data, really missing values or post-processing effects (i.e., some patients did not have LN ADC/D data because they were classified as N0, or some missing values were a consequence of the outlier removal  
Figure 4: HPV-positive OPSCC in a 57-year-old non-smoker non-alcoholic female, correctly classified by all the proposed models.