## 4 Discussion  
The results of the derived models during the 10-fold cross-validation on the training set exhibited varying performance levels (Table 1). The \(k\)NN and LDA models demonstrated overall poor performance. The QDA model excelled in predicting non-immunogenic proteins, while the SVM model best predicted immunogenic ones. Both RF and XGBoost models showed excellent and balanced performance. Based on these results, we selected the QDA, RF, and XGBoost models for further evaluation and implementation in the third version of the Vaxijen web server. Despite the close overall performance of the QDA and SVM models, we chose QDA due to its slightly higher accuracy and MCC metrics. Having an odd number of models in Vaxijen v3.0 allows for the implementation of a majority voting consensus classification. The three selected models showed similar results when evaluated on the external test set (Table 2). This comparison involved the three previously derived models for immunogenicity prediction on peptides (SVM, RF, and XGBoost). It was observed that the peptide models outperformed the protein models across all statistical metrics. One possible explanation is that the immunogenic epitope, the region of the protein recognized by T cells and responsible for immunogenicity, is a small peptide fragment. Thus, models derived from such immunogenic epitopes are more robust. However, assessing the entire protein for immunogenicity may be necessary, making the models derived from whole proteins potentially more advantageous. \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Top 10 Features** \\ \hline
**QDA** & \\ Permutation feature importance & ACC251, ACC223, **ACC344**, ACC257, ACC114, ACC134, ACC131, ACC116, \\  & ACC256, ACC136 \\ Drop-column feature importance & ACC445, ACC334, ACC143, ACC121, ACC543, ACC534, ACC523, ACC442, \\  & ACC424, **ACC344** \\ \hline
**RF** & \\ Permutation feature importance & ACC234, ACC327, ACC344, ACC351, ACC412, ACC253, ACC123, ACC345, \\  & ACC317, ACC311 \\ Drop-column feature importance & ACC445, ACC443, ACC341, ACC523, ACC455, ACC444, ACC441, ACC435, \\  & ACC415, ACC335 \\ \hline
**XGBoost** & \\ Permutation feature importance & **ACC111**, ACC331, ACC131, **ACC123**, ACC337, ACC224, ACC143, ACC124, \\  & ACC153, ACC325 \\ Drop-column feature importance & ACC514, ACC441, ACC254, ACC251, **ACC123**, **ACC111**, ACC541, ACC443, \\  & ACC421, ACC415 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The top 10 attributes are ranked according to their importance for QDA, RF, and XGBoost models. The standard features between each model’s two feature importance techniques are presented in bold. Robustness is the ability of a model to perform well on new and unseen data, not just on the data it was trained on. Y-scrambling, also called Y-randomization, is a method used to test whether the predictions made by the model are not made just by chance. Y-scrambling helps assess model robustness by testing whether the model is learning true patterns or just capturing spurious correlations. It does this by randomizing the target variable (Y) and measuring how the model's performance degrades. If the scrambled model performs similarly to the original model, it suggests that the original model was overfitting or capturing noise instead of real patterns. If the original model performs significantly better than the scrambled one, it confirms that the model is learning meaningful relationships. The Y-scrambling assessment of the three models revealed accuracy scores of around 0.50, similar to the peptide models (Table 3). These results demonstrated that when the feature-target relationship in the data is disrupted, all models lose their predictive capabilities, indicating that their performance is not due to chance. The first step in assessing the importance of each feature for model performance is to identify the standard features calculated using both permutation feature importance and drop-column feature importance techniques for each of the three selected models (Table 4).