RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Results
A total of 125 patients met the inclusion criteria, of which 100 were randomly assigned to the training set and 25 to the test set. The clinical features of the patients are summarized in Table 1 .

The LightGBM model demonstrated the best performance and was selected to construct the radiomics signature, with the area under curve (AUC) values of 0.896 (95% CI [0.825–0.967]) and 0.824 (95% CI [0.658–0.989]) in the training and testing datasets, respectively ( Figure 3A ). The confusion matrix for LightGBM is shown in Figure 3B . The AUC of the clinical signature was 0.728.

In the deep learning models, the performance of M3 was significantly higher than that of other datasets, with an AUC value of 0.826 in the training cohort and 0.674 in the test cohort ( Figure 4 ). Subsequently, the deep transfer learning model trained on the M3 dataset was selected as the deep signature.

Once the base models were trained, we used stacking to combine their predictions ( Džeroski & Ženko, 2004 ). The dataset was split into the training and test sets, and the predictions generated from each base model on the test set were used as input features to train a meta-model for optimally combining the predictions of the three base models. Logistic regression was used as the meta-model for its simplicity and interpretability. Our integrated model demonstrated improved performance compared with individual models, achieving an accuracy of 95%, a sensitivity of 100%, and a specificity of 92.5% in the training dataset, and 84% accuracy, 100% sensitivity, and 76.5% specificity in the test dataset ( Table 2 ). The AUC of the integrated model was 0.949 (95% CI: [0.870–1.000]), which was higher than the AUCs of the radiomics (0.893), clinical (0.728), and deep learning models (0.798) ( Figs. 5A and 5B ).

The performance of the integrated model was also evaluated by the DeLong test and decision curve analysis (DCA). The DeLong test indicated that the integrated model had a higher AUC compared with the individual models (nomogram vs clinical: 0.002; nomogram vs radiomics: 0.073; nomogram vs deep learning: 0.035), demonstrating the superior discriminative ability of the integrated model. The DCA further showed that our integrated model yielded a greater net benefit compared with other models across the majority of practical threshold probabilities ( Figure 6 ). The nomogram of the integrated model is shown in Figure 7 .

The performance of the integrated model was compared with that of clinical experts. Three clinicians with varying levels of experience in oral diseases, i.e ., less than 5 years (D1), less than 10 years but longer than D1 (D2), and more than 15 years (D3), were asked to interpret the imaging and clinical data. The decision of the three clinicians was compared with that of the integrated model by plotting ROC curves. As shown in Figure 8 , the integrated model achieved an AUC of 0.838 compared with 0.654, 0.768, and 0.893 for clinicians D1, D2, and D3, respectively, suggesting differences between the model and the expert evaluation.
Conclusion
The multiomics-based model demonstrated high accuracy, sensitivity, and specificity for predicting OLNM in patients with early-stage tongue cancer. Despite the noted limitations, our integrated model could serve as a valuable tool for improving diagnosis and treatment planning, eventually leading to better patient outcomes.
Figure 1: The workflow of this study.

  Download full-size image
DOI: 10.7717/peerj.17254/fig-1
Figure 2: Features selection process.
(A) Number and ratio of handcrafted features; nonzero coefficients were selected to establish the Rad-score with a LASSO logistic regression model. (B) Coefficients and (D) MSE (mean standard error) of 10 folds validation. (C) The histogram of the Rad-score based on the selected features. (E) Deep learning datasets based on different methods (M1, M2, M3).

  Download full-size image
DOI: 10.7717/peerj.17254/fig-2
Figure 3: Development of radiomics model.
(A) The ROC curves of the radiomics model; (B) the confusion matrix of predicting lymph node metastasis.

  Download full-size image
DOI: 10.7717/peerj.17254/fig-3
Figure 4: The ROC curves of M3.

  Download full-size image
DOI: 10.7717/peerj.17254/fig-4
Figure 5: The ROC curves of the clinical model, radiomics model, deep learning model and combined model in the training cohort (A) and the validation cohort (B).
The integrated model demonstrated significantly higher AUCs in the training and validation cohorts (AUCs of 0.989 and 0.949) than the clinical model (AUCs of 0.815 and 0.728), the radiomics model (AUCs of 0.896 and 0.893) and the deep learning model (AUCs of 0.980 and 0.798).

  Download full-size image
DOI: 10.7717/peerj.17254/fig-5
Figure 6: The DCA of the clinical model, radiomics model, deep learning model and combined model.

  Download full-size image
DOI: 10.7717/peerj.17254/fig-6
Figure 7: The development of the nomogram for clinical use.

  Download full-size image
DOI: 10.7717/peerj.17254/fig-7
Figure 8: The ROC curves of the integrated model and diagnostic results of each radiologist in the test cohort.
The integrated model reached a high AUCs (AUC: 0.838), doctor1 acquired 0.654 of AUC, the AUC of doctor2 was 0.768 and doctor3 demonstrated the highest with 0.893.

  Download full-size image
DOI: 10.7717/peerj.17254/fig-8
The detailed clinical characteristics of the patients.
The performance of different models.