## 3 Results  
### Round 1 Training and Testing (Model Development and Verification)  
Table 2 summarizes the model performance in Round 1 of four deep neural networks (i.e., AlexNet, ResNet-50, MobileNet, and VGG-19) on test datasets from various sources and with four levels of image similarity. At Level 1, all models, except for VGG-19, showed exceptional performance, achieving accuracy rates of over 98%. The AUC, an indicator of a model's capacity to distinguish between classes, was above 99% for all models. ResNet-50 led this metric with an AUC of 99.97%, followed closely by AlexNet, MobileNet, and VGG-19 with 99.94%, 99.93%, and 99.24%, respectively. At Level 2, with X-ray images from a different source, the models maintained high performance, albeit with a slight decrease in most metrics. Although the sensitivity remained high across all models, the specificity showed a minor decrease compared to Level 1, yet it remained sufficiently high, underlining an efficient ability to correctly identify negative cases. Among the four models considered, VGG-19 exhibited the lowest specificity (~90%) when tested on both Level 1 and Level 2 images. The other three models all had a specificity of 94% or higher. However, progressing from Level 2 to Level 3, which comprised outliers from datasets 1 and 2, there was a drastic decline in performance in all models, most noticeably in sensitivity and precision metrics (Figure 2). This decrease was likely attributed to the increased complexity or alterations in image presentations at Level 3. In particular, the sensitivity of MobileNet decreased from 97%+ to 28% (Table 2). The large drop in sensitivity suggested a diminishing ability of the models to correctly identify positive cases when tested on Level 3 or the outlier images. Similarly, a decline in precision from 97% at Level \begin{table}
\begin{tabular}{c c c c c c} \hline
**Network** & **Round 1 (\%)** & **Level 1** & **Level 2** & **Level 3** & **Level 4** \\ \hline \multirow{4}{*}{AlexNet} & Accuracy & 98.91 & 97.93 & 57.44 & 84.11 \\  & AUC & 99.94 & 99.95 & 78.52 & 99.47 \\  & Specificity & 97.71 & 95.75 & 74.10 & 97.78 \\  & Sensitivity & 99.51 & 99.19 & 34.95 & 55.13 \\  & Precision & 98.88 & 97.60 & 50.00 & 92.14 \\ \hline \multirow{4}{*}{ResNet-50} & Accuracy & 99.08 & 97.93 & 58.26 & 87.40 \\  & AUC & 99.97 & 99.91 & 76.46 & 98.22 \\  & Specificity & 97.21 & 94.34 & 72.66 & 91.94 \\  & Sensitivity & 100.00 & 100.00 & 38.33 & 77.78 \\  & Precision & 98.64 & 96.85 & 51.28 & 81.98 \\ \hline \multirow{4}{*}{MobileNet} & Accuracy & 99.24 & 98.28 & 52.89 & 86.30 \\  & AUC & 99.93 & 99.98 & 83.12 & 99.40 \\ \cline{1-1}  & Specificity & 97.91 & 97.17 & 71.22 & 95.56 \\ \cline{1-1}  & Sensitivity & 99.90 & 98.92 & 28.16 & 66.67 \\ \cline{1-1}  & Precision & 98.98 & 98.38 & 42.03 & 87.64 \\ \hline \multirow{4}{*}{VGG-19} & Accuracy & 95.22 & 94.66 & 55.79 & 84.66 \\ \cline{1-1}  & AUC & 99.24 & 99.15 & 73.31 & 98.93 \\ \cline{1-1}  & Specificity & 91.43 & 90.09 & 64.75 & 92.74 \\ \cline{1-1}  & Sensitivity & 97.10 & 97.29 & 43.69 & 67.52 \\ \cline{1-1}  & Precision & 95.82 & 94.47 & 47.87 & 81.44 \\ \hline \end{tabular}
\end{table} 
Table 2: Round 1 performance comparison among models (AlexNet, ResNet-50, MobileNet, and VGG-19) that were trained on Base and tested on samples with varying similarities (Level 1–4) for three-class classifications (normal, COVID-19, pneumonia). 2 to 48% on average (Figure 2) at Level 3 implied a reduced proportion of correct positive identifications. At Level 4, which represented images from an external medical facility, all models showed a decline in classification accuracy compared to Level 2, e.g., from 97% to 86% on average. Although the AUC values remained high for all four models (\(\sim\)99%), both sensitivity and precision experienced a substantial drop.