## 4 Discussion
### Dataset Effects  
In this study, three datasets of X-ray images for normal, COVID-19, and pneumonia were utilized. We demonstrated that, when using the same datasets for training and testing, high classification accuracy (98.1 \(\pm\) 1.6%) and sensitivity (99.1 \(\pm\) 1.2%) were achieved even for the testing images that had been set apart in a separate folder (Level 1), as shown in Figure 2 and Table 2. Furthermore, even for an external dataset (Level 2), impressive accuracy (97.2 \(\pm\) 1.5%) and sensitivity (98.6 \(\pm\) 1.0%) were achieved, indicating the high quality of both datasets as well as the image resemblance between these two datasets.  
The Level 3 dataset contained outlier images collected from the first and second databases, which exhibited evident deformities (rotation, scaling, shifting, incompleteness) and color variations. These images presumably came from data augmentations when the database was small. In this study, we observed drastic declines in model performance when tested on these outlier images--accuracy dropped from 98.1% at Level 1 to 56.1%, sensitivity from 99.1% to 36.3%, and precision from 98.1% to 47.8%. Note that CNN models in Round 1 had not seen these outlier images yet, even though the outlier images were modified from the training images. The drastic drop in performance indicated that features of the outlier images were missing in Round-1-trained models, indicating that the traditional data augmentation techniques would introduce extraneous features and undermine the model development using chest X-rays. Even though data argumentation techniques have been useful when the COVID-19 images were limited, training unmodified images is recommended when COVID-19 X-rays become more available.  
The Level 4 dataset in this study comprised external images that displayed certain disparities from the training images, as evidenced by the moderately reduced classification metrics from Level 1 to Level 4 (i.e., accuracy from 98.1% to 85.6%, sensitivity from 99.1% to 66.8%, Figure 2 and Table 2). Even within the same modality (X-ray), variations could arise due to differences in equipment and operational procedures. The heterogeneity in image quality could introduce potential biases in model training and superfluous misclassifications in testing [55]. In addition, images are susceptible to mislabeling and corruption. Thus, standardization and validation of image databases and repositories, which are lacking in most public COVID-19 X-ray datasets, are needed to avoid future complications.  
The training data were imbalanced in this study. The baseline training images (11,538) include 7791 for normal, 2717 for COVID-19, and 1030 for pneumonia. This might explain the high misclassification rates in normal vs. COVID-19 and normal vs. pneumonia (Figure 7). By contrast, the classification for COVID-19-pneumonia is much rarer. Thus, adopting a training dataset with a balanced distribution among these three categories may further improve the model's performance. It is also noted that COVID-19 has been proven to be highly mutational, and existing datasets cannot fully account for the evolving variants. This will increase the likelihood for a future COVID-19 patient to receive a false negative diagnosis, which makes multi-round training imperative to ensure the accuracy of AI-assisted COVID-19 diagnostic systems.