## 3 Results
### Model Visualization  
Heatmaps provide a visualization of which parts of an image are important for classification decisions. These parts can be correlated to the clinical or empirical observations that physicians rely on to reach their diagnostic decisions. Figure 8a shows the heatmaps in terms of occlusion sensitivity in the sample images of normal, COVID-19, and pneumonia in Round 1. Hotspots are those that, when occluded, will cause a significant drop in the model's prediction. Interestingly, the heatmaps differed remarkably among models, even for an identical image, indicating very different discriminatory features used in different CNN models. Considering the normal image, no hotspots were predicted within the lung by all models except VGG-19. For COVID-19 (second row), scattered zones of intermediate sensitivities (opacities) were predicted within the lungs. By contrast, sporadic hotspots were  
Figure 7: Comparison of misclassification rates between Round 1 and Round 5 for (**a**) ResNet-50 and (**b**) VGG-19. observed in the pneumonia image. In particular, a large zone with intermediate-to-high sensitivities was employed in the lower lobes by ResNet-50. Figure 8b shows the occlusion sensitivity using ResNet-50 in Round 5. After four more rounds of training by exposing them to an incrementally enriched dataset, the heatmaps in Round 5 appeared more indicative and intuitive than in Round 1 with respect to the clinical manifestations, which exhibited a clear lung in the normal case, dispersed opacities and occasional hotspots in COVID-19, and a large region of hotspots in the lower lobes in pneumonia. The improved interpretability from Round 1 to Round 5 underscored the potential benefits of multi-round training on model performance. Next, we examined the impact of multi-round training (Rounds 1-5) on the Grad-CAM heatmap predicted by ResNet-50 (Figure 9). Grad-CAM, short for Gradient-weighted Class Activation Mapping, computed the gradients of the model's class scores with respect to the final convolutional layer's feature maps and only kept features with positive contributions. From the heatmap variations across Rounds 1-5, we could see the progression of ResNet-50's learning behavior by gradually including more diverse images. It was noted that the three images were all correctly predicted. Like the occlusion sensitivity in Figure 8, the Grad-CAM heatmap of the normal image was clear in the lung in all five rounds (the upper panels, Figure 9a-e), which was consistent with the final prediction for the normal class. Considering the COVID-19 image, hotspots of Grad-CAM were observed mainly in the middle and upper lungs. However, the intensity and location of these hotspots varied from Round 1 to Round 5, and this variation did not show a regular pattern in this example. Thus, explicit correlation from heatmaps to clinical manifestations of the disease needs caution, even though it is highly desirable to detect the disease from heatmaps and learned features or to use heatmaps and learned features to better understand the disease. The intermediate activation features at selected layers in residual Block 1 of ResNet-50 are shown in Figure 10. A diagram of the main layers in Block 1 is shown in Figure 10a, which has two branches (i.e., shortcut branch and main branch) and represents the key innovation in ResNet-50, which allows activations to bypass one or more layers. Here \"bn_conv1\" stands for \"batch normalization for convolution layer 1\". The same three source images were passed to the model (Figure 10b), and Figure 10c shows sample extracted features at the first convolution layer \"conv1\", followed by batch normalization (Bn_conv, Figure 10d), Max_pooling (Figure 10e), Bn2a_bracha2a (Figure 10f), Bn2a_bracha2b  
Figure 8: Occlusion sensitivity map for the sample images of normal, COVID-19, and pneumonia at (**a**) Round 1 and (**b**) Round 5. (Figure 10g), and the Block 1 activation layer \"Add_1\" (Figure 10h).