## 3 Results and Discussion
### Explainability of the Classification Models  
With the models derived and exhibiting good classification metrics, we intended to visualize what triggers a trained model to classify a given image as COVID-19-positive or negative. This question is in the context of _explainubility and interpretability_ analysis. Trained CNNs (and other deep learning models in general) are frequently perceived as \"black box\" operators that map inputs (images) onto predicted labels. However, practitioners often do not exactly understand what specific features in the image induced the model to classify the image under a given category. Identifying these features constitutes an important step in the debugging process, which should ensure that the model has learned to look at the pixels (features) that convey the appropriate information to classify the image. It might occur that, in the training process, a model trained to associate certain patterns with the labels can actually mislead the classification process. For instance, one can train a model to distinguish images with cats labeled as \"CATS\" from images with dogs labeled as \"DOGS\". If the images with cats were all obtained outside on sunny days and the images with dogs were all taken indoors, the model could wrongly learn that blue skies should be associated with the label \"CATS\" and that the lack of blue skies should be linked to the label \"DOGS\". If the testing set was obtained from the same set of images--i.e., the testing images of cats also have blue skies and the images of dogs are indoors--the model will likely correctly classify the cats as \"CATS\", but only because of the blue skies rather than because of the cats shown in the images. Likewise, it will classify images with dogs as \"DOGS\" due to the features pertaining to the indoor environment. However, in this case, the same model would probably classify an image of a dog walking in a park on a sunny day as \"CATS\". Therefore, it is important to confirm that the features (pixels) that a given model actually uses to classify images are appropriate. Several approaches have been developed for this purpose [69; 70; 71]. In this work, we applied an approach called GradientExplainer to compute the so-called _Shapley Additive exPlanations_ (SHAP) scores [71; 72; 73; 74; 75]. Given an image with its associated prediction, for each feature in the image, a SHAP score constitutes a measurement of the influence of having a certain value in that feature on the obtained prediction in comparison to the prediction that would be made if that feature took some baseline value. Features with high positive SHAP are features that increase the probability of making such a prediction, whereas negative SHAP scores reduce the probability of classifying the image as such. In other words, the areas of an image with a concentration of high SHAP values constitute areas that convey relevant information to the classification model. Using [74], we implemented this technique to find the most relevant pixels in some of the classified images. Figures 8 and 9 display four original pulmonary radiographs--two of patients diagnosed with COVID-19 and two of uninfected people. Each of these images was properly predicted with the derived classifier. For each of these images, the corresponding SHAP values were computed, and the results are  
to be classified as COVID-19-negative or NORMAL, these areas would attain very low SHAP values, which is illustrated in the bottom images in Figures 10 and 11. Note the side color bar that indicates the mapping from colors to SHAP values. According to the color bar marked as \"SHAP Value\", the more intense the magenta color is, the higher the SHAP scores are and the influence that those pixels had in the classification. Hence, clouds with a high density of magenta points reveal zones in the image that convey significant information used by the classifier. On the other hand, Figures 12 and 13 show the SHAP values computed for the cases of Figure 9, which were classified as NORMAL.