# Accelerating Detection of Lung Pathologies with Explainable Ultrasound Image Analysis
## 4 Model Explainability
### Class Activation Maps
#### 4.1.2 Expert Validation of CAMs for Human-in-the-Loop Settings  
A potential application of our framework is a human-in-the-loop (HITL) setting with CAMs as a core component of the decision support tool that highlights pulmonary biomarkers and guides the decision makers. Since the performance of qualitative methods like CAMs can only be validated with the help of doctors, we conducted a blinded study with two medical experts experienced in the diagnostic process with ultrasound recordings (one physician with \(>\)10 years of clinical experience with LUS and one academic US course instructor). The experts were shown a set of 50 videos comprising non-proprietary video data which was correctly classified by the model. The class activation map for the respective class was computed two times, first with one of the four models that were trained on this video, and secondly only with the model that was not exposed to that video during training (called train- and test-CAMs in the following). Both experts were asked to compare both activation maps for all 50 videos, and to score them on a scale of \(-3\) (\"the heatmap is only distracting\") to 3 (\"the heatmap is very helpful for diagnosis\").  
First, the CAMs were overall perceived useful and the test CAMs were assigned a _higher_ average score of 0.81 than the train CAMs (0.45). Secondly, disagreeing in only 8% of the cases, both experts independently decided for the test-CAM with a probability of 56%. Hence, the test-CAMs are superior to the train-CAMs, however non-significant in a Wilcoxon signed-rank test.  
However, train- and test-CAM both scored best for videos of bacterial pneumonia, lacking performance for videos of healthy and COVID-19 patients. Specifically, test-CAM received an average score of 0.81, divided into \(-\)0.25 for COVID-19, 2.05 for pneumonia, and 0 for healthy patients. Thirdly, the experts were asked to name the pathological patterns visible in general, as well as the patterns that were highlighted by the heatmap. Figure 6 shows the average ratio of pattern that were correctly highlighted by the CAM model, where the patterns listed by the more senior expert are taken as the ground truth for each video. Interestingly, the high performance of our model in classifying videos of bacterial pneumonia is probably explained by the model's ability to detect consolidated areas, where 17 out of 18 are correctly classified. Moreover, A-lines are highlighted in \(\sim\)60% of the normal lung recordings. Problematically, in 13 videos mostly fat, muscles or skin is highlighted, which has to be studied and improved in future work.  
Figure 5: Class activation maps. (**Left**): Interactive scatterplot of the origins of the CAMs across the entire dataset, colored by class. While the data seems rather unstructured, pneumonia-CAMs have lower \(y\)-coordinates than COVID-19 and healthy samples. (**Rest**): Exemplary CAMs for COVID-19 (highlighting a B-line), bacterial pneumonia (highlighting pleural consolidations) and healthy lungs (highlighting A-lines).