# Accelerating Detection of Lung Pathologies with Explainable Ultrasound Image Analysis
## 3 Classification of Lung Ultrasound Data
### Results  
#### 3.2.1 Frame-Based Experiments  
Table 2 shows a detailed comparison of all trained models in terms of recall, precision, specificity and F1-scores, computed in a one-vs.-all fashion for all three classes. Overall, both VGG and VGG-CAM achieve promising performance with an accuracy of \(88\pm 5\%\) on a 5-fold CV of 3234 frames. The results are sufficiently balanced across classes, with the best F1-score achieved for COVID-19 in VGG (0.89) and for healthy data (0.87) in VGG-CAM. Figure 3a visualizes the results of the best model, VGG, for each binary detection task as a ROC curve, showing ROC-AUC scores of 0.94 and above for all classes, while depicting the point where the accuracy is maximal for each class. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline  & **Class** & **Recall (Sens.)** & **Precision** & **F1-Score** & **Specificity** \\ \hline
**VGG** & & & & & \\
**Accuracy: 87.8\%** & COVID-19 & \(\mathbf{0.88\pm 0.07}\) & \(\mathbf{0.90\pm 0.07}\) & \(\mathbf{0.89\pm 0.06}\) & \(\mathbf{0.94\pm 0.05}\) \\ Balanced: 87.1\% & Pneumonia & \(\mathbf{0.90\pm 0.11}\) & \(0.81\pm 0.08\) & \(0.85\pm 0.08\) & \(0.94\pm 0.04\) \\ \#Param: 14.7 M & Healthy & \(0.83\pm 0.11\) & \(0.90\pm 0.06\) & \(0.86\pm 0.08\) & \(0.94\pm 0.03\) \\ \hline
**VGG-CAM** & & & & & \\
**Accuracy: 87.4\%** & COVID-19 & \(0.86\pm 0.11\) & \(0.86\pm 0.07\) & \(0.86\pm 0.08\) & \(0.91\pm 0.04\) \\ Balanced: 86.1\% & Pneumonia & \(0.87\pm 0.15\) & \(\mathbf{0.87\pm 0.06}\) & \(\mathbf{0.86\pm 0.10}\) & \(0.96\pm 0.03\) \\ \#Param: 14.7 M & Healthy & \(\mathbf{0.86\pm 0.11}\) & \(0.88\pm 0.90\) & \(\mathbf{0.87\pm 0.90}\) & \(0.93\pm 0.04\) \\ \hline
**NASNetMobile** & & & & & \\
**Accuracy: 62.5\%** & COVID-19 & \(0.63\pm 0.22\) & \(0.67\pm 0.14\) & \(0.63\pm 0.15\) & \(0.79\pm 0.12\) \\ Balanced: 55.2\% & Pneumonia & \(0.22\pm 0.27\) & \(0.46\pm 0.42\) & \(0.28\pm 0.30\) & \(\mathbf{0.98\pm 0.03}\) \\ \#Param: 4.8 M & Healthy & \(0.80\pm 0.11\) & \(0.58\pm 0.05\) & \(0.67\pm 0.06\) & \(0.60\pm 0.13\) \\ \hline
**VGG-Segment** & & & & & \\
**Accuracy: 85.1\%** & COVID-19 & \(0.81\pm 0.20\) & \(0.86\pm 0.07\) & \(0.82\pm 0.13\) & \(0.91\pm 0.06\) \\ Balanced: 83.9\% & Pneumonia & \(0.86\pm 0.08\) & \(0.84\pm 0.07\) & \(0.85\pm 0.03\) & \(0.95\pm 0.04\) \\ \#Param: 34.0 M & Healthy & \(0.85\pm 0.12\) & \(0.86\pm 0.06\) & \(0.85\pm 0.08\) & \(0.90\pm 0.08\) \\ \hline
**Segment-Enc** & & & & & \\
**Accuracy: 85.7\%** & COVID-19 & \(0.84\pm 0.11\) & \(0.89\pm 0.07\) & \(0.86\pm 0.07\) & \(0.92\pm 0.07\) \\ Balanced: 84.4\% & Pneumonia & \(0.89\pm 0.04\) & \(0.75\pm 0.10\) & \(0.81\pm 0.06\) & \(0.92\pm 0.03\) \\ \#Param: 20.0 M & Healthy & \(0.82\pm 0.18\) & \(\mathbf{0.90\pm 0.05}\) & \(0.85\pm 0.12\) & \(\mathbf{0.94\pm 0.02}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of the tested classification models on 5-fold cross validation for each class. Accuracy abbreviates accuracy, Bal. balanced accuracy, Coefficient and Par. the number of parameters (in millions). For each class and each column the best model is highlighted in bold. The accuracy of 88% of VGG and VGG-CAM cannot be outperformed by pre-trained lung US segmentation models from [57], and unfortunately it cannot be matched by NASNetMobile, a model that is significantly smaller. The false positive rate at the maximal-accuracy point is slightly larger for COVID-19 than for pneumonia and healthy patient. In a clinical setting, where false positives are less problematic than false negatives, this property is desirable. Since the data is imbalanced, we also plot the precision-recall curve in Figure 3b, which confirms that the imbalance is not of concern. In addition, the confusion matrices in Figure 3c,d further detail the predictions of VGG; we observe that the sensitivity of bacterial pneumonia is highest, but confusions occur to similar extent for each pair of classes.