## 3 Evaluation
### Results and Discussion
Interestingly, the results seem more stable in the case of IEM-FD when comparing the performance of fine vs. coarse crackle detection. In any case, it is worth mentioning that the results are worse than the compared approaches and specially limited by the number of false negatives (i.e., crackles events detected as normal), which provokes a clear underperformance in terms of \(P_{r}\). Note that the \(Acc\) can be seen as general metric accounting for both \(S_{e}\) and \(P_{r}\) values. Table 2 shows the mean values in terms of accuracy \(Acc\), sensitivity \(S_{e}\) and precision \(P_{r}\) for the IEM-FD method, TVAR method, and the proposed method for test samples \(\psi\) at SNR from \(-10\) dB and until \(+10\) dB for real and simulated crackles. In contrast to the previous results shown in this document, here we make a distinction by type of crackle within the test sample: ATS, Hoevers, Cohen, IPF, or BE, labeled here as simulated scenarios 1 to 6 and real scenarios 1 and 2. From these results, we can highlight the fact that our proposed method has widely higher percentage values in comparison with the IEM-FD and TVAR methods. It is important to note that all the methods have a clear underperformance in real scenario 2. That is, when dealing with real sounds with coarse crackles (i.e., bronchitis) superposed to simulated breathing. This suggests that the difference between simulated  
Figure 8: Accuracy, sensitivity, and precision average results evaluating all scenarios for each type (fine crackles on the left side and coarse crackles on the right side) of crackles and SNRs from database \(\psi\) by IEM-FD [28] (red color), TVAR [51] (green color), and the proposed method (blue color), where the dashed lines represent the mean value for each metric and method. and real fine crackles allows a suitable modeling valid for both cases whereas in the case of coarse crackles, the parameters may vary between simulated and real cases. Once we have evaluated the robustness of the proposed method against different type of signals and SNR conditions and compared the results with classical approaches, we aim to also compare the results with more recent techniques based on deep learning. In fact, deep learning approaches have recently been widely investigated for the task of adventitious sound detection [41; 62; 63; 64; 65; 66; 67; 68]. The investigation carried out by [55] highlighted the usage of convolutional neural networks (CNNs) as state-of-the-art solutions across various research domains. Leveraging the CNN architecture as detailed in [55], we subjected the same scenarios as previously outlined to testing, subsequently conducting a comparative analysis between the outcomes generated by the support vector machine (SVM) classifier and the CNN classifier. The comparative results, presented in Table 3, delineate the performance contrast in terms of accuracy (_Acc_), sensitivity (\(S_{e}\)), and precision (\(P_{r}\)) across the eight scenarios encompassing our proposed method. As can be observed in Table 3, both classifiers (SVM and CNN) achieve excellent results (>90%) in all metrics except in the real coarse crackle scenario. In any case, a slight improvement can be observed using the SVM-based classifier in terms of accuracy (_Acc_) and sensitivity (\(S_{e}\)). Conversely, CNN performs slightly better in terms of precision (\(P_{r}\)). This suggests that the proposed system is somewhat more robust in determining the occurrence of crackle events, which is very interesting from a clinical perspective. However, it is worth noting that SVM is simpler to train, has fewer parameters, and therefore is more robust to overfitting. Additionally, its parameters are much more interpretable than those of CNN. It is pertinent to underscore the notable performance decline observed in the real-world scenarios examined in this study. As evident in Table 3, these scenarios exhibit discernible decreases across all three metrics. These findings shed light on the potential limitations inherent in the proposed methodology.