## 4 Discussion
However, since such limited performances have been obtained on a limited cohort of patients, we can expect that our preliminary classifier could increase its predictive power when a larger sample (labeled by histological finding and follow up) could be available. Concerning the application of radiomics to 2D DM for calcifications, very few papers have been published. [PERSON] et al. [31] proved that a multimodal radiomic model, consisting of radiomic features from both mammography and dynamic contrast-enhanced magnetic resonance imaging (MRI), combined with a random forest classifier, showed a Sensitivity of 83% and a Specificity of 80% at Leave-One-Out validation, performances comparable to those obtained by our single-modality DBT-based radiomic classier with an ensemble of random forests. Moreover, our performances were validated with a more robust method (nested k-fold cross-validation) and tested also with a permutation test. In literature, radiomics has been also applied to other mammographic techniques such as contrast-enhanced spectral mammography. [PERSON] et al. [32] published a multicenter study showing that a radiomics nomogram of contrast-enhanced spectral mammography was able to predict axillary lymph node metastasis in breast cancer. Other authors showed correlations between mammographic radiomics features and the level of tumor-infiltrating lymphocytes in patients with triple-negative breast cancer [33]. A recent study on Chinese patients published by [PERSON] et al. [34] confirmed the promises of radiomics combined with DBT to distinguish malignant from benign calcifications, showing that, when radiomic features can be extracted from the reconstructed volume of calcifications and integrated into the 2D radiomic features, the model is able to reduce the FP rate up to 20%. To be noted, in our work, the number of FP classified by our radiomic classifier was found reduced by a factor of 5 from the initial number classified by the expert radiologist, thus effectively reducing the risk of inappropriate biopsies. However, we must consider that a reduction in FP rate without a contemporary high NPV (over 98%) has no clinical impact if BI-RADS clinical rules are applied. In order to account for this, we designed a DSS based on both the radiological classification (BI-RADS) and the low FP rate achieved by our radiomic classifier on suspected breast calcifications classified as BI-RADS 3. The final results of such a DSS were very good (Table 4), showing a reduction by a factor of 2 in FP from the initial expert radiologist, at no cost of FN, and an improvement in the PPV by 50%, suggesting potential in our approach, and further validation of these preliminary results with larger datasets. If confirmed by multicenter studies with large sample size, our DSS could reinforce the decision of BI-RADS 3 category to not send to biopsy but to recall for monitoring or to downscale the diagnostic category from BI-RADS 3 to BI-RADS 2. Our study has important limitations. First, the retrospective single-center design, which made it difficult to obtain a larger sample size useful to train and independently test both the radiomic classifier and the DSS developed. However, the different examinations were collected using different DBT units; thus, we can consider the performance of our model to be somewhat robust with respect to independent DBT systems. Second, our study suffers from the lack of a temporal and geographical independent dataset to test the developed models. However, we performed a robust nested k-fold cross-validation on the radiomic model to avoid using testing data during training, and, as already highlighted above, a permutation test to assess the statistical significance of our results and to exclude the presence of false discoveries (\(p<0.05\)). Third, the non-consecutive enrollment of cases, which does not allow a disease-prevalence interpretation of the PPV and NPV obtained in our work, although they can be compared quantitatively.