###### Abstract  
The current guidelines recommend the sentinel lymph node biopsy to evaluate the lymph node involvement for breast cancer patients with clinically negative lymph nodes on clinical or radiological examination. Machine learning (ML) models have significantly improved the prediction of lymph nodes status based on clinical features, thus avoiding expensive, time-consuming and invasive procedures. However, the classification of sentinel lymph node status represents a typical example of an unbalanced classification problem. In this work, we developed a ML framework to explore the effects of unbalanced populations on the performance and stability of feature ranking for sentinel lymph node status classification in breast cancer. Our results indicate state-of-the-art AUC (Area under the Receiver Operating Characteristic curve) values on a hold-out set (67%) while providing particularly stable features related to tumor size, histological subtype and estrogen receptor expression, which should therefore be considered as potential biomarkers.  
sentinel lymph node; imbalanced dataset; data augmentation; breast cancer; machine learning; interpretability +
Footnote †: leftmargin=*] *Nevertheless, the classification of sentinel lymph nodes provides a typical example of an unbalanced classification problem. In healthcare data analysis applications including cancer diagnosis and disease risk prediction, classifying unbalanced datasets has been identified as a major challenge. When the quantity of samples in a dataset is substantially uneven, an unbalanced classification problem emerges. In binary classification tasks, the unbalanced classification problem occurs when one class (i.e., the minority class) has much fewer observations than the other class (i.e., the majority class) [11]. For unbalanced datasets, most of the machine learning (ML) algorithms usually perform poorly since they aim to optimize overall classification accuracy and thus assume an equal incidence of all classes. As a result, these methods exhibit classification bias towards the majority class, failing to identify examples of the minority class [12].  
Several strategies have been proposed to overcome these limitations. Resampling methods have been introduced to rebalance the datasets. Resampling schemes include random oversampling of the minority class, undersampling (or subsampling) of the majority class and some advanced synthetic sampling methods that aim at rebalancing the class distribution at the data level. These rebalancing solutions, however, present several drawbacks. For example, loss of information is an inherent consequence of undersampling [13], but oversampling by randomly repeating the minority class sample could result in overfitting [14].  
Although several works have investigated the effect of different data balancing strategies on the overall classification performance [15; 16; 17; 18], there is a lack of work exploring the effect of these methods on feature ranking. This issue is particularly crucial in clinical contexts, where the interpretability of decisions and therefore the accurate identification of the feature contribution to the decisions of the algorithms is a key requirement [19; 20].  
In this work, we developed a ML framework to explore the effects of unbalanced populations on the performance and stability of feature ranking for sentinel lymph node status classification in breast cancer. In particular, we assessed the predictive power of different clinical and immunohistochemical features by exploiting two classifiers. We embedded a module specifically devoted to analysis the importance of features in relation to the variation of the training set obtained by different sampling techniques. Our work aims to address some questions: (i) How do the different models perform? (ii) What are the most important predictors? (iii) Are they stable across populations?