## 1 Introduction  
Machine learning (ML) techniques are increasingly being adopted in a variety of medical applications for the development of clinical predictive models, i.e., models for the prediction of outcomes of clinical interest, using a set of candidate variables or features. When building a clinical predictive model with ML, two common problems are variable ranking and variable selection. Variable ranking, i.e., the ordering of features based on their importance for outcome prediction [1], is useful both to provide an interpretation of the model, i.e., to compare the predictive ability of different variables, and to perform a feature selection, or model reduction, i.e., to identify the most important features and consequently remove the unnecessary variables from the model. The feature selection is important for several reasons. First, models with a large number of input variables can be more difficult to interpret: noisy features, which are not related to the outcome, can have small and implausible effects in the identified model [2]. Noisy features can also lead to an overfitting of the training set data, with consequent poor generalization ability of the model on new previously unseen datasets. Moreover, the models with many input variables are not easy to implement in the clinical practice because some variables may be difficult to collect in different clinical contexts [3]. A simple and popular approach to perform variable ranking is recursive feature elimination (RFE), an iterative algorithm that starts from the full model, i.e., the model including all the variables, and then removes the variables one at the time, dropping at each iteration the least important one. The order of feature removal determines the variable ranking. Once the ranking is obtained, feature selection can be performed by selecting the set of top ranked variables according to a certain criterion. RFE was originally proposed to perform gene selection for cancer classification using a support vector machine (SVM) model [4]. Then, the algorithm was used for a multitude of applications. For example, RFE and SVM were recently used for classifying attention deficit/hyperactivity disorder [5], multiple sclerosis relapses [6], the management of patients with chronic obstructive pulmonary disease [7], and for selection of heart variability features for cumulative stress monitoring [8]. RFE was also used in combination with other ML models, such as logistic regression (e.g., for [PERSON]'s disease classification [9]), random forest (e.g., for breast cancer classification [10]), and extreme gradient boosting (e.g., for predicting clinical outcomes in young patients with hypertension [11]). Indeed, an advantage of RFE is that it can be applied with any ML model. However, a difficulty of this approach is that the feature ranking can become unstable and sensitive to small perturbations of the training set, potentially resulting in different selected feature sets [12]. In order to assess the stability of variable ranking and perform a robust variable ranking, resampling techniques (with or without replacement) are commonly applied to the original training set in order to produce B different versions of the training set. Then, RFE is performed on each training set version, producing B ranked lists that are combined in order to obtain a global ranked variable list. The most straightforward way to aggregate the B different ranked lists is to use the Borda count method [13], an algorithm well known in voting theory that assigns a score equal to the sum of the number of features with higher position over the B lists to each variable. Then, the global ranked list is obtained by ordering the features according to the Borda count. The Borda count method is equivalent to ranking the features according to their average rank obtained over the B lists. The RFE-Borda count method has been used in many applications [14; 15]. Nevertheless, the RFE-Borda count approach can be affected by the presence of correlated features.