## 2 Methods
Comparison with these models are conducted with the corresponding \"rpart\", \"randomForest\", \"e1071\", \"RSNNS\" and \"stats\" packages in R.

In our experiments, we want all methods to have the same opportunities to achieve their best results, thus the default settings in respective R packages are adopted: for CART, the split criteria is set to Gini; for RF, number of trees \(ntree\) = 500 and split variables \(mtry\) = 3; for SVM, a radial kernel is applied and \(\gamma\) = 1/9; for MLP, the number of units in the hidden layer \(size\) = 5, and maximum of iterations, \(maxit\) = 100; for LR, the error distribution and link function \(family\) = \(gaussian\). The detailed experiment results are given in Table 1.

In Table 1, the highest AUC scores in each fold are highlighted in bold face. From Table 1, we find that CBDF takes the first place three times in five folds, and it beats other learning algorithms with a mean AUC of 0.8051. The mean AUC scores of RF, SVM, CART, MLP and LR are 0.6851, 0.6311, 0.6296, 0.5997 and 0.5579, respectively. In the following, we will compare CBDF with other commonly used balancing techniques, namely the representing under-sampling approach of deletion majority Tomek Links (TL), the representing over-sampling approach SMOTE (SM) [17]. As CBDF is an ensemble of CART decision trees, it is appropriate to choose another ensemble of CART decision trees as the baseline learner in the comparisons. Thus we will use the most popular ensemble of CART decision trees, Random Forest (RF), as the baseline learner in the following experiments. We preprocess the training data in each fold by only removing the majority Tomek Links (TL), by only generating minority samples by SMOTE (SM), or both. Then an RF model is learnt with the processed training data. Experiment results in all folds are shown in Table 2.

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Fold** & **1** & **2** & **3** & **4** & **5** \\ \hline CBDF & 0.7895 & **0.6842** & **0.9444** & 0.6905 & **0.9167** \\ CART



From Table 2, we can find that, CBDF (mean AUC: 0.8051) outperforms TL-RF (mean AUC: 0.6541), SMOTE-RF (mean AUC: 0.7036) and the combination of both (mean AUC: 0.6754) by a large margin. The exclusion of noise Tomek Links, clustering majority samples and bootstrapping minority samples and weighted average of the base classifiers all contributes to CBDF in part. From Tables 1 and 2, we also notice that RF using SMOTE method also does better than CART, RF, SVM, MLP and LR algorithms. However, it is less stable than CBDF and only gets 0.3158 in Fold 1, which is even worse than a random guess (0.5). ### Variable Importance

As mentioned above, one important feature and advantage of CBDF is that variable importance can be calculated easily. Here, we use CBDF to evaluate all variables and find out which are the most important ones in affecting the seminal quality. We record all the spit impurity information in CART in all five folds and the final mean variable scores are given in \[y^{*}=\arg\max_{y}\sum_{C_{i}\in C}w[i]\times C_{i}(\mathbf{x},y)\]0

From Figure 5, we find that the top five variables affecting semen concentration are Age (age at the time of analysis), Accident (accident or serious trauma), Season (Season in which the analysis was performed), Sitting (Number of hours spent sitting per day), Alco \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Fold** & **1** & **2** & **3** & **4** & **5** \\ \hline CBDF & 0.7895 & 0.6842 & **0.9444** & **0.6905** & **0.9167** \\ TL-RF & 0.3158 & 0.5263 & **0.9444** & 0.6786 & 0.8056 \\ SM-RF & 0.3158 & **0.7895** & 0.8333 & **0.6905** & 0.8889 \\ SM+TL-RF & **0.9474** & 0.3158 & 0.5278 & 0.6190 & 0.5278 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison with other balancing methods in terms of AUC. Figure 5: Variable importance scores by CBDF. consumption), and Fever (High fevers in the last year). The least important variable found according to Fertility dataset is Childish (Childish diseases).