## 4 Scikit Learn vs. Tmva  
As seen in Section 2, TMVA is the dominant reference tool for signal-background discrimination at LHC experiments. Therefore, in this section we check, using the generic data presented above, whether TMVA provides the optimal discrimination when compared to Sklearn, assuming that one is using the same datasets and the same algorithm. The exercises below are intended to show the same analysis being carried out with both TMVA and Sklearn libraries, with the goal of helping scientists to switch between the two according to their needs. The main difference between Sklearn and TMVA is the way the data are processed. In Sklearn, we have to transform the ROOT TTrees to a numpy array or a pandas dataframe in order to work with them. At present, libraries such as uproot [48] or rootnumpy [49] allow us to make this change in the data format without extra effort. This option allows for theuse of other libraries, instead of TMVA, even for the usual datasets in LHC experiments, which tend to be ROOT TTrees. Both TMVA and Sklearn offer a simple way of handling algorithms, allowing for easy changes to their hyperparameters and providing useful information about the training time or even the ranking of features. To analyse and explain how the use of different libraries could be advantageous to the users, in Figure 5 and Table 4, we compare both libraries using BDT with AdaBoost for the \(B\to\mu^{+}\mu^{-}\) decay. In this case, we focus in the high-level features and high stats option for training. The comparison was made with the same hyperparameters and the same data for both libraries. This means that all the tunable parameters had the same value and the samples used for training and testing were the same for both cases. These hyperparameters were chosen as the best after performing a grid search, i.e., running through a range of the best score was reached. The results of the classifiers can be seen in Figure 5, where Sklearn is shown to have better results. This is also shown in Table 4, where the AUC scores for \(B\to\mu^{+}\mu^{-}\) and the rest of the decays can be found. In Table 4, we show all the results for the analyses that we defined in the previous section. For all these cases, Sklearn performs better than TMVA. This difference is small and not large enough to make a stronger statement, but it is still noticeable. To obtain a feeling of the statistical significance of the difference, we perform the DeLong test [50; 51], which allows for the AUCs of pairs of classifiers to be compared. This test provides a z score that helps to quantify the probability that the two classifiers perform differently when looking at the same test data. The larger the z score, the more significant the separation between both AUCs. When comparing Sklearn and TMVA, we find that, for all decay channels, the AUC scores are statistically significantly different. Note that this statistical test is limited to pairs of classifiers, so we do not apply it further in the paper, given the large amount of comparisons being performed in Section 5. \begin{table}
\begin{tabular}{c c c c} \hline  & **TMVA AUC** & **Sklearn AUC** & **z Value** \\ \hline \(B\to\mu^{+}\mu^{-}\) & 0.954 & 0.963 & 88.884 \\ \(B\to\pi^{+}\pi^{-}\) & 0.952 & 0.961 & 90.047 \\ \(B\to 3\pi\) & 0.957 & 0.960 & 88.224 \\ \(B\to 4\pi\) & 0.957 & 0.961 & 88.508 \\ \hline \end{tabular}
\end{table} 
Table 4: TMVA vs. Sklearn results: AUC scores for all four decay channels, corresponding to the high-level features and high stats option for training. The z value corresponds to a score [50; 51] comparing the AUCs of pairs of classifiers under the hypothesis that they are the same. The corresponding \(p\)-values are all \(\sim\)0. Figure 5: ROC Curve for the \(B\to\mu^{+}\mu^{-}\) decay, corresponding to the high-level features and high stats option for training. One of the reasons that we think could explain the differences we observe between libraries relates to how these are designed and have evolved. Sklearn was released in 2007, and TMVA in 2009.