## 2 Main MI Algorithms for Signal and Background Discrimination and Their Use at Particle Colliders
In the process of analyzing how to make the most of ML in analyses at particle colliders, as mentioned above, we first compare the two best-known libraries in particle physics, TMVA and Sklearn, to quantify which one is more effective for the same problem, i.e., using the same data with the same characteristics and the same algorithm: a BDT with AdaBoost. For the second part of the paper, we increase the range of methodologies beyond the usual BDTs with AdaBoost to include NNs, and enlarge the range of libraries to PyTorch and Keras. The goal is to determine how all of these methods and libraries perform in terms of signal-background discrimination. To compare the performance of different classifiers, the usual methodology in ML involves the use of the so-called ROC curve. A Receiver Operating Characteristic (ROC) curve is a representation of a classifying model, built to show the discrimination power of the classifier at different thresholds. The two parameters represented in the curve are the rate of true positives vs. the rate of false positives. We built ROC curves for all the classifiers of interest in this paper. For each classifier, one can also integrate the area under the corresponding ROC curve (AUC). The AUC turns out to be a very useful benchmark when one wants to compare the performance of different classifiers, providing a metric to quantify the separation power between different classes. In general, the higher the AUC score, the better the model, i.e., the more often it correctly assigns each instance to the class to which it belongs. This parameter is taken into account to compare the different classifiers. Moreover, we also account for the learning time needed for the algorithms to process the data, as well as the correlations with some key independent or \"spectator\" variables, such as the invariant mass. Before moving to the problem of creating the models, let us examine how different LHC experiments have been using ML in recent years. This is a counting exercise, with which we intend to evaluate the popularity of different ML algorithms and libraries, without directly comparing their performance, which will be performed in the following sections. After looking into all LHCb publications from 2010 to August 2021, we calculated the percentage of papers per year that use TMVA and the percentage of papers that use Sklearn or other popular libraries that deal with NNs, such as Keras and PyTorch. We also generated a label, which we named \"Generic NN\" for those papers in which the use of a Neural Network is implied but the library used is not mentioned. The result can be seen in Figure 1, which shows how the use of TMVA is higher than Sklearn. In Figures 2 and 3, we repeated this for ATLAS [39] and CMS [40], respectively. We observed a similar behaviour to LHCb, where the use of TMVA was preferred to the other libraries. In ATLAS, even though TMVA is still the most used tool, we can appreciate an increase in the use of Keras in the last two years. Note than, in all three experiments, the use of Keras, PyTorch or other types of general NNs usually was not focused on direct signal-background discrimination at the analysis level, but instead on other tasks related to the reconstruction of events at the LHC, such as jet reconstruction and tagging, PID or track reconstruction. Figure 1: Usage of ML at LHCb across the years. We show the number of papers published every year (denoted as “Total number” and corresponding to the right axis), as well as the fraction of them reporting the use of TMVA, Sklearn, Keras, PyTorch and Generic NNs (in the left axis). The latter category corresponds to papers mentioning the use of NNs but never referencing any of the aforementioned libraries. Figure 2: Usage of ML at ATLAS across the years. We show the number of papers published every year (denoted as “Total number” and corresponding to the right axis), as well as the fraction of them reporting the use of TMVA, Sklearn, Keras, PyTorch and Generic NNs (in the left axis).