dataesr/scientific-paragraphs-categorization huggingface dataset:
A. Data analysis Because each detection event in the 3 He PC was recorded by the acquisition system, a very large amount of binary data was generated during the campaign (around 2 TB). Each irradiation cycle, called one "run" was recorded into a single binary file. The first step was obviously to convert the binary files for better handling of the lists of events, for instance in the Matlab environment. Then, several steps were performed to convert the raw data into curves to be fitted by a mathematical model: 1. Energy scaling: the energy histogram of each channel is rescaled, so that the same energy region of interest (ROI) can be applied to all channels. 2. Background discrimination: events outside the region of interest (typically 1000 to 3 10 4 LSB) are discarded. 3. Time synchronization: the fast shutter closing time is used as a reference time to synchronize the cycle.
A concurrent rise in the development of modern machine learning (ML) techniques and the increasing degree of their application to scientific research enabled the LHC experiments to achieve that goal, by squeezing more information from their datasets and improving the quality of their scientific output. In the above context are set the activities of AMVA4NewPhysics, an Innovative Training Network funded through the Marie-Sk lodowska Curie Actions of the European Union Horizon 2020 program. The network, which operated from September 2015 to August 2019, saw the participation of about fifty researchers and students from nine beneficiary nodes among European research institutes and universities, 1 in addition to nine academic and non-academic partners in 1 The involved beneficiary nodes were the Italian Institute for Nuclear Physics and the University of Padova (Italy), the University of Oxford (England), the Université catholique de Louvain (Belgium), the Université Clermont Auvergne (France), the Laboratório de Instru-Europe and the United States.
2 The network, while keeping as its primary goal excellence in training of a cohort of Ph. D. students, conducted cuttingedge research and fostered the use of advanced multivariate analysis methods in physics data analysis for the ATLAS and CMS experiments at the CERN LHC  [8]. The four main pillars, upon which most of the studies performed within AMVA4NewPhysics were based, comprise: In this paper we summarize some of the research outcomes that resulted from work performed by AMVA4NewPhysics members in the four pillars defined above, highlighting the importance of the results for future studies at the LHC and beyond. mentação e Física Experimental de Partículas, Lisbon (Portugal), the CERN laboratories, the Technische Universitat Munchen (Germany), and the Institute for Accelerating Systems and Applications (Greece).
Section 5 is then devoted to describing improvements achieved on the Matrix-Element Method, which is a complex multi-dimensional calculation that approximates the likelihood function to extract SM parameters from the observed data, and resulting applications to Higgs boson searches in ATLAS and CMS data. Section 6 focuses on the new methods we designed to search for new physics in LHC data in model-aspecific ways through the identification of anomalous regions of the feature space of the observed datasets; the section also includes description of a technique developed to improve inference on the presence of new physics signals in invariant mass distributions, and its expected performance in searches for high-mass resonances decaying to jet pairs. Section 7 describes an innovative study of particle showers in the ATLAS forward electromagnetic calorimeter, aimed at the production of a fast simulation of the complex physics processes detected by that instrument.
The complexity of the classification task, combined with the high expertise behind the best proposed solutions, made the HiggsMLChallenge competition a benchmark against which to evaluate and compare different ML approaches for supervised classification, as well as to gauge their applicability on HEP datasets. Triggered by the interest of the challenge and the derived conclusions, we conducted a thorough study published in Ref. [11] , whose results are summarized infra. In parallel, the new Lumin  [12]  software package was developed to provide implementations of the investigated methods, using PyTorch  [13]  as the underlying tensor library.
Challenge details and datasets The data used in the competition were constituted by information on all particles produced in proton-proton collisions simulated under the 2012 LHC run conditions (a centre-of-mass energy of 8 TeV and a typical instantaneous luminosity of 10 34 cm -2 s -1 ), which was fed through a simulation of the ATLAS detector, and from which, after applying state-of-the-art reconstruction algorithms, a set of 30 high-level physics observables were derived per simulated collision. The signal was constituted by direct production of a Higgs boson followed by its decay into a pair of τ leptons, pp → H + X → τ + τ -+ X (where X denotes any additional produced particles) with the subsequent mixed decay of the τ lepton pair, τ + τ -→ e + (µ + )ν e(µ) ν τ + τ - h ν τ (or to the charge-conjugate final state), where τ h denotes the hadronic decay products of one of the tau leptons.
Data preprocessing The training and testing datasets  [14]  consist of 250,000 and 550,000 events, respectively, each a mixture of signal and backgrounds. At the time of the competition, process-identifying labels were only supplied for the training dataset, and the testing dataset was split into two parts: the public test set, for which scores (see Section 2. 2. 2, infra) were supplied to the participant after every submission; and the private test set, for which scores were supplied for the solution selected by the participant once the competition ended, and on which the competition was judged. The analysis discussed infra attempted to reproduce the challenge conditions, by developing the model based on scores on the public dataset, and then checking the performance of the final model on the private dataset.
Dataset features consist of low-level information and high-level information, the latter calculated via (non)-linear combinations of the low-level features or from hypothesis-based fitting procedures. Additionally, events carry a weight to normalize the datasets to a fixed integrated luminosity. Since this weight carries information about the event process (signal or background), weights were not publicly available for the testing datasets, and they cannot be used as an input feature in developing the models. Ref. [14]  provides a compact summary of the dataset features.
g. cosine-annealed LR  [22]  and 1-cycle annealing of LR and momentum  [23] ; • Employing domain-specific data-augmentation techniques (see e. g. [24] ) to improve the performance and generalization of the model; for LHC collisions in symmetrical detectors like ATLAS and CMS, events can be rotated in azimuth and flipped in the transverse and longitudinal plane to create new, valid inputs without affecting the class of the events. This may be applied during training time to artificially increase the available train-ing data and during testing time to reduce residual biases in predictions by averaging over a set of augmented inputs; • Performing advanced ensembling via: Snapshot Ensembling (SSE)  [25] , Fast Geometric Ensembling (FGE)  [26, 27] , and Stochastic Weight Averaging (SWA)  [28] ; These methods can either reduce the time required to train and apply ensembles, or allow a larger ensemble to be trained in a similar time as traditional ensembling; • Using densely connected hidden layers  [11] ; similar to DenseNet  [29] , each hidden layer receives as input the output of all previous layers, meaning that prior representations of the data are never lost (thereby protecting against over-parameterization of the model), and parameters have a more direct connection to the output and subsequent back-propagated gradient.
Performance is measured in AMS (see Section 2. 2. 2). We report the average performance of our model (evaluated via repeated trainings), but for the challenge entrants' solutions we report the AMS of their submitted predictions. The inference times are the recorded time taken to evaluate every event in the full testing dataset (public and private). Accounting for the differences in precision and throughput between the Nvidia Titan and 1080 Ti GPUs, it is estimated that the   • 2nd place: Salimans (https://github. com/TimSalimans/HiggsML), 8core CPU, 64 GB RAM; • 3rd place: Pierre (https://www. kaggle. com/c/higgs-boson/discussion/ 10481), 4-core CPU (2012 laptop). This result highlights the need of optimizing the architecture and training of the ML algorithms, as this has been shown to lead to significant improvement over common default choices for both classification and regression problems.
We consider a dataset of QCD events that feature at least four jets in the final state, and construct in each event a 'transverse thrust axis' using the transverse momentum of the jets. This axis can be defined by the azimuthal angle φ T such that φ T = arg max φ T   j p T,j | cos(φ j -φ T )|   , (1) where the sum runs over all observed jets j; alongside with T we may define the related variable T a = j p T,j | sin(φ j -φ T )|. Once T is defined, the event can be ideally split in two hemispheres by the plane orthogonal to T. The two hemispheres contain two sub-lists of the jets, characterized by the different signs of cos(φ j -φ T ). We may describe each hemisphere by a number of observable features, h(N j , N b , T, M, T a , P z ). In this expression N j is the number of contained jets, N b is the number of b-tagged jets, T (T a ) is their transverse momentum sum along the thrust axis (orthogonal to it), M is their combined invariant mass, and P z is the sum of longitudinal components of the jets.
Results Multi-dimensional statistical tests that employ a complete set of kinematic variables describing the event features prove that the produced artificial events model the multi-dimensional distribution of the original features to sub-percent accuracy, if the dataset is constituted by QCD events. Furthermore, when the data contain a small fraction of events originated by Higgs pair production events the mixing procedure washes out the features of that minority class, such that the artificial data sample still retains accuracy in modelling the QCD properties (see Fig. 2 , which shows the effect of different signal fractions in the modelling). This happens because the probability that two hemispheres in the library, chosen to model a signal event, be both originated from other signal events (and thus retain memory of the peculiarities of the multi-dimensional density of signal in the event feature space) scales with the square of the signal fraction in the original data.
Hence a small signal contamination present in the dataset will not impair the validity of the model. This property makes the hemisphere mixing method an attractive option for the search of rare signals in QCD-dominated datasets. It is of special interest the fact that the user does not need to identify a control sample of data where to perform modelling studies: the method can be directly applied to the same sample where the signal is sought for. This is a considerable simplification of the experimental analysis, which also reduces modelling systematics. Finally, by searching for multiple similar hemispheres to the two that make up the event to be modelled, one may construct a synthetic dataset much larger than the original one, reducing the statistical uncertainties without introducing appreciable systematic biases.
Improvements and Applications of the Matrix Element Method Machine learning techniques employed at the LHC typically rely on the presence of large sets of training data for optimization purposes. The Matrix Element Method (MEM) takes a different approach, and provides a way to approximate the likelihood function for parameters in the SM given observed data. This calculation is performed from first principles, without the need for training. It was first used by the D0 Collaboration for a top quark measurement  [81] , with the original proposal provided by Kunitaka Kondo  [82]. The method can also be used to discriminate between different collision processes by providing powerful observables in searches for rare signals. Ratios of likelihoods describing the probabilities that observed events be consistent with signal or background processes are used in this context.
Main implementation aspects The MadWeight package  [87]  introduced a general way to approach the problem of finding an efficient phase space parameterization for integration purposes. This includes the removal of sharp peaks in the integrand, for example due to resonances. MadWeight is no longer supported, and its lack of flexibility hinders its application. MoMEMta  [83]  has been designed to build upon the ideas of MadWeight. It is a modular C++ software package, introduced to compute the convolution integrals at the core of the method. Its particular modular structure provides the required flexibility, allowing it not only to extend its applicability beyond its use in smaller programs so as to cover the needs of the complex LHC analysis workflows that handle large amounts of data, but also to be open to specific optimizations in the integration structure or engine.
At the same time, since the MEM may be used in both theoretical and experimental high-energy physics problems, with different purposes and use cases in each field, MoMEMta's modular structure constitutes a significant advancement. In terms of accuracy and CPU time, the performance of MoMEMta is similar to that of MadWeight, because they rely on the same algorithmic approach of phase-space parameterization. MoMEMta is however further designed to adapt to any process and allows for implementation in any C++ or Python environment, offering more freedom to the user, who can therefore wrap new modules that handle specific tasks, while benefiting from the existing features of the framework.
Summary The MEM provides a method for evaluating the likelihood of collision events under different hypotheses from first principles. It has applications in parameter measurements and searches for specific collision processes. The MoMEMta software package provides a flexible implementation of the calculations required. It offers flexibility with its modular structure, and enables use of the MEM for a wide range of applications at the LHC. The modularity also allows for extensions to handle novel applications, while taking advantage of the optimizations and convenience features provided by MoMEMta.
Overview Searches for new physics at the LHC proceed by comparing the data collected by the detectors with simulated data sets obtained from software simulations of the underlying physics process, interfaced with a simulation of the detector response. The simulated data describe either the processes predicted by the standard model (background processes) or the processes postulated by the particular theoretical extension of the SM under study (signal processes). These searches may be broadly categorized into model-dependent searches, where the data are compared to both the SM and new physics predictions, and modelindependent searches, where the data are compared only to the SM in order to search for unexplained deviations from it: these deviations (anomalies) may then be attributed to new physics and investigated further.
In this context, the term model-independent refers to independence with respect to a new physics signal model: the SM background model is always assumed. Model-dependent searches are typically approached by producing simulated events for both SM and new physics processes, and hypothesis tests are devised to find which model is favoured by the data. In case of model-independent searches, no particular model for the signal processes is assumed: a simulated dataset describing the signal processes cannot therefore be produced. These searches can be approached as anomaly detection problems, where the data are combed to find any observation that is not consistent with the background model. This setting is an example of semi-supervised learning, given that the two data categories involved are a 'simulated' dataset, generated by Monte Carlo techniques to represent the known background process and therefore considered as 'labelled', and the 'experimental' data sample, which is generated by an a-priori unknown mechanism-possibly comprising contributions from both background and signal processes-and therefore considered as 'unlabelled'.
The presence of a signal in the experimental data is generally inferred through the observation of a significant deviation from the predictions for the background process. Let Y = (y 1 ,. , y m ) , y l ∈ R P , l = 1,. , m, be the experimental data, with y l independent and identically distributed realizations of the random vector with unknown probability density function f BS : R P → R. In addition to the experimental data Y , it is possible to generate with the use of Monte Carlo simulations a large sample X = (x 1 ,. , x n ) , x i ∈ R P , i = 1,. , n. While Monte Carlo simulations in HEP are computationally taxing, a simulated sample of SM events is employed in so many different analyses that the experiments can afford to simulate tens or hundreds of millions of events for SM scenarios: the sample X can therefore be considered arbitrarily large to all practical extents.
H 0 : f BS (•) = f B (•) ⇐ λ = 0 and the alternative as H 1 : f BS (•) = f B (•) ⇐ λ = 0, In Refs. [103, 104, 105] , the IB algorithm was applied to the HEPMASS dataset  [107] , available at http://archive. ics. uci. edu/ml/datasets/HEPMASS: the dataset consists in a background composed by standard model top quark production and a signal composed by a heavy resonance that decays into a top quark pair that would result in a spectrum different from the SM for any observable involving the full final state (e. g. the reconstructed visible mass of the top pair decay products). Refs. [103, 104, 105]  contain the details on the input features that were used. Fig. 9  (left) shows a purity versus efficiency curve for the IB algorithm, as well as for two reference classifiers (relative likelihood, and k-nearest neighbours) that use only event-based information to classify events.
An extensive test  [110]  is performed comparing the IB algorithm to several scenarios, as illustrated in Fig. 10. The case where the signal density is known is represented by two scenarios: a classical likelihood ratio test based on the Neyman-Pearson lemma, and a kernel density estimation performed on the original datasets X and Y. A semi-supervised approach-the most realistic competitor to IB-is represented by a semi-supervised LDA. The IB algorithm is represented by several settings, corresponding to different choices of test statistic (Kolmogorov-Smirnov test  [111] , Mann-Whitney test  [112] ) and of score aggregation (minimum, maximum, mode, or median of the set of test statistics). The best setting for the Gaussian mixture in exam (λ = 0. 06) appears to be the expected value (mean) of the test statistic.
The parametric approach The idea of addressing anomaly detection by semi-supervising parametric clustering was first explored, in the context of new physics searches, in  [115, 116]  under the name of fixed background model (FBM). The authors proposed to specify the distributions f B and f S themselves in Eq. 6 within the family of Gaussian mixture models. Parameter estimation was then conducted via maximum likelihood in two steps. First, the background model fB was obtained based on the background data X. Afterwards, keeping the parameters of fB fixed, the weight λ in Eq. 6 and the parameters characterizing the new possible component f S were estimated using experimental data Y. Maximization of the likelihood function was conducted via a suitable adjustment of the Expectation-Maximization (EM) algorithm  [117].
The proposed penalized anomaly detection (PAD) method builds on a variant of the EM algorithm, derived in the semi-supervised context to estimate the parameters of a mixture model via the maximization of the penalized likelihood function. The PAD methodology was validated both in the unsupervised setting (i. e. to estimate the background distribution) and in the semi-supervised anomaly detection setting (to estimate the whole process density) via simulations designed to account for different aspects of the analysis: different implementations for handling variable selection; performance of competing methods; varying config- urations of the background and possible signal. The study highlighted a general improvement with respect to the state of the art (see  [109]  for details). In the specific context of LHC physics research, the PAD was applied on Monte Carlo data produced to study the experimental signature of processes yielding two energetic jets in the final state.
We refer to the kernel hyper-parameters collectively as θ B = {A, a, b, c, d} and θ S = {C, m, t, k}, respectively. We present procedures that can operate using two or three steps as follows: In the case of the two-step procedure, the background kernel is used in a first step; then, in a second step, a signal kernel identifies a concentrated excess or deficit centred at m with width t. For the three-step procedure, the first two steps are used to accommodate a background with a turn-on; then a third step performs the signal detection. This procedure can be considered as semisupervised: we first model the background-only invariant mass distribution, i. e. the labelled dataset, and then find a model for the unlabelled data distribution where signal could appear.
Modelling the dijet invariant mass: QCD background and artificial signal For testing the two-step procedure, we used as toy dataset a simulated dijet invariant mass spectrum following the event selections described in  [126]. Before reporting the results of the GP optimization, we describe the injection of signal in the invariant mass spectrum. The starting point is the simulated background dataset we generate pseudo-data from, where the signal can be injected. Since the spectrum ranges over several orders of magnitude in the number of events, we prescribe the amplitude of the injected signal via a quantity defined from a signal-over-background ratio (R). We calculate the amplitude of the Gaussian signal from the mean and width: R is the ratio of signal to background events that should be in a window constructed from the interval given by the mean and width.
This is especially relevant for the simulation of the ATLAS forward calorimeter (FCal)  [133, 134]. The FCal is a sampling calorimeter that covers the most forward region of the ATLAS detector, in the range of pseudo- The following modules, made with tungsten, are designed to absorb the hadronic contribution to the particle showers. Since the total energy of a showering particle is predominantly deposited in the sampling calorimeter through a large number of soft electrons and photons, the work presented here focuses on the fast simulation of the FCal response in the electromagnetic module. In ATLAS, the simulation of the FCal detector already makes use of fast techniques aimed at reducing the complexity of the response. The FCal fast simulation is based on a frozen shower library approach  [135] , which consists of storing pre-simulated electromagnetic showers initiated by high-energy particles entering the front face of the calorimeter, subsequently used to quickly model the detector response.