dataesr/scientific-paragraphs-categorization huggingface dataset:
Glossary of terms • Statistical model: This is a synonym for the probability model p(x, y|µ, θ) as in Eq. (  7 ) that includes dependence on the data x and y, the parameters of interest µ and nuisance parameters θ, access to the individual terms and the ability to generate pseudo-(or synthetic-) data (i. e. , "toy Monte Carlo"). • Likelihood: The value of the statistical model for a given fixed dataset as a function of the parameters, e. g. , L(µ, θ) in Eq. (  7 ). • Constraint term: A term in the full statistical model that relates auxiliary data y to a particular nuisance parameter θ. • Observed data the n, x, and y of Eq. (  7 ) needed to construct the likelihood. • Open-world: An approach to statistical modelling that allows users to define and implement custom components in the statistical model.
RooFit provides the core interfaces, modelling language (needed to combine various terms as in Eqs. (  6 ) and (  7 )), as well as a large library of modelling components (needed to represent the individual distributions p ik (x|µ, σ)). The library of modelling components that RooFit provides is finite, but large enough that it stretches the intent of the term closed-world. Moreover, users can extend the library by inheriting from the RooAbsPdf abstract interface, which enables an open-world approach to modelling if desired. In addition to RooFit, the software package RooStats builds on RooFit and provides the statistical tools for hypothesis tests, confidence intervals, and other types of inference  [29]. The factorization of the modelling (i. e. , RooFit) and the inference (i. e. , RooStats) is important conceptually and technically, and the success and stability of the interfaces is an indication that the factorization is robust and mature.
A key innovation was the introduction of the RooWorkspace through the RooStats project, which provides both a convenient interface to RooFit as well as a serialization of statistical models and associated data  [17, 29]. Workspaces can include multiple datasets (e. g. , observed data, Asimov data, pseudo data, modeled with RooAbsData), statistical models (RooAbsPdf), and model metadata (ModelConfig) that identify parameters of interest (POI) µ, the nuisance parameters θ, the primary measurements x, and the auxiliary data y. Workspaces are currently being preserved within the LHC collaborations and form the basis of inter-collaboration combination efforts such as the LHC Higgs Combination Group. Such workspaces represent ready-made data products that could be published on open archives such as HEPData.
Brief guide to commonly used statistical tools • RooFit provides a statistical modelling language and a large library of common modelling components. It also provides the RooWorkspace, which in turn provides a convenient interface to RooFit and is a container for a statistical model, data, and metadata that can be serialized using ROOT's file format and serialization technology. These workspaces are the primary tool for combining statistical models, for example in the ATLAS+CMS Higgs combinations. • RooStats, which is built on top of RooFit, provides tools for statistical inference (hypothesis testing, confidence intervals based on asymptotic properties of the profile likelihood ratio, pseudo-data generation, credible intervals -the Bayesian analog of confidence intervals). • HistFactory is a widely used tool that provides a small subset of RooFit modelling components and defines a declarative specification for binned statistical models in a closed-world approach.
pyhf also permits the pruning of full statistical models in order to simplify them for subsequent statistical analysis (see, for example, Ref. [35] ). The package pyhf introduced a declarative specification using JSON that combined the information from the XML and the ROOT portion of the original schema, such that the entire statistical model and the associated data are represented in a single human and machinereadable JSON file. JSON is ubiquitous as a file format and is supported in most programming languages, including common languages used in particle physics such as C, C++, Python, Fortran, Julia, and Go. Moreover, as an ASCII document it is also a good candidate for longterm archival of data products. In addition to reducing software dependencies and improving long-term archival, the choice of JSON as a format also enhances functionality when it comes to reinterpretation.
Consequently, it is possible to construct a HistFactory statistical model in both ROOT and pyhf from the same JSON file. Likewise, the framework-and computer language-independence of JSON allows for new future implementations of HistFactory. The ATLAS Collaboration has taken a significant step in this direction by starting to publish full statistical models to HEPData associated with published ATLAS analyses. As of the time of writing, in mid-2021, ATLAS has published 8 full statistical models  [20] [21] [22] [23] [24] [25] [26] [27]  in the pyhf JSON format, and has recommended the publication of statistical models for all LHC Run 2 supersymmetry analyses. As demonstrated in Section 4. 3, these published statistical models are already being used by the theory community for new work  [37] [38] [39].
An example user story with a published model To illustrate how these model serialization and analysis tools can be used today, we present an existing example  [40] , based on a published ATLAS statistical model, as a "user story". A small team of physicists would like to download a statistical model from HEPData that was published by an LHC collaboration so that they can investigate and visualize the impacts of different model parameters on the analysis and determine how modifications of the model would impact the final results. The physicists download a pyhf "pallet" from HEPData, which contains all of the JSON and JSONPatch files that serialize the full statistical model for an ATLAS search for bottom-squark pair production  [20, 41]. They are able to use the serialized model patches to build a workspace for a given signal hypothesis using pyhf and, then, using the cabinetry  [42]  library, investigate yields from the model.
The natural place to publish such "digitized" likelihoods and related documentation is HEPData. Lastly, there are recent approaches to simulation-based inference  [15, 51, 52]  such as MadMiner  [53] [54] [55]  that use machine learning to approximate the functional dependence of p(x|µ, θ) or p(x|µ, θ)/p ref (x) on both the data x and the parameters (µ, θ). Unlike the DNNLikelihood approach, which focuses on approximating the full statistical model built using approaches such as RooFit, HistFactory, Combine, etc. , the goal of these tools is to serve as the primary statistical model when the data x are high-dimensional and the traditional modelling approaches are inadequate. Technically, these machine-learning based surrogates go beyond the likelihood function since they depend on the observable data, but, typically, it is not possible to identify the internal details of the model such as signal and background components, etc.
Other related examples HAMMER The HAMMER library  [56]  provides a fast and efficient algorithm to reweight large b → {c, u} τ ντ simulated samples to any desired new physics scenario generated by the relevant ten four-Fermi operators. In addition, changes of hadronic form factors to evaluate uncertainties or float such as additional nuisance parameters in a minimization problem, can be introduced. Although the code itself does not directly construct likelihoods, it provides the LHCb and Belle II experiment with the necessary key tools to present experimental data in a model-independent way -a concrete toy example of which is discussed in Section 4. 4. 3. The code further allows experiments to reuse their large dedicated SM MC samples for new physics interpretations. The algorithm is based on event-weights of the form α,i,β,j c α c † β F i F † j W αiβj , (11) that are proportional to the ratio of the differential rates (and thus depends on the final state kinematics).
Another example, where these tools have been applied by users outside of the Fermi-LAT Collaboration is the characterization of an excess of gamma rays from the center of the Milky Way in terms of dark matter (see, for example, Ref. [64] ). The approach taken by Fermi-LAT is not so much to publish likelihood functions for given models but rather to provide the community with easy to use tools to allow individual scientists to implement their own analysis. Models of universal backgrounds (isotropic, Galactic diffuse gamma-ray emission and point sources) are provided as templates (in fits format or as text files)  [65]. Likelihoods for specific models given specific datasets are not published as part of the Fermitools, but some individual analyses decided to publish likelihood functions in machinereadable format (see e.
g. , it has some negative eigenvalues) or it is ill defined. In this case, the dataset is either discarded or its covariance matrix regularized by some ad hoc procedure. iv The covariance matrix turns out to be sensitive to subtle differences in the assumptions concerning its correlation model. In these cases, the χ 2 in Eq. (  12 ) can deviate markedly from unity even when theory and data are fully compatible. This situation would likely be avoided if the "error on the errors", or sometimes more importantly the "error on the correlations" were to be accounted for. It is important to emphasize that these problems are becoming more acute with the availability of high-precision LHC measurements, which are more often than not limited by systematic rather than by statistical uncertainties. Furthermore, these issues also affect the eventual combination of datasets from a given experiment, which share common systematic error sources, within the global PDF fit.
An example of a low-statistics dataset which in principle provides information required for PDF fits may be found for high-x ZEUS structure function data  [85]. Similar techniques could be applied to low-statistics data in tails of high-energy LHC datasets, if the full statistical models were provided. Some recent examples of category (iv) include issues related to the interpretation of differential top-quark pair production data in terms of the gluon PDF  [86] [87] [88]  and the analysis of inclusive jet and dijet cross sections  [89, 90] , where decorrelation or regularization approaches are required to achieve a sensible PDF fit. Furthermore, one should note that possible tensions observed between datasets might not have an underlying physical origin but may arise instead from the limited available experimental information or affected by the sensitivity to the correlation model described above.
This could have a significant impact on the so-called "tolerance", or artificially inflated ∆χ 2 introduced in some global PDF fits, e. g. , Ref. [78, 79]  in order to account for the tension between different datasets. An example of a dataset which is a source of tension in PDF fits is the ATLAS W, Z 2011 dataset, a high-precision measurement which is omitted from the CT18 global fits because of potential tensions with other datasets, which are absent when a dataset is interpreted independently of others  [91]. Some of these sources of tension may arise from the theoretical uncertainties in the PDFs due to the limits of perturbative QCD calculations. We note the first attempts to understand theoretical uncertainties related to the PDF extractions  [92, 93] , and in theoretical perturbative calculations, e.
Perspective from "official" experimental combinations Within the ATLAS and CMS experimental collaborations, combined Higgs boson measurements have been produced using data from Run 1 of the LHC  [105, 120]. The results published are not combinations of the individual measurements performed by each experiment, but rather a set of combined measurements of Higgs boson properties using simultaneously the Run 1 datasets collected by ATLAS and CMS. While the two experimental collaborations use different software to construct likelihood functions and perform statistical inference, both are based on the RooFit and RooStats packages and the serialization of the data and the statistical model p(x|µ, θ) uses a common container format provided by the ROOT based RooFit package: the RooWorkspace  [17, 29].
The full statistical models are also needed in order to estimate global p-values associated, for example, with the Standard Model hypothesis in view of the BSM search results. An attempt to do this was made in Ref. [136]  based on an entire database of simplified model results from nearly 100 ATLAS and CMS searches. Here, the statistical models of the analyses were sampled treating every signal region in every analysis as if it were independent of the others. This approximation holds only in the case where the experimental uncertainties are dominated by statistical rather than by systematic effects. With full statistical models for all results at hand, in which it is possible to identify parameters that have the same meaning across analyses, a single global model could be constructed, which could then be used to generate pseudo data for all analyses simultaneously.
The module FlavBit uses SuperIso  [141] [142] [143] , which provides the theoretical predictions of the flavor observables together with their uncertainties and correlations in the SM and in BSM scenarios, and compares in detail the theoretical predictions against all relevant experimental data. Another widely used example is the flavio  [144]  package. It includes a large database of experimental measurements and has an easy to use interface to carry out fits for new physics scenarios. The package allows for both frequentist and Bayesian fits and implements likelihoods via YAML files using either Gaussian approximations (with or without asymmetric uncertainties) or, where available, numerical evaluations of the profile likelihood. If more experimental information were made available in a comprehensive form, i.
e. , as a full likelihood, its inclusion would be straightforward and highly beneficial. There are additional subtle points that are important to make correct interpretations of flavor anomalies. For instance, the interpretation of other measurements characterizing the b → s transition require theoretical input for form factors. This creates a non-trivial dependence between, e. g. , fitted Wilson coefficients or new physics couplings and the theoretical input that was used. V ub and V cb from semileptonic B hadron decays. Such measurements can be carried out with selections that yield statistically independent datasets. For instance, the most precise value of V ub from exclusive B → π ν decays is obtained when using a global average prepared by HFLAV  [145, 146]  of the measured differential branching fraction as a function of the four-momentum transfer squared q 2 = (p Bp π ) 2 , shown in Fig.
Left: Electron (blue), oxygen (red), and tungsten-recoil bands (green). The bands have been fitted to the neutron calibration data and are required for the definition of the acceptance region (yellow) in the blind data. Right: The fraction of accepted events in the nuclear recoil bands, per nuclear species as a function of the reconstructed energy. Plots taken from  [156]. rameters such as the assumed dark matter density or velocity distribution, nuclear physics form factors and yield parameters common for certain target materials. Unless the scaling is trivial, such as changing the local dark matter density, reinterpreting experimental results will require the full statistical model used by the experiments. In the following sections, we describe two such efforts, one by the CRESST-III Collaboration, and the other by XENON1T, specifically their ionization-only analysis.
CRESST-III public results In the CRESST experiment particle interactions are measured by means of scintillating calorimeters. The conventional target material is CaWO 4 in form of crystals. An incoming dark matter particle may collide with any of the individual nuclei (with probability proportional to its respective interaction cross section). The data consist of a set of 2D points representing the measured deposited energy in the detector and the scintillation efficiency, Light Yield, associated with the particle interaction. The Light Yield permits the identification of the nuclear recoils and it is used to define the region of interest (ROI). For a given model f (E R |ω), the probability for a nuclear recoil of energy E R to be reconstructed at energy E reco reads f (E reco |ω) = Θ(E reco -E thr,reco ) • ε(E reco ) • ε acc (E reco ) × +∞ 0 f (E R |ω)N (E reco -E, σ 2 )dE, (15) where Θ(E reco -E thr,reco ) is a hard cut representing the trigger (only energies above E thr,reco are taken into account), ε(E reco ) is the fraction of events surviving the cut selection, ε acc (E reco ) the fraction of events occurring in the acceptance region in the Light Yield -Energy plane and N (E reco -E, σ 2 ) is a normal distribution accounting for the finite energy resolution σ of the detector.
Example combination Open statistical models in direct detection will allow combining results from multiple experiments either with different and complementary detector technologies or with shared nuisance parameters for similar experiments. Assumptions on the dark matter velocity distribution can be updated as new and more precise data becomes available, and signal models can be easily extended beyond the customary spin-dependent and spin-independent WIMP-nucleon interactions. As an example, a low-threshold detector with a higher background rate may  (Left) Constraints on the spin-independent WIMP-nucleon cross section using CRESST-III  [157]  and XENON1T ionization-only  [159]  data, altering the assumption on the Milky Way escape velocity v esc. Results from CRESST-III (black), with a lower energy threshold are approximately constant, while the higher energy threshold means the XENON1T limit is weakened with a lower escape velocity.
In Fig. 14  (Right) predictions of light mediator models for different experiments are reported together. The CRESST-III sensitivity (not present in the data release) was determined using the data release and interpreted in the light of the light mediator models developed for the XENON1T analysis. Having the full likelihood available allows for the determination of the sensitivity at any desired confidence level. In the example presented in Fig. 14  (Left), the sensitivities (at each WIMP mass) of the individual searches were chosen such that the combined p-value equals 0. 1.
The only available information (in case of a Bayesian credible interval) is that the integrated likelihood over a quoted range has a certain value. A further, common, example of insufficient information is when (external) input parameters were used in a measurement but their values are not quoted in a publication. Sometimes certain assumptions were made in some measurements, but not in others or assumptions differ in different measurements. Such information is essential in order to perform a valid combination of measurements. Another area of frequent difficulties are correlations. Correlations between measurements by different collaborations as well as correlations between measurements by the same collaboration with different methods or datasets must be known if correct averages are to be produced.
Unfortunately, this information is often not or only partly available. It is for this reason that the averaging collaborations are composed of members of various experimental collaborations who may be able to obtain the required information. Correlations arise from common parameters. While these are usually well defined for physics parameters, such as daughter particle branching fractions, the situation is often more challenging for nuisance parameters describing systematic effects. For example, consider the trigger or reconstruction efficiencies for two datasets taken with evolving detector performances or configurations or processed with different reconstruction algorithms. Quantifying the correlation can be hard even for those with access to internal information. The systematic publication of likelihoods would address major problems in the combination of measurements.
In the last few years, in particular, global fits turned into a challenging undertaking, due to the breadth of available data from particle physics, astrophysics and cosmology, their associated uncertainties, and the breadth of new physics models under consideration. State-ofthe-art BSM global fits are now performed by large collaborations comprising experimental and theoretical physicists, using dedicated software frameworks such as GAMBIT  [181] , MasterCode  [182] , HEPFit  [183]  and Fittino  [184]. As the preceding sections and, in particular, the examples in Figs. 5, 6, 8 and 11 show, insufficient likelihood information can significantly distort the result of a parameter estimation analysis. Since a BSM global fit aims at a statistical combination of all experimental results relevant for a new theory -from measurements of the Higgs boson, heavy flavor and electroweak precision observables, to searches for new particles at the LHC and in dark matter experiments -these likelihood challenges compound in global fit analyses.
[191, 192]  for efforts in this direction. Before closing this section, we note that in addition to using public data products including statistical models, global fits create them. The results of a global fit, be it a collection of posterior samples created using MCMC sampling or otherwise, are valuable and computationally expensive to obtain, and could be preserved for future scrutiny and reuse. See, e. g. , Refs. [193, 194]  for examples published by the GAMBIT Collaboration using Zenodo. In summary, open statistical models from particle physics, astrophysics and cosmology would make it easier and faster to perform reliable global fits, alleviate numerical pathologies introduced by otherwise necessary approximations, enable simulation-based approaches, and, lastly and perhaps most importantly, allow fits to harness the full power of the breadth of experimental data available to us today and in the decades to come.
Figure14: (Left) Constraints on the spin-independent WIMP-nucleon cross section using CRESST-III [157]  and XENON1T ionization-only [159]  data, altering the assumption on the Milky Way escape velocity v esc. Results from CRESST-III (black), with a lower energy threshold are approximately constant, while the higher energy threshold means the XENON1T limit is weakened with a lower escape velocity. The dashed black line shows a combined result. (Right) Spin-independent WIMP-nucleon cross sections in the case of a light mediator mass. The CRESST-III result has be determined according to Ref. [159]  making use of the data public release, LUX results are presented in Ref. [161] , PandaX-II in Ref. [162].
Acknowledgements This work was supported in part by the  National Science Foundation  under Cooperative Agreement OAC-1836650 (KC, MF and MN); the  U. S. Department of Energy  Award No. DE-SC0010102  (HP); the  IN2P3  master project " Théorie -BSMGA " (SK); the  Buchman Scholarship Funds  and the  Azrieli Foundation (IB) ; the  Knut and Alice Wallenberg Foundation  and the  Swedish Research Council (JC) ; the  Science and Technology Facilities Council, STFC , through grants  ST/S000844/1  (GC),  ST/T000856/1  (RT) and  ST/N003985/1  (NW); the  DFG -Germany 's excellence strategy EXC -2094-390783311 (NFI); the  NSFC Research Fund for International Young Scientists  grant  11950410509  (AF); and the  National Science Foundation  under Award no. 1719286  (KDM).