dataesr/scientific-paragraphs-categorization huggingface dataset:
1. The rest of this paper is organized as follows. Section 2 discusses in detail our definition of proto-models, their parameters and the assumptions they are based on. Section 3 describes the method used for computing the LHC constraints for a given proto-model as well as how we use this information to construct an approximate likelihood. The MCMC walk and the algorithm used for building new proto-models is detailed in Section 4. Finally, in Section 5, we apply the MCMC walk to the full SModelS database and discuss the results obtained. We conclude in Section 6 with a short summary and charting out future developments and extensions of this work. A set of appendices provides complementary information on the MCMC-type walk over the proto-models parameter space (Appendix A. 1), posterior densities of proto-model parameters and distributions of the test statistic (Appendix A.
As mentioned above, the lightest BSM particle is required to be the X 1 Z , hence all masses must satisfy m(X) ≥ m(X 1 Z ). In a given proto-model only particle masses below 2. 4 TeV are considered, so they are within the LHC reach. States which appear in two-or three-fold multiplicities are mass ordered, e. g. , m(X 2 t ) > m(X 1 t ). Some additional requirements are necessary for the masses of the colored new particles, to avoid that our machinery "discovers" light states in regions which are poorly covered in the database. Concretely, for the X g and X q we disallow masses below 310 GeV, since most particle decay channels particle decay channels X q qX j Z , q X i W , qX g X 1 W W X j Z X 1 t tX j Z , bX i W , W X 1 b , tX g X 2 W W X j Z , ZX 1 W , hX 1 W X 1 b bX j Z , tX i W , W X 1 t , bX g X j =1 Z W X i W , ZX k Z , hX k Z X 2 t tX j Z , bX i W , ZX 1 t , W X 1 b , tX g X X j Z , ν X i W X 2 b bX j Z , tX i W , ZX 1 b , W X 1 t , bX g X ν ν X j Z , X i W X g q qX i Z , q q X i W , b bX i Z , t tX j Z , btX i W , qX q , bX b , tX t Table  1.
LHC Results In order to quickly confront individual proto-models with a large number of LHC results, we make use of the SModelS  [14, [24] [25] [26] [27] [28] [29]  software package. This allows us to evaluate LHC constraints for a given input model without the need of Monte Carlo simulation. The experimental results stored in the SModelS database fall into two main categories: • Upper Limit (UL) results: these correspond to the observed limits on the production cross section (times BR) for simplified model topologies as a function of the BSM masses obtained by the experimental collaborations, σ UL obs. Some results also include the expected upper limits, denoted as σ UL exp. All limits are given at 95% confidence level (CL). • Efficiency Map (EM) results: these correspond to signal efficiencies (more precisely, acceptance × efficiency, A × , values) for simplified topologies as a function of the BSM masses for the signal regions considered by the corresponding experimental analysis.
These results also include information about the number of observed and expected events (incl. uncertainties thereon) for each signal region. While providing conservative results, the SModelS tool is computationally cheap, making it feasible to test hundreds of thousands of proto-models within reasonable time. The current SModelS database v1. 2. 4  [25, 28, 29]   Key to this is the construction of an approximate likelihood for the signal, L BSM (µ|D) = P (D|µ + b + θ) p(θ) , (3. 1) which describes the plausibility of the signal strength µ, given the data D. Here, θ denotes the nuisance parameters describing systematic uncertainties in the signal (µ) and background (b), while p(θ) corresponds to their probability distribution function. 2 In the following two subsections, we explain the two main steps to arrive at L BSM : computing the individual likelihoods for each applicable analysis, and combining the individual likelihoods into a global one.
In this context, CMS sometimes provides covariance matrices, which allow for the combination of signal regions as discussed in  [32]. (N. b. , this is still a simplified likelihood assuming Gaussian uncertainties. ) So far, however, a covariance matrix together with simplified model EMs is available for only one analysis  [34]. ATLAS, on the other hand, has recently started to publish full likelihoods using a plain-text (JSON) serialization  [35]  of the likelihood, which describes statistical and systematic uncertainties, and their correlations across signal regions, at the same fidelity as used in the experiment. 3 At present, EM results with JSON likelihoods are available in the SModelS database for three ATLAS SUSY analyses  [38] [39] [40]  for full Run 2 luminosity (139 fb -1 ), leading to clear improvements in the statistical evaluation for these analyses  [29].
The Walker Let us now turn to the algorithm for performing an MCMC-type walk in the proto-model's parameter space in order to identify the models that best fit the data. This algorithm, which we dub the walker, is composed of several building blocks, or "machines" that interact with each other in a well-defined fashion: 1. Starting with the Standard Model, a builder creates proto-models, randomly adding or removing particles and changing any of the proto-model parameters (see Appendix A. 1 for details). 2. The proto-model is then passed on to the critic which checks the model against the database of simplified-model results to determine an upper bound on an overall signal strength (µ max ). 3. The combiner identifies all possible combinations of results and constructs a combined likelihood for each subset (see Section 3.
2 for details). 4. Using the combinations provided by the combiner, the walker computes a test statistic K for the proto-model (see Section 4. 1 below). If the K value is higher than the one obtained in the last step, the new proto-model is kept. If it is lower than the previous K, the step is reverted with a probability of exp 1 2 (K i -K i-1 ) , where i is the index of the current step. By many iterations of the above procedure, the walker is able to identify proto-models which evade all simplified-model limits and at the same time can explain dispersed signals in the data. This will be illustrated by means of a toy walk in Section 4. 2 and by walking over the full SModelS database in Section 5.
Toy Walk In order to illustrate the walker algorithm, we first apply it to an extremely reduced version of the database with only three 13 TeV analyses: the ATLAS search for stops in the 1 lepton channel (ATLAS-SUSY-2016-16)  [48] , the CMS search for hadronically decaying tops (CMS-SUS-16-050)  [49] , and the generic CMS search in the multi-jet plus E miss  of the particles present in the proto-model every 10 steps. Each point is scaled according to the proto-model test statistic K, so bigger points correspond to higher K values. Since the database considered only contains results for simplified models with tops and bottoms in the final state, proto-models without top and bottom partners find no corresponding data and no test statistic can be computed; hence they are set to zero. Around step 20, a protomodel with a top partner (X 1 t ) is created, thus resulting in an increase in the test statistic.
Results using the full SModelS Database In this section we present the results of running the walker algorithm over the full SModelS Interpreting the relevance of the K values obtained for the proto-models created by the algorithm requires knowledge about the K distribution under the SM-only hypothesis. To this end, we first replace the observed data by values sampled from the expected SM backgrounds, resulting in an ensemble of "fake databases", which are then used to estimate the density of the test statistic K under the SM hypothesis. This procedure is presented in Section 5. 1. In Section 5. 2, we then apply the walker to the real data. Using the K distribution obtained under the SM hypothesis, we can evaluate the significance of the results and estimate a global p-value. Finally, in Section 5.
3, we will test the closure of our procedure by generating another "fake" database, where the signal of a given proto-model has been injected in the data. This way we can verify the behavior of the algorithm and make sure that it is indeed capable of correctly reconstructing a signal present in the data. For all results presented in this section, we used 50 parallel walkers, each with 1, 000 steps. For reasons of CPU time consumption, we ran the code without making use of the correlations across signal regions provided for a few ATLAS and CMS analyses (cf. section 3. 1); this is justified because the EM-type results, for which a covariance matrix or pyhf JSON files are presently available, do not noticeably influence the results. 9 With this setup, each walker has a runtime of 12h to 24h on a single core.
Walking over "fake" Standard Model Data In order to compute the significance of the test statistic and, ultimately, its global p-value, we produce "MC toy" versions of the database, in which we replace the observed data by values sampled from the SM background models. More specifically, • for EM-type results we sample a normal distribution with a mean of the background estimate, and a variance of the squared error of the background estimate. The sampled value is then entered as the lambda parameter of a Poissonian distribution. Finally, the value drawn from the Poissonian is used as the "fake" observation, i. e. the fake event yield; • for UL-type results, for which expected upper limits are known to SModelS, we estimate the SM uncertainty on the expected limit assuming a large number of observed events and small systematical uncertainties  [30].
The observed upper limits are then sampled from a Gaussian distribution: σ UL obs ∼ Norm σ UL exp , δσ UL exp 2 , (5. 1) where δσ UL exp = σ UL exp /1. 96 is the estimated width of the distribution; • UL-type results, for which only observed upper limits are available, enter the fake database as they are, since it is not possible to estimate the SM uncertainty in this case. Using the sampled values described above, we produce 50 fake databases and apply the walker algorithm to all of them. The proto-models with the highest K values from each of these runs are then used to estimate the density of the test statistic K under the SM hypothesis via a Kernel density estimator: ρ(K) := 1 N w N i=1 kern w K -K i fake (5. 2) Here, N = 50 is the number of runs with fake databases, and K i fake the test statistic for the i-th run; kern w denotes the choice of Kernel in the Kernel density estimation.
We choose a Gaussian with a width w determined by Scott's rule  [50]. Figure  6  shows the K i fake values generated for each run as well as the result obtained for ρ(K). As we can see, under the SM hypothesis, values up to K ≈ 11 are expected, with the density peaking at K ≈ 4. As discussed in Section 4, the walker algorithm looks for dispersed signals, with K computed as the maximum over all possible combinations of results. Therefore, large K values are expected to be found even when considering the SM hypothesis, where upward background fluctuations can mimic dispersed signals. In addition, it has been suggested  [51]  that the experimental results tend to be conservative and overestimate the background uncertainties. We can strengthen this claim by considering the p-values computed for all signal regions of the EM results contained in the database.
If the observed data is well described by the estimated backgrounds and the uncertainties thereon, one would expect a flat distribution of p-values. In fact, for the fake databases generated under the SM hypothesis, this is the case, as shown by the red histogram in Fig. 7. When we consider the real data, however, the distribution is not flat, as seen from the blue histogram in the same figure. This is expected if the background uncertainties are overestimated. As a consequence, the density ρ(K) of the SM hypothesis in Fig. 6  is conservative; we can safely assume that the actual density would be shifted towards lower values of K and that our results generally lie on the conservative side. In order to avoid distortions due to small numbers of events, only signal regions with at least 3. 5 expected events were included.
Walking over the LHC Data We are now ready to apply the walker algorithm to the actual SModelS database with the real LHC results. We perform 10 runs, each employing 50 walkers and 1, 000 steps/walker. The results are summarized in Fig. 8 , where we display the proto-models with the highest K value from each run. Besides the X 1 Z LBP, all models include one top-partner, X 1 t , and one light-flavor quark partner, X d,c , and their test statistics are at K = 6. 76 ± 0. 08 thus showing the stability of the algorithm. The X µ particle introduced in run 5 is due to small ≈ 1σ excesses in the CMS search for sleptons, CMS-SUS-17-009  [52]  and the ATLAS search for electroweakinos, ATLAS-SUSY-2016-24  [53] ; the prior's penalty for introducing the additional particle, ∆K = -1. 25, is overruled by the increase in the likelihood ratio, ∆K = 1.
Analysis (all excesses observed in the 13 TeV CMS and ATLAS stop searches  [48, 49] , which lead to the introduction of a top partner (X 1 t ) with a mass around 1. 2 TeV. Despite corresponding to small excesses, identifying the presence of such potential dispersed signals is one of the main goals of the algorithm presented here. Furthermore, the fact that these excesses appear in distinct ATLAS and CMS analyses and can be explained by the introduction of a single top partner is another interesting outcome of the whole procedure. It is worth noting that the signal injected by the winning proto-model is typically smaller than the one favored by the excesses, as seen when comparing the Obs and the Signal columns in Table  3. This is due to a tension with the constraints imposed on the signal strength by all the other analyses in the database (the critic).
The model discussed above is mostly constrained by the 13 TeV CMS multijet analyses for 36 fb -1  [57, 58]  and 137 fb -1  [15] , which all observed under-fluctuations with respect to the expected background, see Table  4. The most serious challenges comes from the 2 jets+E miss T and t t+E miss T limits of  [15] , for which the proto-model under consideration exceeds both the observed and expected UL. Concretely, the analysis  [15]  gives r obs (r exp ) = 1. 3 (1. 11) for the X d , and r obs (r exp ) = 1. 28 (1. 26) for the X 1 t. 10 Finally, we point out that very similar conclusions are drawn for all the high score proto-models shown in Fig. 8. Of course, all these conclusions may be challenged (or strengthened) by new results for full Run 2 luminosity, which are not yet included in the SModelS database, e.
g. the stop searches  [60, 61]. Some comments are in order on the significance of the excesses and K values found for the models in Fig. 8. In Fig. 9  we show the K distribution found under the SM-only hypothesis (see the discussion in Section 5. 1) and the values for the 10 highest-scoring protomodels obtained with the "real" database (green stars). As we can see, despite being at the tail of the distribution, the values still seem compatible with the SM-only hypothesis. In order to quantify this compatibility, we compute the p-value given by the relative frequency of the K values under the SM hypothesis which were found above the observed K values (K obs ): p global := lim N →∞ 1 N N i=1 1 [ Kobs ,∞) (K i fake ) ≈ ∞ Kobs dKρ(K). (5. 3) Here N = 50 is the number of fake SM-only runs, 1 X (x) denotes the indicator function that equals unity for all x ∈ X and zero otherwise, Kobs is the average of K obs of the ten runs over the "real" database, and ρ(K) refers to the estimated density of the test statistic as defined in Eq.
(5. 2). The resulting value corresponds to the shaded area under the curve shown in Fig. 9. Using the K values in Fig. 8  we obtain the global p-value for the SM hypothesis: p global ≈ 0. 19. (5. 4) Our results thus indicate a very mild disagreement with the SM hypothesis. As already 10 Recall that we accept mild transgressions up to r 1. 3 to account for the fact that when simultaneously checking limits from a large number of analyses, a few are statistically allowed to be violated. shows the distribution obtained under the SM-only hypothesis (see Fig. 6 ). The shaded green area corresponds to the global p-value for average K from the runs over the real database. pointed out in Section 5. 1, we expect this to be conservative due to potentially overestimated background uncertainties. Furthermore, this result requires no look-elsewhere correction, since the SM density ρ(K) was derived through the exact same procedure as applied for the real database.
Walking over "fake" Signals As discussed above, the K values obtained for the real database are in a mild disagreement with the SM hypothesis. It is relevant, however, to investigate whether the procedure proposed in this work would be able to reproduce the underlying model. For this kind of closure test, we employ a scheme analogous to the one used to produce fake databases under the SM-only hypothesis in Section 5. 1. This time, however, using the highest-score model from Fig. 8 , we create fake observed data under the "BSM hypothesis". Concretely, through the proto-model parameter space, adding or removing new particles, and randomly changing their cross sections and branching ratios. This is coupled to a combinatorial algorithm, which identifies the set of analyses and signal regions that maximally violates the SM hypothesis (from a total of 86 ATLAS and CMS search results).
Running over the SModelS v1. 2. 4 database, the proto-model builder identified small excesses at the level of ≈ 2σ in ATLAS and CMS stop searches, as well as in multijet + E miss T searches. These may be explained by introducing SUSY-like top and quark partners X 1 t and X q at a mass scale of roughly 1 TeV, together with a neutralino-like lightest particle X 1 Z below about 400 GeV, while staying in agreement with constraints from other searches. The highest-score proto-model has a test statistic of K = 6. 9, while the global p-value for the SM hypothesis is p global ≈ 0. 19. The test statistic is effectively K ≈ ∆χ 2 + 2 log π(M), where ∆χ 2 is a measure of the violation of the SM hypothesis, and π(M) penalizes for new degrees of freedom introduced in the protomodel. Akaike-type information criterion has been used to judge the quality of a proto-model.
It will be interesting and relevant to try other information criteria, such as the Bayesian information criterion, or the deviance information criterion, systematically comparing their performances  [63]. Furthermore, we plan to work towards making the global, combined likelihood differentiable. This will allow for using gradient-based methods, which will be a major performance boost. Note here that pyhf also aims at full differentiability, which is another argument in favor of communicating full statistical models via this scheme. In this respect it will also be interesting to machine learn the SModelS database, e. g. via neural networks or symbolic regression, as these methods automatically come with a gradient. Regarding the statistical interpretation, future work will concern, for instance, a more quantitative assessment of the fitted proto-models, including estimations of posterior distributions for the relevant parameters.
We will try and remedy such artefacts by producing larger maps ourselves, similar to what has been done for gluino-squark production in  [28]. We remind the reader that, when only observed ULs are available, no proper likelihood can be constructed; instead only a step function at the 95% CL limit can be used. CMS-SUS-14-021  [136]  ISR jet + 1-2 soft 19. 7 Table  8. List of CMS Run 1 analyses and their types of results in the SModelS v1. 2. 4 database. The column "comb. " is omitted as no information on signal region correlations is available for any of the analyses.
-1 ] UL obs UL exp EM ATLAS-CONF-2013-007 [104] 2 SS + (b-)jets 20. -2013-23 [118] WH (1 + bb or γγ) 20. 3 ATLAS-SUSY-2014-03 [119] ≥ 2(c-)jets 20. List of ATLAS Run 1 analyses and their types of results in the SModelS v1. 2. 4 database. The column "comb. " is omitted as no information on signal region correlations is available for any of the analyses. ID Short Description L [fb -1 ] UL obs UL exp EM CMS-EXO-12-026 [-SUS-13-015 [122] ≥ 5(b-)jets, top tag 19. 4 CMS-PAS-SUS-13-016 [123] 2 OS + (b-)jets 19. 7 CMS-PAS-SUS-13-018 [124] 1-2 b-jets, M CT 19. 4 CMS-PAS-SUS-13-023 [125] 0 stop 18. 9 CMS-SUS-12-024 [126] 0 + ≥ 3 (b-)jets 19. 4 CMS-SUS-12-028 [127] 0 + (b-)jets, α T 11. -13-019 [134] ≥ 2 jets, M T2 19. 5 CMS-SUS-14-010 [135] b-jets + 4 W s (0-4 ) 19. 5.
Acknowledgements The computational results presented were obtained using the CLIP cluster [65]. We thank  Nishita Desai ,  Sylvain Fichet ,  Jan Heisig ,  Suchita Kulkarni ,  Harrison Prosper  and  Helga Wagner  for discussions related to this work. The work of S. K. was supported in part by the  IN2P3  master project " Théorie -BSMGA ".
Conclusions and Outlook In view of the null results (so far) in the numerous channel-by-channel searches for new particles, it becomes increasingly relevant to change perspective and attempt a more global approach to find out where BSM physics may hide. To this end we presented in this paper a novel statistical learning algorithm, that is capable of identifying potential dispersed signals in the slew of published LHC analyses. The task of the algorithm is to build candidate protomodels from small excesses in the data, while at the same time remaining consistent with all other constraints. At present, this so-called proto-model builder is based on the concept of simplified models, exploiting the SModelS software framework and its large database of simplifiedmodel results from ATLAS and CMS searches for new physics.
7 ATLAS 3 Table 7. 7 CMS 7377 -1 ] UL obs UL exp EM ATLAS-CONF-2013-007 [104] 2 SS + (b-)jets 20. -2013-23 [118] WH (1 + bb or γγ) 20. 3 ATLAS-SUSY-2014-03 [119] ≥ 2(c-)jets 20. List of ATLAS Run 1 analyses and their types of results in the SModelS v1. 2. 4 database. The column "comb. " is omitted as no information on signal region correlations is available for any of the analyses. ID Short Description L [fb -1 ] UL obs UL exp EM CMS-EXO-12-026 [-SUS-13-015 [122] ≥ 5(b-)jets, top tag 19. 4 CMS-PAS-SUS-13-016 [123] 2 OS + (b-)jets 19. 7 CMS-PAS-SUS-13-018 [124] 1-2 b-jets, M CT 19. 4 CMS-PAS-SUS-13-023 [125] 0 stop 18. 9 CMS-SUS-12-024 [126] 0 + ≥ 3 (b-)jets 19. 4 CMS-SUS-12-028 [127] 0 + (b-)jets, α T 11. -13-019 [134] ≥ 2 jets, M T2 19. 5 CMS-SUS-14-010 [135] b-jets + 4 W s (0-4 ) 19. 5.
Furini et al. (2019)  defined the NICP, in which for a given graph G and an interdiction budget k ≥ 1 , the decision maker has to find a subset of at most k vertices to remove from G so that the clique number in the remaining graph is minimized. The authors designed an exact algorithm CLIQUE-INTER , which is able to report optimal solutions for most instances on randomly generated graphs with up to 150 vertices. Optimal solution values are also reported for some larger networks from the SNAP database. 1 Pajouh, Boginski,  and Pasiliao (2014) defined the MVBCP, in which one searches for a subset of vertices of minimum cardinality to be removed from a graph G, so that the maximum (weighted) clique in the remain- ing graph is bounded from above. The authors provided an analytical lower bound for the MVBCP and gave a MIP formulation with an exponential number of constraints, along with the characterization of conditions under which they are facet-defining.