dataesr/scientific-paragraphs-categorization huggingface dataset:
Event loop Internally, Rivet is primarily a framework for executing analysis routines on the incoming stream of events. The top-level structure of this framework in terms of code objects, userfacing scripts, and data flows is illustrated in Figure  1. As with most such frameworks, this is broken into three phases: initialisation, execution, and finalisation. Rivet analysis objects, which inherit from the Rivet::Analysis base class, have three methods (functions), which are called at each stage: init() during initialisation, analyze(const Rivet::Event&) for each event during the execution loop, and an optional finalize() called once at the end of the run. Initialisation and finalisation are used for the set-up and pull-down phases of the analysis, most notably for creating histogram objects with appropriate types and binnings during initialisation using the Analysis::book() methods, and scaling or normalising them (or performing arbitrarily more complicated post-processing manipulations and combinations) in the finalize() step.
Analysis-routine compilation & loading Analyses are compiled separately from the core Rivet library, as "plugin" shared libraries which are loaded explicitly by searching the internal analysis path (set either programmatically or via the RIVET_ANALYSIS_PATH environment variable) for libraries matching the pattern Rivet*. so. A rivet-build script is supplied as a plugin-building frontend for the C++ compiler, both for simplicity and to ensure exact compatibility of compiler options with the core library. This script can build multiple analyses into a single plugin, and as well as for user convenience is used to build the more than 700 included analysis codes into plugins grouped by experiment or collider. 3 New features.
Here we summarise first the Cuts system, and then its extension to filtering functors. In the original API, many functions used lists of several, perhaps optional, floating-point valued arguments, e. g. FinalState(double etamin, double etamax, double ptmin) or fs. particles(double ptmin). Not only were these inflexible, not allowing "inline" cuts other than the hard-coded ones, and sometimes annoyingly verbose -as with nearly always symmetric rapidity cuts, or often having to supply DBL_MAX values to apply a p T cut onlybut at the compiled code level they were ambiguous. It was easy to forget if the η cuts or the p T cut came first, and accidentally require something like η > 10, with clear consequences: the compiler has no way of telling one double from another and warning the analysis author of their error.
Re-entrant finalize and run merging It is quite common to divide up Rivet analyses of very large event samples in smaller runs, and merge the produced Yoda files. Previously this has been done with the yodamerge Python script distributed with Yoda. However, this script was completely ignorant of the way the different analysis objects were produced. In order to better handle this Rivet has since version 2. 7 introduced the Python script rivet-merge which does the same thing but using the complete knowledge of how the analysis objects were filled in Analysis::analyze (. ) and manipulated in Analysis::finalize(). The way this is implemented means that all analysis objects come in two different instances. One is called raw and is used only for filling in the Analysis::analyze(. ) functions. Before a call to Analysis::finalize() the raw analysis objects are copied to the final instances which are then used for manipulating the objects into their final form to be plotted.
In this way the Analysis::finalize() function can be run several times and is then called re-entrant. The user will notice that the output Yoda file contains both sets of analysis objects, one with the standard naming, and one with the same name prefixed by /RAW/. 2  In this way the rivet-merge script can read in Yoda files, create and run the Analysis ::init() for the corresponding analyses, merge all raw analysis objects together and run Analysis::finalize(). When doing this it is important to note that there are two different kinds of merging possible. In one situation we have run Rivet on several completely equivalent event samples, and in the other the runs have been on different kinds of event samples which should be combined in another way. An example of the latter is when the same analysis is run on event samples with different √ s, and in the end will produce e.
Now if the runs to be merged have exactly the same process, the weights in the combined histogram will simply be the added weights, S w = i S w,i , S [b] w = i S [b] w,i , and S [b] w 2 = i S [b] w 2 ,i (1) and the cross-section for the combined files will be a weighted average σ = 1 S w i S w,i σ i. Alternatively, to be on the safe side, if the files have identical processes, but different weight variations, we might want to use the effective number of entries, N = S 2 w /S w 2 , as weights, σ = 1 N i N i σ i and δσ 2 = 1 N 2 i N 2 i δσ 2 i. ( 2 ) 2 Handlers are provided in the Python interface to identify these histograms when reading Yoda files for post-processing. For each bin we will e. g. have the plot value σ [b] = σS [b] w /S w with an estimated error of δσ [b] = σ S [b] w 2 /S w as for the individual histograms.
Deep-inelastic ep scattering and photoproduction Although Rivet traces its conceptual origins directly back to HZTool  [72, 73] , a Fortran package developed by the H1 and ZEUS collaborations at the HERA ep collider to facilitate the comparison of their measurements to MC predictions and each other, rather few deep inelastic scattering (DIS) or photoproduction measurements have been implemented in Rivet to date. This is partly because of the existing and extensive library of such analyses in HZTool. Such measurements contain important and unique information, principally on high-energy QCD dynamics and hadronic structure, which remains relevant to current and future analyses. The need to preserve access for use with modern event generators, and to exploit the ongoing benefits of new development in Rivet-several of which are informed by lessons learned from HZTool-has grown over the years, and has been further stimulated by the active community work toward a future electron-ion collider.
Detector emulation Rivet was initially developed to encode unfolded analyses, i. e. those for which the biases and inefficiencies introduced by detector material interactions and imperfect reconstruction algorithms have been corrected, making the experiment's published observables the best possible estimate of what happened at the fundamental interaction point, independent of any particular detector. It remains our firm belief that unfolded measurements -while requiring a significant investment of time and effort to understand and invert the detector biases, and to minimise model-dependence in the derivation of such corrections -are the gold standard form in which to publish collider physics measurements. This is particularly the case when the fiducial analysis phase-space (i. e. the allowed kinematic configurations at truth-particle level) has been carefully designed to minimise extrapolation beyond what the detector could actually (if imperfectly) observe.
The problem becomes particularly acute when, as is often the case, the analysis contains many parallel search regions, all of which have their own cut-flow. On top of all this, one needs to then code a print-out of the cut-flow stages, including raw event counts -possibly normalised to cross-section and luminosity, or to a fixed reference -as well as step-wise and cumulative efficiency fractions. Rivet's Cutflow object exists to solve these problems. It acts like a fillable histogram, augmented with extra methods to attach human-readable labels to each cut stage, to track the current cut stage rather than make the user do so manually via a histogram's fill variable, to be able to fill multiple cut stages at once, and to return the result of a fill such that it can simultaneously update the cutflow and a signal-region counter, e.
For this reason Rivet now includes an option machinery. For any analysis name it is possible to specify one or more options that can take on different values. This is encoded by supplying suffixes to the analysis names on the form rivet -a AnalysisName:Opt1=val1:Opt2=val2 In the analysis class it is then possible to retrieve the specified value of a particular option with the function string Analysis::getOption(string). It is possible to specify the options and values that are allowed for a given analysis in the. info file, but it is also possible communicate other options to an analysis. Note that it is possible to use several versions of the same analysis in the same Rivet run as long as the versions have different options. Handlers are provided in the Python interface to extract or strip the options from histogram paths.
Python 3 While the Rivet core library and the analysis implementations are written in C++, there is a full featured Python wrapper around them built using the Cython system. In fact, the rivet executable is written in Python and uses this interface. Cython is not a requirement to install the Rivet Python modules if building from a release tarball, but is necessary when building from the git repository directly. Recent Rivet versions are compatible both with Python 2. 7 and Python 3. If multiple Python versions are available in a system, you can choose which one to use for the Rivet installation by prefixing the configure command with e. g. PYTHON=/usr/bin/python2. /configure. It is also possible to install modules for both versions in the same installation location by running the appropriate PYTHON=<python>.
First Rivet runs Now to run analyses on simulated events, using pre-written analyses. Let's say we want to analyse some top-quark pair (t t) events at √ s = 13 TeV, using routines that both compare to real data, and generically characterise the jet and partonic top distributions in the events. Using the metadata search system on the command-line we find the CMS_2018_I1662081 data analysis, and also the MC_JETS and MC_TTBAR routines which have no experimental analysis counterpart. These analyses are located by the Rivet analysis loader system, which by default looks in the Rivet library install path under $prefix/lib/Rivet/. If the $RIVET_ANALYSIS_PATH environment variable is set, or search paths are specified via the Rivet library API, these are used by preference with fallback to the default unless the path variable ends with a double-colon, ::.
Plotting and manipulating results The usual next step is to plot the results. The final. yoda file written by the rivet run (named Rivet. yoda by default) is the principle input to this plotting, optionally along with equivalent files from other MC runs. If multiple MC runs -either for separate MC processes or to split a large single homogeneousprocess run into smaller chunks -need to be combined into a single. yoda file for this purpose, the rivet-merge script can be used to read in these independent contributions and re-run the analyses' finalize() methods to give a final, statistically exact combined. yoda file, as described in Section 3. 2. 4. Cross-section and number-of-events scalings will be automatically calculated from information stored in the input files. Should any manual scaling be needed in addition, the yodascale script or a custom manipulation using the YODA Python API are also possible.
Raw materials Our first step is to generate the file templates into which we will insert analysis data and logic. The starting point should always be the rivet-mkanalysis script, run like rivetmkanalysis EXPT_2019_I123456 where the analysis name follows the three-part structure described earlier. Particularly critical is the third part, encoding the Inspire database key with which the script can automatically extract reference data and publication metadata. Running the script in this way generally results in four template files: a. cc file containing a generic template for your analysis code, with "boilerplate" features pre-filled; a. info metadata file in YAML format, used to generate documentation and constrain applicability; a. plot file used to apply plot styling directives to sets of output histograms; and a.
yoda reference data file in the YODA format, downloaded if possible from the HepData database. The only essential file for private running is (of course) the. cc in which the analysis logic will be written, but any analysis submitted to Rivet for official inclusion in the repository must also complete the other files. In particular it is critically important that the. yoda file be compatible with HepData's YODA output, so updates can be synchronised with subsequent Rivet releases.
This latter form also has an integer-triplet shorthand, expanding to the daa-xbb-ycc dataset/axes format output by HepData. Counters and Scatter*D objects, which do not have a variable binning, have simpler book() method overloads. Note that analyses which run in several modes, e. g. making the same kinds of observable histograms for event runs at two different √ s energies, so not need different histogram pointer variables for each mode -simply pass different additional arguments to the book() methods depending on the context of, for example, a call to Analysis::sqrtS()/GeV. Within the analysis, the Ptr objects are used in a straightforward fashion, most notably calls like e. g. _h_myhist->fill(x). Users of Rivet v2 will note that the event weight is no longer part of this function signature.
Contributing analysis routines We encourage all users to contribute validated analyses to the official Rivet collection: this helps the whole particle physics community, and your efforts will be accredited through public analysis authorship declarations. After submission, the Rivet core authors will be responsible for maintaining the analysis code compatibility with any future changes in the core API. In addition to the. cc code file, the metadata. info file, plot styling. plot file, and if appropriate. yoda reference data files must be provided, along with information (ideally including plots from rivet-mkhtml) illustrating the pre-submission validation procedure performed by the analysis author. The info file must include a ReleaseTests entry indicating how a short analysis-validation behavioural regression run should be performed, using the example 1000-event HepMC event files located at http://rivetval.
web. cern. ch/rivetval/. If no existing event file is suitable for the analysis, a new. hepmc. gz analysis file should be supplied along with the analysis code and metadata upload. In the past, contribution of analyses and this supporting validation information has been done through a mix of email submissions to the Rivet developer mailing list and (for official experiment representatives) upload to a special "contrib" area on the Rivet website. Since version 3. 0. 2, a more formal procedure is in operation, whereby new analyses are to be contributed via merge requests on the https://www. gitlab. com/hepcedar/rivet code repository. Validation plots and similar information, and new HepMC event samples if appropriate, should be contributed to the Rivet core team separately from the repository merge request.
Funding information This work was supported in part by the  European Union  as part of the  Marie Sklodowska-Curie Innovative Training Network MCnetITN3  (grant agreement no. 722104 ). AB thanks The  Royal Society for University Research Fellowship  grant  UF160548. AB and CP thank the  University of Glasgow  for postdoctoral funding through the  Leadership Fellow scheme. LL was supported in part by  Swedish Research Council , contract number  2016-03291. FS was supported by the  German Research Foundation  under grant No. SI 2009/1-1. Many thanks to  Xavier Janssen ,  Markus Seidel ,  Alex Grecu , and  Antonin Maire  for (in addition to members of the author list) acting as LHC experiment contacts.
The stochastic paradigm, by contrast, considers biological communities as products of random processes, such as dispersal  (Hubbell 2001). However, it is likely that niche and dispersal processes are not mutually exclusive but constitute the end points of a continuum, similar to the continuum of niche and neutral models proposed by  Gravel et al. (2006). Within the dispersal-niche continuum, compositional variation of biological communities can be driven to a greater or lesser extent by each process  (Holyoak et al. 2005 , Mouillot 2007 , Vellend 2010 , Gibert and Escarguel 2019). As patterns in community composition are scale-dependent  (Chase et al. 2018 ) and underlying drivers are generally difficult to distinguish  (Gotelli and Colwell 2001) , quantifying the main assembly processes in various systems and across different datasets has remained challenging  (Viana and Chase 2019).
e. E-metric distribution from the 'dispersal' model). Thus, DNCI offers a way to both quantify and compare the strengths of the main assembly processes across datasets (Fig. 1 ). Positive or negative DNCI values indicate that niche or dispersal assembly is the main process structuring the studied site groups consisting of biological communities, respectively. Higher absolute values of the index represent greater potential strength of the dominant assembly process. To test the robustness of the DNCI, we performed robustness tests with simulations. We then calculated the DNCI for two example datasets to test the applicability of the DNCI with empirical data. Using stream monitoring data from the USA, we compared whether the DNCI values differ among metacommunities of organisms varying in trophic level and dispersal capacity, including diatoms, macroinvertebrates and fish.
Using a field study dataset from China, we addressed whether the DNCI values of stream bacteria and macroinvertebrates differ from each other and whether they vary with Negative DNCI values indicate that dispersal assembly is the dominating process, while positive DNCI values indicate that niche assembly is the dominating process behind community dissimilarities among site groups. With increasing absolute values, the strength of the dominating process increases. DNCI values close to zero indicate that both niche and dispersal assembly processes affect distributions. In dataset A, the DNCI does not significantly differ from zero, indicating that both dispersal and niche processes control the empirical species distributions. In dataset B, DNCI is significantly greater than zero, indicating that niche assembly is the main driver of the empirical species distribution.
In datasets C and D and the illustrated dataset, DNCI is significantly less than zero, indicating that dispersal assembly is the main driver of the empirical species distributions. Dataset D and the illustrated dataset do not have significantly different DNCI values, i. e. the control of the empirical species distributions by dispersal assembly is similar. Dataset C has a DNCI significantly higher than dataset D and the illustrated dataset, i. e. dispersal assembly control in dataset C is significantly weaker than in dataset D and the illustrated dataset. SES d = standard effect size of E d. SES n = standard effect size of E n. Modified from Gibert and Escarguel's (2019) Supporting information. spatial distance. Finally, we discussed the strengths and weaknesses of the DNCI in inferring community assembly processes and suggested potential research avenues.
DNCI robustness test with simulated communities PER-SIMPER's original qualitative approach has shown accurate identification of the main assembly process underlying the composition of simulated communities, even when the analysed groups have different number of taxa or result from various sampling intensities. Furthermore, PER-SIMPER's qualitative identifications are particularly robust when one assembly process plays a role significantly larger than the other one. Conversely, when both processes are simulated, PER-SIMPER is more sensitive to sampling biases. In particular, the number of sampled sites (less than five sites in a site group) within the analysed dataset may bias the PER-SIMPER inference toward the niche-based expectation by underestimating the effect of dispersal processes  (Gibert and Escarguel 2019).
Here, the effects of number of taxa and number of sites are tested and measured on simulated datasets prior to using the new DNCI on empirical datasets. To ensure consistency between the quantitative and qualitative robustness tests, the simulated communities are obtained from cellular automata detailed by  Gibert and Escarguel (2019). The automata are designed for simulating the different displacement potential of various taxa (dispersal) as well as the influence of local environmental conditions on taxonomic richness (niche). The dispersal potential of taxa and the taxonomic richness limit inside cells can be tuned by, respectively, changing the number of individuals moving along the automaton grid (i. e. the more individuals, the more dispersal) and by changing the pattern limiting maximum richness within the cells.
Observational field survey data on diatoms, macroinvertebrates and fish from the United States Datasets including stream diatoms, macroinvertebrates and fish were obtained from the National Water-Quality Assessment (NAWQA) Program of the U. S. Geological Survey. We selected four subregions of the Mid-Atlantic Region (hydrologic unit HUC02), situated in the northeastern part of the United States, including the Upper Hudson (HUC0202), Delaware (HUC0204), Susquehanna (HUC0205) and the Potomac (HUC0207) (map in Supporting information). In total, there were 131 sites sampled for diatoms, 128 sites sampled for macroinvertebrates and 43 sites sampled for fish across the four subregions (i. e. four site groups). Each of the subregions comprised a sufficient number of sites for the analyses (a minimum of eight sites in Delaware for fish and a maximum of 66 sites in the Potomac for macroinvertebrates), given that at least five sites per group is recommended by  Gibert and Escarguel (2019).
Observational field survey data on bacteria and macroinvertebrates from China In October 2014, 89 individual streams were sampled for bacteria and macroinvertebrates in the southeastern part of the Tibetan Plateau, Yunnan province, China. The sampled streams include 52 tributaries of the Salween River and 37 tributaries of the Mekong River (see map in Supporting information). The elevational ranges were pronounced, spanning from 570 to 1592 m a. s. l. in the Salween catchment and from 1270 to 2100 m a. s. l. in the Mekong catchment. Macroinvertebrate samples were identified to genus level when possible. Bacterial OTUs (i. e. operational taxonomic units) were determined based on the 16S rRNA genes using bacterial universal primers 515F and 806R that target the V4 region. The bacterial dataset was rarefied to 10 000 sequences.
Statistical methods We developed an R function ('DNCI_multigroup') for the calculation of DNCI values. The function first performs the PER-SIMPER procedure  (Gibert and Escarguel 2019)  and, then, as described earlier, calculates pairwise DNCI values among site groups based on the E-values from the PER-SIMPER analysis. For the US data, we used the four subregions as site groups for all three presence/absence community matrices (i. e. diatoms, macroinvertebrates and fish). Due to differing numbers of sites per group, we selected the size of the smallest group (Delaware with eight sites in the fish dataset) and randomly resampled sites from all other groups to eliminate a potential group size bias. We performed the random resampling of eight sites from each of the four groups 200 times. For the Chinese data, we used elevational bins, each containing 12 or 13 sites, as site groups in our analyses (Supporting information).