dataesr/scientific-paragraphs-categorization huggingface dataset:
Pipeline validation using designed mock communities The mock communities from BEI are used as standards by many research groups for molecular microbiology and metagenomic sequencing. This type of mock community is a useful tool to compare performance with published studies. Most of the organisms of the BEI mock communities are particularly common bacterial strains that are well represented in the taxonomic databases used in bioinformatics. However, biological tissue samples contain mostly less common bacteria that are poorly represented or absent from the available databases. For this reason, we also tested the pipeline using mock communities composed of random bacteria representative of different phyla of interest (Table  1 ). We purposely included strains of bacteria absent from the RDP database (Geofilum rubicundum, a genus and species of bacteria discovered in 2012  [40] ) and strains of Verrucomicrobia (Prosthecobacter fusiformis), a phylum that cannot be amplified with our primers according to in silico simulations.
The resulting assignments were: 23. 2% Enterobacteriaceae, 25. 7% Bifidobacteriaceae and 49. 0% Burkholderiaceae (Fig 1C ). We then tested a mix of 14 plasmids each of which carried a copy of the 16S gene of different strains (Table  1 ). Except for Prosthecobacter fusiformis, which, as confirmed by in silico analysis, cannot be amplified by the primers used, we identified all strains at the family level, including the recently discovered Geofilum rubicundum (Fig 1D ), and 11 strains at the genus level (Fig  1E ). The Geofilum genus is absent from the RDP database and is considered by the Naive Bayesian classifier as a bacterium from the Alkaliflexus genus, the genus closest to Geofilum  [40]. The assigned proportions of the different bacteria were generally very close to the real copy numbers present in the samples (Fig 1D  and 1E ).
Pipeline versatility: example of primer optimization We took advantage of the versatility of the pipeline to design and incorporate a new set of primers to amplify the Verrucomicrobia phylum in addition to the bacterial taxa recognized by the former set of primers ( 95% of Ribosomal Database Project). With the new set of primers, all 14 strains of the designed mock community, including Prosthecobacter fusiformis, were sequenced and assigned to the correct family and 12 of them to the genus level (S1 Fig).
Some tissues, (e. g. feces, Ileum and adipose tissue), displayed a very high diversity in terms of bacterial content which could not have been assessed correctly with other lower throughput techniques. Bar plot representations of the results (Fig 4D  and 4E ) confirmed the previous observations and showed that the pipeline allowed effective assignment for this type of sample: 99. 3% -100% of assigned reads at the phylum level (Fig 4D ) and 88. 0% -98. 3% at the family level (Fig  4E ). Assignments were more variable at the genus level, between 34. 4% and 88. 8%, probably reflecting a higher number of unknown genera in the reference taxonomy database for those samples (data not shown).
We process this problem using six RT codes participating in this benchmark effort, and compare the results to a reference solution computed with the publicly available dust emission code DustEM. Results. The participating codes implement different heuristics to keep the calculation time at an acceptable level. We study the effects of these mechanisms on the calculated solutions, and report on the level of (dis)agreement between the participating codes. For all but the most extreme input fields, we find agreement within 10% across the important wavelength range 3 µm ≤ λ ≤ 1000 µm. Conclusions. We conclude that the relevant modules in RT codes can and do produce fairly consistent results for the emissivity spectra of SHGs. This work can serve as a reference for implementors of dust RT codes, and it will pave the way for a more extensive benchmark effort focusing on the RT aspects of the various codes.
This is also the computationally most demanding part of the calculation. In the current epoch, performing a full treatment of vibrational quantum modes for the dust grains in each cell is computationally prohibitive. In practice, RT codes use the continuous cooling assumption, which was shown to provide a good approximation in areas relevant for RT through the interstellar medium  (Draine & Li 2001). The reference solutions in this paper were produced with the public version of the DustEM code  (Compiègne et al. 2011). DustEM determines the grain temperature distribution by iteratively solving an integral equation  (Desert et al. 1986 ). We compare the reference solutions with the dust emission spectra calculated by six distinct RT codes. These codes determine the grain temperature distribution by solving a set of linear equations  (Guhathakurta & Draine 1989).
Data files The data files defining the dust properties described above can be downloaded from the web site indicated in footnote 2. They are contained in the DustModel directory, which is organized in the following subdirectories: -GrainInputs: optical and calorimetric properties for each of the three grain types k. Optical properties include the absorption efficiencies Q abs k (a, λ) tabulated on a grid of 1201 wavelengths λ and 28 (for PAH) or 121 (for graphite and silicate) grain sizes a. Calorimetric properties include the specific enthalpy h(T ) and the specific heat capacity c(T ) of the grain material, tabulated on a grid of 1000 temperatures T. The calorimetry data for graphitic grains should be used for PAH molecules as well. -SizeInputs: tabulated grain size distributions Ω k (a) for each grain type k, on a size grid with 24, 62, and 63 samples for the PAH, graphite, and silicate component, respectively.
DustEM The DustEM code is described in  Compiègne et al. (2011). DustEM is a stand-alone code (i. e. it is not a RT code) that calculates the emission and extinction of dust grains given their size distribution and their optical and thermal properties. It determines the grain temperature distribution P(T ) using the formalism of  Desert et al. (1986) , and it then computes the dust SED and associated extinction for given dust types and size distributions. To correctly describe the dust emission at long wavelengths the original algorithm has been adapted to better cover the low temperature region. Using an adaptive temperature grid, DustEM iteratively solves the integral equation Eq. 25 from  Desert et al. (1986)  in the approximation where the grain cooling is fully continuous. The temperature distribution calculation is performed for all grain populations and sizes including those for which the thermal equilibrium approximation would apply.
We produced the reference solutions in this paper with the public version of  DustEM (v3. 8, dated Spring 2010). To this end, we converted the dust properties defined in Sect. 2 and the input radiation fields defined in Sect. 3 into the data format expected by DustEM. We adjusted the values of the physical constants in the DustEM code, raised the maximum number of grain size and temperature bins to accommodate our input data, and fixed a minor issue in the routine that imports the grain size distribution. More importantly, to obtain accurate reference solutions, we substantially increased the number of temperature bins and the number of numerical iterations (see Sect. 6. 2). Other than this, the DustEM code was used without modifications. Our use of DustEM for producing reference solutions should not be understood to imply that it necessarily produces the physically correct results.
The DustEM implementation relies on the continuous cooling assumption just like the RT codes participating in this benchmark. However, since it is not a RT code by itself, DustEM's focus is solely on calculating dust properties and emission, and it has a neutral status in the context of this benchmark. The following sections describe each of the participating codes, with a focus on the specific heuristics employed for calculating the results presented in this paper. Table  3  offers a (very) concise overview of this information.
This computation can be accelerated substantially by precalculating and storing the elements of the matrix A f,i defined in Sect. 4. 3, insofar as they don't depend on the radiation field. The memory requirements scale with BN 2 , just like the computation time. More importantly, pre-calculating these values requires a predefined temperature grid that remains fixed for all emissivity calculations. However, performing all calculations on the finest grid would be very inefficient. The SKIRT implementation handles these conflicting requirements as follows. We predefine three separate temperature grids (A, B, and C) that can be used for any of the emissivity calculations, and we pre-calculate and store all radiation-fieldindependent values on each of these three grids for each size bin. The temperature range is the same for all grids; it is usually set to [2 K, 3000 K] but if needed the upper limit is decreased to the largest temperature for which enthalpy data is available in the dust properties.
DIRTY DIRTY  (Gordon et al. 2001; Misselt et al. 2001 ) is a Monte Carlo RT code designed to study dust and its effect on radiation in arbitrary astrophysical systems. DIRTY is a fully 3D code allowing for the specification of arbitrary density distributions of both dust and radiation sources. It implements an adaptive mesh allowing for the efficient allocation of computing resources amongst regions in the model space depending on the physical characteristics of the system. Dust absorption, temperature distribution, and emission are handled self-consistently and multiple, anisotropic scattering is implemented. The dust heating implementation supports both equilibrium and stochastic processes based on the local radiation field and dust properties at each grid in the model space. Like other codes presented here, DIRTY follows the approach presented in Sect.
TRADING TRADING  (Bianchi 2008 ) is a 3D Monte Carlo dust continuum RT code with characteristics similar to those of SKIRT and DIRTY. Originally designed to study the effects of clumping in the disks of spiral galaxies, it uses an binary-tree adaptive grid (octree) for the dust distribution. TRADING computes stochastic heating following the method described in Sect. 4. A single temperature grid is used to precompute the field-independent terms of the matrix elements (Eq. 25 and most factors in Eq. 23). Since the RT models of clumpy galactic disks in  Bianchi (2008)  need a few million dust cells, and each cell requires the calculation of dust heating for the full grain size distribution,  Bianchi (2008)  uses a limited temperature grid of 80 logarithmically spaced bins between 2. 7 and 2000 K.
CRT CRT  (Juvela & Padoan 2003; Juvela 2005; Lunttila & Juvela 2012 ) is a Monte Carlo dust continuum RT program. It solves the RT equation self-consistently with a full treatment of scattering, absorption and emission of radiation in 1D, 2D, and 3D geometries. The program allows using an external component, for instance DustEM, for the calculation of the dust emission spectrum for a given input radiation field. This feature has been used, e. g. in  Ysard et al. (2011) , to study the microwave emission from spinning dust grains. However, CRT itself also has optimized routines for fast dust emission calculations, including the treatment of stochastically heated grains. Although CRT can calculate dust emission with fully discrete cooling, the results presented in this paper use the continuous cooling approximation, and the following discussion is focused on the algorithms for continuous cooling computations.
Acknowledgements. We thank the authors of DustEM (Compiègne et al. 2011) for making their code publicly available. This work fits in the CHARM framework (Contemporary physical challenges in Heliospheric and AstRophysical Models), a phase  VII Interuniversity Attraction Pole (IAP)  programme organised by  BELSPO , the  BELgian federal Science Policy Office. TL acknowledges the support from the  Swedish National Space Board (SNSB). GN is supported by the  Leverhulme Trust  research project grant  RPG-2013-418. MJ acknowledges the support of the  Academy of Finland  Grant No. 250741. JS acknowledges support from the  ANR  ( SEED ANR-11-CHEX-0007-01 ).
ECHOSim is the end-to-end time-domain simulator of the Exoplanet Characterisation Observatory (EChO) space mission. ECHOSim has been developed to assess the capability EChO has to detect and characterize the atmospheres of transiting exoplanets, and through this revolutionize the knowledge we have of the Milky Way and of our place in the Galaxy. Here we discuss the details of the ECHOSim implementation and describe the models used to represent the instrument and to simulate the detection. Software simulators have assumed a central role in the design of new instrumentation and in assessing the level of systematics affecting the measurements of existing experiments. Thanks to its high modularity, ECHOSim can simulate basic aspects of several existing and proposed spectrometers for exoplanet transits, including instruments on the Hubble Space Telescope and Spitzer, or groundbased and balloon borne experiments.
Signals are small, at the level of 1 × 10 -4 of the star, and detections therefore require an exquisite control of observational and instrumental systematics both at instrument level and during data analysis. The risk is in coupling the star signal into the signal of the exoplanet atmosphere. When observing from the ground, the Earth's atmospheric emission can also couple to the much smaller planet's signal if systematics are not sufficiently under control. This is not the first time we face such a problem, which can be effectively addressed only through dedicated instrumentation. The Exoplanet Characterisation Observatory  [21] , EChO, is a proposed space mission designed for photometric stability over a spectral band spanning from the visible to the mid-IR part of the electromagnetic spectrum and ECHOSim is the end-to-end software simulator developed to aid the instrument definition and to validate the mission concept.
Because of the high modularity, and execution efficiency, ECHOSim can also be used for a number of different applications such as the design of novel ground-based and balloon-borne exoplanet spectroscopic experiments, or to study the reliability of an existing detection by specifying a suitable instrument model in a parametric form. In this work, we review the algorithms implemented in ECHOSim and the approximations made to make the execution of these simulations efficient so that run times of the order of a few tens of seconds are achieved on normal laptop-size computers. The code is fully implemented in Python, with standard numerical libraries for portability, and simulations can be run on Windows, Linux and MacOS based machines. Each simulation begins with the generation of the frequency and time-dependent astronomical signal expected from the extrasolar system, which is then propagated to a cascade of processes that are involved in the instrument detection.
The ECHOSim simulator The purpose of the ECHOSim simulator is to provide a software tool to assess all aspects of the EChO mission concept baseline, and alternative solutions, by using realistic, time-domain simulations of the astronomical scene and instrument. The instrument definition within the simulator is highly configurable, and it is possible to implement instruments other than EChO, including existing ground-based and space instruments. The simulator design is optimized for computational efficiency without compromising the fidelity of the simulated detection. This allows each simulation to be run in seconds, opening up the possibility of Monte Carlo analyses of instrumental and astrophysical effects. For this reason, the simulator implementation is highly modular (see Figure  2 ). A parametric description of the instrument and of the astronomical scene constitute the ECHOSim inputs.
Exoplanets on a line of sight with higher Zodiacal column density are not considered, as extinction (in the visible) or foreground emission (in the IR) becomes prohibitively high for the detection. The multiplicative factor of the Zodiacal model can also be user-defined to represent Zodiacal emission towards a particular target's line of sight. This is estimated from the best fit of equation 4 to a realistic Zodiacal model  [25] , obtained at the ecliptic latitude and longitude of the target, and at the expected time of observation. Because ECHOSim can simulate instruments other than EChO, Earth atmospheric emission and transmission models can be used for simulations of ground-based and balloon-borne experiments. Atmospheric spectra can be pre-computed by the user with Modtran 4 or similar software and used in ECHOSim simulations.
Noise module The Noise module simulates the main sources of instrumental and astrophysical noise. In a real instrument, noise sources act at every stage of the detection, but ECHOSim implements addition at the very end of each simulation. This is required for computational efficiency as the random processes of noise require a larger (temporal) bandwidth, which is very different from the bandwidth of the astronomical signal. The outputs from the instrument module simulations are signal-only timelines which are sampled with a cadence chosen to be the minimum required to Nyquistsample the time-varying astronomical signal, i. e. the modulation in the light curve. The first task of the noise module is to re-sample the timelines to the user-defined detector sampling rate. The following noise components are then added to the signal-only timelines: -Photon noise; -Detector dark current and dark current noise; -Detector readout noise; -Detector inter-pixel gain variation; -Telescope Pointing jitter.
Detector gains are expected to be measured both before launch and during operations, and are used by the data reduction pipeline to flat-field the focal plane array when reconstructing the exoplanet's spectrum. ECHOSim implements pixel-to-pixel detector gain variation (or inter-pixel response) to simulate uncertainties in the flat-field operation. A detector-specific gain, G ij , is randomly assigned to each pixel with a user-defined RMS indicating the level of precision obtained in the calibration process. The stability of the instrument attitude and orbital control system (AOCS) is quantified in terms of mean performance error (MPE), performance reproducibility error (PRE) and relative performance error (RPE). These are ESA defined pointing error terms 5  affecting the measured timelines via mainly two mechanisms: 1) the drifting of the spectrum along the spectral axis of the detector array, from here on referred to as spectral jitter; 2) the drift of the spectrum along the spatial direction (or spatial jitter).
The effect of jitter on the observed timelines is the introduction of correlated noise, characterized by the power-spectrum of the telescope pointing. The amplitude of the resultant photometric scatter depends on the amount of spectral/spatial displacement of the spectrum, the PSF of the instruments, the detector intra-pixel response and the amplitude of the inter-pixel variations. The effects of spectral jitter are not simulated by ECHOSim. This is motivated by the reasoning that drifts in the spectrum along the spectral axis of the array can be effectively removed during data reduction using the several stellar emission and absorption lines detected with high significance (see e. g. [27][28] ). The spatial component of the jitter affects the point source signal only and is simulated through a second-order variation between the position of the sampled PSF in the spatial direction in Equation 15 and the location of the detector pixel: Q J (i, j, t) = Q P (i, j, t) 1 p s f eff ∂p s ∂y j < δθ > ∆T + + f 2 eff 2 ∂ 2 p s ∂y 2 j < δθ 2 > ∆T , where < δθ > ∆T is the time-averaged angular displacement of the telescope's line-of-sight from the target, and ∆T is the detector sampling interval.
The jitter δθ is simulated as a Gaussian random process with an user-defined power spectrum (and bandwidth) which defines the electromechanical response of the AOCS. When the AOCS loop is closed on the information provided by a star sensor (in the case of EChO this is the FGS in Figure  1 ), pointing information is also available. ECHOSim provides this housekeeping information with a user-defined accuracy on the absolute pointing knowledge and update rate. This information can be used by a data reduction pipeline to de-correlate the effects of pointing jitter on the reconstructed exoplanet spectrum. The reason to model the pointing jitter as a first and second order effect on the detected timelines is for computational efficiency. It would be more computationally onerous to simulate the jitter early on in the process because of the different bandwidths characterizing the astronomical signal, the detector sampling rates and the AOCS.
Use of ECHOSim ECHOSim simulates all aspects of the spectroscopic detection of the exoplanet atmosphere. The simulation is done in the time-domain, and includes all major systematic aspects known to challenge the confidence with which a detection is obtained. This is an invaluable tool in a wide range of experimental activities and scientific analyses. A data reduction pipeline is the first tool required in order to reconstruct the exoplanet's atmospheric spectrum from the raw simulated data. ECHOSim allows the development of novel data reduction pipelines which can be tested against the simulation's inputs. This allows the validation of the reduction pipeline in reconstructing the signal in the presence of random correlated and uncorrelated noise sources and systematics. A data reduction pipeline has been developed to analyse ECHOSim simulations in the context of the EChO mission study.
The pipeline is distributed along with ECHOSim and it is fully discussed in  [29]. Case studies discussed in the next section have been analysed using this reduction software. ECHOSim is used to validate the ability of a proposed instrument concept and mission scenario to deliver its scientific goals. Similarly, the proposed scientific goals and mission scenario can be used by ECHOSim to obtain technical instrument requirements such as pointing stability, detector noise and stability, temperature stability, etc. , required for a successful detection. Therefore this simulator plays a central role when designing an instrument or when formulating a mission concept as, for instance, EChO. ECHOSim simulations of exoplanet primary and secondary eclipses are used by  [30]  to demonstrate how EChO would be effective in constraining atmospheric models using the methodology discussed in  [31]  and  [32].
This is particularly severe because of the interplay between intra-pixel and interpixel response and pointing jitter. In this paper, we have presented a detailed description of the algorithms implemented by ECHOSim to deliver realistic simulations. The software is very efficient, with full simulations run in seconds on modern laptop computers, hence enabling the possibility of Monte Carlo analyses. ECHOSim has been used to assess the EChO space mission, and can be adopted as a tool to develop novel mission concepts, or to investigate detection confidence for existing suborbital and space instrumentation.
The Linked Open Data (LOD) cloud is a massive source of data in different domains. However, these data might be incomplete or outdated. Furthermore, there are still a lot of data that are not published as static linked data such as sensors data, on-demand data, and data with limited access patterns, that are in general available through web services. In order to use web services as complementary sources of data, we introduce LIDSEARCH (Linked Data and Services Search), a SPARQL-driven framework for searching linked data and relevant semantic web services with a single user query.