dataesr/scientific-paragraphs-categorization huggingface dataset:
For SGA, the links file was created using the sga-bam2de command with parameters '-n 5 -m 500 --mina 31 -k 31'. The astat file was generated setting '-m 500'. The scaf file and the corresponding FASTA file were both created with parameters '-m 500'. For BESST, we chose the optimal k-mer size used for the contig assembly as '-K 91' and ran the program with parameter '-e 5'. For each program, we selected the scaffolds that were over 2 kb in length. For OPERA-LG, we set the k-mer size for scaffolding with option 'kmer=91'. The minimum contig size required for the scaffolding step was fixed as 500 bp with the parameter 'contig_size_threshold=500'. Finally, the number of links to validate a connection between two contigs was assigned with the parameter 'cluster_threshold=5'. The source code of QUAST was modified to avoid, as much as possible, the detection of misassemblies (relocation, translocation, and inversion) that correspond to false positives.
More { 𝑔𝑎𝑝 ++ = 𝜇 -𝑝𝑜𝑠 1 -𝑝𝑜𝑠 2 -2𝑅 𝑔𝑎𝑝 +-= 𝜇 -(𝑝𝑜𝑠 1 + 𝑅) -(𝑙𝑒𝑛 2 -𝑝𝑜𝑠 2 ) 𝑔𝑎𝑝 --= 𝜇 -(𝑙𝑒𝑛 1 -𝑝𝑜𝑠 1 ) -(𝑙𝑒𝑛 2 -𝑝𝑜𝑠 2 ) 𝑔𝑎𝑝 -+ = 𝜇 -(𝑙𝑒𝑛 1 -𝑝𝑜𝑠 1 ) -(𝑝𝑜𝑠 2 + 𝑅) We validate the link if 0 ≤ 1 𝑛 ∑ 𝑔𝑎𝑝 (𝑜𝑟𝑖𝑒𝑛𝑡 1 ,𝑜𝑟𝑖𝑒𝑛𝑡 2 ) ≤ 𝜇 + 3. sd, where 𝜇 and 𝑠𝑑 are the mean size and the standard deviation of the MP library fragment size respectively, and 𝑛 is the number of supporting pairs for the scaffolds link with the following orientation (𝑜𝑟𝑖𝑒𝑛𝑡 1 , 𝑜𝑟𝑖𝑒𝑛𝑡 2 ). Finally, all validated links were formatted for the SGA program to perform the final scaffolding.
Analysis of A. thaliana WGP data We used the WGP data produced from the A. thaliana col-0 BAC library by Keygene (Wageningen, The Netherlands)  [9]. WGP tags were ordered by an automated procedure that performed the following steps. First, fingerprinted contig data were read with contig and position information per BAC. Then, BACs were sorted on their left and right positions in the contig and assigned a rank number (identical left and right positions lead to identical ranks). Next, tag information from the WGP tag file was read and occurrences of tags per BAC were listed. For a given contig, a tag position was calculated as the mean value of BAC rank numbers on which the tag occurred. If BAC ranks were too far apart, the tag was identified as an outlier and put aside. The remaining tags were ranked according to their mean BAC rank value, possibly with equal rank scores for equal average BAC rank values.
Based on the QUAST NA50 and NA75 metrics, we ranked the assemblies from the highest to lowest quality as BESST, OPERA-LG, SSPACE, SOAPdenovo2, and SGA. We used the WGP map to provide a reference-free approach that evaluates the quality of the five assemblies. We applied the wgp2map and map2qc modules of MaGuS to calculate the length of all collinear segments (Figure1b) and generated Anx values (Table  1 , Figure  2a ). Considering the MaGuS An50 and the An75 metrics, the ranking of the assemblies was the same as the ranking using the QUAST NA50 and NA75 metrics. The NAx and Anx values were strongly correlated (R 2 >0. 96) for the five assemblies (Figure  2c ), which allowed us to consider using the MaGuS Anx metrics to compare assembly quality. Selecting the appropriate bioinformatics tools to perform genome de novo assembly is difficult and often depends on the genome complexity and on the sequencing technology used.
Conclusions The method presented here and implemented in MaGuS enabled the evaluation of the quality and the scaffolding of a draft genome assembly using a physical map and HTS data. Its application to Arabidopsis with a WGP map provides a first example of its efficiency in reconstructing a eukaryotic genome. Evaluating the quality of a genome assembly is necessary in order to increase the accuracy of downstream analyses, such as genome annotation or comparative genomic analyses. De novo assembly projects often lack a genome reference and different ways to assess the assembly quality have been investigated  [2, 33]  using either the HTS data used for the assembly or a genome map. The latter remains a very good independent source of information for this task. From this perspective, we developed the map2qc module of MaGuS to provide assembly quality metrics.
Acknowledgments We thank the two anonymous reviewers which comments improved the initial version of this manuscript. This study has been primarily supported by the  French National Research Agency  ( ANR-VUL 005 :  IPER-RETRO  project) with additional funding from the  "Association des Pays de Savoie " to JJ, the  INEE (CNRS France) , who supported the PEPS program ADNmessager to ID and OS, the "innovative projects" and "invited professors" fundings from  INRA  to BA, OS, and  ID , and  Université de Savoie  to VB.
III. EXPERIMENT RESULTS In this section, we show the evaluation of our proposed detection model, which is a combination between spatialbased and temporal-based detections, using Matlab tool. We compare its performances with a temporal-based detection, a spatial-based detection, and one of the most popular learning algorithms SVM (Support Vector Machines). We check the robustness of each algorithm in term of Detection Rate (DR) and False Positive Rate (FPR) defined as follows:  Detection Rate (DR):the ratio of correctly identified malicious nodes over total number of attackers,  False Positive Rate (FPR):the ratio of normal nodes that are classified as malicious over total number of normal nodes. Table  I  summarizes the main simulation parameters. As illustrated in Fig. 3 , in term of detection rate, the learning algorithm is the best since it relies on a sophisticated technique that results in more time and computing requirements  [6].
This work led to further refinements of the catalogue creation algorithm for the analysis of the second public BOSS data release DR10  (Ahn et al. 2014) , which coincided with an internal release (called DR11). In particular, the code was rewritten into a modular version, called MKSAMPLE, new weights were used to correct for fluctuations in the expected target density, and new masks were used for "bad" areas. These refinements were presented alongside the BAO results for the CMASS sample in  Anderson et al. (2014b)  and the LOWZ sample in  Tojeiro et al. (2014) , and were confirmed to be robust to colour  (Ross et al. 2014 ) and against possible systematics in the fit  (Vargas-Magana et al. 2014). As for DR9, the results were extensively used, further testing the catalogues: RSD measurements have been made in a number of different ways  (Beutler et al.
These effects are often corrected by applying a weight to the galaxies (e. g. Ross et al. 2012) , but could instead be incorporated into the mask. The quality of the DR12 data is such that we can now observe systematic effects that couple radial and angular fluctuations, and we introduce 3D corrections for these. The manner adopted to deal with these effects for BOSS is described in Section 6. BOSS includes a number of galaxy catalogues with different selection functions, some of which spatially overlap. The combination of these to optimally quantify the underlying matter overdensity field is non-trivial, and we present the method adopted by the BOSS team in Section 7. The MKSAMPLE code will be released upon publication of this paper, and we will also publish the resulting Large-Scale Structure catalogues, with a full datamodel describing each.
2), which may differ from the quantities available for an object in the DR8 catalog. BOSS targeting was performed using three different versions of the reduction software that resolves the catalogues from overlapping imaging data (RESOLVE; see  Aihara et al. 2011). Chunks 1-4 used a version of the RESOLVE software tagged on 14-06-2009, chunks 5-11 used a version tagged on 16-11-2009, and chunks 12 onwards used the same version as that used to produce DR8. In total, 17% of targets were targeted with pre-DR8 RESOLVE versions. Because these different versions of the software selected different imaging data to be designated as "primary" (i. e. either the only or the best observation of this object; see the DR8 documentation for more details), approximately 9% of the imaging data used for targeting CMASS galaxies is now designated as secondary 1  in the DR8 database.
The difference between the two is therefore a measure of the extendedness of the galaxy. In summary, the LOWZ selection algorithm, as implemented after commissioning, is as follows: r cmod < 13. 5 + c || /0. 3 (9) |c ⊥ | < 0. 2 (10) 16 < r cmod < 19. 6 (11) r psf -r cmod > 0. 3 , (12) The galaxies in the LOWZ sample may be selected from the DR12 database using the following flags, whose definitions can be found on the SDSS website 4 : • BOSS TARGET1 && 2 0 Objects targeted by the LOWZ algorithm. • SPECPRIMARY == 1 Objects with spectra, removing duplicate observations. • ZWARNING NOQSO == 0 Objects whose spectroscopic redshifts are cleanly measured. • CLASS NOQSO == 'GALAXY' Objects whose spectra are those of a galaxy (as opposed to a quasar or star). The basic properties of the LOWZ sample are presented in  Parejko et al.
CMASS galaxies can be selected from the DR12 database using the following flags:   (2013)  for details. The green dashed line shows the sliding cut adopted for the CMASS SPARSE sample (see Section 3. 4). Vertical solid lines show the magnitude limits. On both panels, the small fraction of targets that lie outside of the selection cut are due to differences in the targeting and final photometry, see Section 2. Only chunks greater than 6 are shown. • BOSS TARGET1 && 2 1 • SPECPRIMARY == 1 • ZWARNING NOQSO == 0 • CLASS NOQSO == 'GALAXY' The basic clustering properties of the CMASS sample are presented in  White et al. (2011) , which fitted the small-scale clustering of the galaxies using HOD modelling. They showed that these galaxies lie in massive haloes, with a mean halo mass of 2. 6 × 10 13 h -1 M , a large-scale bias of ∼ 2.
Spectroscopic Reductions Each "tile" output from the tiling algorithm specifies a central location on the sky and the list of targets to be observed. Physical plates are drilled at the University of Washington based on the anticipated airmass of observation. Multiple plates can cover the same tile, and plates may be observed on multiple nights until the desired signalto-noise ratio is reached  (Dawson et al. 2012 ). The BOSS spectroscopic reduction pipeline is detailed in  Bolton et al. (2012) , with minor updates given in  Alam et al. (2015). The final DR12 catalogues used the v5 7 0 tag of the IDLSPEC2D software package 5 for spectroscopic calibration, extraction, classification, and redshift analysis. We restrict the large-scale structure catalogues to only include data from plates with PLATEQUALITY set to "good.
After full footprint data and random catalogues are produced, we trim each catalogue back to its original targeted region (i. e. , LOWZE2 in chunk 2, LOWZE3 in chunks 3-6, and LOWZ in chunks 7) using the mutually exclusive masks discussed above. Our algorithm to generate the combined catalogue from the four different samples (CMASS, LOWZ, LOWZE2, LOWZE3) is as follows: • Renormalize the CMASS galaxy systematic weights wsystot,i ∝ wsystot,i such that 1 = i wsystot,i(wcp,i + wnoz,i -1) i (wcp,i + wnoz,i -1). ( 54 ) This ensures that in the combined catalog, a CMASS target and a LOWZ target on average have equal weight in each of the three distinct regions. The functional form chosen for wsee and wstar does not guarantee this normalisation. Fibre collision and redshift failure weights are left the same as in the original CMASS-only catalogue and the parameters for the systematic weights are identical to the ones in the CMASS-only catalogue (apart from the renormalisation).
DISCUSSION The small statistical errors achievable on cosmological measurements from BOSS data require removal of potential systematic issues to an unprecedented level. Spectroscopic target selection and mask creation are key areas where systematic problems can be introduced if care is not taken to fully understand both. In this paper we have presented the target selection for the three primary spectroscopic galaxy catalogues within BOSS: LOWZ, CMASS and Sparse, and for variations on these used for some early data. Each sample has different sky coverage and expected redshift distribution. We have also presented the methods used to turn the target catalogue and redshift measurement data into galaxy and random catalogues, which enable clustering measurements to be quickly made, as well as methods to mitigate potential systematics.
ACKNOWLEDGEMENTS Funding for SDSS-III has been provided by the  Alfred P. Sloan Foundation , the Participating Institutions, the National Science 20 Reid et al. Foundation, and the  U. S. Department of Energy Office of Science. The  SDSS-III  web site is http://www. sdss3. org/. SDSS-III is managed by the  Astrophysical Research Consortium  for the Participating Institutions of the  SDSS-III Collaboration  including the  University of Arizona , the  Brazilian Participation Group ,  Brookhaven National Laboratory, Carnegie Mellon University, University of Florida , the  French Participation Group , the  German Participation Group, Harvard University , the  Instituto de Astrofisica de Canarias , the  Michigan State/Notre Dame/JINA Participation Group ,  Johns Hopkins University, Lawrence Berkeley National Laboratory, Max Planck Institute for Astrophysics, Max Planck Institute for Extraterrestrial Physics, New Mexico State University, New York University, Ohio State University, Pennsylvania State University, University of Portsmouth, Princeton University, the Spanish Participation Group, University of Tokyo, University of Utah, Vanderbilt University, University of Virginia, University of Washington , and  Yale University.
In order to solve this kind of scattering problem we have first pursued the development of an already existing home made FEM (Finite Elements Method) software based on the resolution of the weak form of the Helmholtz equation and used in different configurations  [34]. In this software the unknowns are expanded onto P 1 Lagrange basis functions associated to a conformal triangular mesh of the scene. The amplitude of the scattered field in a given direction is then computed thanks to the Huygens principle. One of the key point of this method is to properly introduce the sources responsible for the incident field. In the case of an incident field of finite extent, the use of a Bayliss-Turkell boundary conditions on the external boundary yields a good approximation of the incident field  [35].
A current overview of synchrotron radiation (SR) in macromolecular crystallography (MX) instrumentation, methods and applications is presented. Automation has been and remains a central development in the last decade, as have the rise of remote access and of industrial service provision. Results include a high number of Protein Data Bank depositions, with an increasing emphasis on the successful use of microcrystals. One future emphasis involves pushing the frontiers of using higher and lower photon energies. With the advent of X-ray free-electron lasers, closely linked to SR developments, the use of ever smaller samples such as nanocrystals, nanoclusters and single molecules is anticipated, as well as the opening up of femtosecond time-resolved diffraction structural studies. At SR sources, a very high-throughput assessment for the best crystal samples and the ability to tackle just a few micron and sub-micron crystals will become widespread.
The topical review presented here concentrates on developments since then. A recent comprehensive overview of phasing methods in crystallography including MX and the use of multiple and single wavelength methods, including in a historical context, is given in the book by  Giacovazzo (2013). Recently, Abad-Zapatero (  2014 ) undertook an analysis of the growth rate in Protein Data Bank (PDB;  Berman et al. , 2000 ; http://www. rcsb. org/pdb/) depositions over the decades, reporting that the appearance of third-generation facilities, beginning with the ESRF (Grenoble, France) in 1994, has helped to maintain PDB data-deposition rates which otherwise might well have slowed down as more and more complex 'molecular machines' were studied. The last four years have seen the maturing of MX data collection and data processing at third-generation synchrotron beamlines into a highthroughput and largely automated technique.
Automation The automation revolution created for MX by institutes like the ESRF in partnership with the EMBL-Grenoble, by Stanford Synchrotron Radiation Laboratory, and by the more recent synchrotrons such as Diamond (Didcot, UK), SOLEIL (Saclay, France) and the SLS (Swiss Light Source, Switzerland) is a key point in the continued rise of synchrotron crystallography. These robotic developments and smart soft-topical reviews ware pipelines, with their database and data-delivery frameworks, are having a major impact. Reliable sample changers and automated intelligent software are routine at every synchrotron MX beamline. In Europe, the SPINE hardware standard gives consistent sample-mounting pins and bases. A new generation of pins is being developed under the banner of another European project, BioStructX, for the transport of a larger number of samples per cryo-dewar.
, 2011). This serial method leads to a myriad of still diffraction patterns, but with the advent of XFELs new software has been developed (e. g. CRYSTFEL;  White et al. , 2012)  to handle these data efficiently and produce complete diffraction data sets. The serial delivery method was also recently proven for the synchrotron using lysozyme as the model system, with over 40 000 crystals exposed and merged to a data set of 2. 1 A resolution  (Stellato et al. , 2014). Taking this concept further, recent work has used the ESRF's ID13 micro/nanofocus diffraction beamline with a sample injector system, again originally developed for use at XFELs but specifically designed for lipidic cubic phase crystals. Another method aims to use many crystals cryo-cooled in one sample support and then raster-scan to locate crystals, collect diffraction data and combine the partial diffraction data sets to form the full data set.
SR MX leads the way for commercial industry access Industrial use of the synchrotron research infrastructure is a core mission of most such facilities. At the SRS Daresbury, commercial access to MX was the largest share of the Daresbury Analytical Research Technical Service (DARTS)  (Maclean et al. , 2006). Nowadays, most synchrotrons have an industry or business development office, managing and developing links with industry and creating economic value and impact from industrial access [see, for example, Cutler (2014),  Shotton et al. (2014)  and  Mitchell et al. (2011) ]. Drug discovery is still a significant income generator for all facilities, yielding millions of euros per year in industrially derived income flowing into the facilities. Industry requirements have driven aspects of SR MX automation and, in particular, the development of metadata models and database systems to track data collections and processed results and to make all the information available via a secure web link for remote access and download of results.
Availability of raw diffraction images at SR facilities Retaining raw diffraction data has become an increasingly debated topic in recent years. The Australian synchrotron is leading on data archival with its Store. Synchrotron data storage service. As well as diffraction image data archiving, it also supports users in their publications with linking to raw data sets via DOI registrations and, finally, the release of data sets for public analysis -something that, in the neutron community, the ILL is doing as well. There are also fine examples like Diamond that has so far retained all of its measured data. The ESRF has published a summary of its views on the era of Big Data at SR facilities in general and the challenges involved today, as exemplified by ESRF itself  (ESRF, 2013). The imgCIF dictionaries continue to be developed in a way that will facilitate interoperability with NeXus/ HDF5 workflows at synchrotron radiation facilities, and topical reviews IUCrJ (2015).
Summary and outlook The excellent SR infrastructure now established for MX spans high-brilliance SR sources of front-line capabilities at national and international level. It is amazing that time-resolved protein crystallography is undertaken at the nanosecond time scale using Laue diffraction at, for example, the ESRF and the APS, and now at the femtosecond time scale at the LCLS. The choice of photon wavelength is still widening; the usage of longer wavelengths up to $5 A ˚is set to become a regular feature of our data-collection repertoire. Special experiments such as high-pressure protein crystallography have been routinely using wavelengths as short as 0. 3 A ˚  (Fourme et al. , 2011) , and even wavelengths down to 0. 2 A ˚have been investigated for MX  (Jakoncic et al. , 2006). The speed of routine MX data collection, facilitated by such brilliant X-ray beams and the new generation of pixel detectors, is now minutes per data set, benefitting the entire academic and industrial user community for high-throughput fragment-based drug discovery or simply for selecting the best diffracting sample out of many.
Acknowledgements We heartily thank the SR and neutron facilities, and their academic and industrial user communities with whom we have interacted, for their collaboration. In particular, EM thanks the  ESRF, ILL  and the PSB teams in Grenoble. With reference to JRH's IUCr Montreal Congress Keynote Lecture, delivered in August 2014, the Chair of the Programme Committee,  Professor Jim Britten , requested a presentation on the history of SR MX up to the present day and projecting into the future. This IUCrJ Topical Review, with which it is linked, has focused on just the last three years. Nevertheless, JRH wishes to state that he is grateful to his PhD students and post-doctoral assistants over the last 35 years, to the  Daresbury Laboratory , the  ESRF  and the  ILL , to DLS and to the  Universities of York, Oxford, Keele  and  Manchester.
A new, two-layer canopy module with thermal inertia as part of the detailed snow model SNOWPACK (version 3. 2. 1) is presented and evaluated. As a by-product of these new developments, an exhaustive description of the canopy module of the SNOWPACK model is provided, thereby filling a gap in the existing literature. In its current form, the two-layer canopy module is suited for evergreen needleleaf forest, with or without snow cover. It is designed to reproduce the difference in thermal response between leafy and woody canopy elements, and their impact on the underlying snowpack or ground surface energy balance. Given the number of processes resolved, the SNOW-PACK model with its enhanced canopy module constitutes a sophisticated physics-based modeling chain of the continuum going from atmosphere to soil through the canopy and snow.
Focus is mostly on snow-covered environments, but not exclusively. For our purpose, we develop a two-layer canopy representation (leaves and wood) in the aforementioned SNOWPACK model, which proposes a very detailed physical and microphysical representation of the snowpack. Before our work, it included a simple, onelayer canopy module where radiation and precipitation interception by forest elements were represented (e. g. ,  Musselman et al. , 2012) , but the equivalence between air and canopy temperature was assumed. Because SNOWPACK was initially developed for alpine environment and is still mostly used in alpine or boreal context where conifers are dominant, our new canopy module is for now only suited for needleleaf, evergreen forest. Micro-meteorological and sub-canopy, stand-scale radiation data collected during the SnowMIP2 experiment provide a proper data set for the evaluation of our developments.