dataesr/scientific-paragraphs-categorization huggingface dataset:
For each hit, 100 nucleotides upstream and downstream of the sequence are collected. In order to define the structure of the inserted sequence and to classify them into four groups ("Alone," "Repeat," "PolyA," and "3 0 -truncated"), the program performs successively three steps of analysis. In the first step, the program identifies the potential TSD signature with a local alignment using EMBOSS wordmatch, which identifies exact matches between two sequences  (Rice et al. 2000). The first and last ten nucleotides of the hit itself are also included in the TSD search to allow for some boundary imprecision. By default, TSDs were required to be at least ten nucleotides long. If no match is found, the downstream sequence is extended to 6 kb in order to find more distant TSD, and the program runs successively EMBOSS wordmatch and wordfinder, which allows mismatches in the alignment.
Next, if TSD is found, the program tests for the presence of repeat sequences by performing BLAST (with default parameters) against RepBase Update, a library of mobile elements  (Jurka et al. 2005). When found, the repeat family is specified in the result. When not found, we have to manually look for the origin of the sequence (see below). Finally, a search of poly(A) sequences is performed and is positive if more than six adenosines are present in a sliding window of ten nucleotides. After each step, if the result is positive, the downstream sequence is rebuilt and the boundary of the signature is recorded. Next to the four defined groups, we added two other categories: "To Check" and "Gaps. " The "To Check" category includes sequences that are 5 0 -truncated and sequences that are 3 0 -truncated without TSD.
Acknowledgments The authors thank  Lise Corriger ,  Marjorie Côte ,  Sandrine Enjalbert , and  Oussama Meziane  for their help in the initial screening of snRNA sequences,  Ned Lamb  and  Nicolas Philippe  for helpful discussion;  Guillaume Gielly ,  Jacques Faure  and Alfred Vriese, members of the computing facility at the  Institute of Human Genetics , for their help with computers and servers. A. J. D. was the recipient of fellowships from the  French government (Ministere de l'Enseignement Sup erieur et de la Recherche) , from  Association pour la Recherche contre le Cancer (ARC) , and from  Fondation pour la Recherche M edicale (FRM). O. S. was the recipient of a fellowship from the  Deutsche Forschungsgemeinschaft (DFG). J. A. was the recipient of a fellowship from  FRM. Work in the laboratory of N.
General-Purpose Xeon E5. The Intel Xeon E5 is a 64-bit x86-64 processor. In this study we used a Xeon E5-4640 Sandy Bridge-EP, which has 8 physical cores running at 2. 40 GHz. Each core has 32 KB instruction and 32 KB data L1 caches and 256 KB of L2 cache. All the 8 cores share a 20 MB L3 cache and the platform has 32 GB of DDR3 memory. Altix UV 2000. SGI Altix UV 2000 (Figure  1 ) is a Non-Uniform Memory Access (NUMA) platform designed by SGI. The platform is composed of 24 NUMA nodes. Each node has a Xeon E5-4640 Sandy Bridge-EP processor (with the same specifications of the Xeon E5 platform) and 32 GB of DDR3 memory. This memory is shared in a ccNUMA fashion through SGI's proprietary NUMAlink6 (bidirectional). This high-speed interconnection provides a point-topoint bandwidth of 6. 7 GB/s per direction.
This is an expected phenomenon because Xeon E5 runs a multithreaded version of the applications whereas Altix UV 2000 runs their distributed counterparts. Distributed versions experience periods of low processor usage as, for example, those during the task request/response cycle and those related to load imbalance. This will be further discussed in Section 5. 5). The power consumed by each processor was obtained using the same approach. Both Xeon E5 and Altix UV 2000 feature Intel Sandy Bridge microarchitecture, which has Running Average Power Limit (RAPL) energy sensors. This allows us to measure the power consumption of CPU-level components through Machine-Specific Registers (MSRs). We used this approach to obtain the energy consumption of the whole CPU package including cores and cache memory (named RAPL PKG domain).
e. , with no inter-peer communications. We measure, therefore, the scalability of the distributed version of the algorithms and not that of the of the multi-threaded version. Detailed scalability evaluation analysis for the multi-threaded algorithms can be found in the base works presented in Section 3. Overall, the distributed version of the applications scaled considerably well and execution times showed similar trends on both platforms. However, Altix UV 2000 was from 9x up to 13x Moreover, in order to avoid NUMA e↵ects on the Altix UV 2000 platform and ensure good execution performance, we had to employ some additional runtime optimizations. For all applications we employed thread-pinning  [33]. Since the TSP was implemented using POSIX threads,  we used Linux's specific system calls to ensure that threads would not be migrated during the execution of the application.
K-Means and Ondes3D used OpenMP to implement parallelism, therefore in these cases we employed the GOMP CPU AFFINITY environment variable to ensure threads were correctly bound to the available cores. Additionally, we specially modified the initialization phase of the applications so that the first-touch strategy (Linux's default) could suitably place the allocated memory on the NUMA nodes. For that, each application would make each thread initialize (either with the actual value or with a dummy value when initialization had to be centralized) each private region of memory.
At the beginning (Figure  11b ), one thread in each peer asks a master peer for partitions and starts populating the local pool of tasks. As tasks become available, other threads in the same peer can start the computation. Once the thread assigned to populate the local pool of tasks finishes its job, it also starts the computation. Afterwards, as the peers run out of work, they ask a master peer weak and strong scalability on a cluster of 96 ARM Cortex-A9 dual-core processors and demonstrated that the ARM-based cluster can be more e cient in terms of energy-to-solution compared to a cluster of Intel Xeon X5550 processors. Similarly, Ou et al. [35]  compared the energy efficiency of a ARM-based cluster against an Intel X86 workstation on three applications: a web server, an in-memory database and a video transcoding.
Acknowledgments The authors would like to thank  CAPES  for funding this research through project  CAPES/ Cofecub 660/10  and through a  PNPD/CAPES scholarship. This work was done in the context of LICIA and Mont- Blanc  project (funded from the  European Union 's  Seventh Framework Programme  under grant agreement # 288777 ), being partially supported by  CNPq ,  FAPEMIG ,  FAPERGS  and  INRIA.
We detail the components of hsim and show the importance of the key stages and their effects on observations. We then showcase the pipeline with two studies: calculations of point source sensitivities for the available observing modes, and a case study of measuring kinematics of star forming galaxies at z ∼ 2-3. The paper is organised as follows: Sections 2 and 3 detail the motivation, goals and methodology of the simulations; in Section 4 we present an overview of the pipeline stages; in Section 5 we present our point spread function parameterisation and demonstrate its importance; Section 6 shows our verification crosschecks against existing software; in Section 7 we present predicted point source sensitivities; Section 8 presents our high redshift galaxy simulations; and we finally conclude in Section 9.
SIMULATION PIPELINE OVERVIEW We first presented the simulation pipeline in  Zieleniewski et al. (2014)  and here we provide an overview of each stage, highlighting several novel features. The overall process is presented in Fig. 1. hsim is written in python and makes use of the astropy package  (Astropy Collaboration et al. 2013) , as well as the numpy and scipy packages. It handles input and output data in fits format, as this is the usual file format for 3D spectrographs. The input data-cube is uploaded as a 3D array of two spatial and one spectral dimension. The fits headers contain all the relevant information about the object (e. g. spatial and spectral sampling, flux units, spectral resolution). The user chooses suitable observing parameters and the data-cube is then processed through several steps:.
EFFECTS OF THE AO POINT SPREAD FUNCTION The effects of the telescope and AO system on observations are encoded in the spatial PSF. The E-ELT PSF is predicted to be a strong function of wavelength as well as other parameters including seeing, off-axis distance from reference AO star, and guide star magnitude. Its complex form means it needs to be incorporated carefully into the simulations not simply modelled as a simple Gaussian or Moffat function. We have incorporated a continuously varying AO PSF as a function of wavelength and seeing, for both LTAO and SCAO, in our simulation pipeline, using the eltpsffit program developed by J. Liske 1. This allows the user to fit a 1D average radial profile of a PSF with a set of analytical functions. We used a set of simulated long-exposure LTAO and SCAO The importance of the wavelength variation is worth emphasising.
In Fig. 9  we show the rotation curves for galaxy B (extracted along a ∼ 1 kpc wide slit aligned along the semi-major axis)  Also plotted in the centre subplot is the rotation curve for a 1 hour observation at the 30 × 60 mas scale (blue squares), which has been binned up to 60 mas spaxels. We also show a simulated 10 hour VLT (SINFONI) observation at 100 mas (green triangles). All curves are extracted along a ∼ 1 kpc slit aligned along the semi-major axis. From these simulations the E-ELT with HARMONI offers a 10 fold improvement in observing efficiency compared to current telescopes, i. e. the E-ELT could observe 10 times as many objects in equal observing times. It also offers finer resolution, with 5× more independent data points at the 20 mas scale, and improved sensitivity for equal observing times on the same object for the same galaxy properties, but with a star-formation rate of 3.
Comparing this to the rotation curve derived from a simulated 10 hour VLT (SINFONI) observation at 100 mas we see that the E-ELT offers higher resolution data at better sampling with 10× greater efficiency in observing time. For equal observing time the VLT curve underestimates the true curve at all radii and also only extends to 3r hl , compared with 4. 2r hl from HARMONI at 20 mas sampling. Deriving a simple dynamical mass estimate from each observation gives ∼ 9 × 10 9 M from E-ELT (HARMONI) and ∼ 4 × 10 9 M from the VLT (SINFONI) simulation. Comparing to the input value of 8. 6 × 10 9 M this represents both an accurate estimate and a factor of 2 improvement by HARMONI af-ter 10 hours observing. From these simulations we find that the E-ELT with HARMONI offers a 10 fold improvement in observing efficiency compared to current telescopes, i.
4. We perform simulations of z ∼ 2-3 star forming emission-line galaxies and find that the E-ELT with HAR-MONI will be capable of obtaining velocity maps of these galaxies down to Milky-Way SFRs at these redshifts, with a factor ∼ 5 improvement in spatial resolution over current  generation instruments. By deriving rotation curves we find that the E-ELT with HARMONI offers improved resolution data with a 10 fold improvement in observing efficiency compared to current telescopes, i. e. in equal observing times the E-ELT could observe 10 times as many objects. For equal observing times on the same galaxy the increased sensitivity of the E-ELT with HARMONI gives a factor of ∼ 2 improvement in dynamical mass estimates. 5. HARMONI with LTAO will provide exquisite resolved spectroscopy of z ∼ 2-3 galaxies, allowing the detection of individual star forming complexes and measurements of their properties.
ACKNOWLEDGMENTS We are grateful to the anonymous referee for helpful feedback which improved this paper. The authors also thank  N. Schwartz  for providing AO PSFs,  M. Puech  for helpful feedback while testing our pipeline, and  J. Liske  for help using eltpsffit. SZ, NT, SK, RH, MT and FC are supported by  STFC-HARMONI  grant  ST/J002216/1. RH was also supported by  STFC  grant numbers  ST/H002456/1 & ST/K00106X/1. AMS acknowledges an  STFC Advanced Fellowship  ( ST/H005234/1 ) and the  Leverhulme foundation.
2) Infrastructure configuration: The experiments were conducted on Grid'5000 Nancy, by using 7 physical machines linked by a 20 Gbit/s Ethernet switch. Each machine has two 2. 8GHz Xeon processors (4 cores per CPU) and 15GB of RAM, running Linux 2. 6 Machines. The used cloud computing platform is Openstack Grizzly 1. 0. 0, which requires one dedicated physical machine, the cloud controller, to manage the system. Consequently, 6 physical machines are dedicated to host virtual machines, which in turn, are pre-configured to run Ubuntu 12. 04. 3) Autonomic Manager: Our autonomic manager process is hosted on the cloud controller machine. It monitors the average response time of received and processed requests by aggregating the lbt's Nginx logs in a given time window (in our case 15 seconds). According to a comparison of this value with two threshold values, two event patterns can be issued : • "High Response Time" occurs when the monitored value is higher than an upper threshold.
B. Cross-layer Elasticity Management According to our knowledge, only few studies have focused on expanding system's elasticity capability to the software layer. [23]  is interested in architecturebased self-adaptation. They developed Rainbow  [24] , a framework which is similar to a MAPE-K model  [2] , using software architectures and a reusable infrastructure to support runtime self-adaptation of software systems. For illustration, they rely on the Znn. com case study (also known as ZAP. com or Z. com), a web-based system that serves multimedia news content. The objective is to serve news contents within a reasonable response time while keeping the resources' associated cost under a predefined threshold. In the case of sudden and unanticipated workload increase, to prevent service unavailability due to resource elasticity limitations, Znn.
The cctbx also featured many of the restraints we needed, in the module cctbx. restraints. For our project, we contributed to it a Small-Molecule Toolbox (smtbx) which features, in particular, the constraint framework presented in detail in x3 and Appendix C, and the computation of structure factors presented in Appendix A. We also contributed several new restraints to cctbx, restraints which are discussed in x4 and Appendix D. Finally, both the cctbx and the smtbx use the Scientific Toolbox (scitbx) for arrays, matrices, special functions and other purely mathematical matters. We have contributed a new least-squares toolbox (lstbx) to the scitbx, which provides flexible tools to deal with generic linear and non-linear problems, with or without an unknown overall scale factor. It implements the method based on normal equations presented in Appendix B.
The program Olex2 relies either on SHELX or on the smtbx to refine structures. If using the latter, Olex2 converts its internal model of a structure into the objects used by the smtbx and the cctbx to represent the unit cell, reflections, crystal symmetry, constraints, restraints etc. Once the smtbx has produced the desired results they are sent back to Olex2 which converts them to its own representations, so as to perform diverse post-processing and eventually display the results to the user of the program. Fig. 1  summarizes the dependencies between the various modules and programs that have just been described.
, 2011; Watkin, 2008). This term L restraints may of course be absent. Conversely, the data term could be absent in a refinement against geometrical terms only [cf. the program DLS-76 by  Baerlocher et al. (1978)  for example]. In equation (  1 ), the crystallographic parameters [i. e. atomic positions, atomic displacement parameters (ADPs) and chemical occupancy in routine refinements] that each component of Y c and T c depends upon will be collectively denoted as the vector y ¼ ðy 1 ;. ; y p Þ and we can therefore denote those dependencies with the compact notations Y c ðyÞ and T c ðyÞ. The parameters that are actually refined may be a different set x 1 ;. ; x n , collectively denoted as a vector x. We assume that the dependency of y upon x is known analytically. Since the scale factor K is unknown, our problem is the minimization of L with respect to all of K; x 1 ;.
We decided not to put arbitrary limits on that graph. Indeed, we could have made a closed list of reparametrizations and of reparametrization combinations that our framework would accept but instead we decided to write our code so that it could correctly handle the computation of parameter values and of partial derivatives for arbitrary reparametrizations, combined in arbitrary ways. This framework is therefore open as new types of reparametrizations can be added, without the need to change the basic infrastructure in any way, and without the risk of breaking existing reparametrizations. This has proven very useful to the authors as this enabled them to incrementally add the wealth of constraints now available, some of which are unique to olex2. refine, as discussed in Appendix C. Furthermore, it enables third parties to develop their own constraints without the need for the involvement of the original authors beyond documenting how the reparametrization framework works.
Constraint matrix There is a wealth of algorithms designed to minimize least squares, but crystallographic software has only implemented a few of them. The two most popular methods have historically been the full matrix 3  (all small-molecule programs, including Olex2, as explained in Appendix B) and conjugate gradient LS (CGLS, see e. g. Bjo ¨rck, 1996) which SHELXL offers as an option along with full matrix. The macromolecular community later introduced the limited-memory Broyden-Fletcher-Goldfarb-Shanno method  (LBFGS, Nocedal, 1980) , phenix. refine (Afonine et al. , 2012)  and a sparse Gauss-Newton algorithm, REFMAC  (Murshudov et al. , 2011, x4 and references therein). All methods mentioned above have in common the fact that they require only the computation of the value and the first-order derivatives of the calculated F or F 2 , that we have denoted as Y c in this paper.
Such twins belong to the taxonomies pseudo-merohedral or merohedral. olex2. refine is able to refine a twinned structure input in this manner. SHELXL users would handle the general case by using a reflection file in the HKLF5 format. Such a file is created by the integration software and this approach can deal with the most complex situations. By contrast, CRYSTALS always requires a list of twin laws. olex2. refine can handle a more general case: when general and merohedral twinning are simultaneously present. For example, one may have four domains 1, 2, 3 and 4. Domains 1 and 3 are related by a twin law R, and so are domains 2 and 4, but the relative orientation of domains 1 and 2 does not correspond to any twin law. Thus the measured frames exhibit two lattices of Bragg peaks, with some overlaps, and the integration software will then output a list of (F 2 o , h 1 ), (F 2 o , h 2 ) and (F 2 o , h 1 , h 2 ).
The refinement engine needs to expand this to a list of (F 2 o , h 1 , h 3 ¼ h 1 R), (F 2 o , h 2 , h 4 ¼ h 2 R) and (F 2 o , h 1 , h 2 , h 3 ¼ h 1 R, h 4 ¼ h 2 R). olex2. refine performs this task on the fly, while passing Miller triplets to the code computing structure factors and their derivatives. The theory and phenomenology of twinning extends far beyond our exposition, but from the narrow point of view of refinement our presentation is sufficient, since, in this paper, we do not concern ourselves with the computation of the twin law or with the indexing of superposed lattices.
Refinement results olex2. refine uses CIF  (Hall et al. , 1991)  as its main output. The CIF contains information regarding the space group, the data indicators such as merging indices, the refinement indicators such as the R factors, goodness of fit, residual electron density, refinement convergence indicators and tabulated structure information, including tables of atomic parameters, bonds and angles. olex2. refine produces a table describing the restraints using the CIF restraints dictionary. Moreover, the olex2. refine CIF always contains a verbal description of the refinement model -hydrogen-atom treatment, constraints, restraints and their targets. Optionally, the refinement model file (SHELX RES file) and the reflections can also be included in the final CIF for deposition. It should be noted that the constraints and restraints unique to olex2.
Conclusion We have presented herein the full mathematical derivations of the concepts used within olex2. refine. This new refinement engine is feature-wise on a par with the established software in the field such as SHELXL and CRYSTALS. It actually provides a richer wealth of constraints than those classic suites. olex2. refine is immediately useful to the practising crystallographer since it is presently available from the program Olex2 by default. It can also be used independently as a library of components to write short scripts as well as more complex programs, dealing with any aspect of small-molecule refinement. Indeed, olex2. refine is largely based on the Small-Molecule Toolbox (smtbx) that is part of the Computational Crystallography Toolbox (cctbx), which is usable as a library for the Python programming language, thus providing great expressiveness, conciseness and ease of coding combined with an immense wealth of tools covering all of crystallography.
This is mitigated by computing the structure factor and its derivatives together, as the cosine and sine then only need to be computed once for each reflection and scatterer. This optimization is the reason why we have written our own new code into the smtbx instead of reusing the original code in the cctbx. Indeed, in the latter, structure factors are computed separately from the derivatives, resulting in two separate loops over the reflections and the scatterers, and in sines and cosines being computed twice. This is rather well suited to the optimization algorithm used in phenix. refine (LBFGS) because it does not need the derivatives at every step. But it does not fit well with full-matrix least squares as is prevalent in small-molecule crystallography. Our code also provides two options to compute the sines and cosines: either by using the trigonometric functions of the standard C++ library, or by using a cctbx function that interpolates between tabulated values of sines and cosines.
The resulting database was enlarged with African isolates to include most human tuberculosis diversity. Assignations were obtained using TB-Lineage, MIRU-VNTRPlus, SITVITWEB and an algorithm from Borile et al. By identifying the recurrent concordances between the alternative taxonomies, we proposed a consensus including 22 sublineages. Original and consensus assignations of the all isolates from the database were subsequently implemented into an ensemble learning approach based on Machine Learning tool Weka to derive a classification scheme. All assignations were reproduced with very good sensibilities and specificities. When applied to independent datasets, it was able to suggest new sublineages such as pseudo-Beijing. This Lineage Prediction tool, efficient on 15-MIRU, 24-VNTR and spoligotype data is available on the web interface "TBminer.