dataesr/scientific-paragraphs-categorization huggingface dataset:
There is evidence to suggest that a surprising proportion of published experiments in science are difficult if not impossible to reproduce. The concepts of data sharing, leaving an audit trail and extensive documentation are fundamental to reproducible research, whether it is in the laboratory or as part of an analysis. In this work, we introduce a tool for documentation that aims to make analyses more reproducible in the general scientific community. The application, lpEdit, is a cross-platform editor, written with PyQt4, that enables a broad range of scientists to carry out the analytic component of their work in a reproducible manner-through the use of literate programming. Literate programming mixes code and prose to produce a final report that reads like an article or book. lpEdit targets researchers getting started with statistics or programming, so the hurdles associated with setting up a proper pipeline are kept to a minimum and the learning burden is reduced through the use of templates and documentation.
lpEdit: a literate programming editor Many of the tools available for literate programming do not provide a graphical editor, which is a barrier for adoption by nonspecialists. Other tools depend on a particular operating-system and only a handful of tools can switch freely between several programming languages. The motivation to build lpEdit arose because there was no apparent library/tool that fit these three criteria in a simple and intuitive way. We have developed here an environment for literate programming, based on the model-view-controller (MVC) software architecture pattern. The only major difference from conventional realizations of MVC patterns is that instead of the user interacting directly with the controller in a non-GUI mode, we have developed a convenience class called NoGuiAnalysis for this purpose.
The GUI editor portion of lpEdit is written with PyQt4 http: //www. riverbankcomputing. com/software/pyqt, which are Python bindings to the widget toolkit Qt http://qt. digia. com. For the basic editing component of the software we use the Qt port of Scintilla http://www. scintilla. org called QScintilla http:// www. riverbankcomputing. com/software/qscintilla. The additional prerequisites are the Python packages for numeric computing (NumPy)  [Oliphant07]  and the ubiquitous documentation tool Sphinx http: //sphinx-doc. org. The software is available under the GNU General Public License version 3. 0 or later from http://bitbucket. org/ajrichards/ reproducible-research. The accompanying documentation can be found at http://ajrichards. bitbucket. org/lpEdit/index. html.
L A T E X and reStructuredText Perhaps the most widely used literate programming tool is Sweave  [Leisch02]  which embeds R code into a L A T E X document. Due to its popularity and because Sweave is now part of the R project  [RCore12] , the Sweave environment may be used from within lpEdit. Another notable projects that mixes R and L A T E X is knitr http://yihui. name/knitr. RStudio  [RStudio]  is a graphical editor that supports Sweave and knitr. R is a standard language for statistics, but for other common computational tasks, like text processing and web-applications, it is used less frequently than scripting languages. We opted to add Python, a scripting language, because it is being increasingly used in the life-sciences  [Bassi07]  and because it has a clean syntax that ultimately aids transparency and reproducibility.
Several wellfeatured literate programming tools exist for Python including PyLit http://pylit. berlios. de and like PyLit our software uses re-StructuredText (reST) http://docutils. sourceforge. net/rst. html, although we additionally allow arbitrary Python code to be included in L A T E X source documents. Another powerful tool for reproducible research using Python is the IPython notebook  [Perez07]. There are three types of file extensions currently permitted for use with lpEdit: the Sweave extension ( *. rnw); a Noweb  [Ramsey94]  inspired syntax ( *. nw); and the reST file extension ( *. rst). By selecting an embedded language and a file type there are a number of different workflows available as shown in Figure  1. 1 : Summary of the possible workflows using lpEdit. First, a language, either R or Python is selected then it is embedded into a specific document ( *.
rnw, *. nw or *. rst). Next a L A T E X or Sphinx project is built for the document, which then allows for both HTML and PDF output formats. lpEdit as a library lpEdit has a simple API, which facilitates the use of unit testing and exposes the functions of this library for those who are not in need of a text editor. In this section, we explain how to create a project and build reports using the command line, in order to illustrate the basic mechanics of lpEdit. The following example script, BasicPython. nw, is bundled with the package lpEdit. To build a project and compile it into report form only a few commands are needed. 1 from lpEdit import NoGuiAnalysis 2 nga = NoGuiAnalysis() 3 nga. load_file("BasicPython. nw",fileLang="python") 4 nga. build() 5 nga. compile_pdf() 6 nga. compile_html() First the class is imported (line 1) from the module lpEdit and then it is instantiated (line 2).
The file is then loaded and the language may be specified (line 3). The build() method creates a directory to contain the project in the same folder as BasicPython. nw. The build-step also creates a *. tex document. This directory is what lpEdit refers to as a project and it is where both reST and L A T E X projects are managed. The compile_pdf() command either uses sphinx-build or pdflatex. The compile_html() command defaults to sphinx-build or latex2pdf depending on the project type. In most cases the default paths for pdflatex, python, R, and sphinx-build are found automatically, however, they may be customized to a user's preference. To modify these variables without the GUI, there is a configuration file corresponding to the current version of lpEdit located in the user's home directory.
Documenting by example Like Sweave, lpEdit uses a Noweb  [Ramsey94]  inspired syntax. The advantages are that due to a simplified syntax, the flow of the document is only minimally interrupted by the presence of code. Also, to reduce the learning burden on new users, we suggest they concentrate on learning L A T E X, reST and the embedded programming language of choice instead of lpEdit-specific tricks to embed plots, tables or other convenient features. For *. rnw, *. nw and *. rst documents, we embed code in the following way. <<label=code-chunk-1>>= print("Hello World!") @ Although this particular example may not be executed in lpEdit because it is not a valid L A T E X or reST document, it illustrates that code, in this case just a print statement, is included by placing it between "« txt »=" and "@", where txt is any arbitrary string, preferably something informative.
Note that under Sweave txt is a place where options may be passed. Refer to the official documentation for more comprehensive examples. Documents written in L A T E X, or reST are written as they normally would be although now there is a way to execute embedded code within the document. There is no limit to the number of code chunks and lpEdit will execute them in sequential order, preserving the variable space. The building step is where code chunks are executed and output gathered. There is one thing to keep in mind when working with projects, and that is the idea of scope. Suppose, there are two documents document1. rst and document2. rst. If we build document1. rst then document2. rst, the results from document1. rst will be preserved, which is convenient when there are code chunks that take significant time to run.
Involved analyses Analyses can take the form of long complicated pipelines, that may not reasonably be reproduced at the click of a button. This may happen if, for example, a database needs to be populated before an analysis can be carried out or perhaps there is a hardware constraint, such as the requirement of a high-performance computing infrastructure. In these cases, lpEdit or another documentation software may still be used to document details that would not normally be present in the methods section of a published manuscript. For analyses that are accompanied by substantial code and/or data, we provide the keyword INCLUDE which simply tells lpEdit that a given file is part of the current project. For example, files may be included in a *. nw or *. rnw document by %INCLUDE MyFunctions.
py, MyData. csv where the INCLUDE statement is preceded by a comment indicator. For reST documents ". " is used. At build time symbolic links are created. For a reST document, INCLUDE is preceded by the comment indicator. With increasingly involved analyses, the readability of documentation should not deteriorate and to this end prose may be simplified by including code and data as links. Other than INCLUDE and the syntax to embed code, reST and L A T E X, documents are written as they normally would be, which has the important benefit of minimizing the learning burden.
Analysis database and environment Because P. brassicae is a species without a reference genome, the assembled isotigs must be compared to species that have functional descriptions. In order to make time-efficient comparisons we first created a database using PostgreSQL http://postgresql. org (version 9. 1. 9). The database contained gene, accession, taxon, and functional ontology information all of which is available through the National Center for Biotechnology Infomation (NCBI) FTP site http://www. ncbi. nlm. nih. gov/Ftp. The database is detailed in Figure  2. The interaction with tables in the database was simplified through the use of the object relational mapper available as part of the python package SQLAlchemy http: //www. sqlalchemy. org. The schema figure was generated using the Python package sqlalchemy_schemadisplay https://pypi.
Functional characterization of the transcriptome For each isotig, functional annotations were found by using the Basic Local Alignment Search Tool (BLAST) [Altschul90] via NCBI's BLAST+ command line interface  [Camacho09]. Specifically, each isotig was locally aligned to every sequence in the Swiss-Prot database [UniProtConsortium12] then using our local database, accession names were mapped to gene names and corresponding functional annotations were gathered. The handling of sequence data was done using the classes and functions provided by BioPython  [Cock09]. Of the nearly 17,000 isotigs that were examined, 11,846 were considered hits (E-value ≤ 0. 04). The isotigs were then mapped to 6901 unique genes. The appropriate Gene Ontology  [Ashburner00]  annotations were then mapped back to the isotigs.
Conclusions and future work The RNA-seq example demonstrates that involved analyses may be well-documented in a way that is interesting for those who understand the technical details of the analysis and those who do not. In the future, more languages, even compiled ones, may be integrated into the project, which is feasible because lpEdit uses the Python package subprocess to make arbitrary system calls. It is not our intention for lpEdit to evolve to be a replacement for already established tools, like Org-mode. Rather, it is meant as a simple tool to help newcomers with programming and statistics. With the API version of lpEdit there remains the possibility that it may be adapted as a plug-in or extension to existing text editors. Given that the target user-base for lpEdit are those with limited computing background, there are a number of power-user features left out of the current version for the sake of a nearly 'push button approach'.
Acknowledgments We would like to thank  Eric Pante  and  Michel Baguette  for helpful comments and discussion. The research carried out here was partially supported by the  Duke Cancer Institute (DCI). Additional support for this work was provided by the  Agence Nationale de la Recherche  ( ANR; France) MOBIGEN  [ ANR-09-PEXT-003 ]. The opinions, findings and recommendations expressed in this work are those of the authors and do not necessarily reflect the views of the DCI, CNRS or other affiliated organizations.
General Concept The STT detector consists of 4636 tubes arranged in 27 layers (see sec. 2. 3). The maximum counting  To fulfill all these requirements, the proposed straw tube read-out organization comprises 3 stages: 1. Analog Front End Electronics (FEE) cards hosting, depending on radial distance from the beam, 36-80 channels. FEE is composed of preamplifier, amplifier with analog signal shaping and discriminator unit with differential output. 2. Digital Board (DB) for time and amplitude (or charge) measurements, local logic resources for noise suppression, fast hit detection, memory buffer for hit storage, serial Gbit optical links for the data transmission and slow control. The data from the DCBs will be transferred via fast optical links to Compute Nodes for the on-line track reconstruction and subsequently, after merging with the information from other PANDA detector systems, for the event selection.
It will be possible to perform some local correlations on the data inside the single DB to suppress noise and reduce the amount of data sent from the board. The PANDA data acquisition and filtering systems will implement a trigger-less architecture. Instead of having a hardware trigger signal, which indicates the occurrence of a valid event, each DB will receive a precise clock signal distributed centrally from a single source: the Syncronization Of Data Acquisition (SODA). The DB boards will continuously monitor the detector channels and will generate data packets whenever the number of the input signals exceeds programmed thresholds. These data will be tagged with time-stamps obtained from the SODA. The data acquisition system will profit from the structured running mode of the HESR operation.
Periods of 2µs with antiproton interactions will be interleaved with periods of 400 ns of idle time. The information on the accelerator activity will be distributed to DCBs via the Clock and Timing Distribution System. The data recorded during the interaction intervals will be grouped together in DCB to form a burst which will be then uniquely tagged. Grouping of data from many bursts into predefined epoques (e. g. 500 µs) inside DB is also considered in order to reduce network traffic. Data from all PANDA detectors tagged with the same burst identification number will be grouped together and will be made accessible to filtering algorithms implemented in the Compute Nodes (CN). Decisions produced by these algorithms will thus be based on the complete detector data with full granularity.
Thus a 5 byte word is generated for each hit on the DB. Assuming 800 khits per second a data rate of 4 MB/s is generated in each TDC channel. This results in a 784 MB/s data rate from one DB. This data rate can be handled by four 3. 2 GBit/s optical serial links. Twenty four 196-channel DBs will be necessary for the full CT read-out. It seems reasonable to merge the data from the straw layers belonging to one STT-sector and sent them to a common DCB. Such a layout can be more favorable for a cluster search but is not mandatory since the DB have also features of the Detector Concentrator (grouping of bursts in epoques). A board which could be considered as a prototype of the PANDA DCB has also been developed by the HADES DAQ group (optical hub module) and is currently installed in the Krakow straw tube test set-up.
Detector Control System The Detector Control System (DCS) of the STT is one branch of the general PANDA Experiment Control System (ECS). The control system has to continuously collect the actual parameters from the supply and electronic readout systems and compare them with their set values and predefined tolerances. In addition, parameters of the detector environment, like for instance temperatures and humidities at several locations must be monitored. In case of certain deviations a specific alarm mes- sage should be generated to inform the detector operators. Some of the data must be stored on disk and added to the experiment event data for possible offline corrections of the detector measurements. The DCS system will manage the database with the mapping of the physical channel source (voltage, current, temperature, aso.
The individual time offsets from the electronic readout system have to be corrected only once. In the following, algorithms for the isochrone calibration are described, which can be easily adapted for the PANDA-STT. The methods presented were checked with experimental data from different straw test systems. In particular the COSY-STT detector is considered as an ideal test system for the whole calibration method and the performance results can be extrapolated to the PANDA-STT due to the similar technique of close-packed straw layer-modules. The test systems and measurements are described in detail in Sec. 5. ber of events outside the drift time window gives an estimate of the random, constant noise level over time range (see Fig. 4. 1)  [1]. For each tube, the parameters of the drift time distribution are derived from a fit performed with the following empirical function  [1, 2, 3] :.
6. 14): • specific simulation and reconstruction classes for the detectors; • I/O Manager based on ROOT TFolder, TTree and TChain; • geometry readers: ASCII and ROOT (also CAD files converted to ROOT are usable); • track follower (GEANE 1  [20] ); 1. GEANE is a track follower distributed within the geant3 package. It is written in FORTRAN and a C++ interface has been developed in FairRoot and is used also in the extrapolation step of the Kalman fit. • event display based on TEve; • database for geometry and parameters handling.
Acknowledgments We acknowledge financial support from the  Bundesministerium für Bildung und Forschung (bmbf) , the  Deutsche Forschungsgemeinschaft (DFG) , the  Forschungszentrum Jülich GmbH , the  University of Groningen, Netherlands , the  Gesellschaft für Schwerionenforschung mbH (GSI) ,  Darmstadt , the  Helmholtz-Gemeinschaft Deutscher Forschungszentren (HGF) , the  Schweizerischer Nationalfonds zur Förderung der wissenschaftlichen Forschung (SNF), the Russian funding agency "State Corporation for Atomic Energy Rosatom" , the  CNRS/IN2P3  and the  Université Paris-sud, the British funding agency "Science and Technology Facilities Council" (STFC) , the  Instituto Nazionale di Fisica Nucleare (INFN) , the  Swedish Research Council , the  Polish Ministry of Science and Higher Education , the  European Community FP6 FAIR Design Study: DIRAC secondary-Beams , contract number  515873 , the European Community  FP7 Integrated Infrastructure Initiative :  HadronPhysics2 , contract number  227431 , the  INTAS , and the  Deutscher Akademischer Austauschdienst (DAAD).
-We show that DynaSoRe is especially efficient compared to our baselines when assuming a memory budget, which is an important practical goal due to cost of rack space in modern data centers. Roadmap. The remainder of this paper is structured as follows. We specify in Section 2 the model and requirements of an efficient in-memory store. We present the design of DynaSoRe in Section 3 and evaluate it in Section 4. We discuss related work in Section 5 and present our conclusions in Section 6.
DynaSoRe spans multiple servers, as the views of all users cannot fit into the memory of a single server. Distributing the workload across servers is critical for scalability. Our applications reside in data centers. Servers inside a data center are typically organized in a three-level tree of switches, which has a core tier at the root of the tree, an intermediate tier, and an edge tier at the leaves of the tree  [1, 7, 8]  as shown in Figure  1. The core tier consists of the top-level switch (ST), which connects multiple intermediate switches. The intermediate tier consists of intermediate switches (SI) and each of them connects a subset of racks. The edge tier consists of racks and each rack is formed by of a set of servers connected by a rack switch (SR). The network devices, i. e. , switches at different levels of the tree architecture, only forward network traffic.
The views of users are maintained in the servers (S), connected directly to rack switches. The servers have a bounded memory capacity and we established its capacity by the number of views it can host. We use b to denote the number of bytes we use for a view. Brokers (B) are also servers connecting directly to rack switches and they are in charge of reading and writing views on the different servers of the data center. Note that DynaSoRe could be deployed on several data centers by adding a virtual switch representing communications between data centers, which would then be minimized by DynaSoRe. In practice, Web companies such as Facebook do not have applications deployed across data centers. Instead, they replicate the content of each data center through a master/slave mechanism  [13]. Thus in this work we focus on the case of a single data center.
Missing updates because of crashes, however, is highly undesirable, so we guarantee that updates to the system are durable. To do this, we rely upon a persistent store that works independently of DynaSoRe. Updates to the data are persisted before they are written to DynaSoRe to guarantee that they can be recovered in the presence of faulty DynaSoRe servers. Since we replicate some views in our system, copies of data might be readily available even if a server crashes. In the case of a single replica, we need to fetch data from the persistent store to build it. In both cases, single or multiple replicas, crashes additionally require live servers to dynamically adjust the number of replicas of views to the new configuration.
API DynaSoRe is an in-memory store used in conjunction with a persistent store. The API of DynaSoRe matches the one used by Facebook for memcache  [13]. It consists of a read request that fetches data from the in-memory store, and a write request that updates the data in the memory store using the persistent store. Consequently, DynaSoRe can be used as a drop-in replacement of memcache to cache user views and generate social feeds. Read(u, L): u is a user id and L is a list of user ids to read from. For each id u ′ in L, it returns view(u ′ ). Write(u): u is a user id. It updates view(u) by fetching the new version from the persistent store.
In this case, DynaSoRe has no incentive to add data to the new servers since no requests will originate from there. When adding a new branch to the data center, we consequently need to move some views and proxies onto the new servers to bootstrap it. This procedure is, however, not detailed in this paper. Removing servers on the other hand requires the views hosted by the servers to be relocated. Before removing a server, the views that have no other replica should be moved to a near server. The views that exist on multiple servers can simply be deleted as DynaSoRe will recreate them if needed. Managing the social network As described earlier, DynaSoRe does not maintain the social network, and instead receives the list of users from which data need to be retrieved when executing a read request.
Baseline Random In-memory storage systems, such as Memcached and Redis, rely on hash functions to randomly assign data to servers. This configuration is static in the sense that it is not affected by the request traffic. For this scheme, the proxies of a user are deployed on the broker located in the same rack in which the user view is located. This is the simplest baseline we compare against, as it ignores the topology of the data center, the structure of the social graph, and does not leverage free memory through replication. METIS Graph data can be statically partitioned across servers using graph partitioning. This leverages the clustering properties of social graphs and increases the probability that social friends are assigned to the same sever. We rely on the METIS library to generate partitions, and randomly assign each of them to a server.