dataesr/scientific-paragraphs-categorization huggingface dataset:
3C-seq experimental procedures and data analysis workflow. Formaldehyde cross-linked chromatin is digested with a six-cutter restriction enzyme and ligated under dilute conditions. After de-cross-linking, DNA is digested with a four-cutter enzyme and again ligated under dilute conditions to create small circular fragments representing individual ligation events. Inverse PCR using viewpoint-specific primers containing Illumina sequencing adapters is used to generate a viewpoint-specific 3C-seq library. After high-throughput sequencing, reads are trimmed and mapped to the reference genome, after which they are loaded into the r3Cseq software. method by choosing a power-law reference distribution with an exponent of a = À1. 35 and an n 0 = 1 million offset. The normalization functions are used to transform the read counts from all samples into normalized reads per million (RPM).
The reverse-cumulative fitted values of the power-law normalization method described in this study were implemented as an improvement over the simple RPM normalization that is most often used in count data analysis to remove bias because of unequal sequencing depth. In this study, we demonstrated that this method performs better than those not applying normalization or those applying the simple RPM normalization. However, normalization techniques are still immature for most types of count data generated by next-generation sequencing technologies. Although indepth development of such methods for 3C-seq is beyond the scope of this study, the r3Cseq package can easily be expanded with new normalization methods as they become available.
P-values can then be assigned to each Z-score and transformed into a q-value for false discovery rate (FDR) analysis using the qvalue package from Bioconductor  (20) , with a 0. 05 FDR level (which can be changed by the user) and using bootstrap as the selected method [qvalue(interactions. p. values, fdr. level = 0. 05, pi0. method = 'bootstrap')]. The method we applied to identify interactions in cis has successfully been used for 5C data analysis  (26) , and a similar method has been used for the detection of interactions in 4C data analysis  (8, 15). Figure  4  shows the analysis results of two data sets using the mouse Myb promoter as viewpoint, showing that this method successfully identifies 3C-seq interaction regions in cis. These experiments were performed under two experimental conditions: (i) fetal liver (FL) erythrocytes expressing high levels of Myb and (ii) fetal brain (FB) cells expressing low levels of Myb  (24).
Analysis of 3C-seq replicate experiments To investigate 3C-seq data reproducibility among replicates, we performed additional 3C-seq experiments using the Myb promoter as the viewpoint in FL erythrocytes and FB cells (3C-seq data are available at http://r3cseq. genereg. net). When considering the entire data set, including signals with low read counts, interacting regions (!1 RPM, calculated from restriction fragmentbased, 5-and 10-kb window-based sizes) across the whole genome in general show low reproducibility, as low intensity signals in 3C-based methods are likely the product of random contacts between DNA fragments (9,15). Remarkably, reproducibility (defined as the percentage of detected interactions present in both replicates against all detected interactions) is extremely low in trans (<1% of detected interactions were reproducible, Supplementary Table  S1 ); interactions found in trans almost always exhibit very low read counts and are, therefore, likely to be caused by random ligation events.
To determine significant interactions among replicates, we first performed r3Cseq data analysis for each individual sample. We then combined the detected interactions across biological replicates, providing 'union' and 'intersection' options for this purpose. The union method combines all significant interactions across samples, whereas the intersection method takes only significant interactions present across all samples into account. Read counts and RPM values across samples are averaged to obtain representative values for the final list of detected interactions. The assigned P-values across samples are combined using Fisher's combined probability test as implemented in R (  29 ), and q-values are calculated using the qvalue package with FDR level 0. 05 (which can be changed by the user), using bootstrap as the selected method.
Functionality available in r3Cseq The r3Cseq package was built on and extends the functionality of the Bioconductor packages BSgenome, GenomicRanges, Rsamtools and rtracklayer  (30). It contains functions for the following groups of tasks: Importing aligned reads r3Cseq can read BAM files and converts this file and its related information into an object-oriented core class for the r3Cseq package. r3Cseq can also load aligned reads from the GRanges object generated by the GenomicRanges package in R. A detailed description of input parameters can be found in the r3Cseq software documentation.
Data processing After class initialization, processing functions getRawReads, getBatchRawReads, getReadCountPerRes-trictionFragment, getReadCountPerWindow, getBatch-ReadCountPerRestrictionFragment and getBatchRead-CountPerWindow are performed. The getRawReads function retrieves aligned reads from BAM files and transforms them to GRanges objects that can be stored in an r3Cseq object, whereas getRawReadsInBatch processes the data in batch mode and stores the aligned reads GRanges in R files (. rdata). To count the number of reads used for further analysis, r3Cseq provides two ways to count the number of reads per region; (i) count the number of reads per restriction fragment (using the getReadCountPerRestrictionFragment function) and (ii) count the number of reads per non-overlapping defined window (using the getReadCountPerWindow function), whereas the getBatchReadCountPerRestrictionFragment and getBatchReadCountPerWindow functions perform the same analysis for replicate data sets.
These functions provide options for counting all reads or only the informative reads (i. e. those that are mapped exactly adjacent to restriction sites). The latter method will exclude reads generated from inappropriately digested chromatin by the restriction enzyme and randomly sequenced DNA fragments (see r3Cseq software documentation). Data normalization and the identification of interacting regions calculateRPM and calculateBatchRPM are functions that normalize the number of RPM for each restriction fragment or window. Users can select different RPM calculation methods, as described in the 'Materials and Methods' section, by defining the normalization method parameters (see r3Cseq software documentation). After normalization, the getInteractions and getBatchInteractions functions are used to calculate Z-scores, estimate P-values and assign q-values to detect significant interactions, respectively.
Window-based approaches will reduce the detected resolution (depending on the selected size range), but they will improve reproducibility, which can be convenient when long-distance cis-interactions or trans-interactions are of particular interest. The r3Cseq interaction detection method provides a better interaction score (q-values) than was used in our previous analysis  (13). This analysis did not include background signal correction, which resulted in assigning overly significant interaction scores to low interaction signals (Supplementary Figure  S6 ). Although our previous method detected $4000 contacts, our current methods detected $600 significant contacts in FL (q 0. 05) using the same cut-off. To show how the r3Cseq package can also be used to analyze data sets generated by other laboratories, we used r3Cseq to identify long-range interactions at the mouse b-globin locus using data obtained from a recent 4C-seq study (9).
DISCUSSION We developed the R/Bioconductor package r3Cseq, and in this study, we describe its functionality and demonstrate its use and power for the identification of chromatin interaction regions generated by 3C-seq experiments. The software provides functionality for pre-processing, analyzing and visualizing interaction regions with any given viewpoint of interest. The package can process BAM files, which are generated by mapping software, such as Eland and Bowtie. We provided r3Cseq with functions to support the BAM file format, as it is a compressed binary file  (21) , which allows users to perform 3C-seq data analysis on a regular personal computer (CPU $2 GHZ with $4 GB of random access memory, Supplementary Table  S3 ). However, it is recommended, when possible, to use more powerful computer hardware when performing 3C-seq data analysis on replicate data sets, as this will require more random access memory and storage space.
However, the window-based strategy is helpful for detecting large interaction domains, which can significantly promote the identification of novel interaction regions. Larger window sizes show a higher reproducibility (Supplementary Table  S1 ), especially at large distances (>500 kb) from the viewpoint and may also be preferred for replicate data analysis. To facilitate both fragment-based and window-based analysis, r3Cseq enables users to easily switch between both types of analysis and promptly provides these results as text files and interpretable plots (see http://r3cseq. genereg. net for more details). r3Cseq also supports data analysis of replicate 3C-seq experiments, as it can combine detected interactions across biological replicates to produce a final list of significant interactions.
Users can select a union or an intersection operation to obtain the final list of interactions, as described in the 'Materials and Methods' section. Both these options are useful, although only the intersection method allows for the detection of truly consistent interaction regions across samples. In summary, we have shown that r3Cseq can remove signals originating from random ligation events and provide data normalization, the accurate detection and powerful visualization of both existing and novel significant interaction regions present across multiple biological replicates.
AVAILABILITY AND IMPLEMENTATION The r3Cseq package has been implemented in R and is available as part of the Bioconductor (www. bioconductor. org) distribution, as version 2. 9. As such, it also gives users and software developers the opportunity to extend and customize the pipeline to their needs. We have developed a website to host the r3Cseq package, which can be found at http://r3cseq. genereg. net. The website provides downloadable data sets presented in this article and the current version of the r3Cseq package (version 1. 5. 0). The website also describes the R code examples for the 3C-seq data analysis pipeline. Additional guidelines and typical workflows can be found in the package's vignette in R. The ChIP-seq and 3C-seq data used here were deposited in the sequence read archive (SRA) database.
A number of techniques are available for deriving spatial filters so as to capture distinct signal sources in the data (  [1]  and see Discussion). Spatial filtering can be highly effective in detecting signal features in multi-sensor data, even at the single-trial level  [2]. In signal processing a common technique for detecting the presence of a known signal, s, in noisy data, is to simply use s itself as a filter. Typically a different label, h, is used to refer to s when treated as a filter. In this context, h (= s) is referred to as a "matched filter" because the filter is matched to (identical to) the shape of the known signal that we are trying to detect, but that might be hidden in noise. Filtering is performed by simply correlating the filter, h, with a segment of noisy data, x. A matched filter is the optimal filter for revealing the presence of a known signal (i.
The objective function is simply a computational representation of the data analysis question, which might, for example, concern the relationship between the sensor data and a behavioral variable such as reaction-time. The MatLab toolbox that we provide includes standard objective functions, but also allows for the objective function to be user-defined. This makes the method highly versatile (see Discussion). For some purposes one might want to compute a single spatial filter and apply the same filter at all time points in the trial epoch, and this is also supported by our toolbox. By doing so it is possible to examine the time course of a specific stationary topography. For instance, if one was interested in the time course of the P300 component of the ERP in EEG data, one could compute a single topography over a time window centered on the peak of the P300.
MEG recordings and pre-processing While subjects performed the cognitive tasks, we continuously recorded brain activity using a 306-channel whole-head magnetoencephalography system (Elekta Neuromag®) with 102 gradiometers and 102 pairs of orthogonally oriented planar gradiometers. The subject's head position was measured at the beginning of each run and this information was used during data pre-processing to compensate for differences in head position between runs. Electro-oculogram (EOG) and electrocardiogram (ECG) were recorded simultaneously for offline rejection of eye movements and cardiac artifacts. Signal Space Separation, head movement compensation, and interpolation of bad channels were applied using the MaxFilter Software (Elekta Neuromag®). Epoching, trial rejection, and baseline correction were then applied using the Fieldtrip software package (http://fieldtrip.
Performance Since different software was used for the EMS filtering and SVM analyses (MatLab and Python, respectively), performance comparisons can only be considered descriptive and approximate. All analyses were tested on a Dell Precision T3400 PC (64-bit dual Intel Core-2 Duo CPU E8500 3. 16 GHz, 6144 KB cache) running Ubuntu Linux 10. 04. All processes were single-threaded, so only one processor was used. In all cases, results are reported for the lag-9 versus control conditions (classification or contrast between means) with ~30 to 75 trials per condition per subject. The mean processing time of the SVM analysis with five-fold cross validation was ~1 to 2 minutes per subject (avg 1. 44 seconds per trial, +/-0. 37s stdev). The mean running time of the SVM analysis with leave-one-out cross validation was ~6 to 8 minutes per subject (avg 6.
For the analysis of stimulus-evoked responses, it is convenient for both the topography of the weight vector, and the resulting time course to be directly interpretable in terms of the particular experimental effect under investigation. Thus we have not incorporated the noise covariance into EMS filtering by default (although the option is available in our MatLab toolbox). Given that EMS filtering yields a significant improvement in signal quality over the most commonly-used techniques (single-sensor and ROI), foregoing an additional margin of separability at the single-trial level is a reasonable compromise in favor of readily interpretable weight vectors. Computing the noise covariance matrix also lengthens the computation time, especially in the context of a LOO cross validation.
We used MEG data to illustrate the method, but the method can be applied to any kind of multivariate time-series data, such as slow event-related functional magnetic resonance imaging (fMRI), or data from domains outside of neuroimaging. We have demonstrated the effectiveness of the method in dramatically improving the quality of single trial data vis-à-vis a given experimental effect, and specifically in revealing the time course of the psychological refractory period. An implementation of the method is freely available as a MatLab toolbox at http:// bitbucket. org/emsf/emsf_matlab.
Acknowledgements AS was supported by a  Marie Curie post-doctoral fellowship  from the  European Commission  ( FP7-PEOPLE-2009-IIF , project number  252665 ). SM was supported by a grant from the  Human Frontiers Science Program. Additional support came from a  European Research Council  senior grant " NeuroConsc " to SD. The Neurospin MEG facility was sponsored by grants from  INSERM, CEA, FRM , the  Bettencourt-Schueller Foundation , and the  Région île-de-France. Special thanks to  Bertrand Thirion  for assistance with the mathematical derivation in Appendix B, and for technical guidance. Special thanks to  Lucas Parra  for technical guidance and comments on an earlier draft of the manuscript. Special thanks also to  Gaël Varoquaux  for technical guidance.
A first series of tests was performed to examine the influence of the technique used to measure residual strength on safety factors. This is a key issue for a test program, so both straight tensile pull and cycling to failure on the sheave at increasing load levels were examined. An example of the results is shown in Figure  9. Residual strength is plotted versus the number of cycles N normalized by the cycles to failure N f. These results are very important as they clearly show that it is dangerous to rely on residual tensile strengths to evaluate the influence of cyclic bending damage, as they will provide an over-optimistic value of residual strength. Cyclic bending on the sheave at increasing load was therefore used in all subsequent tests to evaluate remaining lifetime. The determination of data such as those in Figure  9  can then be used in an empirical life prediction model.
In this paper, we describe our experiences in implementing and applying Coarray Fortran (CAF) for the development of data-intensive applications in the domain of Oil and Gas exploration. The successful porting of reverse time migration (RTM), a data-intensive algorithm and one of the largest uses of computational resources in seismic exploration, is described, and results are presented demonstrating that the CAF implementation provides comparable performance to an equivalent MPI version. We then discuss further language extensions for supporting scalable parallel I/O operating on the massive data sets that are typical of applications used in seismic exploration.
I. INTRODUCTION New programming models are needed to meet the challenges posed by ever-increasing data processing requirements. They must provide mechanisms for accessing and processing large-scale data sets. Moreover, we believe they must be simple and intuitive to use, particularly for non-expert programmers. A large fraction of the codes that are used in the Oil and Gas industry are written in Fortran, and it is important that new programming models can work interoperably with these existing codes. In the latest Fortran 2008 standard, new features were added which can potentially transform Fortran into an effective programming language for developing data-intensive, parallel applications  [1]. These features, collectively, are based on Co-Array Fortran (CAF)  [2] , a Fortran extension proposed in the 1990s that incorporated a Partitioned Global Address Space (PGAS) programming model into the language.
However, there has been an unfortunate lack of available implementations for these new language features, which therefore prompted us to develop our own implementation in the opensource OpenUH compiler. Our initial evaluations revealed that additional support for handling large data sets is essential if coarrays are to be used on very large scale comptuations. As the size of such data sets, and the computational capacity of platforms, continues to grow there is an increasing gap between this and the rate at which data is input and output from and to disk. In order to accommodate it, we believe that there is a need for parallel I/O features in Fortran which complement the existing parallelprocessing features provided by the coarray model. In addition to our work to identify suitable features, we are also exploring approaches that would allow Fortran applications to share data in memory in order to avoid the slowdown implied by disk storage.
This paper is organized as follows. In section II we provide an overview of the CAF features that were adopted in Fortran 2008. We then describe our implementation of these features in section III. We present our experience in porting an existing reverse time migration (RTM) application, widely used by Oil and Gas exploration teams for seismic images processing, to CAF in section IV. In section V, we discuss our thoughts on extending the language for enabling scalable, parallel I/O. We present related work in section VI, and close with our conclusions in section VII.
II. COARRAY FORTRAN The CAF extension in Fortran 2008 adds new features to the language for parallel programming. It follows the Single-Program, Multiple-Data (SPMD) programming model, where multiple copies of the program, termed images in CAF, execute asynchronously and have local data objects that are globally accessible. CAF may be implemented on top of shared memory systems and distributed memory systems. The programmer may access memory of any remote image without the explicit involvement of that image, as if the data is "distributed" across all the images and there is a single, global address space (hence, it is often described as a language implementation of the PGAS model). The array syntax of Fortran is extended with a trailing subscript notation in square brackets, called cosubscripts.
A list of cosubscripts is used when the programmer intends to access the address space of another image, where an image index may range from 1 to the total number of images that are executing. Figure  1  shows the logically shared but partitioned memory view that characterizes PGAS programming models. Specifying A Coarray(n) without cosubscripts (square brackets) accesses only the local coarray. Since the remote memory access is explicit, it provides a clearly visible marker for potentially expensive communication operations in the code. This differentiates PGAS implementations like CAF from shared memory models which do not distinguish between local and remote data. Only objects declared as coarrays (i. e. one or more codimensions) can be accessed from a different image. Coarrays can be global/static or dynamically allocated, but in any case they must exist on all images.
This has many consequences described in the standard, among them being that the allocation of an allocatable coarray is a collective operation with an implicit barrier. Coarrays may be declared with multiple codimensions, in which case the number of images are logically organized into a multi-dimensional grid. CAF provides both a global barrier synchronization statement (sync all) and a partial barrier synchronization statement (sync images) which may be used to to synchronize with a specified list of images. Critical sections, locks, and atomic operations are also part of the language. Additionally, CAF includes several intrinsic functions for image inquiry such as returning the image index of the executing process (this image), the total number of running images (num images), and the image index holding a coarray with specified cosubscripts (image index).
III. CAF IMPLEMENTATION We have implemented support for coarrays in OpenUH  [3][4] , an open-source research compiler based on the Open64 compiler suite. CAF support in OpenUH comprises three areas: (1) an extended front-end that accepts the coarray syntax and related intrinsic functions/subroutines, (2) back-end translation, optimization, and code generation, and (3) a portable runtime library that can be deployed on a variety of HPC platforms. Additionally, we have implemented a number of (currently) non-standard features that would be beneficial in developing more scalable parallel codes.
A. OpenUH Compiler We have used OpenUH to support a range of research activities in the area of programming model research. OpenUH, depicted in Fig. 2 , includes support for C/C++ and Fortran, inter-procedural analysis, a comprehensive set of state-ofthe-art loop nest optimizations, an SSA-based global scalar optimizer, and code generators for a variety of target platforms. OpenUH can emit source code from an optimized intermediate representation and also selectively instrument the lowered intermediate code for low-overhead performance profiling.