dataesr/scientific-paragraphs-categorization huggingface dataset:
The CLI and the API The first AMI client was written in Java. It was not convenient because it was too dependent on server code changes, and clients had to update too frequently. In 2004, it was decided to move to a classic Simple Object Access Protocol (SOAP) "Web Service" approach [11]. This was actually an approach strongly supported by the EGEE Grid middleware at the time, and a workshop was held at CERN in June 2004 with contributions from the four LHC experiments  [12]. The web service approach had many advantages in theory, since users can use utilities to build a client in any language they wish. However in the ATLAS environment this feature was not exploited. ATLAS allows only a Python client. The principle of a client which is as light as possible was retained, thus minimising client changes.
The latest version of our client (pyAMI version 5) has dropped the SOAP web service in favour of HTTPS. The syntax has been changed, and the client has been split into modules. There is one basic client, and the other modules are experiment specific. This change of AMI was not backward compatible. It was rendered necessary when experiments other than ATLAS adopted AMI. In the same way, a new C/C++ client based on HTTPS and OpenSSL is currently under development.
Filling the Databases The first version of the AMI dataset bookkeeping used a "push" method of filling the databases. Executing jobs generated commands for the AMI clients, and the users executed the commands at remote sites. This did not work very well for several reasons. Once the ATLAS production system took over control of user jobs on the data grid, the "push" mechanism did not work at all, and the survival of AMI was in doubt. In 2006 an ATLAS internal review decided that AMI should be adopted for physics metadata bookkeeping, and accorded AMI reading rights on the necessary databases. AMI switched to a "pull" mode of filling the databases for ATLAS with two dedicated task servers. Other users of AMI are free to push data into the catalogues, and client functions are provided for this.
Hardware Evolution It was clear from the beginning of the project that MySQL was not a longstanding choice, for both technical and political reasons. Oracle is a much better choice for a large database, and it was available at both CERN and the French Tier 1 site of the Centre de Calcul IN2P3 (CC-IN2P3), Lyon. AMI had rapidly outgrown the single web and database server at Grenoble. In 2004, it was time to change. After consultation with the ATLAS database coordination, ATLAS France and the CC-IN2P3, the move to Lyon became possible. Two servers were purchased and the database backend was extended to work with Oracle using the Oracle cluster. Part of the AMI application still runs on MySQL, but it was moved to the CC-IN2P3 managed cluster. In 2008, an Oracle cluster of 4 nodes (dual processor, quad-core with 32GB RAM) was put in place at CC-IN2P3 for hosting the AMI database as well as the ATLAS database and two new web servers were bought and installed at CC-IN2P3 to cope with the high activity and load periods.
At the beginning of AMI, the database size was only a few gigabytes (~12 GB) but with an activity continuously growing, the database size reached in 2015 more than 450 GB, see figure  1. AMI is very critical for ATLAS. The CC-IN2P3 implemented a data guard solution in 2014. Data Guard is the name for Oracle's standby database solution, used for disaster recovery and high availability. In case of crash or maintenance of the production database at CC-IN2P3, all users will be redirected to the standby CC-IN2P3 AMI database with a minimal interruption of service. In production, this solution allows the service downtime to be minimized to a few minutes during a major ORACLE upgrade. In 2015, Oracle servers come to the end of their warranty and should be replaced during the summer. This replacement will be a good opportunity for using ORACLE Data Guard.
Tools During the period of AMI development the size of the developer team at Grenoble has doubled, and there were several short term collaborations from ATLAS colleagues based in other research institutes. Work methods had to be made more formal. Development collaboration needs a shared code repository and a tool to manage builds and integration. A bug tracker is also essential. A fairly standard set of tools was chosen: SVN for code versioning, Jenkins  [14]  for building and deployment, and Redmine [15] for tracking bugs. Redmine is about to be abandoned because ATLAS has decided to adopt JIRA [16] for bug tracking. It is planned to migrate to GitHub, which seems more suitable for the future developments, see section 8. Jenkins is an extremely satisfactory tool, and, in the near future, the only change will be to move from  ANT [17]  to  Maven [18]  for the Jenkins scripts.
Outlook AMI is now a well-established and mature application, with about 2000 active users from the ATLAS collaboration. Since 2013 another experiment, nEDM [19], is using AMI for run bookkeeping. Agreement was reached with the SuperNEMO [20] collaboration for use of AMI, and at least one other experiment has expressed interest. This expansion of AMI prompted some refactorisation of the core code, since the frontier with experiment specific code had become slightly fuzzy over time. The new preliminary core code is opensource and is available on GitHub  [21]. We will work on making it much easier for experiments to set up AMI databases with a minimum of support from the core team.
International Conference on Computing in High Energy and Nuclear Physics (CHEP2015) IOP Publishing Journal of Physics: Conference Series 664 (2015) 042040 doi:10. 1088/1742-6596/664/4/042040 By an ATLAS request, since June 2005, a partial site failover was implemented at CERN, so that AMI users can switch to CERN in case of a CC-IN2P3 failure. An AMI Tomcat server is available at CERN with its own database replicated in real time from CC-IN2P3. Replication was done via Oracle Streams technology configured for one directional replica set read-only on the destination database at CERN. After the official announcement by Oracle in 2013 regarding the deprecation of the stream technology, it was decided jointly with the CC-IN2P3, AMI and the CERN IT to replace the stream technology by Oracle Golden Gate software.
21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015) IOP Publishing Journal of Physics: Conference Series 664 (2015) 042040 doi:10. 1088/1742-6596/664/4/042040 By an ATLAS request, since June 2005, a partial site failover was implemented at CERN, so that AMI users can switch to CERN in case of a CC-IN2P3 failure. An AMI Tomcat server is available at CERN with its own database replicated in real time from CC-IN2P3. Replication was done via Oracle Streams technology configured for one directional replica set read-only on the destination database at CERN. After the official announcement by Oracle in 2013 regarding the deprecation of the stream technology, it was decided jointly with the CC-IN2P3, AMI and the CERN IT to replace the stream technology by Oracle Golden Gate software.
The DNA methylation percentage of a given CpG is calculated by dividing the surface area of the peak representing the methylated fragment by the total surface area of the peaks of both the methylated and unmethylated fragment. For an effective application of EpiTYPER, a streamlined process of assay design, assay validation, bisulfite conversion, data generation, and data processing is necessary. These steps involve switching between lab and computer work. We will provide a step-wise description of all these steps and also include various tips. These steps are an addition to the protocol provided for EpiTYPER by Agena Biosciences.
Materials and Equipment Prerequisites Agena Bioscience Inc. provides several automated pipelines for the measurement of nucleotides (SNPs, gene-expression etc. ) on the MassARRAY R system. If a MassARRAY system is already available EpiTYPER only requires an additional proprietary software suite, and some additional fine-tuning of the MALDI-TOF mass spectrometer by a specialized service engineer to function. If a MassARRAY system is not yet available EpiTYPER will require a significant investment in hardware, including a compatible provided MALDI-TOF mass spectrometer, an automated liquid handling system to transfer samples from 384 plates to the SpectroCHIP II and a dedicated server to process and store the data generated. 7. MassCLEAVE ™ , T Cleavage Kit (Agena Bioscience Inc).
Software Several of the sections in this protocol use R (R Development Core Team, 2011), a free-to-use open-source software for data analysis. Several packages were designed in R with useful tools to evaluate assay designs prior to the measurement on the MassArray system, assess coverage and bisulfite conversion rate and pre-process EpiTYPER R data  (RSeqMeth, 2008; MassArray, 2009). One of these, RSeqMeth, requires an older version of R. We added the relevant functions of this package for download on Github, so that they are usable with any R version above 2. 13. R scripts can be opened, read and edited using "script editors": Tinn-R or RStudio ™ are both excellent choices since they are free to download and use for non-commercial purposes, have good manuals and a large user base. 1. Install a recent version of R using the CRAN website: http:// cran.
Start the latest version of R as administrator (in Microsoft Windows: right click on R in start menu or desktop icon and select "run as administrator") and run the following lines in R: source ("http://bioconductor. org/biocLite. R") biocLite("MassArray") install. packages(c("reshape", "ggplot2", "gplots", "devtools")) Select a CRAN repository near to your location, install packages. Of note: R may prompt you to update certain packages. This is often not required for MassArray to work properly, but is advisable to perform. 3. Next, download the files with our custom scripts from Github: https://git. lumc. nl/molepi/EpiTyper.
Procedures Amplicon Design Whether EpiTYPER is suitable for a study depends on the number of regions and samples you wish to measure. One experimental run entails 384 reactions resulting in 126 measurements in triplicate with additional genomic DNA (gDNA) and water control measurements. An EpiTYPER PCR amplicon ranges from 100 to 600 bp, but is typically between 250 and 450 bp. For tips on how to select a region to measure, see Note 1. After selecting a region of interest, the reference sequence is required as input for assay design software. The sequence of the region of interest can be obtained from various databases. The sequence should be annotated for deviations from the reference sequence (e. g. , Single Nucleotide Polymorphisms (SNPs)) that may affect the measurement. If a specific strain of a (model) organism is used, Sanger sequencing of the region of interest can be used to identify deviations from the reference sequence.
The steps outlined below are based on the human genome. 1. Browse to the UCSC genome browser http://genome. ucsc. edu/, go to "Genome" and select the most recent human genome build  (Kent et al. , 2002). 2. Select the region of interest with 300 base pairs upstream and downstream of the target region to allow some flexibility for the assay design software. 3. To control for genetic variation and repeat sequences load the latest common SNP (dbSNP ≥1% of samples, under the "Variation" header) and RepeatMasker tracks (under the "Repeats" header) and set them to "full" view. 4. Go to the "View" tab at the top right of the genome browser and select "DNA". 5. Press the extended options tab. Highlight the SNPs and repeats. Make sure that each feature is easily and uniquely identifiable. 6. Press "submit" and copy the DNA sequence into a text editor.
When the software has trouble designing primers please see: Note 2. 15. Export the designs at the bottom of the page as "Order" and as "Detailed". 16. In the "Order" file rename the primers with suitable and unique identifiers. The small letters in the primer sequence denote tags added for the EpiTYPER biochemistry. They are required for the transcription step, do not remove them. 17. From the "Detailed" file take the genomic sequences for each chosen amplicon and save them in separate tab delimited files using a text editor like notepad. 22. This file contains the sequences and masses of each so-called "CpG unit", that is a fragment containing one or multiple CpG sites. The CpG sites are numbered from the 5 ′ end of the genomic sequence. After each unit there might be a warning message. If the fragment has another (non-) CpG containing fragment within a few Dalton in the mass spectrum a warning is given.
To automatically check for such fragments start R (see Materials and Equipment) and load the MassArray package in R. Run the following line in R: library(MassArray) 24. Open the tab delimited file of the suitable amplicons and copy the sequence. Denote the end and start of your forward and reverse primer with > and < respectively. Run in R: Name_of_amplicon <-"paste_the_amplicon_sequence_here" Example: Gene1 <-"ATTCCTGGG>ATCGATCGGGAAAATTTCGC GAAAA<GCCCTAATTAA" 25. Then run in R: ampliconPrediction(Gene1) A figure now appears showing multiple fragmentation profiles of your amplicon. Look at the T(-) fragmentation picture. CpG sites which can be measured are denoted with numbers in blue. Those who cannot be measured are denoted in red. More importantly look for a green fragment. These are fragments that can be used to assess the bisulfite conversion, a key experimental parameter.
• When uploading the sample list to the plate editor software the software will automatically sort the sample list. We advise the following naming convention of your sample list: a number from 001 to 384 denoting the well on the 384 PCR plate, followed by a number unique for each sample (note that the software does not handle text), followed by a number denoting if it is the first, second or third measurement for that sample. These numbers are separated by an underscore. Examples: 001_5186_01; 212_5186_02; 310_5186_03. Well 001-012 should denote the first column on the 384 plate. Use of this format allows a quick automated creation of the plate in the EpiTYPER software and allows the use of an R script we wrote to automatically perform the data pre-processing (Note 6).
Data Pre-processing and Clean Up After the measurement on the mass spectrometer the data needs to be pre-processed. 1. Open your measurement in the EpiTYPER R Analyser software and select one PCR amplicon. 2. Manually inspect the spectra of multiple samples for irregularities by eye. For details on what to look for see Note 5. Please note that manual inspection of the spectra for measurement noise and peaks not expected based on the amplicon sequence is essential for reliable and high quality EpiTYPER data. 3. Go to "File", "export grid" and check the boxes at "rotate table data" and "Include comments". Save the file as "Tab separated values ( *. txt)" with a unique name and in a unique directory folder. This is the file to perform data pre-processing and quality control on. 4. Exported data for plates with multiple primers will have many NAs for the wells on which another PCR was performed.
Analyzing DNA Methylation Data The optimal way to analyze the DNA methylation depends on the project, sample size, and distribution of your DNA methylation data and covariates of interest. Bisulfite conversion plate and also the EpiTYPER experimental plate may be included as covariates in the statistical analysis of larger projects, as they may have an influence on DNA methylation values. For single CpG sites linear regression often suffices. When one wishes to analyze a region as a whole two approaches are often seen in literature. First, the average of the CpG sites can be calculated per sample. This approach causes some information to be lost, but is straightforward in its analysis and interpretation. Linear mixed models (LMMs) may be employed  (Loke et al. , 2013). These take into account all information on all single CpG units.
BS-PCR Validation of LEP Next, we tested the PCR of this amplicon using 8 test gDNA samples that were extracted from whole blood and bisulfite converted as described earlier. We included additional controls, namely non-converted genomic DNA and several PCR reactions with water added instead of gDNA. One clear band was observed on a 1. 5% agarose gel (data not shown), but the intensity of the band was not that strong. Therefore, the PCR was redone with a 3 • C lower annealing temperature (50 • C) in the last step of the step down protocol PCR protocol. This yielded a strong and still unique PCR product on gel (data not shown) and we chose to measure this amplicon on a small number of samples using EpiTYPER.
Four out of 5 CpGs are measured in separate fragments. The mass of the fragments range from 2260 Da till 4525 Da and no other fragments are located with 15. 9 Da, indicating that the measurement will not be influenced by the signal for another fragment. Also no warnings are given of possible overlaps between CpG units. Also no CpG unit has the same mass as another. When looking at some CpG units in more detail the spectra were smooth (Figure  5B ), again indicative of a good quality measurement. Next, we exported the raw data from the EpiTYPER software and found that the EpiTYPER software has given a warning for the CpG site that cannot be measured (Table  5 ); its mass is too low to be measured. For CpG unit 3 an additional remark is given, namely "SN3", indicating that a non-CpG fragment overlaps the unmethylated CpG unit 3 fragment.
Notes Note 1. Advice on Where to Measure In many instances one starts with a gene or set of genes. The genomic regions in such instances are too large to assess in their totality and it is currently largely unknown which CpG sites have a regulatory potential. For some genes a thorough literature search may yield some insights. However, for most genes little is known. The standard deviation and filtered final methylation values (ratio) is given. CpG unit 1 has a low mass and cannot be measured (data is all missing: NA) while CpG unit 3 has a warning message ("SN3") denoting a non-CpG containing fragment overlapping the unit. This results in an overall higher technical variability between the triplicates, resulting that only 3 out of 10 samples have a measurement passing the QC steps. many epigenetic datasets can be visualized.
The Roadmap project also has a dedicated browser. All links can be found at: http:// www. roadmapepigenomics. org/. ENCODE has an "experiment matrix", while the roadmap has a similar "epigenome atlas" containing overviews of the available data types and datasets. The data is continually evolving as both projects continue. Therefore, we limit ourselves to some general tips on which type of data might be a good starting point in your effort to prioritize regions for EpiTYPER measurements. • One may start by visualizing tracks containing DNA methylation data in tissues and cell lines relevant for the planned study. Regions exhibiting variation between tissues and cell types may be prioritized. Note 2. Problems with Designing a Bisulfite PCR Amplicon Sometimes it is difficult to design primers for some regions.
Note 5. Inspecting EpiTYPER ® Mass Spectra It is important to manually inspect a proportion of the generated EpiTYPER spectra in the Analysis software to identify possible issues with your biochemistry, mass spectrometer, or automated liquid handlers. • Open the EpiTYPER R software and check that the "T reverse" tab is activated. • Set the "Spectrum View" tab on "recalibrated. " • Look at multiple samples. Does the baseline of the spectrum run parallel with the x-axis? The vast majority of samples should have a low baseline. If lots of samples have arched shaped baselines there might be something wrong with the laser intensity or the spotter. Arched shaped spectra will negatively influence the quality of the measurement by inflating the background signal, especially in the lower mass ranges.
Consult with the manufacturer to verify the mass spectrometer settings are still optimal. • Zoom in to the lower mass ranges (1000-2000 Da) and higher mass ranges >7000 Da. Normally some variation is seen in outer mass ranges, indicating a slightly less optimal signal-tonoise ratio. Especially in the higher mass ranges the optimal detection cut-off may vary depending on the conditions during the spotting of the sample on the mass spectrometry plates. When the baseline becomes jagged and many additional peaks are called by the EpiTYPER software changes in the spotting conditions and/or mass spectrometer are needed. Note 6. Running the R Data Cleaning and Pre-processing Script. To aid data pre-processing, we provide an R script that handles the data pre-processing and data cleaning. • Scripts are available in GitHub using the following link: https://git.
lumc. nl/molepi/QC_Epityper. • Set your R working directory to the location of your exported DNA methylation data, have one unique directory with all DNA methylation data from all plates belonging to a particular project. Run the following line in R: setwd("C:/directory_with_DNA_methylation_data") • Load the scripts downloaded from Github to R by referring to the file as follows, run in R: source("C:/location_of_Supplemental_file/EpiTYPER. r") • Define samples you wish to exclude from analysis. For instance, the genomic DNA and negative controls. Run the following line in R, denoting any samples you wish to exclude with brackets: Donots <-c("negative_control", "gDNA1", "gDNA2") • Define the directory where the script can find all relevant EpiTYPER data for one project. Run the following line: directory <-getwd() • Do the pre-processing and QC.
Note 7. Qualitative Assessment of Bisulfite Conversion For a large project it is not feasible to do an assessment of the bisulfite conversion rate for all samples with Sanger Sequencing. After having confirmed the bisulfite conversion for a subset of your samples by Sanger Sequencing it is possible to get a qualitative assessment for each sample if you have a PCR amplicon with a so-called conversion control fragment. An additional data export from the EpiTYPER software is required to perform bisulfite conversion assessment using the MassArray package. For additional info and more details please refer to the Bioconductor MassArray page (on http://www. bioconductor. org/ search for "MassArray"). • Make sure to have the genomic sequence of your amplicon with conversion control available. Verify that no common SNP is present in this conversion fragment.
Use the latest common, ≥1% of samples, dbSNP version to this end. • Open the EpiTYPER R Analysis software. • Load one amplicon of your plate of interest. This script can only be performed per amplicon and per individual 384 plate. • Once you have opened an individual amplicon in the EpiTYPER R application, enable display of both the "T reverse" and "C reverse" tabs. • Then go to "View", "Enable/Disable Pane", make sure that both the "Show All Matched Peaks" and "Show All Missing Peaks" options are checked. • Go to "File", "Export Grid" and save the tab delimited file in a unique directory with a unique name. • Open R and run the script: library(MassArray) • Annotate your amplicon sequence with the start and end of your forward and reverse primers using > and < respectively. Example: Gene1 <-"ATTCCCAAAT>AATTTACGCGAATGGGGCC GCG<TAACCAACCTTTGG" Enter your amplicon in R and run the script: amplicon <-"paste_sequence_of_your_amplicon_here" • Set the work directory of R to the work directory of your exported tab delimited file by running the following lines in R: directory_name="C:/Directory_name_of_exported_file" setwd(directory_name) • Then processes your data, and run: data <-new("MassArrayData", amplicon, "Name_of_ Exported_file.