TITLE: Instructions for recreating elPrep 4.0.0 WES benchmarks
AUTHORS: Charlotte Herzeel

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">Instructions for recreating the elPrep4.0.0 WES benchmarks used in the following paper:</div><div class = "text-block">Herzeel C, Costanza P, Decap D, Fostier J, Verachtert W. elPrep: A multithreaded framework for sequence analysis. BioRxv https://doi.org/10.1101/492249  </div></div>

[STEPS]
?. Configuration1.1 Hardware1.2 Software
These instructions have been tested with elPrep v.4.0.0. The following assumes that everything is performed from a working directory WORKDIR.
* 2x18-core Intel Xeon processor E5-2699v3 Haswell @ 2.3GHz* 256 GB RAM* 2x400 GB SSD
* Ubuntu 14.04.5 LTS* elPrep 4.0.1
?. Installation
The following steps are required to run elPrep:1. Download the elPrep binary distribution from https://github.com/ExaScience/elprepDirect download link: https://github.com/ExaScience/elprep/releases/download/v4.0.0/elprep-v4.0.0.tar.gz  2. mdkir elprep-v4.0.0 3. mv elprep-v4.0.1.tar.gz elprep-v4.0.04. cd elprep-v4.0.05. tar xvf elprep-v4.0.0.tar.gz6. PATH=$WORKDIR/elprep-v4.0.0:$PATH
?. Data preparation3.1 Required tools3.2 Required data3.3 Data preparation steps3.3.1 Create the reference index:3.3.2 Align the FASTQ files to create a BAM file:3.3.3 Create hg19 elfasta file:3.3.4 Create elsites files from vcf files:3.3.5 Unzip the BED file with captured regions:
Our WES benchmark uses the public data provided by the Genome in a Bottle Consortium (GIAB). This data consists of unaligned FASTQ files, but we offer an aligned BAM file for this data on our demo repository (see https://github.com/ExaScience/elprep/tree/master/demo). Otherwise, the following steps describe how to download and align the data yourself using BWA mem (version 0.7.17). Similarly, our benchmark requires the reference genome, databases with known SNPs and BED files to be converted into an elPrep-specific format. Again, these files can be downloaded from our demo repository. Otherwise, the following steps describe how to download the data from public repositories and creating the elPrep-specific conversions.
1. Ensure GCC installed (version 4.8.4 recommended)2. Download BWA source code from https://github.com/lh3/bwa Direct link: https://github.com/lh3/bwa/releases/download/v0.7.17/bwa-0.7.17.tar.bz23. tar xvf bwa-0.7.17.tar.bz24. cd bwa-0.7.175. make6. cd $WORKDIR
FASTQ and BED files* Download GIAB whole-exome NA12878, FASTQ and BED files from https://github.com/genome-in-a-bottleDirect links: ftp://ftp-trace.ncbi.nih.gov/giab/ftp/data/NA12878/Garvan_NA12878_HG001_HiSeq_Exome/NIST7035_TAAGGCGA_L001_R1_001.fasq.gzftp://ftp-trace.ncbi.nih.gov/giab/ftp/data/NA12878/Garvan_NA12878_HG001_HiSeq_Exome/NIST7035_TAAGGCGA_L001_R2_001.fastq.gzftp://ftp-trace.ncbi.nih.gov/giab/ftp/data/NA12878/Garvan_NA12878_HG001_HiSeq_Exome/nexterarapidcapture_expandedexome_targetedregions.bed.gzReference files* Download the hg19 reference files from https://software.broadinstitute.org/gatk/download/bundleDirect links:ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/hg19/ucsc.hg19.*When attempting a download, this may result in an error message that the login is incorrect. This is because the ftp site only allows a maximum of 25 users at the same time. If this happens, try again.
Known variants* Download the database with known SNPs from https://software.broadinstitute.org/gatk/download/bundleDirect links:ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/hg19/Mills_and_1000G_gold_standard.indels.hg19.sites.vcf.gzftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/hg19/dbsnp_138.hg19.vcf.gz
?. Benchmarking elPrep4.1 elPrep filter mode4.2 elPrep sfm mode
elPrep provides a lot of filtering options, as well as two modes to execute it. The following benchmark implements a pipeline that executes the following four steps:1. Sorting by coordinate order (equivalent to, for example https://software.broadinstitute.org/gatk/documentation/tooldocs/current/picard_sam_SortSam.php)2. Marking PCR and optical duplicates (equivalent to, for example, https://software.broadinstitute.org/gatk/documentation/tooldocs/current/picard_sam_markduplicates_MarkDuplicates.php)3. Base quality score recalibration (equivalent to, for example, https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_hellbender_tools_walkers_bqsr_BaseRecalibrator.php)4. Applying base quality score recalibration (equivalent to, for example, https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_hellbender_tools_walkers_bqsr_ApplyBQSR.php)Please see the elPrep documentation at https://github.com/ExaScience/elprep for further filtering options.As for the execution options, elPrep can be run in an in-memory mode (filter) or a mode that first splits the input into smaller chunks, operates on each chunk to produce partial results, and merges the partial results into a final output file (sfm). The filter mode uses significantly more RAM than the sfm mode, but also runs significantly faster.
Each invocation produces the same BAM file as output. See the Statistics section below to double-check whether the BAM file is correctly processed. Please delete the output files before each rerun of elPrep.