DOI: 10.5061/dryad.h988s
title: Data from: Prevention, diagnosis, and treatment of high-throughput sequencing data pathologies

description: High Throughput Sequencing (HTS) technologies generate millions of
 sequence reads from DNA/RNA molecules rapidly and cost-effectively,
 enabling single investigator laboratories to address a variety of “omics”
 questions in non-model organisms, fundamentally changing the way genomic
 approaches are used to advance biological research. One major challenge
 posed by HTS is the complexity and difficulty of data quality control
 (QC). While QC issues associated with sample isolation, library
 preparation, and sequencing are well known and protocols for their
 handling are widely available, the QC of the actual sequence reads
 generated by HTS is often overlooked. HTS-generated sequence reads can
 contain various errors, biases, and artefacts whose identification and
 amelioration can greatly impact subsequent data analysis. However, a
 systematic survey on QC procedures for HTS data is still lacking. In this
 review, we begin by presenting standard “health check-up” QC procedures
 recommended for HTS datasets and establishing what “healthy” HTS data look
 like. We next proceed by classifying errors, biases and artifacts present
 in HTS data into three major types of “pathologies”, discussing their
 causes and symptoms, and illustrating with examples their diagnosis and
 impact on downstream analyses. We conclude this review by offering
 examples of successful “treatment” protocols and recommendations on
 standard practices and treatment options. Notwithstanding the speed with
 which HTS technologies–and consequently their pathologies–change, we argue
 that careful QC of HTS data is an important–yet often neglected–aspect of
 their application in molecular ecology, and lay the groundwork for
 developing a HTS data QC “best practices” guide.
description: The high quality
 Illumina short read dataset used for the illustration in Figure
 1The
 high quality Illumina short read data was generated on Illumina HiSeq2000
 platform with 100bp read length for a fungal genomic sequencing
 experiment. The first 7bp of each read is barcode sequence and has been
 removed.96.131.4.filtered.fastq.bz2Commands used for all QC tool evaluationscommands.docxSimulated Illumina short read datasets used for the evaluation of adapter trimming toolsAll data are 100bp paired-end Illumina reads simulated from the human genome (hg19.fa.masked) using pIRS which can simulate reads with realistic sequencing error profile and GC-bias. For paired-end reads simulation, pIRS selects randomly select insert sizes from a normal distribution with preset mean and standard deviation values. To simulate adapter contamination, the pIRS was modified so that when selected insert sizes are shorter than the preset read length (100bp in this study), adapter sequences will be concatenated at ends of genomic sequences before subsequent simulation of sequencing errors. Multiple simulations were performed with mean values of insert size ranging from 80 to 130 in order to approximate high to low levels of adapter contamination (the mean value used for each simulation was included in the file name of simulated dataset). All simulations were done with standard deviation of 10bp and 0.1x coverage.simulated_reads.tar.bz2Illumina short read dataset used for the illustration in Figure 3The Illumina short read dataset was generated for a fungal genomic sequencing experiment and likely contains cross contamination from other samples prepared in the same experiment.contamination_dataset.fq.bz2