FASTQ format
FASTQ format is a text-based format for storing both a biological sequence (usually nucleotide sequence) and its corresponding quality scores. Both the sequence letter and quality score are each encoded with a single ASCII character for brevity. It was originally developed at the Wellcome Trust Sanger Institute to bundle a FASTA formatted sequence and its quality data, but has recently become the "de facto" standard for storing the output of high-throughput sequencing instruments such as the Illumina Genome Analyzer. Format. A FASTQ file normally uses four lines per sequence. A FASTQ file containing a single sequence  might look like this: The byte representing quality runs from 0x21 (lowest quality; '!' in ASCII) to 0x7e (highest quality; '~' in ASCII). Here are the quality value characters in left-to-right increasing order of quality (ASCII): <pre> Color space. For SOLiD data, the sequence is in color space, except the first position. The quality values are those of the Sanger format. Alignment tools differ in their preferred version of the quality values: some include a quality score (set to 0, i.e. '!') for the leading nucleotide, others do not. The sequence read archive includes this quality score. Simulation . FASTQ read simulation has been approached by several tools. A comparison of those tools can be seen here. Compression. General compressors. General-purpose tools such as Gzip and bzip2 regard FASTQ as a plain text file and result in suboptimal compression ratios. NCBI's Sequence Read Archive encodes metadata using the LZ-77 scheme. General FASTQ compressors typically compress distinct fields (read names, sequences, comments, and quality scores) in a FASTQ file separately; these include Genozip, DSRC and DSRC2, FQC, LFQC, Fqzcomp, and Slimfastq. Reads. Having a reference genome around is convenient because then instead of storing the nucleotide sequences themselves , one can just align the reads to the reference genome and store the positions (pointers) and mismatches; the pointers can then be sorted according to their order in the reference sequence and encoded, e.g., with run-length encoding. When the coverage or the repeat content of the sequenced genome is high, this leads to a high compression ratio. Unlike the SAM/BAM formats, FASTQ files do not specify a reference genome. Alignment-based FASTQ compressors supports the use of either user-provided or "de novo" assembled reference: LW-FQZip uses a provided reference genome and Quip, Leon, k-Path and  KIC perform de novo assembly using a de Bruijn graph-based approach. Genozip can optionally use a reference if the user provides one, which may be a single- or multi-species reference file. Explicit read mapping and "de novo" assembly are typically slow. Reordering-based FASTQ compressors first cluster reads that share long substrings and then independently compress reads in each cluster after reordering them or assembling them into longer contigs, achieving perhaps the best trade-off between the running time and compression rate. SCALCE is the first such tool, followed by Orcom and Mince. BEETL uses a generalized Burrows –Wheeler transform for reordering reads, and HARC achieves better performance with hash-based reordering. AssemblTrie instead assembles reads into reference trees with as few total number of symbols as possible in the reference. Benchmarks for these tools are available in. Quality values. Quality values account for about half of the required disk space in the FASTQ format (before compression), and therefore the compression of the quality values can significantly reduce storage requirements and speed up analysis and transmission of sequencing data. Both lossless and lossy compression are recently being considered in the literature. For example, the algorithm QualComp performs lossy  compression with a rate (number of bits per quality value) specified by the user. Based on rate-distortion theory results, it allocates the number of bits so as to minimize the MSE (mean squared error) between the original (uncompressed) and the reconstructed (after compression) quality values. Other algorithms for compression of quality values include SCALCE and Fastqz. Both are lossless compression algorithms that provide an optional controlled lossy transformation approach. For example, SCALCE reduces the alphabet size based on the observation that “neighboring” quality values are similar in general. For a benchmark, see. As of the HiSeq 2500 Illumina gives  the option to output qualities that have been coarse grained into quality bins. The binned scores are computed directly from the empirical quality score table, which is itself tied to the hardware, software and chemistry that were used during the sequencing experiment. Genozip uses its DomQual algorithm to compress binned quality scores, such as those generated by Illumina or by Genozip's own "--optimize" option which generates bins similar to Illumina. Encryption. Genozip encrypts FASTQ files (as well as other genomic formats), by applying the standard AES encryption at its most secure level of 256 bits ("--password" option). Cryfa uses  AES encryption and enables to compact data besides encryption. It can also address FASTA files. File extension. There is no standard file extension for a FASTQ file, but .fq and .fastq are commonly used.