Fixed the compressed header. The second block is now correctly
labelled as using method RAW not GZIP, and also uses \0 instead of #
as the padding byte.


git-svn-id: svn://svn.code.sf.net/p/staden/code/io_lib@3693 ba6f2710-ca9f-427c-89d3-5bf716dbc8e1
Experimental support for LZ4.
Deemed not particularly worth it as the RANS codecs are sufficient for
fast encoding.

A fairly exhaustive test on a set of PacBio and Illumina reads:

Scramble/CRAM      Ppacbio        Illumina
(-0)           1279035384 17.1 1264171548 10.6
LZ4             892970642 21.1  532076895 13.3
LZ4-1MB         842081646 22.4  517941513 14.4
LZ4->RANS0      603214678 28.9  392491440 17.7
RANS0           448346050 20.7  356666227 14.9
RANS1           405014990 23.7  309875733 18.4
RANS0|RANS1     409071073 22.1  305988028 17.5
LZ4|RANS0       444212301 25.7  349938914 14.8
LZ4|RANS1       401810473 24.5  308939789 18.2
GZIP_RLE        467484322 36.8  365212371 27.3
(-1)            463002372 37.3  342941521 27.0
RLE|RANS0|RANS1 409061304 27.3  305794646 18.3
(-1i -J)        406583970 21.1  305735590 17.9
(-1 -J)         406332918 25.3  302421923 18.8
RLE|RANS0       448336281 23.3  354670548 17.4
RLE|RANS1       403952590 27.0  305292281 18.9
(-J)            399355251 51.1  250455744 29.1
(-jJ)           359153769 84.0  243842159 45.8

Samtools/BAM
samtools -1     545534074 24.8  493027166 21.3
samtools -1i    743555031 11.0  610226168 10.7

(args) implies scramble with those arguments, otherwise it comes from
directly modifying method and methodF in cram_encode.c to experiment.
"-1i" implies Intel's Zlib -1 (via LD_LIBRARY_PATH change).

For CRAM maybe -2 should be GZIP_RLE|RANS0/1 and -1 should be RANS0 only?



git-svn-id: svn://svn.code.sf.net/p/staden/code/io_lib@3694 ba6f2710-ca9f-427c-89d3-5bf716dbc8e1
Experimental support for LZ4.
Deemed not particularly worth it as the RANS codecs are sufficient for
fast encoding.

A fairly exhaustive test on a set of PacBio and Illumina reads:

Scramble/CRAM      Ppacbio        Illumina
(-0)           1279035384 17.1 1264171548 10.6
LZ4             892970642 21.1  532076895 13.3
LZ4-1MB         842081646 22.4  517941513 14.4
LZ4->RANS0      603214678 28.9  392491440 17.7
RANS0           448346050 20.7  356666227 14.9
RANS1           405014990 23.7  309875733 18.4
RANS0|RANS1     409071073 22.1  305988028 17.5
LZ4|RANS0       444212301 25.7  349938914 14.8
LZ4|RANS1       401810473 24.5  308939789 18.2
GZIP_RLE        467484322 36.8  365212371 27.3
(-1)            463002372 37.3  342941521 27.0
RLE|RANS0|RANS1 409061304 27.3  305794646 18.3
(-1i -J)        406583970 21.1  305735590 17.9
(-1 -J)         406332918 25.3  302421923 18.8
RLE|RANS0       448336281 23.3  354670548 17.4
RLE|RANS1       403952590 27.0  305292281 18.9
(-J)            399355251 51.1  250455744 29.1
(-jJ)           359153769 84.0  243842159 45.8

Samtools/BAM
samtools -1     545534074 24.8  493027166 21.3
samtools -1i    743555031 11.0  610226168 10.7

(args) implies scramble with those arguments, otherwise it comes from
directly modifying method and methodF in cram_encode.c to experiment.
"-1i" implies Intel's Zlib -1 (via LD_LIBRARY_PATH change).

For CRAM maybe -2 should be GZIP_RLE|RANS0/1 and -1 should be RANS0 only?



git-svn-id: svn://svn.code.sf.net/p/staden/code/io_lib@3695 ba6f2710-ca9f-427c-89d3-5bf716dbc8e1