Squashed 'src/contrib/htslib/' changes from 1c631e9..063bee7

6a5b9fd vcf_parse: Detect incorrect number of columns
53c3ba8 bcf_write: Error message instead of assert + propagate the error up
7d61caf Remove val_unused -- use HTS_UNUSED instead
252d27d Add sam_open_mode() API function
aa68a05 vcf: Change of bcf_hdr_add_sample() usage.
97c1e18 synced_reader: Removed unused code
49610fc Set NM/MD creation to be on by default.
e8806c7 Fix endianness problems outwith CRAM code
0c360be Check index file magic numbers and I/O
34082df sam_hdr_write() now calls cram_load_reference if hts_set_fai_filename() has been called.  Also refs_load_fai() now copes with being given foo.fa or foo.fa.fai and does the right thing.
d3ec3e2 Merge API functions for index statistics (PR #83)
416f02a Return 0/-1 rather than bool; parenthesise C macro
885bba0 Improve hts_idx_get_stat to work for more file types
6622a18 Add functions for idxstats
be61038 Removed bogus "Unable to find ref name"* errors when converting a SAM file UR:local/path/name syntax and without finding an MD5 reference to use. The code previously went on to work fine as a subsequent refs2id call worked.
b62e668 Fix "unrecognized CIGAR operator" diagnostic
7e059de Add a comment to deobfuscate bam_cigar_type
e7bd15c Check CIGAR string has fewer than 2^16 operators
b3a7868 Remove RAZF source code
7d11028 Fix forgotten s/razip/bgzip/
8d283ed Sanitised the setting of fd->empty_container into one place. This also fixes the samtools bug #146.
e76b489 idx: Don't create broken .tbi or .csi when file unsorted (VCF/BCF/BAM)
191be31 Changed the initial CRAM EOF value to be 1 (expected EOF) and then explicitly set it to 0 at the start of each slice decode.
d49dfa6 vcf: New bcf_dup() function
2cc7ea4 bcf_sr: New missed_reg_handler and bcf_sr_regions_flush() function
165850e bgzf: Support the remaining optional GZIP header fields
02721b7 bgzf: Support for FCOMMENT gzip header
dd58092 Minor comment change
dc9a0fe bgzf: Support for reading gzip-compressed files.
8deb11d Merge branch 'bcftools+calling' into develop
fc4a18b New bcf_hdr_[sg]et_version API; bcf_hdr_subset preserves VCF version
9db893a For bcftools to work with CSI indexes up to 2^31-1
f99d5ba Fix bam_write1() error handling
70be28c Fix bgzf_write() and bgzf_flush_try() error handling
f1f2950 Merge branch 'develop' into bcftools+calling
5003f7b Add hts_set_threads() file I/O threading API
a1fb3bd vcf_parse: Detect missing INFO values in vectors
0557506 Merge CRAM indexing bug fixes (PR #85)
9ed346d Return "" from bam_flag2str(0) rather than NULL
eb71ced Better fix for container memory leak.
09ba271 Revert one of the previous memory leaks as it breaks multi-threading. I need to find a more suitable way to solve this. The memory leak is back, but it's tiny; essentially leaking the last (empty) container.
d3bf5b6 Fixed two small memory leaks.
1ce5791 bcf_write: sanity check the number of samples, resolves #55
996b3c0 Check sam_open() and sam_close() return values
104adb5 Silence -Wstrict-prototypes warning
41e8793 Merge branch 'bcftools+calling' into develop
db8055a bcf1_sync: Update rlen for new records as well
35f4642 Exit statuses are non-negative
e45b3f3 Fix bgzip and tabix dependencies and link commands
6e2ebc4 synced_reader: Regions operate on two as well as three columns again
5adb883 bcf_subset: Reset n_fmt fields when all samples are dropped
f38b5e2 Merge bgzip.c and tabix.c/.1 history from github.com/samtools/tabix
bd6ed8b .gitignore tabix and bgzip
f5c96ca Added bgzip and tabix
e321182 Bug fix to CRAM indexing and index usage.
4a5dcf4 bcf_remove_alleles to work with BCF output
388abc7 Fixed BCF output after bcf1_sync_alleles, simplified
df2c538 Silence -Wstrict-prototypes warnings
eed28cb bcf_remove_alleles: Fix indexing range for Number=A alleles
52ecc97 bcf_sr_seek to seek to start with seq=NULL
128d97d bcf_hdr_subset: To allow sanity checks, mark missing samples
3e52ddb Migrate documentation from samtools bam.h to sam.h and add a few more items
42d6aa1 Merge bam_dup1() addition (PR #80)
bb4721c faidx: Set error status if unknown seq is queried
503c95a Support for =X cigars in mpileup overlaps.
5c80322 Merge pull request #82 from sm15/feature/bcf_calc_ac_bugfix
acfeb8a make sure we have found values for AC _and_ AN before using
f7953db hts_readlist modifed to explicitly indicate file name vs list
b26a445 Propagate error from bcf_hdr_printf
9dec00e Construct HTS_IDX_REST and HTS_IDX_NONE iterators without an index
9d640ff Fixed off-by-one in bcf_get_format_values
3490604 vcf: Propagate error in bcf_hdr_append
d66571e Propagate error codes for CRAM_OPT_RANGE
cf693ad This resolves #49
cd410e1 Add duplication function for the bam1_t interface
cc207d8 hts_idx: fixed indexing of NO_COOR blocks
22dedbe Merge pull request #79 from spalte/comment_typo
ec3a15a Fixed comment typo
0180921 Define __attribute__ macros in a new (semi-public) header
d7f0ae1 Add MIT/Expat license boilerplate
fe173b6 Make hfile.h part of the public API
d91c9bb Merge branch 'bcftools+calling' into develop
fb23174 Fix typo in comment
321c402 Reinstate bam_header_dup as bam_hdr_dup
1435a5d Merge branch 'develop' into bcftools+calling
c6f8531 Add ks_release() to kstring.h
fca50b8 Merge branch 'develop' into bcftools+calling
5bdc06d synced_reader: Init position correctly
fee84e0 synced_reader: Init position correctly
20df6b9 Merge sam_format1() and __skip_tag() fixes (PR #73)
ab84296 Add sam aux field tests
30b4033 Rewrite __skip_tag() as skip_aux()
5c65920 vcf: Allow omitted trailing FORMAT fields
692bb41 Fix sam_format1 failure on lines ending in integer/float tags and issue #63
78e237b Fixed a bug when loading in fasta files consisting of all sequence on a single line and in lowercase.
1069b17 Merge #70, fetching of remote BAM/VCF indices
babaaf3 Use hFILE rather than knet in test_and_fetch()
cb7f6e9 fix remote fetching of regions for bam and vcf
0d95c21 Add "data:blahblah" in-memory hFILE backend (read-only)
3ea22a1 Add 'a' (append) mode letter; propagate mode letters
21ca653 Merge #68, fix incorrect length given to MD5 calculation
bdb6a4a Add HTS_IDX_NONE and document HTS_IDX_*
f6c37a5 Fixes incorrect MD5 calculation in cram_write_SAM_hdr
c38cf60 Keep the style of the surrounding code
bcbfe37 Ignore operation-not-supported errors in fd_flush
c336894 More sensible Quals of matching bases in overlaps
39558ce bcf_translate: Catch unchecked errors
5a340a1 Check cram_eof() before calling cram_close()
d4d37c0 synced_reader: New bcf_sr_region_done macro
728ad50 Merge pull request #62, minor CRAM updates
5e03160 Make synced reader aware of bcf_hdr_set_sample(), expanded docs
aac5df5 hts_idx_load: Print warning when index is older than data
85b864f To avoid confusion, make bcf_get_*_int and bcf_update_*_int symetric
2eb509e Reduced the number of realloc calls zlib_mem_inflate(), while hopefully not overallocating much either.
2279b3e Merge pull request #60, CRAM EOF block improvements
c62982e hts_expand0: type ptr properly, it can be of different type
0038f9e vcf: Don't let unpack_*_core forget the to-be-freed pointer
bae6c41 vcf: Abort on broken VCFs with multiple/trailing spaces/tabs
dc45cfc Bug fix to the SAM header padding. It could go negative and then fail to write the header.
322b28b Fixed EOF check. The usual code path had the correct check, but an empty cram triggered the incorrect code.
b932f95 The default CRAM version is now 2.1. This is now the official release for supporting EOF blocks and the Java implementation only enforces them for CRAM 2.1 and above.
0e76a59 Merge pull request #46, CRAM EOF blocks
025ecb3 Also initialise tid in case user code looks at it
fde4bd9 vcf: New bcf_get_info_flag() API
ad17bb8 bcf_sr_set_samples to use the new --sample convention
843a063 Fixed the iterator to be 1-based inclusive coordinates for CRAM.
c68d523 Initialise beg and end values in the hts iterator. Without these mpileup simply bails out after fetching the first column as end is 0.
19ccc09 Fixed initialisation of CRAM indexing when the CRAM file contains refids that aren't sequential.
939f6ea Merge branch 'develop' of https://github.com/samtools/htslib into develop
42152a6 synced_reader: Better handling of target alleles
655edab Add sam_itr_*() dealing with BAM and CRAM iterators interchangeably
4d123f2 hts_itr_querys() calls hts_itr_query() via a function parameter
f0fb825 Add basic test exercising bcf_itr_queryi and bcf_itr_querys
a8be0b8 Write strcmp() comparisons comprehensibly
a60ff3f Make search key arguments *const* char pointers
fd657c4 Store readrec function pointer within hts_itr_t
68848e2 bcf_hdr_set_samples() recognise lists vs file names
0eb4c88 Merge branch 'bcftools+calling' into develop
650202c Strict matching of strings in bam_str2flag
2daf614 Make filename argument a *const* char pointer
b729954 vcf: new experimental bcf_hdr_set_samples() for subsetting
9dbf592 Fix hseek() return value test
4622eaf vcf: New bcf_has_filter() API function
60c22d4 idx: Support for "." queries to retrieve all records
1d963d7 Initial version of bgzip and tabix with htslib.
ec581d6 Merge tabix and samtools bgzip.c and tabix.c
e7ffc82 vcf: New bcf_translate() and bcf_hdr_combine() API
600acc3 synced_bcf_reader: New bcf_sr_seek() function
96132f4 vcf: New vcf_write_line() function
2d0423a vcf: New bcf_hdr_id2hrec() and bcf_hrec_format() functions
b2aab8b hts indexing: Allow arbitrary chromosomal order.
b57379f vcf: Tip to workaround missing contig tags
4855f43 vcf: fix in bcf_get_genotypes, sanity check too strict for GT
03f3fd7 Merge branch 'bcftools+calling' into develop
36a8ce9 Merge branch 'develop' into bcftools+calling
c7c3ada vcf: new function bcf_remove_filter()
0caf597 Allow overriding default columns indexes in bcf_sr_regions_init
2b801d7 add support for Number=R fields when trimming alleles
f78d671 vcf: New API bcf_get_format_string()
312add9 vcf: New bcf_get_info_string() wrapper
78a6ef3 Merge branch 'develop' of git://github.com/samtools/htslib into feature/number_R
ea4dbc6 add support for Number=R fields when trimming alleles
76cfc12 Updated the Cram EOF block detection to avoid falsely considering the lack of an EOF block as an error when doing a sub-range query.
edb12c3 Merge branch 'develop' of https://github.com/samtools/htslib into develop
dc4d0f1 Added a check to EOF in Cram in hts_close(). Note that this isn't the same place that BAM checks as it does it in hts_open(), however the benefit of doing on close is that this check can work when inputting from a pipe.
cfdc6fd vcf: Make sure fileformat is added only once
06208f2 Updated test
b5ccf9b Merge branch 'develop' into bcftools+calling
997c519 Output VCFv4.2 header
3a4aac5 Bug fix to EOF block writing when in multi-threaded mode (not yet supported in htslib's cram interface). It was writing the EOF block before flushing any pending threads.
ae92627 Bug fix to cram_next_slice().  The new while loop checking for non-empty blocks caused a crash when multi-threading.
a6cbed2 Added some EOF block creation and checking.
a09bef9 -C to output lines not overlapping BED
34227f8 Added missing return code checks.  Fixed 'x' mode of mfreopen.
1a39edb Add mfsteal to mFILE, and use it in cram_populate_ref.
a935e3c Added refs_t->last_id (note distinct from ->last) for use when reading from MD5 server. This reduces excessive reloading of the file in certain cases.
70efcd9 Bug fix to the zlib strategy tuning in multi-threaded mode. It could (legitimately) drive the number of trials negative, but this meant it never left the trial phase and consequentially encoding speed more or less halved the instant we asked for more than one encoding thread.
3e96dd7 Added implementation of cram_external_encode(). It's not yet used except in experimental versions, but opens up the route to storing more CORE data in external blocks to make partial decoding simpler (eg CRAM->FASTA).
ebd9b94 Cope with empty containers internal to a CRAM stream. Technically empty containers shouldn't exist as the specification states they must contain at least one block. However this is a useful precursor to the EOF block checking.
9d6defa vcf: qual now set to missing on bcf_clear; Avoid warnings about type-punned pointers
9029156 vcf.c: Fixed a renaming artifact, bcf_clear1 should be bcf_clear
0c084f2 vcf: Output . for missing INFO, FMT and samples
f192318 vcf: Support for IDX BCF tag
5e770b3 vcf: bcf_update_format now sets the number of samples in VCF record which previously was responsibility of the user; Fix in bcf_itr_querys macro, non-existent bcf_name2id renamed to bcf_hdr_name2id
0908d4c synced_reader: Improved regions/targets to be faster with large number of sequences; More userfriendly regions_overlap() function
d49f20e faidx: Detect compressed BGZF fasta files and create .gzi index without asking to allow proper functioning of samtools faidx fasta.fa.gz
7092725 Merge branch 'bcftools+calling' into develop
5d609bc Merge branch 'develop' into bcftools+calling
5d68967 Merge branch 'bcftools+calling' into develop
cf12fba sam: New bam_str2flag and bam_flag2str functions for a more user-friendly setting of bitmasks. Removed the default pileup filters (except for BAM_FUNMAP) and moved the filtering to BAM-reading callbacks
ea407fe Merge branch 'develop' into bcftools+calling
626a888 Use the exact type in realloc(N * sizeof(TYPE*))
c8ec7cb Merge branch 'bcftools+calling' into develop
18a5382 vcf: Update rlen on BCF output, this is important in BCF indexing, rlen is used as it is. Todo: also END should be checked with symbolic alleles
715f5db bcf_remove_alleles: Assume diploid genotypes if the ploidy cannot be determined, better to proceed than crash.
6f5473a Merge branch 'develop' into bcftools+calling
06eeee1 Add HTS_NORETURN and annotate fail() as such
13b445b Fix pointer arithmetic [minor]
d6e1ad0 Use the exact type in malloc(N * sizeof(TYPE*)) etc
4e4501c Use $(LDFLAGS) & $(LDLIBS) when linking shared objects
bb71572 Ensure r is clearly always initialised [minor]
923191c l_qname is unsigned, so is always >= 0
bc53b5d Merge cram_index_build() from github.com/jkbonfield/htslib
7994a4b Merge branch 'bcftools+calling' into develop
20b8bf5 Resolved merge conflicts
223459a vcf: new APIs bcf_alleles2gt and bcf_gt2alleles; bcf_synced_reader: recognise compressed BED files
a8bc3b1 pileup: Detect non-recoverable errors and exit with proper status
69778ad Fix bgzf_write() for large (> 2GB) blocks
66c4f41 synced reader: Give a hint when tabix was used with wrong column indexes for regions/targets
ae0de02 synced reader: regions and targets now support VCF; Fix for match_alleles by reverting 0-terminating chr modification
d038d8e Change hts_close() to return int (error indication)
5b22605 synced reader: Support for BED regions/target files
ff10faf Add hclose_abruptly() for stack unwinding after errors
19beee0 bcf reader: Fix in chr:pos type of regions
65450dd hts_idx: check return status of bgzf_read
9cef38d tbx: Sanity check if the file type meets the expectation, detect non-numeric fields
69cf8a2 vcf: Support for regions in the form chr|chr:pos|chr:from-to|chr:from-
f1d9f6f Merge branch 'develop' into bcftools+calling
c16f20b Merge pull request #33 from sm15/develop
d1f589d bcf_gt_type can now, optionally, give the index of the 2nd alt allele for GT_HET_AA genotypes
0f9a088 Check return status of IO calls to prevent gcc 4.6.3 warnings
2b7806d Fixed return status checks of fread calls
6561746 Compile cleanly without warnings with gcc 4.6.3
f010407 Reinstate distinct bgzf_check_EOF() status for seek failure
1344221 Added the ability to do samtools index on a CRAM file.
bef7597 Further fixing of auxiliary tag decoding.
3e18c41 Merge branch 'develop' of https://github.com/samtools/htslib into develop
2591937 Remove now-unused fn_aux hts_open() parameter
2402fc0 Add hts_set_fai_filename(); parse .fai in sam_hdr_read()
3f3758d Removed old comment from earlier code.
1b2bc6e Added decoding checks for the most obvious of sam parsing failures.
941b400 Merge branch develop of github.com/jkbonfield/htslib
bebdfea Merge cram_set_header() from github.com/jkbonfield/htslib
6fd937f Robustness improvements following fuzz testing. (See io_lib r3500).
a3fd2b0 Merge branch 'develop' of https://github.com/samtools/htslib into develop
b3d93ad A couple more clang reported errors, which are duplicates of previous ones. I assume omitted before due to duplication.
b185d95 Fixed a bunch of code warnings produced by clang's static analyser.
d80899b Merge CRAM copyright notices and data race fixes from jkbonfield
18f6183 Handle erroneous modes with neither 'r' nor 'w'
c7612be Fixed a couple complaints from clang.
f149e78 Added prototype for cram_set_header()
808b36f Samtools. The test harness worked, but the samtools interface needed an additional cram_set_header function as we don't pass this over when opening the new cram file. (Try#2 - previous commit accidentally rolled back an upstream change from 2 hours before.)
c83a9c0 Merge branch 'develop' of https://github.com/samtools/htslib into develop
e2e0a4b Reverted a fix of "chr:pos-pos" index queries, the original version now works.
116ccd2 Fixed a variety of multi-threading data races, detected through 1000 repeated runs of the same command at approximately the 2% failure level on unsorted data only.
c413e8a Merge branch 'develop' into bcftools+calling
43586c9 The file type not necessary any more when opening synced reader
7eb91ef bcf_sr_open_reader(): Auto-detect input file format
44fc956 Format detection: non-compressed binaries; is_compressed setting
d37f6df hts_open(fname, "r") now works in both cases; no need for "rb"
091f27e test.pl: fail if any tests fail
1282758 Fix tbx_itr_next() when given a textual (is_kstream) htsFile
ba4e3fc Limit input hFILE buffers to 32K [temporary hack version]
65a3ed4 Require lowercase [rw] mode letters in bgzf_open() etc
b92504a Tab, newline, etc are not control characters
b02146d Merge branch 'io' into develop
3b9ba8e hts_open(fname, "r") detects format by peeking at the input
a1e76ad vcf: Print a comprehensible error message if unsupported version of BCF is encountered
6416ebf Use hFILE underneath cram_fd, and add cram_dopen()
dc77561 Move cram_seek() from cram_index.c to cram_io.c
ab5077b Merge branch 'bcftools+calling' into develop
d3803f3 Fixed errors in copyright due to pasting the MRC one and forgetting to change the organisation name. Oops!
b5f5f16 Added copyright notices.
10ef097 vcf: New bcf_hdr_printf() call
4294c7f Merge pull request #27 from jkbonfield/develop
d359060 Fixed CRAM to code with the 0x800 supplementary flag.
cf02167 Return bytes >= 0x80 from hgetc() correctly
638083f Get rid of %ld compilaiton warnings
1c9239f Add BAM_FSUPPLEMENTARY flag bit
1ee29f4 Add SAM and VCF man pages extracted from samtools.1
43e8305 hfile: Removed accidently commited local change in blksize
23d64b8 Merge branch 'develop' of github.com:samtools/htslib into develop
95321be Merge branch 'bcftools+calling' into develop
bc86c10 vcf module: Made the naming more consistent to avoid confusion about which of the vcf/bcf_* function variant to use.
3f4ecdf vcf: Tidied up vcf.h, made API more consistent and extended documentation
35621c0 synced_reader: Fixed an off by one error in bcf_sr_next_line() to properly match target alleles
07d5457 vcf.c: Call bcf_unpack() on user's behalf in all bcf1_get/update routines
aa50834 Disentangle CRAM header loopiness [minor]
9f6f905 vcf.h: New API call: bcf_get_genotypes()
a4736dc vcf.h: Add out of range check into bcf_idinfo_exists() as it always has to be checked elsewhere.
59c317e New feature in bcf_synced_reader: negate the sample selection in bcf_sr_set_samples by exclamation mark
bce13c4 vcf.c: Proper freeing of memory taken by dirty info/fmt tags when update routine called multiple times
037860b Add bam_endpos(const bam1_t *)
c6a1c65 Fixed a rather surprising bug in CSI index access via hts_itr_next, before it was not possible to obtain BCF record at chr:pos-pos. The fix may have unexpected consequences in other situations and file formats, beware!
e806853 Remove -Wc++-compat for the sake of the CRAM code
2ea61c2 bcf_index_build: Check if the BCF is compressed
7b85b13 Proper initialization in hts_readlines
26425fa Merge branch 'cram' into develop
de484cc vcf: Added bcf_get_variant_type[s]() call to avoid the need for explicit call of bcf_set_variant_types(); The mpileup's X allele is not a SNP; Get rid of htfile warnings
bbd6647 synced_reader: Added support for alleles in targets files to select best matching line out of possibly duplicate VCF records.
30cf9e9 (io_lib r3470) Fixed re-calling refs2id so it clears cached r->last; the pointer becomes invalid.
616a0a5 Enable warnings only if -Wno-unused-result works
fb94dac Build test_view with -pthread
7fe5680 Merge the CRAM test harness into current htslib
159a498 Add makefile dependencies for cram code
c41394f Move cram/ directory to the top-level
b3dccc2 Merge branch develop of github.com/jkbonfield/htslib into cram
265e2e7 Silenced a "pointer targets in initialization differ in signedness" warning.
8a76b46 (io_lib r3467)
95d7e53 (io_lib r3461 minus arithmetic coder & last_name experimental parts)
70a227b (io_lib r3460)
525c5bd (io_lib r3439)
f716f35 (io_lib r3438/7)
67a9231 Merge branch 'develop' of github.com:samtools/htslib into develop
b0312f8 bcf_index_build: Check return status of helper calls
d5bd42c Properly ignore hclose() in bgzf_[d]open functions
5ec2456 bgzf: Added ad-hoc checks to all hfile calls to keep compiler happy
1de141b Merge branch 'develop' of github.com:samtools/htslib into develop
4352ff1 Minor docs comment added
fd13d8c Fixes in targets/regions VCF synced reading, now regions work when streamed. Removed redundant get_fmt API calls
9dfcc35 Merge branch 'io' into develop
ac7844a Add bgzf_raw_read() and bgzf_raw_write()
29290bc Use hFILE instead of stdio for uncompressed output
193ac2f Fix -Wc++-compat warning / C++ compilation error
663f12e Replace "void *fp" by a union of the file pointers
aca31de vcf: Non-critical vcf_parse1() errors must not go unnoticed by vcf_write1(). Fixed offending \0\n added by bcf_hdr_subset()
e7a2de6 Change *_itr_next() to take htsFile*, not BGZF*
c9734f7 VCF header parsing: Handle pathological cases where less-than and greater-than signs are used in place of double quotes
799ed4a vcf: Support for string values in FORMAT; handling missing BCF_BT_CHAR values in bcf_fmt_array()
5e277dd synced_reader: Expanded docs - example of usage
4be3a31 Don't #include bgzf.h from other htslib headers
5a5fe13 Merge branch 'develop' of github.com:samtools/htslib into develop
78bb557 On Windows, call knet_win32_init before using knet
c0a0035 Remove unneeded cast
25459e9 Merge branch 'develop' of github.com:samtools/htslib into develop
a3dae45 vcf: prevent segfault when sample columns do not exist but FORMAT column does
3fa19a7 Lift mallocing & freeing to the generic hFILE code
5bbaed6 Fix typo [minor]
c371324 Also run our tests
5750966 Fix vcf_sweep.h dependencies
c88645c Merge branch 'io' into develop
655227b Add hfile*.[ch] to $(HTSLIB_ALL)
f20bae1 Use hFILE underneath BGZF, and add bgzf_hopen()
3c3d80a BGZF worker threads do no I/O themselves
1a7dae0 Add hdestroy_buffer() for use in hopen_*() cleaning up
5c1e1bb bgzip: Do not overwrite input files when -b or -s are given
d49e3f6 kseq,hts: Enable seek() on kstreams and hts_useek(),hts_utell() on htsFiles. New module which allows sweeping BCF/VCF files both fwds and bwds
a6e1c9f vcf: Minor bug fixes and API touches
60f315f synced_vcf_reader: new bcf_sr_get_line macro and fix in vcf/bcf index interator initialization
fd086ed bgzf: Fixed typo in block_address calculation in bgzf_seek
cb5cacc kfunc: Moved the kt_fisher_exact test from the original bcftools into htslib
441af2e Maintain our own hFILE offset within the stream
99f05d3 Flush or discard the buffer in hseek()
2b9a9d2 Add hseek() and htell() test cases
009dfb7 Refactoring [minor]
70b088e Warn if hFILE function return statuses are ignored
ff1465e This commit fixes issues in hts_open and bgzf with uncompressed BCF input, BGZF now detects uncompressed streams on the fly. Fixes in FILTER and FMT part of bcf1_sync.
1013c35 synced_reader,vcf: Big rewrite of synced reader to allow more flexible handling of regions and targets in future.
85ecdc2 Add buffered low-level input/output streams
55d6f39 vcf: Changed semantics of bcf_get_[info|format]*() ndst parameter: interpret as number of elements in the array, not the size in bytes. Allow overlapping memory blocks in bcf1_update_alleles
14a22ef vcf: Make sure GT comes always first in bcf1_update_format; Do not loose last newline in vcf_hdr_read; Updated tests.
8e00f34 Merge branch 'develop' of github.com:samtools/htslib into develop
818a5ed hts_open,bgzf_open: Changed semantics of the "u" flag. Originally "u" was equivalent to "0", which on output yielded uncompressed data in the zlib format. Newly "u" results in plain uncompressed output.
aac4601 Alphabetise, fix whitespace [minor]
b68bfbf Merge branch 'develop' of github.com:samtools/htslib into develop
f59d50f vcf.h: New bcf_get_info_* and bcf_get_format_* calls
12c677b Add faidx (FASTA index) description man page
919fd54 Keep the makefile non-recursive
1f9b3db Use static inline, not just inline
9946dc7 Fix union-based type-punning buglet
fb60f0c vcf.h: Reverted back to bcf_float_is_* macros as the inlined unions break vcfmerge code, found a solution with the original version.
c485827 Resolved merge conflict in Makefile, merged.
6af2c1d [hts_open,bgzf,vcf] New flags for uncompressed input/output to distinguish between compressed/uncompressed BCF/VCF. Updates and fixes in BCF update functions. Removed old tests (now in samtools/bcftools repo) and added new.
842b21c Actually $(NUMERIC_VERSION) is more generic
c2e9391 Replace HTS_VERSION with hts_version()
750f564 Use config.h for configuration options
fcb9563 constify input parameter to faidx_fetch_seq
8256ea8 Use strtoul() for other unsigned 'B' types too
83a9bdd Use strtoul instead of strtol for XX:B:I tags so that integers >2.14 billion work.
4b8c9b5 Bug fix to stop sam view from converting 'H' into 'Z' types.
ab92989 Bug fix to handle blank headers in SAM files
6d9e20e Update kstring.[ch] from upstream sources
039620b Update khash.h from upstream sources
13698ea Added tests/test_view
716070d tview and bgzip: Check that the files are bgzipped, not gzipped.
956f45b bgzf.c: Check return status of bgzf_read_block in bgzf_useek to catch gzipped (as opposed to bgzipped) files.
90ae62f Added BGZF indexing and reindexing capability to bgzip. Fixed -b seeking.
9ec7860 New API for BGZF indexing w.r.t uncompressed data. Extended BGZF to read uncompressed files. Moved from RAZF to BGZF in FAI indexing.
eeb41a7 Bug fix to khashification of process_one_read.
1e6f3cf Added some synthetic SAM test data and a test perl script to trial conversion between SAM, BAM and CRAM. A work in progress.
7012805 A bit of a hack to set the header and references up properly for cram. This is a hack right now as it requires accessing internals of the fp, but it should be fixed in future by properly supporting methods to manipulate these fields for all data types rather than just cram (eg see scram.c in io_lib).
0649ecb Bug fix to khash-ification of sam_hdr_add_lines
e25066b Better handling of unknown references.
016efd5 Use strtoul instead of strtol for XX:B:I tags so that integers >2.14 billion work.
5500ef0 Bug fix to stop sam view from converting 'H' into 'Z' types.
fa5d1a1 Bug fix to handle blank headers in SAM files.
417fc59 Switch from modifying CFLAGS to CPPFLAGS so we can do "make CFLAGS=-g".
47b7241 Further removal of various unusued portions of Staden io_lib.
6df8103 Merge branch 'develop' of https://github.com/samtools/htslib into develop
e79e112 Major overhaul of CRAM code (originally imported from Samtools/htlib branch, and further back from Staden Package io_lib).
2424c14 Merged with attractivechaos/klib: - add upstream kvsprintf and rework ksprintf - add upstream printf attribute checks - use our kputw (has MIN_INT bug fix) - keep our extra functions (kputc_, kputsn_, kputl)
6799348 Build shared libraries, add "make install" target
f455afd New non-recursive build infrastructure
dd9c5a2 Lift library source to the top-level directory
ad5ee37 Remaining application code also moving repository
7413bed Deleted files which were moved to the new samtools/bcftools.git repository
7508806 Merge branch 'develop' of github.com:samtools/htslib into develop
3c4259c vcfcheck: re-enabled stats by QUAL
cbc4478 Test with both Clang and GCC
518619b Added CRAM support to htslib and htscmd samview. This is just a port over from the samtools io_lib subdirectory and has all the same nasties and duplication of abstract data types.
bdb550f Move format specifications to sam-spec repository
779e243 Merge remote branch 'remotes/sm15-mod/develop' into develop
3ecd0cb vcfsubset: Make --apply-filters consistent with other tools
dbcc082 Resolved vcfmerge.c merge conflict
e47feba vcf core lib, vcfsubset, vcfmerge: First step towards binary BCF output. Functional but probably buggy.
348c5a1 Merge branch 'master' into develop
0fb04c2 Add Travis CI control file
eedca8d Merge branch 'razf_sync' of https://github.com/mp15/htslib into develop
0670753 Merge branch 'for-samtools' into develop
c3d530e vcffilter: Change in neighbourhood function in SOM training, seems more robust with respect to random seed
8496905 Avoid conflicting with samtools's bam_index()
b267fb4 Expose bam_hdr_init() in the public API header
7a5992c vcfnorm: Removed two assertions to allow realignment of MNPs
efa9530 vcfcheck: Fixed a bug in initialization of frame-shift calculation; More detailed FS output
a634ce9 vcffilter: FILTER column of hard-filtered sites now lists failed filters; Unset the FILTER of skipped sites only with -u switch, not by default.
cf03c20 Use htslib's bgzf.h, faidx.h, and razf.h
0c275f6 Maintain the same API regardless of -D BGZF_MT
fdcfa2f synced read and vcftools: Changed the -f, --apply-filters switch so that it accepts a list of allowed FILTERs. This is useful for example in cases when PASS has to be distinguished from "."
e567ef0 Add klist.h, as used by samtools
1994af5 vcfsubset: New -i,-e options for general filter expressions. In this version, only QUAL and INFO columns are supported.
4523040 plot-vcfcheck: Produce the overlap by AF graphs
45be6fe vcffilter,gtcheck: Added sanity checks; Extended help message; Changed the order of columns on gtcheck output
72b5b28 vcfcheck: Changed output format of Ts/Tv
d30f1aa resolved test/check.chk conflict
8876289 Replace with samtools's ksort
3f76a40 Updated tests
3164242 vcffilter &friends: Filtering of indels now supported. Changes in vcfcheck, plot-vcfcheck and vcfquery to reflect that. More changes and fixes to come, but this version seems stable and fully functional.
49f9e95 vcfmerge, vcfnorm: Append command and version string to VCF header
5da7aef vcf.[hc]: API to update FILTER column. BCF output not supported yet in this version
db1edf3 vcfutils: bcf_gt_type should not be declared as static
c420b54 add tests for vcfsubset
a7f4597 add new tool vcfsubset
a7f24e2 deal with Number=A and Number=G INFO and FORMAT fields when trimming alleles
c9873a3 correct typos
e91d10b tbx: Allow indexing of empty files, do not segfault
b281180 vcfview: Fix in bcf_subset; force unpacking on VCF output
ac479c3 Merge changes from samtools's kstring
a458ed8 Fix <stdarg.h> include
c3622b8 Revert to kstring.h with a separate kstring.c
f15f7bc vcfcheck: More readable output, replace "-" by "<STDIN>" when reading from standard input
8b095aa Port fixes from samtools razf.c/h to htslib.
8e14b56 vcfcheck: create additional ts/tv stats which counts 1st alternate allele only
f344f7a vcffilter: skip the two additional score fields when applying SOM filters
222fe6c vcffilter: First step towards indel filtering - new switch to select from good sites mask
fb0d88d vcfmerge: Fixed subtle bugs revealed when merging FMT fields of different lengths; Append version string and command to the output VCF header
5b49a22 vcf core lib: Added sanity check to catch up common problem makers, such as PL declared as Number=. instead of Number=G; Support for output of more than 64 alternate alleles; API to access bcfinfo_t header line information.
12b827f vcfcheck: Fixed a bug introduced by recent changed in synced reader
429216a vcffilter: backup commit, bad SOM now working
f6c95a1 Merge remote branch 'mp15/develop' into develop
5d717a2 vcfmerge, vcfisec: Change in synced reader's API, allow unlimited number of input files. (Previously the limit was sizeof(int)*8)
99b8b89 Add const to char *'s to ensure they can be called with fixed values without triggering compiler warnings
ce6022e Import header comments from samtools
46a4b6a Use sizeof() and NULL instead of numbers
974d20b Fix crash on legacy files
6a5a672 vcffilter: Plot also bad sites in annotation distributions. Updated test files.
f4bfe4a vcfquery: Enable -p option again
1dbfae8 vcfisec: in the sites output mode print also file mask
7d76f62 vcffilter: check to prevent division by zero in the python plot script
89f7e5d gtcheck: fixed incorrect output, instead of uncertainty, the opposite (confidence) was reported before
944c4b0 vcfmerge: fix of copy-and-paste error in format field merging, use correct integer ranges
35ed1b5 vcfgtcheck: Fix in INFO/DP parsing, wrong ID was used before
54868e8 vcffilter: Plot also cropped annotation distributions to show the range of values actually used in filtering
1ed4f54 vcfmerge: do not crash with non-overlapping chromosomes
aae973b vcfview: Restored the subset functionality (continued)
eaee02d vcfview: Restored the subset functionality
b1c5fea vcffilter: plot annotation distributions
19ca371 vcffilter: allow arbitrary order in -f and -l filtering options (X>value is same as value<X)
d01fcee vcffilter: extended help message
84901c3 bcf1_update_info fix: initialize bcf_info_t pointer, do not pass NULL
4cf9350 vcffilter: fixed a typo, output file names now named correcly based on filter type
7141bd5 vcfcheck: Do not run in streaming mode with the -t option
d09058b vcfcheck: complain if tabix index not available with -t option
2f5843a vcfquery fix: Allow a mask file with the -v option
73f147b A few bugfixes from the previous two commits. New -S option in gtcheck and -H made work also in the cross-check mode
eef8320 gtcheck: Removed forgotten debugging line
baa5708 VCF/BCF core lib: Allow missing values in vectors. Note that this is a major change which, in contrast to BCFv2.1 specification, allows missing values in vectors. For integer types, the values 0x80, 0x8000, 0x80000000 are interpreted as missing values and 0x81, 0x8001, 0x80000001 as end-of-vector indicators.  Similarly for floats, the value of 0x7F800001 is interpreted as a missing value and 0x7F800002 as an end-of-vector indicator. This trial BCF version (v2.2) is compatible with the VCF specification and enables to handle correctly vectors with different ploidy in presence of missing values.
c49a4ec bgzip now compiles
5b0880e vcfcheck: Do not crash at non-variant sites. Restore checking of two VCFs.
6d906ba core vcf: added API for modifying/deleting/adding INFO fields, currently only VCF output supported; vcfmerge: AN,AC tags now updated correctly.
8af2d0a Fix: do not drop first record with both -B and -h
f14716c More informative error message about missing or out-of-date tabix index
1344d39 Fix: allow custom formats
a33c73e Even better, do this only for reading
8abdff6 Assume GFF format when the file type cannot be guessed, as in the previous version
c2a5899 Guess the filetype from the file extension when -p not given
46b2463 Added option for printing header lines only
83afb06 updated BGZF
8ff4c17 Querying remote files required -f option on some systems
bec709b Release tabix-0.2.5 (r964)
046c761 Release tabix-0.2.4 (r949)
e4477ce update version number
87f69fe fixed two bugs due to recent changes
70e9ba8 update to the latest bgzf.*
7d48a19 BED support
1cdcb39 New -r (reheader) option for efficient header replacement.
4dedb83 Fix: Complain only when VCF is newer, not newer or same mtime
08d08a2 Prevent the common user mistake and check the timestamps of the vcf and index file
bbcc0b2 Release tabix-0.2.3
4a6e786 Disable "unknown target name or minus interval" warning.
2c0b7ee Added -h option to print header lines
128523b patches by Peter Chines
3c60871 fixed a bug in C/Java when n_off == 0
07f01ed Fix: Exit with an error rather than segfault when index is not present and region is queried
31c8bd3 Complain about not-bgzipped files and check for noncontinuous chromosome blocks
a39b546 Release tabix-0.2.2
a79e73a Release tabix-0.2.1
bccdc09 Release tabix-0.2.0
6fa709d * improved C/Perl APIs * added test for Perl * added an tiny example
c6770bb Release tabix-0.1.6
cf389c2 If nothing bad happens, this will become 0.1.6
50f5b5c * Release tabix-0.1.5 (r560) * Improve seeking efficiency. Index file needs to be rebuilt.
9f16a45 Release tabix-0.1.4 (r559)
9b14f2c * tabix-0.1.3-3 (r556) * fixed a small memory leak in knetfile * fixed a minor bug for remote downloading
4fb1b67 * tabix-0.1.3-2 (r555) * do not overwrite index file by default * a little code cleanup
79d6239 * added the format specification * fixed a typo in bgzip
7cacea8 The behaviour changed slightly to mimic gzip. Detect if std descriptors are connected to the terminal.
ac9fc9f Fix in src/dst file detection and slight change of behaviour
15159c7 Added the -l option for listing chromosomes
f79c886 removed a line of debugging code
81170d8 * tabix-0.1.3 (r543) * fixed another off-by-one bug
a2d85ec Release tabix-0.1.1
f667bfc Release tabix-0.1.0
446b6e4 documentation
e7dc13d * tabix-0.0.0-5 (r504) * fixed a critical bug in fetching data (a typo in fact) * support SAM (tested on ex1.sam) and VCF (not tested) * improve the command-line interface
05df526 * tabix-0.0.0-4 (r503) * index files are bgzf compressed
0f56c58 * tabix-0.0.0-3 (r502) * support meta lines (not tested) * I am going to make the index file in the BGZF format
e0afbb5 * tabix-0.0.0-2 (r501) * accelerate ti_readline()
dcd2f78 * tabix-0.0.0-1 (r500) * apparently working
7d22a01 bgzip is more like gzip in its command-line interface
0d51797 initial source code. It is BUGGY!
4c42d43 Changed the mode for newly created files to 0666. This allows less strict permissions with umask properly set (e.g. 0002 vs. 0022).
8b14e7f Change copyright notices now that MIT has approved open source distribution.
20b6f6d Create trunk copy

git-subtree-dir: src/contrib/htslib
git-subtree-split: 063bee7d7a66dc7acad6e987d9261ba84ed22586
Merge commit '835c6ff2167f6486446dffdc8a1bcd9bbe8d5f4f' into newcaller
Squashed 'src/contrib/htslib/' changes from 063bee7..894ebcf

894ebcf On open failure, set errno instead of printing message
fcd0bd8 vcf: Support for mpileup's symbolic <X> allele
b7c2a46 Merge REF_PATH fixes and libcurl removal (PR #111)
42775eb Suppress compression suffix search on default EBI MD5 service
642783e faidx: Exit with an error if run on gzip-ed files
56d740e bcf_hdr_combine: Check for tags of conflicting lengths
663b9cf bgzf: Translate windows line endings
7d31af0 synced_reader: Logical complements of target regions
eb27d19 vcf: Detect broken structured header lines
cc23df1 Fixed REF_PATH to handle URLs better.
5f406e5 Merge pull request #108 from MauricioCarneiro/develop
c158790 Add extern "C" around API to enable C++ linking
049f877 synced_reader: Consider non-variant sites as matching in comparisons
3b87c0e Improved bug fix of bcf1_sync() from the commit 2504243:
2504243 Fixed bug in bcf1_sync. The main cause of the bug is as follows: bcf_unpack(bcf_rec, BCF_UN_STR); bcf_update_id(bcf_rec, id); This causes bcf_rec->d.shared_dirty to be set. duplicate = bcf_dup(bcf_rec); This calls bcf1_sync(). Since bcf_rec->d.shared_dirty is set, the shared data is re-created in tmp.s, bcf_rec->shared.s is freed and bcf_rec->shared.s = tmp.s. However, bcf_rec->unpack_ptr and bcf_rec->d.info[i].vptr still point to the old shared.s region which is invalidated. The fix is done by assigning bcf_rec->unpack_ptr and the vptr fields after tmp.s is fully constructed in bcf1_sync. I use unpack_offset and offset_array to track the offsets of these fields with respect to tmp.s
a969d0f Fix in index initialization
9074f89 bcf_update_alleles: Reference length with INFO/END
aff3226 bcf_trim_als: Propagate error instead of assert
c8c5b31 vcf: Fix bcf_hdr_id2hrec, col_type parameter is needed as well
0d4e155 bgzf: Do not end prematurely on non-critical Z_BUF_ERROR zlib error
3cdd62f vcf: Fixes in bcf_translate
6172a08 vcf: Fix BCF syncing when only partially unpacked

git-subtree-dir: src/contrib/htslib
git-subtree-split: 894ebcfc275db6a57abd9f5054f46c4f57703930
Merge commit '910ea44abdc1b5ca3b4c04b6cc42336a6c765093' into newcaller